mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
perf(engine): content-addressed extraction cache for video frames (#446)
## What Adds a content-addressed cache for extracted video frames, keyed on the tuple `(path, mtime, size, mediaStart, duration, fps, format)`. Repeat renders of the same composition (studio edit → re-render, preview → final) skip the ffmpeg extraction entirely. ## Why Video frame extraction is the dominant non-capture phase for video-heavy compositions. Studio iteration workflows extract the same frames over and over — each render burns ffmpeg time that adds no value. Validated on `/tmp/hf-fixtures/cfr-sdr-cache`: ``` Cold (miss): extractMs=69, videoExtractMs=70, totalElapsedMs=2052 Warm (hit): extractMs=1, videoExtractMs=2, totalElapsedMs=1964 cacheHits: 0→1, cacheMisses: 1→0 ``` The fixture is tiny (3s CFR SDR @ 30fps), so the wall-clock delta is small; the extraction-time delta (69→1ms, 98%) scales linearly with source length. For heavy-iteration workflows (a user rendering the same composition while tuning encoding params), extraction time goes to zero on every repeat render. Depends on #444 (instrumentation surface) and #445 (segment-scope HDR preflight — otherwise cache keys would be unstable across renders on mixed-HDR compositions). ## How - New `packages/engine/src/services/extractionCache.ts`: - SHA-256 key over a stable JSON encoding of `(path, mtime_ms, size, mediaStart, duration, fps, format)`. Infinity duration is normalized to `-1` so unresolved natural-duration sources still produce stable keys. - Truncates to 16 hex chars in the entry directory name — 64 bits of entropy is plenty at cache scale and keeps `ls` output short. - `hfcache-v2-` schema prefix — bumping it invalidates old entries (callers own gc policy; the cache owns keys). - `.hf-complete` dotfile sentinel. An entry dir without the sentinel is treated as a miss (covers crash-mid-extract and abandoned writes); the next render re-extracts over the partial frames with `-y`. - `FRAME_FILENAME_PREFIX = "frame_"` shared with the extractor — future refactors only need to touch one place to rename frames. - `EngineConfig.extractCacheDir` (env: `HYPERFRAMES_EXTRACT_CACHE_DIR`) gates the feature. Undefined disables caching — extraction runs into the render's workDir and cleanup removes it on render end, preserving the prior behaviour exactly. No default root is chosen by the engine; the caller (CLI, app, studio) owns the location policy. - `ExtractedFrames.ownedByLookup` flag prevents `FrameLookupTable.cleanup` from rm'ing a shared cache dir at render end. Set to `true` on both hits and misses (misses own the directory they wrote into, but hand it over to the cache rather than deleting it). - Phase 3 extractor flow: 1. Snapshot `(videoPath, mediaStart, start, end)` per resolved video BEFORE Phase 2a/2b preflight mutates them — so cache keys are stable across renders that use workDir-local normalized files (those files have fresh mtimes every render). 2. Compute key, `lookupCacheEntry`. 3. On hit: rebuild `ExtractedFrames` from the cache dir plus the Phase 2-probed `VideoMetadata` — no re-ffprobe. 4. On miss: `ensureCacheEntryDir`, extract with `extractVideoFramesRange(..., outputDirOverride)`, then `markCacheEntryComplete` (the sentinel write is the last step so a crash leaves the dir un-sentineled). - `extractVideoFramesRange` gains an `outputDirOverride` parameter so cache-miss writes land directly in the keyed dir (no `join(outputDir, videoId)` wrapping). ## Test plan - [x] 19 unit tests in `extractionCache.test.ts` covering key determinism, mtime/size invalidation, format/fps/mediaStart/duration invalidation, Infinity normalization, sentinel semantics, missing-file tolerance - [x] 2 integration tests in `videoFrameExtractor.test.ts`: - "reuses extracted frames on a warm cache hit" — asserts `cacheHits=1`, `extractMs<50ms` on second call against a CFR SDR fixture - "invalidates the cache when fps changes" — different fps on second call forces a new miss - [x] End-to-end validation with `HYPERFRAMES_EXTRACT_CACHE_DIR` set, two runs of the same fixture - [x] Lint + format (oxlint + oxfmt) - [x] Typecheck (engine + producer)
This commit is contained in:
@@ -72,6 +72,30 @@ export interface EngineConfig {
|
||||
/** Custom manifest path for Hyperframe runtime. */
|
||||
runtimeManifestPath?: string;
|
||||
|
||||
// ── Cache ────────────────────────────────────────────────────────────
|
||||
/**
|
||||
* Directory where the content-addressed extraction cache persists frame
|
||||
* bundles keyed on (path, mtime, size, mediaStart, duration, fps, format).
|
||||
* Undefined disables caching — extraction runs into the render's workDir
|
||||
* and cleanup removes it when the render ends, preserving the pre-cache
|
||||
* behaviour.
|
||||
*
|
||||
* **Single-writer.** The cache is not safe for concurrent renders pointing
|
||||
* at the same directory. A `.hf-complete` sentinel prevents another render
|
||||
* from serving an entry that hasn't finished extracting, but individual
|
||||
* frame files are written non-atomically — a second render reading during
|
||||
* the write window can observe a truncated frame. Give each concurrent
|
||||
* render pipeline its own `extractCacheDir`, or gate with an external mutex.
|
||||
*
|
||||
* **Network filesystems.** `mtime` resolution on NFS/SMB mounts can be
|
||||
* coarser than expected (seconds rather than nanoseconds), which may
|
||||
* produce spurious cache hits if a source file is overwritten within the
|
||||
* same mtime tick. Local filesystems are the intended deployment target.
|
||||
*
|
||||
* Env fallback: `HYPERFRAMES_EXTRACT_CACHE_DIR`.
|
||||
*/
|
||||
extractCacheDir?: string;
|
||||
|
||||
// ── Debug ────────────────────────────────────────────────────────────
|
||||
debug: boolean;
|
||||
}
|
||||
@@ -203,6 +227,8 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
|
||||
|
||||
verifyRuntime: env("PRODUCER_VERIFY_HYPERFRAME_RUNTIME") !== "false",
|
||||
runtimeManifestPath: env("PRODUCER_HYPERFRAME_MANIFEST_PATH"),
|
||||
|
||||
extractCacheDir: env("HYPERFRAMES_EXTRACT_CACHE_DIR"),
|
||||
};
|
||||
|
||||
// Remove undefined values so they don't override defaults
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
import {
|
||||
COMPLETE_SENTINEL,
|
||||
FRAME_FILENAME_PREFIX,
|
||||
SCHEMA_PREFIX,
|
||||
cacheEntryDirName,
|
||||
computeCacheKey,
|
||||
ensureCacheEntryDir,
|
||||
lookupCacheEntry,
|
||||
markCacheEntryComplete,
|
||||
readKeyStat,
|
||||
type CacheKeyInput,
|
||||
} from "./extractionCache.js";
|
||||
|
||||
const keyFor = (videoPath: string, overrides: Partial<CacheKeyInput> = {}): CacheKeyInput => {
|
||||
const stat = readKeyStat(videoPath);
|
||||
if (!stat) throw new Error(`keyFor fixture missing on disk: ${videoPath}`);
|
||||
return {
|
||||
videoPath,
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
mediaStart: 0,
|
||||
duration: 3,
|
||||
fps: 30,
|
||||
format: "jpg",
|
||||
...overrides,
|
||||
};
|
||||
};
|
||||
|
||||
describe("extractionCache constants", () => {
|
||||
it("exposes the v2 schema prefix", () => {
|
||||
expect(SCHEMA_PREFIX).toBe("hfcache-v2-");
|
||||
});
|
||||
|
||||
it("exposes the frame filename prefix shared with the extractor", () => {
|
||||
expect(FRAME_FILENAME_PREFIX).toBe("frame_");
|
||||
});
|
||||
|
||||
it("uses a dotfile sentinel so ls-without-A hides it", () => {
|
||||
expect(COMPLETE_SENTINEL.startsWith(".")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("computeCacheKey", () => {
|
||||
let tmpRoot: string;
|
||||
let sourceFile: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpRoot = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
|
||||
sourceFile = join(tmpRoot, "clip.mp4");
|
||||
writeFileSync(sourceFile, "fake-video-bytes", "utf-8");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (existsSync(tmpRoot)) rmSync(tmpRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const base = (videoPath: string): CacheKeyInput => keyFor(videoPath);
|
||||
|
||||
it("returns the same key for identical inputs", () => {
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey(base(sourceFile));
|
||||
expect(a).toBe(b);
|
||||
});
|
||||
|
||||
it("produces a 64-char hex SHA-256 digest", () => {
|
||||
const key = computeCacheKey(base(sourceFile));
|
||||
expect(key).toMatch(/^[0-9a-f]{64}$/);
|
||||
});
|
||||
|
||||
it("changes when path changes (moved files re-extract)", () => {
|
||||
const other = join(tmpRoot, "other.mp4");
|
||||
writeFileSync(other, "fake-video-bytes", "utf-8");
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey(base(other));
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it("changes when mediaStart changes", () => {
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey({ ...base(sourceFile), mediaStart: 1 });
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it("changes when duration changes", () => {
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey({ ...base(sourceFile), duration: 5 });
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it("changes when fps changes (different frame count invalidates key)", () => {
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey({ ...base(sourceFile), fps: 60 });
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it("changes when format changes", () => {
|
||||
const a = computeCacheKey(base(sourceFile));
|
||||
const b = computeCacheKey({ ...base(sourceFile), format: "png" });
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it("normalizes non-finite duration so Infinity doesn't produce unstable keys", () => {
|
||||
const a = computeCacheKey({ ...base(sourceFile), duration: Infinity });
|
||||
const b = computeCacheKey({ ...base(sourceFile), duration: Infinity });
|
||||
expect(a).toBe(b);
|
||||
});
|
||||
|
||||
it("changes when file content changes (mtime+size bump)", () => {
|
||||
const before = computeCacheKey(base(sourceFile));
|
||||
// Force an mtime change by waiting 5ms then overwriting with different bytes.
|
||||
// 5ms is well above the Linux mtime resolution (typically nanoseconds) and
|
||||
// below any Windows cache coherency window. Using a longer sleep pads against
|
||||
// coarse filesystem mtime granularity without slowing the suite.
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < 5) {
|
||||
/* spin */
|
||||
}
|
||||
writeFileSync(sourceFile, "different-bytes-longer-than-before", "utf-8");
|
||||
const after = computeCacheKey(base(sourceFile));
|
||||
expect(after).not.toBe(before);
|
||||
});
|
||||
|
||||
it("readKeyStat returns null for a missing source (callers skip the cache)", () => {
|
||||
// Previously readKeyStat returned a `{mtimeMs: 0, size: 0}` sentinel for
|
||||
// missing files; two unrelated missing paths then shared the same cache
|
||||
// key tuple and polluted the cache. The contract now returns null so
|
||||
// callers can explicitly skip the cache path and let the extractor
|
||||
// surface the real file-not-found error.
|
||||
const missing = join(tmpRoot, "does-not-exist.mp4");
|
||||
expect(readKeyStat(missing)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("cacheEntryDirName", () => {
|
||||
it("prefixes with the schema and truncates to 16 hex chars", () => {
|
||||
const full = "a".repeat(64);
|
||||
expect(cacheEntryDirName(full)).toBe(`${SCHEMA_PREFIX}${"a".repeat(16)}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe("lookupCacheEntry / markCacheEntryComplete", () => {
|
||||
let tmpRoot: string;
|
||||
let sourceFile: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpRoot = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
|
||||
sourceFile = join(tmpRoot, "clip.mp4");
|
||||
writeFileSync(sourceFile, "fake-video-bytes", "utf-8");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (existsSync(tmpRoot)) rmSync(tmpRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const base = (videoPath: string): CacheKeyInput => keyFor(videoPath);
|
||||
|
||||
it("misses on an empty cache root", () => {
|
||||
const lookup = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
expect(lookup.hit).toBe(false);
|
||||
expect(lookup.entry.dir.startsWith(tmpRoot)).toBe(true);
|
||||
});
|
||||
|
||||
it("hits after ensureCacheEntryDir + markCacheEntryComplete", () => {
|
||||
const first = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
ensureCacheEntryDir(first.entry);
|
||||
markCacheEntryComplete(first.entry);
|
||||
|
||||
const second = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
expect(second.hit).toBe(true);
|
||||
expect(second.entry.dir).toBe(first.entry.dir);
|
||||
});
|
||||
|
||||
it("treats an in-progress dir without the sentinel as a miss", () => {
|
||||
const lookup = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
ensureCacheEntryDir(lookup.entry);
|
||||
// Simulate abandoned extraction — frames written but sentinel never marked.
|
||||
writeFileSync(join(lookup.entry.dir, "frame_00001.jpg"), "x", "utf-8");
|
||||
const again = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
expect(again.hit).toBe(false);
|
||||
});
|
||||
|
||||
it("places entries under the cache root, not the source parent", () => {
|
||||
const subroot = join(tmpRoot, "cache-root");
|
||||
mkdirSync(subroot, { recursive: true });
|
||||
const lookup = lookupCacheEntry(subroot, base(sourceFile));
|
||||
expect(lookup.entry.dir.startsWith(subroot)).toBe(true);
|
||||
});
|
||||
|
||||
it("uses the same directory for identical inputs across lookups", () => {
|
||||
const a = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
const b = lookupCacheEntry(tmpRoot, base(sourceFile));
|
||||
expect(a.entry.dir).toBe(b.entry.dir);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* Content-Addressed Extraction Cache
|
||||
*
|
||||
* Video frame extraction is the single most expensive phase of a render
|
||||
* after capture. Repeat renders of the same composition (preview → final,
|
||||
* studio iteration) re-extract identical frames from the same source file,
|
||||
* burning ffmpeg time that adds no value. This module keys extracted frame
|
||||
* bundles on the (path, mtime, size, mediaStart, duration, fps, format)
|
||||
* tuple so re-renders resolve to a pre-extracted directory instead of
|
||||
* re-invoking ffmpeg.
|
||||
*
|
||||
* ### Scheme
|
||||
*
|
||||
* - The key is the SHA-256 of a stable JSON encoding of the tuple above.
|
||||
* - Cache entries live under `<rootDir>/<SCHEMA_PREFIX><key[0..16]>/` so
|
||||
* `ls` output and tracing logs stay short. Truncation to 16 hex chars
|
||||
* leaves 64 bits of entropy — collision risk at cache scale is negligible.
|
||||
* - A completed entry is marked by writing the `.hf-complete` sentinel file
|
||||
* after all frames are on disk. A dir without the sentinel is treated as
|
||||
* absent (stale/abandoned) and re-extracted into a fresh key (the old dir
|
||||
* is left for external gc — the cache owns keys, not deletion policy).
|
||||
*
|
||||
* ### Versioning
|
||||
*
|
||||
* `SCHEMA_PREFIX` bumps when the cache-contents invariant changes (e.g.
|
||||
* extraction format, frame layout). Old entries under the previous prefix
|
||||
* become inert and can be gc'd by the caller.
|
||||
*/
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdirSync, readdirSync, statSync, writeFileSync } from "node:fs";
|
||||
import { existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import type { VideoMetadata } from "../utils/ffprobe.js";
|
||||
|
||||
/** Filename prefix for extracted frames. Shared with the extractor. */
|
||||
export const FRAME_FILENAME_PREFIX = "frame_";
|
||||
|
||||
/** Sentinel filename written after a cache entry is fully populated. */
|
||||
export const COMPLETE_SENTINEL = ".hf-complete";
|
||||
|
||||
/** Current schema version. Bump when cache-entry layout changes. */
|
||||
export const SCHEMA_PREFIX = "hfcache-v2-";
|
||||
|
||||
/** Truncated hex chars of SHA-256 used for the entry directory name. */
|
||||
const KEY_HEX_CHARS = 16;
|
||||
|
||||
export type CacheFrameFormat = "jpg" | "png";
|
||||
|
||||
export interface CacheKeyInput {
|
||||
/** Absolute path to the source video file. Part of the key so moved files
|
||||
* re-extract rather than match by (size, mtime) alone. */
|
||||
videoPath: string;
|
||||
/** Source file modification time in ms (floored). Invalidates the key on edit. */
|
||||
mtimeMs: number;
|
||||
/** Source file size in bytes. Invalidates the key on content change. */
|
||||
size: number;
|
||||
/** Seconds into source the composition starts reading (video.mediaStart). */
|
||||
mediaStart: number;
|
||||
/** Seconds of source the composition uses. Infinity is normalized to -1
|
||||
* so callers that pass an unresolved "natural duration" still produce a
|
||||
* stable key across invocations. */
|
||||
duration: number;
|
||||
/** Target output frames-per-second. */
|
||||
fps: number;
|
||||
/** Output image format. */
|
||||
format: CacheFrameFormat;
|
||||
}
|
||||
|
||||
export interface CacheEntry {
|
||||
/** Absolute path to the cache entry directory. */
|
||||
dir: string;
|
||||
/** Full 64-char SHA-256 hex digest (parent of the truncated key). */
|
||||
keyHash: string;
|
||||
}
|
||||
|
||||
export interface CacheLookup {
|
||||
/** Cache entry information — always returned even on a miss so the caller
|
||||
* can extract directly into `dir` then call `markCacheEntryComplete`. */
|
||||
entry: CacheEntry;
|
||||
/** True when the entry exists AND carries the completion sentinel. */
|
||||
hit: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read `(mtimeMs, size)` for a path. Returns `null` if the file is missing —
|
||||
* callers should skip the cache path for that entry so the extractor surfaces
|
||||
* the real file-not-found error. Returning a zero-stat sentinel would let two
|
||||
* missing files share the same `(0, 0)` tuple and pollute the cache with an
|
||||
* orphaned entry.
|
||||
*/
|
||||
export function readKeyStat(videoPath: string): { mtimeMs: number; size: number } | null {
|
||||
try {
|
||||
const stat = statSync(videoPath);
|
||||
return { mtimeMs: Math.floor(stat.mtimeMs), size: stat.size };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function canonicalKeyBlob(input: CacheKeyInput): string {
|
||||
const durationForKey = Number.isFinite(input.duration) ? input.duration : -1;
|
||||
return JSON.stringify({
|
||||
p: input.videoPath,
|
||||
m: input.mtimeMs,
|
||||
s: input.size,
|
||||
ms: input.mediaStart,
|
||||
d: durationForKey,
|
||||
f: input.fps,
|
||||
fmt: input.format,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the SHA-256 hex digest for a cache key input.
|
||||
*/
|
||||
export function computeCacheKey(input: CacheKeyInput): string {
|
||||
return createHash("sha256").update(canonicalKeyBlob(input)).digest("hex");
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive the truncated cache-entry directory name from a full key hash.
|
||||
* Exposed so tests and the entry dir resolver share one truncation rule.
|
||||
*/
|
||||
export function cacheEntryDirName(keyHash: string): string {
|
||||
return SCHEMA_PREFIX + keyHash.slice(0, KEY_HEX_CHARS);
|
||||
}
|
||||
|
||||
/**
|
||||
* Look up a cache entry by key input. Returns the resolved entry path plus a
|
||||
* `hit` flag. On miss, callers should extract frames into `entry.dir`
|
||||
* (after calling `ensureCacheEntryDir`) and then call `markCacheEntryComplete`
|
||||
* once the extraction succeeds.
|
||||
*/
|
||||
export function lookupCacheEntry(rootDir: string, input: CacheKeyInput): CacheLookup {
|
||||
const keyHash = computeCacheKey(input);
|
||||
const dir = join(rootDir, cacheEntryDirName(keyHash));
|
||||
const complete = existsSync(join(dir, COMPLETE_SENTINEL));
|
||||
return { entry: { dir, keyHash }, hit: complete };
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure a cache entry's directory exists so the extractor can write into it.
|
||||
* Idempotent: `mkdirSync({recursive:true})` is a no-op when the dir exists.
|
||||
*/
|
||||
export function ensureCacheEntryDir(entry: CacheEntry): void {
|
||||
mkdirSync(entry.dir, { recursive: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the completion sentinel so subsequent lookups treat this entry as a
|
||||
* hit. Must be called only after every frame has been written.
|
||||
*
|
||||
* Concurrency: lookup→populate→mark is non-atomic. Two concurrent renders of
|
||||
* the same key may both miss, both extract into the same dir, and the later
|
||||
* writer's frames win. The result is correct (identical inputs yield identical
|
||||
* frames) but wasteful. Acceptable for a single-process render pipeline;
|
||||
* anyone running concurrent renders against a shared cache root should front
|
||||
* it with an external lock.
|
||||
*/
|
||||
export function markCacheEntryComplete(entry: CacheEntry): void {
|
||||
writeFileSync(join(entry.dir, COMPLETE_SENTINEL), "", "utf-8");
|
||||
}
|
||||
|
||||
/**
|
||||
* Rebuild the in-memory frame index for a cached entry. Called on cache hits
|
||||
* so the extractor's caller receives the same `ExtractedFrames` shape it
|
||||
* would get from a fresh extraction — without re-running ffmpeg or ffprobe.
|
||||
*
|
||||
* The `metadata` argument is the `VideoMetadata` probed in the extractor's
|
||||
* Phase 2 (pre-preflight). Passing it here avoids an extra ffprobe on the
|
||||
* hit path.
|
||||
*/
|
||||
export interface RehydrateOptions {
|
||||
videoId: string;
|
||||
srcPath: string;
|
||||
fps: number;
|
||||
format: CacheFrameFormat;
|
||||
metadata: VideoMetadata;
|
||||
}
|
||||
|
||||
export interface RehydratedFrames {
|
||||
videoId: string;
|
||||
srcPath: string;
|
||||
outputDir: string;
|
||||
framePattern: string;
|
||||
fps: number;
|
||||
totalFrames: number;
|
||||
metadata: VideoMetadata;
|
||||
framePaths: Map<number, string>;
|
||||
}
|
||||
|
||||
export function rehydrateCacheEntry(
|
||||
entry: CacheEntry,
|
||||
options: RehydrateOptions,
|
||||
): RehydratedFrames {
|
||||
const framePattern = `${FRAME_FILENAME_PREFIX}%05d.${options.format}`;
|
||||
const framePaths = new Map<number, string>();
|
||||
const suffix = `.${options.format}`;
|
||||
const files = readdirSync(entry.dir)
|
||||
.filter((f) => f.startsWith(FRAME_FILENAME_PREFIX) && f.endsWith(suffix))
|
||||
.sort();
|
||||
files.forEach((file, idx) => {
|
||||
framePaths.set(idx, join(entry.dir, file));
|
||||
});
|
||||
return {
|
||||
videoId: options.videoId,
|
||||
srcPath: options.srcPath,
|
||||
outputDir: entry.dir,
|
||||
framePattern,
|
||||
fps: options.fps,
|
||||
totalFrames: framePaths.size,
|
||||
metadata: options.metadata,
|
||||
framePaths,
|
||||
};
|
||||
}
|
||||
@@ -196,6 +196,137 @@ describe.skipIf(!HAS_FFMPEG)("extractAllVideoFrames on a VFR source", () => {
|
||||
expect(result.phaseBreakdown.vfrPreflightMs).toBeGreaterThan(0);
|
||||
}, 60_000);
|
||||
|
||||
it("reuses extracted frames on a warm cache hit", async () => {
|
||||
const CACHE_DIR = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
|
||||
const SRC = join(FIXTURE_DIR, "cache-src.mp4");
|
||||
|
||||
// Synthesize a clean CFR SDR clip — bypasses VFR preflight so the cache
|
||||
// key is stable across the two runs.
|
||||
const synth = await runFfmpeg([
|
||||
"-y",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc2=s=320x180:d=2:rate=30",
|
||||
"-c:v",
|
||||
"libx264",
|
||||
"-preset",
|
||||
"ultrafast",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
SRC,
|
||||
]);
|
||||
if (!synth.success) {
|
||||
throw new Error(`Cache fixture synthesis failed: ${synth.stderr.slice(-400)}`);
|
||||
}
|
||||
|
||||
const video: VideoElement = {
|
||||
id: "cv1",
|
||||
src: SRC,
|
||||
start: 0,
|
||||
end: 2,
|
||||
mediaStart: 0,
|
||||
hasAudio: false,
|
||||
};
|
||||
|
||||
const outDirA = join(FIXTURE_DIR, "out-cache-miss");
|
||||
mkdirSync(outDirA, { recursive: true });
|
||||
const miss = await extractAllVideoFrames(
|
||||
[video],
|
||||
FIXTURE_DIR,
|
||||
{ fps: 30, outputDir: outDirA },
|
||||
undefined,
|
||||
{ extractCacheDir: CACHE_DIR },
|
||||
);
|
||||
expect(miss.errors).toEqual([]);
|
||||
expect(miss.phaseBreakdown.cacheHits).toBe(0);
|
||||
expect(miss.phaseBreakdown.cacheMisses).toBe(1);
|
||||
|
||||
const outDirB = join(FIXTURE_DIR, "out-cache-hit");
|
||||
mkdirSync(outDirB, { recursive: true });
|
||||
const hit = await extractAllVideoFrames(
|
||||
[video],
|
||||
FIXTURE_DIR,
|
||||
{ fps: 30, outputDir: outDirB },
|
||||
undefined,
|
||||
{ extractCacheDir: CACHE_DIR },
|
||||
);
|
||||
expect(hit.errors).toEqual([]);
|
||||
expect(hit.phaseBreakdown.cacheHits).toBe(1);
|
||||
expect(hit.phaseBreakdown.cacheMisses).toBe(0);
|
||||
// extractMs on a hit is only the cache-lookup bookkeeping; asserting <50ms
|
||||
// is loose enough to survive CI jitter but tight enough to catch a
|
||||
// regression that accidentally triggered ffmpeg again.
|
||||
expect(hit.phaseBreakdown.extractMs).toBeLessThan(50);
|
||||
expect(hit.extracted).toHaveLength(1);
|
||||
expect(hit.extracted[0]!.totalFrames).toBe(miss.extracted[0]!.totalFrames);
|
||||
|
||||
rmSync(CACHE_DIR, { recursive: true, force: true });
|
||||
}, 60_000);
|
||||
|
||||
it("invalidates the cache when fps changes", async () => {
|
||||
const CACHE_DIR = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
|
||||
const SRC = join(FIXTURE_DIR, "cache-fps-src.mp4");
|
||||
|
||||
const synth = await runFfmpeg([
|
||||
"-y",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc2=s=320x180:d=1:rate=30",
|
||||
"-c:v",
|
||||
"libx264",
|
||||
"-preset",
|
||||
"ultrafast",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
SRC,
|
||||
]);
|
||||
if (!synth.success) {
|
||||
throw new Error(`Cache-fps fixture synthesis failed: ${synth.stderr.slice(-400)}`);
|
||||
}
|
||||
|
||||
const video: VideoElement = {
|
||||
id: "cv2",
|
||||
src: SRC,
|
||||
start: 0,
|
||||
end: 1,
|
||||
mediaStart: 0,
|
||||
hasAudio: false,
|
||||
};
|
||||
|
||||
const outA = join(FIXTURE_DIR, "out-cache-fps-30");
|
||||
mkdirSync(outA, { recursive: true });
|
||||
const first = await extractAllVideoFrames(
|
||||
[video],
|
||||
FIXTURE_DIR,
|
||||
{ fps: 30, outputDir: outA },
|
||||
undefined,
|
||||
{ extractCacheDir: CACHE_DIR },
|
||||
);
|
||||
expect(first.phaseBreakdown.cacheMisses).toBe(1);
|
||||
|
||||
const outB = join(FIXTURE_DIR, "out-cache-fps-60");
|
||||
mkdirSync(outB, { recursive: true });
|
||||
const second = await extractAllVideoFrames(
|
||||
[video],
|
||||
FIXTURE_DIR,
|
||||
{ fps: 60, outputDir: outB },
|
||||
undefined,
|
||||
{ extractCacheDir: CACHE_DIR },
|
||||
);
|
||||
expect(second.phaseBreakdown.cacheMisses).toBe(1);
|
||||
expect(second.phaseBreakdown.cacheHits).toBe(0);
|
||||
|
||||
rmSync(CACHE_DIR, { recursive: true, force: true });
|
||||
}, 60_000);
|
||||
|
||||
// Regression test for the segment-scope HDR preflight fix: pre-fix,
|
||||
// convertSdrToHdr re-encoded the entire source, so a 30-minute SDR source
|
||||
// contributing a 2-second clip took ~200× longer than needed. Post-fix the
|
||||
|
||||
@@ -18,6 +18,15 @@ import {
|
||||
import { downloadToTemp, isHttpUrl } from "../utils/urlDownloader.js";
|
||||
import { runFfmpeg } from "../utils/runFfmpeg.js";
|
||||
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
|
||||
import {
|
||||
FRAME_FILENAME_PREFIX,
|
||||
ensureCacheEntryDir,
|
||||
lookupCacheEntry,
|
||||
markCacheEntryComplete,
|
||||
readKeyStat,
|
||||
rehydrateCacheEntry,
|
||||
type CacheFrameFormat,
|
||||
} from "./extractionCache.js";
|
||||
|
||||
export interface VideoElement {
|
||||
id: string;
|
||||
@@ -37,6 +46,13 @@ export interface ExtractedFrames {
|
||||
totalFrames: number;
|
||||
metadata: VideoMetadata;
|
||||
framePaths: Map<number, string>;
|
||||
/**
|
||||
* True when the extractor owns `outputDir` and cleanup should rm it when
|
||||
* the render ends. Cache hits set this to false so the shared entry isn't
|
||||
* deleted by a single render's cleanup — the cache dir is owned by the
|
||||
* caller's gc policy, not any one render.
|
||||
*/
|
||||
ownedByLookup?: boolean;
|
||||
}
|
||||
|
||||
export interface ExtractionOptions {
|
||||
@@ -181,15 +197,22 @@ export async function extractVideoFramesRange(
|
||||
options: ExtractionOptions,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout">>,
|
||||
/**
|
||||
* Override the output directory for this extraction. When provided, frames
|
||||
* are written directly into `outputDirOverride` (no per-videoId subdir).
|
||||
* Used by the cache layer to materialize frames straight into the keyed
|
||||
* cache entry directory.
|
||||
*/
|
||||
outputDirOverride?: string,
|
||||
): Promise<ExtractedFrames> {
|
||||
const ffmpegProcessTimeout = config?.ffmpegProcessTimeout ?? DEFAULT_CONFIG.ffmpegProcessTimeout;
|
||||
const { fps, outputDir, quality = 95, format = "jpg" } = options;
|
||||
|
||||
const videoOutputDir = join(outputDir, videoId);
|
||||
const videoOutputDir = outputDirOverride ?? join(outputDir, videoId);
|
||||
if (!existsSync(videoOutputDir)) mkdirSync(videoOutputDir, { recursive: true });
|
||||
|
||||
const metadata = await extractMediaMetadata(videoPath);
|
||||
const framePattern = `frame_%05d.${format}`;
|
||||
const framePattern = `${FRAME_FILENAME_PREFIX}%05d.${format}`;
|
||||
const outputPattern = join(videoOutputDir, framePattern);
|
||||
|
||||
// When extracting from HDR source, tone-map to SDR in FFmpeg rather than
|
||||
@@ -253,7 +276,7 @@ export async function extractVideoFramesRange(
|
||||
|
||||
const framePaths = new Map<number, string>();
|
||||
const files = readdirSync(videoOutputDir)
|
||||
.filter((f) => f.startsWith("frame_") && f.endsWith(`.${format}`))
|
||||
.filter((f) => f.startsWith(FRAME_FILENAME_PREFIX) && f.endsWith(`.${format}`))
|
||||
.sort();
|
||||
files.forEach((file, index) => {
|
||||
framePaths.set(index, join(videoOutputDir, file));
|
||||
@@ -353,6 +376,21 @@ async function convertSdrToHdr(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the used-segment duration for a video, falling back to the source's
|
||||
* natural duration when the caller hasn't specified bounds (end=Infinity) or
|
||||
* the bounds are nonsensical (end<=start).
|
||||
*/
|
||||
function resolveSegmentDuration(
|
||||
requested: number,
|
||||
mediaStart: number,
|
||||
metadata: VideoMetadata,
|
||||
): number {
|
||||
if (Number.isFinite(requested) && requested > 0) return requested;
|
||||
const sourceRemaining = metadata.durationSeconds - mediaStart;
|
||||
return sourceRemaining > 0 ? sourceRemaining : metadata.durationSeconds;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-encode a VFR (variable frame rate) video segment to CFR so the downstream
|
||||
* fps filter can extract frames reliably. Screen recordings, phone videos, and
|
||||
@@ -414,7 +452,7 @@ export async function extractAllVideoFrames(
|
||||
baseDir: string,
|
||||
options: ExtractionOptions,
|
||||
signal?: AbortSignal,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout">>,
|
||||
config?: Partial<Pick<EngineConfig, "ffmpegProcessTimeout" | "extractCacheDir">>,
|
||||
compiledDir?: string,
|
||||
): Promise<ExtractionResult> {
|
||||
const startTime = Date.now();
|
||||
@@ -469,6 +507,28 @@ export async function extractAllVideoFrames(
|
||||
|
||||
breakdown.resolveMs = Date.now() - phase1Start;
|
||||
|
||||
// Snapshot the pre-preflight key inputs so the extraction cache keys on the
|
||||
// user-visible source (original path, original mediaStart, original segment
|
||||
// bounds) rather than the workDir-local normalized file produced by
|
||||
// Phase 2a/2b preflight. Without this, every render would write a new
|
||||
// normalized file with a fresh mtime → fresh cache key → perpetual misses.
|
||||
const cacheKeyInputs = resolvedVideos.map(({ video, videoPath }) => {
|
||||
const stat = readKeyStat(videoPath);
|
||||
// Missing files return null — skip the cache path for that entry. The
|
||||
// extractor will surface the real file-not-found error downstream, and we
|
||||
// avoid polluting the cache with a `(mtimeMs: 0, size: 0)` tuple that two
|
||||
// unrelated missing paths would otherwise share.
|
||||
if (!stat) return null;
|
||||
return {
|
||||
videoPath,
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
mediaStart: video.mediaStart,
|
||||
start: video.start,
|
||||
end: video.end,
|
||||
};
|
||||
});
|
||||
|
||||
// Phase 2: Probe color spaces and normalize if mixed HDR/SDR
|
||||
const phase2ProbeStart = Date.now();
|
||||
const videoMetadata = await Promise.all(
|
||||
@@ -565,6 +625,10 @@ export async function extractAllVideoFrames(
|
||||
resolvedVideos.splice(i, 1);
|
||||
videoMetadata.splice(i, 1);
|
||||
videoColorSpaces.splice(i, 1);
|
||||
// Added by the extraction-cache commit: keep cacheKeyInputs aligned
|
||||
// with the other parallel arrays so Phase 3's `cacheKeyInputs[i]`
|
||||
// lookup doesn't point at a stale slot after the splice.
|
||||
cacheKeyInputs.splice(i, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -615,25 +679,85 @@ export async function extractAllVideoFrames(
|
||||
}
|
||||
breakdown.vfrPreflightMs = Date.now() - vfrPreflightStart;
|
||||
|
||||
// Phase 3: Extract frames (parallel)
|
||||
const phase3Start = Date.now();
|
||||
const cacheRootDir = config?.extractCacheDir;
|
||||
const cacheFormat: CacheFrameFormat = options.format ?? "jpg";
|
||||
|
||||
async function tryCachedExtract(
|
||||
video: VideoElement,
|
||||
videoPath: string,
|
||||
videoDuration: number,
|
||||
i: number,
|
||||
): Promise<ExtractedFrames | null> {
|
||||
if (!cacheRootDir) return null;
|
||||
const keyInput = cacheKeyInputs[i];
|
||||
const probedMeta = videoMetadata[i];
|
||||
if (!keyInput || !probedMeta) return null;
|
||||
|
||||
const keyDuration = resolveSegmentDuration(
|
||||
keyInput.end - keyInput.start,
|
||||
keyInput.mediaStart,
|
||||
probedMeta,
|
||||
);
|
||||
const lookup = lookupCacheEntry(cacheRootDir, {
|
||||
videoPath: keyInput.videoPath,
|
||||
mtimeMs: keyInput.mtimeMs,
|
||||
size: keyInput.size,
|
||||
mediaStart: keyInput.mediaStart,
|
||||
duration: keyDuration,
|
||||
fps: options.fps,
|
||||
format: cacheFormat,
|
||||
});
|
||||
|
||||
if (lookup.hit) {
|
||||
breakdown.cacheHits += 1;
|
||||
const rehydrated = rehydrateCacheEntry(lookup.entry, {
|
||||
videoId: video.id,
|
||||
srcPath: keyInput.videoPath,
|
||||
fps: options.fps,
|
||||
format: cacheFormat,
|
||||
metadata: probedMeta,
|
||||
});
|
||||
return { ...rehydrated, ownedByLookup: true };
|
||||
}
|
||||
|
||||
breakdown.cacheMisses += 1;
|
||||
ensureCacheEntryDir(lookup.entry);
|
||||
const result = await extractVideoFramesRange(
|
||||
videoPath,
|
||||
video.id,
|
||||
video.mediaStart,
|
||||
videoDuration,
|
||||
options,
|
||||
signal,
|
||||
config,
|
||||
lookup.entry.dir,
|
||||
);
|
||||
// Mark complete only AFTER frames are on disk — a crash mid-extract
|
||||
// leaves the entry un-sentineled so the next lookup re-extracts over it.
|
||||
markCacheEntryComplete(lookup.entry);
|
||||
return { ...result, ownedByLookup: true };
|
||||
}
|
||||
|
||||
const results = await Promise.all(
|
||||
resolvedVideos.map(async ({ video, videoPath }) => {
|
||||
resolvedVideos.map(async ({ video, videoPath }, i) => {
|
||||
if (signal?.aborted) {
|
||||
throw new Error("Video frame extraction cancelled");
|
||||
}
|
||||
try {
|
||||
let videoDuration = video.end - video.start;
|
||||
|
||||
// Fallback: if no data-duration/data-end was specified (end is Infinity or 0),
|
||||
// probe the actual video file to get its natural duration.
|
||||
if (!Number.isFinite(videoDuration) || videoDuration <= 0) {
|
||||
const metadata = await extractMediaMetadata(videoPath);
|
||||
const sourceDuration = metadata.durationSeconds - video.mediaStart;
|
||||
videoDuration = sourceDuration > 0 ? sourceDuration : metadata.durationSeconds;
|
||||
const probedMeta = videoMetadata[i] ?? (await extractMediaMetadata(videoPath));
|
||||
const videoDuration = resolveSegmentDuration(
|
||||
video.end - video.start,
|
||||
video.mediaStart,
|
||||
probedMeta,
|
||||
);
|
||||
if (video.end - video.start !== videoDuration) {
|
||||
video.end = video.start + videoDuration;
|
||||
}
|
||||
|
||||
const cached = await tryCachedExtract(video, videoPath, videoDuration, i);
|
||||
if (cached) return { result: cached };
|
||||
|
||||
const result = await extractVideoFramesRange(
|
||||
videoPath,
|
||||
video.id,
|
||||
@@ -800,6 +924,10 @@ export class FrameLookupTable {
|
||||
|
||||
cleanup(): void {
|
||||
for (const video of this.videos.values()) {
|
||||
// Cache-hit / cache-write entries are owned by the extraction cache —
|
||||
// a single render must not delete them, or the next render's lookup
|
||||
// would miss and re-extract unnecessarily.
|
||||
if (video.extracted.ownedByLookup) continue;
|
||||
if (existsSync(video.extracted.outputDir)) {
|
||||
rmSync(video.extracted.outputDir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
@@ -1427,7 +1427,7 @@ export async function executeRenderJob(
|
||||
projectDir,
|
||||
{ fps: job.config.fps, outputDir: join(workDir, "video-frames") },
|
||||
abortSignal,
|
||||
undefined,
|
||||
{ extractCacheDir: cfg.extractCacheDir },
|
||||
compiledDir,
|
||||
);
|
||||
assertNotAborted();
|
||||
|
||||
Reference in New Issue
Block a user