perf(engine): content-addressed extraction cache for video frames (#446)

## What

Adds a content-addressed cache for extracted video frames, keyed on the tuple `(path, mtime, size, mediaStart, duration, fps, format)`. Repeat renders of the same composition (studio edit → re-render, preview → final) skip the ffmpeg extraction entirely.

## Why

Video frame extraction is the dominant non-capture phase for video-heavy compositions. Studio iteration workflows extract the same frames over and over — each render burns ffmpeg time that adds no value.

Validated on `/tmp/hf-fixtures/cfr-sdr-cache`:
```
Cold (miss): extractMs=69,  videoExtractMs=70,  totalElapsedMs=2052
Warm (hit):  extractMs=1,   videoExtractMs=2,   totalElapsedMs=1964
cacheHits: 0→1, cacheMisses: 1→0
```
The fixture is tiny (3s CFR SDR @ 30fps), so the wall-clock delta is small; the extraction-time delta (69→1ms, 98%) scales linearly with source length. For heavy-iteration workflows (a user rendering the same composition while tuning encoding params), extraction time goes to zero on every repeat render.

Depends on #444 (instrumentation surface) and #445 (segment-scope HDR preflight — otherwise cache keys would be unstable across renders on mixed-HDR compositions).

## How

- New `packages/engine/src/services/extractionCache.ts`:
  - SHA-256 key over a stable JSON encoding of `(path, mtime_ms, size, mediaStart, duration, fps, format)`. Infinity duration is normalized to `-1` so unresolved natural-duration sources still produce stable keys.
  - Truncates to 16 hex chars in the entry directory name — 64 bits of entropy is plenty at cache scale and keeps `ls` output short.
  - `hfcache-v2-` schema prefix — bumping it invalidates old entries (callers own gc policy; the cache owns keys).
  - `.hf-complete` dotfile sentinel. An entry dir without the sentinel is treated as a miss (covers crash-mid-extract and abandoned writes); the next render re-extracts over the partial frames with `-y`.
  - `FRAME_FILENAME_PREFIX = "frame_"` shared with the extractor — future refactors only need to touch one place to rename frames.
- `EngineConfig.extractCacheDir` (env: `HYPERFRAMES_EXTRACT_CACHE_DIR`) gates the feature. Undefined disables caching — extraction runs into the render's workDir and cleanup removes it on render end, preserving the prior behaviour exactly. No default root is chosen by the engine; the caller (CLI, app, studio) owns the location policy.
- `ExtractedFrames.ownedByLookup` flag prevents `FrameLookupTable.cleanup` from rm'ing a shared cache dir at render end. Set to `true` on both hits and misses (misses own the directory they wrote into, but hand it over to the cache rather than deleting it).
- Phase 3 extractor flow:
  1. Snapshot `(videoPath, mediaStart, start, end)` per resolved video BEFORE Phase 2a/2b preflight mutates them — so cache keys are stable across renders that use workDir-local normalized files (those files have fresh mtimes every render).
  2. Compute key, `lookupCacheEntry`.
  3. On hit: rebuild `ExtractedFrames` from the cache dir plus the Phase 2-probed `VideoMetadata` — no re-ffprobe.
  4. On miss: `ensureCacheEntryDir`, extract with `extractVideoFramesRange(..., outputDirOverride)`, then `markCacheEntryComplete` (the sentinel write is the last step so a crash leaves the dir un-sentineled).
- `extractVideoFramesRange` gains an `outputDirOverride` parameter so cache-miss writes land directly in the keyed dir (no `join(outputDir, videoId)` wrapping).

## Test plan

- [x] 19 unit tests in `extractionCache.test.ts` covering key determinism, mtime/size invalidation, format/fps/mediaStart/duration invalidation, Infinity normalization, sentinel semantics, missing-file tolerance
- [x] 2 integration tests in `videoFrameExtractor.test.ts`:
  - "reuses extracted frames on a warm cache hit" — asserts `cacheHits=1`, `extractMs<50ms` on second call against a CFR SDR fixture
  - "invalidates the cache when fps changes" — different fps on second call forces a new miss
- [x] End-to-end validation with `HYPERFRAMES_EXTRACT_CACHE_DIR` set, two runs of the same fixture
- [x] Lint + format (oxlint + oxfmt)
- [x] Typecheck (engine + producer)
This commit is contained in:
James Russo
2026-04-24 00:35:31 -04:00
committed by GitHub
parent 9912d3730a
commit 57cdf7d80e
6 changed files with 715 additions and 15 deletions
@@ -196,6 +196,137 @@ describe.skipIf(!HAS_FFMPEG)("extractAllVideoFrames on a VFR source", () => {
expect(result.phaseBreakdown.vfrPreflightMs).toBeGreaterThan(0);
}, 60_000);
it("reuses extracted frames on a warm cache hit", async () => {
const CACHE_DIR = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
const SRC = join(FIXTURE_DIR, "cache-src.mp4");
// Synthesize a clean CFR SDR clip — bypasses VFR preflight so the cache
// key is stable across the two runs.
const synth = await runFfmpeg([
"-y",
"-hide_banner",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
"testsrc2=s=320x180:d=2:rate=30",
"-c:v",
"libx264",
"-preset",
"ultrafast",
"-pix_fmt",
"yuv420p",
SRC,
]);
if (!synth.success) {
throw new Error(`Cache fixture synthesis failed: ${synth.stderr.slice(-400)}`);
}
const video: VideoElement = {
id: "cv1",
src: SRC,
start: 0,
end: 2,
mediaStart: 0,
hasAudio: false,
};
const outDirA = join(FIXTURE_DIR, "out-cache-miss");
mkdirSync(outDirA, { recursive: true });
const miss = await extractAllVideoFrames(
[video],
FIXTURE_DIR,
{ fps: 30, outputDir: outDirA },
undefined,
{ extractCacheDir: CACHE_DIR },
);
expect(miss.errors).toEqual([]);
expect(miss.phaseBreakdown.cacheHits).toBe(0);
expect(miss.phaseBreakdown.cacheMisses).toBe(1);
const outDirB = join(FIXTURE_DIR, "out-cache-hit");
mkdirSync(outDirB, { recursive: true });
const hit = await extractAllVideoFrames(
[video],
FIXTURE_DIR,
{ fps: 30, outputDir: outDirB },
undefined,
{ extractCacheDir: CACHE_DIR },
);
expect(hit.errors).toEqual([]);
expect(hit.phaseBreakdown.cacheHits).toBe(1);
expect(hit.phaseBreakdown.cacheMisses).toBe(0);
// extractMs on a hit is only the cache-lookup bookkeeping; asserting <50ms
// is loose enough to survive CI jitter but tight enough to catch a
// regression that accidentally triggered ffmpeg again.
expect(hit.phaseBreakdown.extractMs).toBeLessThan(50);
expect(hit.extracted).toHaveLength(1);
expect(hit.extracted[0]!.totalFrames).toBe(miss.extracted[0]!.totalFrames);
rmSync(CACHE_DIR, { recursive: true, force: true });
}, 60_000);
it("invalidates the cache when fps changes", async () => {
const CACHE_DIR = mkdtempSync(join(tmpdir(), "hf-extract-cache-test-"));
const SRC = join(FIXTURE_DIR, "cache-fps-src.mp4");
const synth = await runFfmpeg([
"-y",
"-hide_banner",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
"testsrc2=s=320x180:d=1:rate=30",
"-c:v",
"libx264",
"-preset",
"ultrafast",
"-pix_fmt",
"yuv420p",
SRC,
]);
if (!synth.success) {
throw new Error(`Cache-fps fixture synthesis failed: ${synth.stderr.slice(-400)}`);
}
const video: VideoElement = {
id: "cv2",
src: SRC,
start: 0,
end: 1,
mediaStart: 0,
hasAudio: false,
};
const outA = join(FIXTURE_DIR, "out-cache-fps-30");
mkdirSync(outA, { recursive: true });
const first = await extractAllVideoFrames(
[video],
FIXTURE_DIR,
{ fps: 30, outputDir: outA },
undefined,
{ extractCacheDir: CACHE_DIR },
);
expect(first.phaseBreakdown.cacheMisses).toBe(1);
const outB = join(FIXTURE_DIR, "out-cache-fps-60");
mkdirSync(outB, { recursive: true });
const second = await extractAllVideoFrames(
[video],
FIXTURE_DIR,
{ fps: 60, outputDir: outB },
undefined,
{ extractCacheDir: CACHE_DIR },
);
expect(second.phaseBreakdown.cacheMisses).toBe(1);
expect(second.phaseBreakdown.cacheHits).toBe(0);
rmSync(CACHE_DIR, { recursive: true, force: true });
}, 60_000);
// Regression test for the segment-scope HDR preflight fix: pre-fix,
// convertSdrToHdr re-encoded the entire source, so a 30-minute SDR source
// contributing a 2-second clip took ~200× longer than needed. Post-fix the