mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 17:30:50 +00:00
* feat(engine): static-frame dedup for screenshot capture (opt-in) Skip re-seeking + re-screenshotting frames byte-identical to their predecessor. A frame is dedupable iff no GSAP tween or clip cut is active in it or its predecessor (predicted from window.__timelines + clip schedule) AND an empirical anchor-compare confirms it. Opt-in HF_STATIC_DEDUP=true, default off. Correctness (designed for the multi-worker / distributed render paths): - Reuse is keyed by the ABSOLUTE composition frame (derived from the frame's time), NOT the captureFrameCore frameIndex arg — chunked/parallel callers pass a chunk- relative index. Validated lossless (PSNR=inf) on both single- and multi-worker renders of a static-hold comp. - verifyStaticFramesSafe checks EVERY run (no longest-first budget truncation that left runs armed-but-unverified), and samples each run's FIRST reused frame, its END, and interior points at a stride; a hard cap disables dedup rather than trust an unverified set. - Conservative arming: skipped when capture mode != screenshot (BeginFrame tick semantics + the verifier's screenshot path wouldn't transfer), when a before-capture hook is set (per-frame video injection), when page-side compositing is active (shader / drawElement composite the plain verification screenshot can't reproduce), and when any data-start is a non-numeric reference expression the clip-boundary parser can't protect, or duration is unknown/zero. - Session reuse (prepareCaptureSessionForReuse) resets lastFrameBuffer + dedup counter so a probe/prior-render buffer can't bleed into the first static frame; the armed set is kept (same-composition reuse). Cost calibration bypasses dedup for its sparse, non-contiguous sample sweep, then restores the armed set. - HF_STATIC_DEDUP_SAMPLES is NaN-guarded. Disqualifies on signals the GSAP predictor can't see: video, canvas/webgl, zero tweens, running CSS/WAAPI animation. Pays on static-hold content (title cards, slideshow/kiosk loops, data-viz pauses); no-op on continuously-animated comps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(engine): static-frame dedup default-on + render telemetry Flip dedup from opt-in (HF_STATIC_DEDUP=true) to default-on (opt-out HF_STATIC_DEDUP=false). Verification (verifyStaticFramesSafe) is the safety net that keeps reuse sound at scale. Add end-to-end dedup observability. The capture session records enabled / armed / skipReason / predicted; these surface via CapturePerfSummary -> a dedupPerfs accumulator (disk sequential + parallel AND streaming sequential + parallel) -> aggregated into RenderPerfSummary.staticDedup (OR armed, SUM frames across workers) -> render_complete props static_dedup_{enabled,armed,skip_reason, predicted_frames,reused_frames}. skip_reason is a low-cardinality code: capture_mode | video_injection | page_composite | ineligible | verification_failed. Distributed chunks run on Linux/beginframe where dedup never arms, so they pass a throwaway dedupPerfs sink (no per-chunk reporting). Tests: aggregation logic (OR/SUM/skip-reason) + opt-out passthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(engine): address review on dedup default-on + telemetry Review feedback (miga-heygen) + self-review fixes: - Retry double-count: executeDiskCaptureWithAdaptiveRetry pushed worker dedup perf inside the retry loop, so an adaptive retry counted frames twice (reused/predicted could exceed totalFrames). Reset dedupPerfs at the start of each attempt — retry now REPLACES rather than accumulates; common no-retry path is unchanged. - Opt-out parsing: HF_STATIC_DEDUP now disables on {false,0,off} case/space-insensitive (was strict !== "false", so `False`/`0` silently kept dedup on — the kill-switch could no-op). - Verification budget vs drift: verifyStaticFramesSafe returns {badFrame, budgetExhausted}; armStaticDedup reports a distinct `verification_budget` skip reason so a telemetry spike means "raise HF_STATIC_DEDUP_SAMPLES", not "compositions are non-static". - Index idiom: captureFrameCore now uses Math.floor(time*fps + 1e-9) (matches quantizeTimeToFrame) so the dedup lookup agrees with the frame the seek lands on even for non-exact times. - Stale "opt-in HF_STATIC_DEDUP=true" comments -> "opt-out HF_STATIC_DEDUP=false" across frameCapture.ts + types.ts. - Extract pushWorkerDedupPerfs helper (perfSummary.ts), used by the disk and streaming parallel paths — removes the duplicated push loop and drops captureStreamingStage back under the complexity threshold. - dedupPerfs is now required (not optional) on executeDiskCaptureWithAdaptiveRetry — a missing arg silently dropped telemetry. - Test: captureStreamingStage createInput() now provides the required dedupPerfs field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(engine): address deferred dedup-review items - Derivable state: drop session.staticDedupArmed/staticDedupPredicted; derive both from session.staticFrames in getCapturePerfSummary (armed ⟺ non-empty set, predicted === size) so they can't desync. - Config altitude: HF_STATIC_DEDUP now resolves into EngineConfig.staticFrameDedup (resolveConfig, opt-out on {false,0,off}), alongside forceScreenshot/browserGpuMode — armStaticDedup reads config instead of process.env. Default-on preserved (missing config → enabled). - Lossy aggregation: aggregateDedup now reports DISTINCT skip reasons (sorted, `|`-joined) across diverging unarmed workers instead of just the first. - discardWarmupCapture: also snapshot/restore staticDedupCount and lastFrameBuffer so a warmup capture can't leak a phantom reuse or a stale buffer anchor into the real summary. - Convention: perfSummary-dedup.test builds its job via createRenderJob instead of `as unknown as RenderJob`. - Docs: verification_budget added to skip-reason lists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
77 lines
3.2 KiB
TypeScript
77 lines
3.2 KiB
TypeScript
import { describe, it, expect } from "vitest";
|
|
import { captureFrameToBuffer, type CaptureSession } from "./frameCapture.js";
|
|
|
|
/**
|
|
* Regression lock for the static-dedup reuse index.
|
|
*
|
|
* `captureFrameCore` must key the static-frame reuse set on the ABSOLUTE
|
|
* composition frame — derived from `time` (`round(time * fps)`) — NOT the
|
|
* `frameIndex` argument. Distributed / per-worker-range / parallel callers pass
|
|
* a chunk-RELATIVE `frameIndex` (captureStage passes the loop `i`,
|
|
* parallelCoordinator passes `i - outputFrameOffset`) while `staticFrames` is
|
|
* keyed in absolute frames. A prior bug used `frameIndex`, so a chunk with
|
|
* `startFrame > 0` reused the wrong frames (and the right frames missed).
|
|
*
|
|
* The reuse branch returns BEFORE any page interaction, so we can exercise the
|
|
* decision with a stub session whose `page` throws if touched: a dedup HIT
|
|
* returns the cached buffer (page untouched); a MISS proceeds to the page and
|
|
* rejects. Both assertions below FAIL on the pre-fix (relative-index) code.
|
|
*/
|
|
|
|
const SENTINEL = Buffer.from("cached-anchor-frame");
|
|
|
|
// ponytail: minimal stub of the 40-field CaptureSession — only the fields the
|
|
// reuse decision reads are real; `page` is a trap that throws on any access so
|
|
// a dedup MISS (which falls through to prepareFrameForCapture) rejects loudly.
|
|
function makeSession(staticFrames: Set<number>, fps: { num: number; den: number }): CaptureSession {
|
|
const pageTrap = new Proxy(
|
|
{},
|
|
{
|
|
get() {
|
|
throw new Error("PAGE_TOUCHED");
|
|
},
|
|
},
|
|
);
|
|
return {
|
|
page: pageTrap,
|
|
options: { fps, format: "jpg" },
|
|
captureMode: "screenshot",
|
|
isInitialized: false,
|
|
staticFrames,
|
|
lastFrameBuffer: SENTINEL,
|
|
staticDedupCount: 0,
|
|
} as unknown as CaptureSession;
|
|
}
|
|
|
|
describe("static-dedup reuse keys on absolute frame index (time), not relative frameIndex", () => {
|
|
const fps30 = { num: 30, den: 1 };
|
|
|
|
it("HIT: relative frameIndex=0 but absolute time=90/30 reuses the anchor", async () => {
|
|
const session = makeSession(new Set([90]), fps30);
|
|
// Pre-fix used frameIndex (0) ∉ {90} → would miss → page trap throws.
|
|
const result = await captureFrameToBuffer(session, 0, 90 / 30);
|
|
expect(result.buffer).toBe(SENTINEL);
|
|
expect(session.staticDedupCount).toBe(1);
|
|
});
|
|
|
|
it("MISS: relative frameIndex=90 but absolute time=0 does NOT reuse", async () => {
|
|
const session = makeSession(new Set([90]), fps30);
|
|
// Pre-fix used frameIndex (90) ∈ {90} → would wrongly reuse the anchor.
|
|
await expect(captureFrameToBuffer(session, 90, 0)).rejects.toThrow();
|
|
expect(session.staticDedupCount).toBe(0);
|
|
});
|
|
|
|
it("non-integer fps (29.97) recovers the absolute index exactly", async () => {
|
|
const fps2997 = { num: 30000, den: 1001 };
|
|
const session = makeSession(new Set([100]), fps2997);
|
|
const time = (100 * fps2997.den) / fps2997.num; // absolute frame 100 → time
|
|
const result = await captureFrameToBuffer(session, 7, time);
|
|
expect(result.buffer).toBe(SENTINEL);
|
|
});
|
|
|
|
it("no reuse when the absolute frame is not in the static set", async () => {
|
|
const session = makeSession(new Set([10, 11, 12]), fps30);
|
|
await expect(captureFrameToBuffer(session, 0, 50 / 30)).rejects.toThrow();
|
|
});
|
|
});
|