Files
hyperframes/packages/engine/src/index.ts
T
Vance IngallsandClaude Fable 5 b3493a7b61 feat(engine,producer): verified interleaved parallel drawElement streaming (opt-in)
Step 2 of the DE engagement plan: multi-worker drawElement capture through
the streaming encoder, with the full runtime self-verification net riding
along — the confinement rule that kept the parallel clamp in place is now
satisfied on this path. Opt-in via HF_DE_PARALLEL_STREAM=true; default
routing (including the #2026 single-worker inversion) is unchanged.

Mechanism:
- distributeFramesInterleaved + WorkerTask.frameStride: worker i captures
  frames i, i+N, i+2N... — seek-based capture makes stride free and the
  ordered writer's reorder window shrinks from totalFrames/N to N (contiguous
  chunks serialize workers behind the writer).
- Depth-2 pipelined worker-encode produce in the parallel worker loop (the
  same shape as the sequential loop; frame k's in-page encode overlaps
  k+stride's produce). HF_DE_PAR_DEBUG=1 traces the first frames per worker.
- Drain guard extracted to createDrainFrameGuard (session-parameterized):
  every parallel frame gets the SAME blank-guard + PSNR self-verify as the
  sequential drain, against its owning worker's pre-injection ground truth
  (all sessions arm identical sample indices from
  CaptureOptions.compositionDurationSeconds).
- FrameReorderBuffer.abort(err): a failed worker (e.g. verification error)
  rejects all parked and future waiters — without this, peers park forever
  in waitForFrame and the pool (which awaits ALL workers before surfacing
  errors) deadlocks. Found by the verify-trip test; unit-tested.
- The typed DrawElementVerificationError is preserved past the pool's
  error-string flattening so the orchestrator's verify-retry recognizes it.
- Static-dedup stride hazard fixed: lastEncodeResult reuse now requires EVERY
  frame in (lastEncodeResultFrame, i] to be predicted-static (sequential
  capture reduces to the old has(i) check).
- Workers get separate browser PROCESSES under the flag: pages co-tenant in
  one browser starve non-active pages of BeginFrames on the paint-wait path
  (measured 86s vs 30s on a 3,245-frame rAF comp).

Validation:
- Happy path W3: verify samples pass across workers (4x inf on the 2,381f
  comp), output vs single-worker DE = 59.3dB (encode noise floor) — the
  interleave + dedup-stride produce identical pixels.
- Verify-trip (marginal comp + HF_DE_VERIFY_MIN_DB=45): fails at frame 649
  (32.2dB < 45), peers abort instead of deadlocking, whole render retries
  via parallel screenshot, RENDER_OK in 42.6s.
- Canary suite 7/7 with the flag off (default paths untouched); producer
  orchestrator tests 99/99; engine suite 909 passed (14 pre-existing main
  failures, stash-A/B verified); reorder-buffer abort unit tests.

Perf note: capture-only parallel speedup measured 1.38x (W2) / 1.52x (W3)
over single-worker DE in the spike; end-to-end numbers on this machine are
currently noisy (separate-browser init overhead + bench load) — clean
benchmarks before any default routing change. The flag stays explicit
opt-in; promoting it into the router replaces the #2026 W=1 pin for the
same cohort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:11:46 -07:00

292 lines
9.3 KiB
TypeScript

/**
* @hyperframes/engine
*
* Seekable web page to video rendering engine.
* Framework-agnostic: works with GSAP, Lottie, Three.js, CSS animations,
* or any web content that implements the window.__hf seek protocol.
*
* ## Error Convention
*
* Engine services use three error strategies depending on the operation type:
*
* - **Orchestration services throw on failure.** Browser launch, session init,
* frame capture, and CDP operations propagate errors as thrown exceptions.
* Callers are expected to catch and handle (e.g. frameCapture, browserManager,
* screenshotService, videoFrameExtractor.extractVideoFramesRange).
*
* - **FFmpeg process wrappers return `{ success, error? }` result objects.**
* Encoding, muxing, audio mixing, and streaming encode operations never reject.
* They resolve with a result that includes `success: boolean` and an optional
* `error` string (e.g. chunkEncoder, audioMixer, streamingEncoder).
*
* - **Cleanup and teardown functions never throw.** Browser close, session close,
* temp directory removal, and resource release swallow errors via `.catch(() => {})`
* to avoid masking the original failure (e.g. releaseBrowser, closeCaptureSession,
* FrameLookupTable.cleanup).
*
* - **Optional lookups return `T | undefined` or `T | null`.**
* Functions that may legitimately find nothing (resolveHeadlessShellPath,
* getFrameAtTime, detectGpuEncoder) return a nullable value instead of throwing.
*
*/
// ── Protocol types ─────────────────────────────────────────────────────────────
export type {
HfProtocol,
HfMediaElement,
HfTransitionMeta,
CaptureOptions,
CaptureVideoMetadataHint,
CaptureResult,
CaptureBufferResult,
CapturePerfSummary,
SubTimelineWaitOutcome,
} from "./types.js";
// ── Configuration ──────────────────────────────────────────────────────────────
export {
resolveConfig,
DEFAULT_CONFIG,
scaleProtocolTimeoutForComposition,
type EngineConfig,
} from "./config.js";
export {
DEFAULT_VP9_CPU_USED,
MAX_VP9_CPU_USED,
MIN_VP9_CPU_USED,
normalizeVp9CpuUsed,
} from "./services/vp9Options.js";
export {
getSystemTotalMb,
isLowMemorySystem,
LOW_MEMORY_TOTAL_MB_THRESHOLD,
} from "./services/systemMemory.js";
// ── Browser management ─────────────────────────────────────────────────────────
export {
acquireBrowser,
releaseBrowser,
drainBrowserPool,
resolveHeadlessShellPath,
resolveBrowserGpuMode,
buildChromeArgs,
ENABLE_BROWSER_POOL,
type BuildChromeArgsOptions,
type CaptureMode,
type AcquiredBrowser,
} from "./services/browserManager.js";
// ── Frame capture pipeline ──────────────────────────────────────────────────────
export {
createCaptureSession,
initializeSession,
closeCaptureSession,
captureFrame,
captureFrameToBuffer,
captureFrameToBufferPipelined,
captureFramesBatchPipelined,
DrawElementVerificationError,
isDrawElementVerificationError,
recaptureDrawElementFrameForVerify,
completeDeferredDrawElementInit,
writeCapturedFrame,
discardWarmupCapture,
getCompositionDuration,
getCapturePerfSummary,
prepareCaptureSessionForReuse,
type CaptureSession,
isTransientBrowserError,
isMemoryExhaustionError,
type BeforeCaptureHook,
type DiscardWarmupInnerCapture,
} from "./services/frameCapture.js";
// ── Screenshot (BeginFrame) ─────────────────────────────────────────────────────
export {
beginFrameCapture,
pageScreenshotCapture,
getCdpSession,
injectVideoFramesBatch,
syncVideoFrameVisibility,
cdpSessionCache,
probeBeginFrameLiveness,
initTransparentBackground,
captureAlphaPng,
applyDomLayerMask,
removeDomLayerMask,
DOM_LAYER_MASK_STYLE_ID,
type BeginFrameResult,
} from "./services/screenshotService.js";
// ── Encoding ───────────────────────────────────────────────────────────────────
export {
buildEncoderArgs,
encodeFramesFromDir,
encodeFramesChunkedConcat,
muxVideoWithAudio,
applyFaststart,
detectGpuEncoder,
ENCODER_PRESETS,
getEncoderPreset,
type GpuEncoder,
} from "./services/chunkEncoder.js";
export type { EncoderOptions, EncodeResult, MuxResult } from "./services/chunkEncoder.types.js";
export {
spawnStreamingEncoder,
createFrameReorderBuffer,
type StreamingEncoder,
type StreamingEncoderOptions,
type StreamingEncoderResult,
type FrameReorderBuffer,
} from "./services/streamingEncoder.js";
// ── Media processing ───────────────────────────────────────────────────────────
export {
parseVideoElements,
parseImageElements,
extractVideoFramesRange,
extractAllVideoFrames,
resolveProjectRelativeSrc,
getFrameAtTime,
createFrameLookupTable,
FrameLookupTable,
analyzeClipMediaFit,
type VideoElement,
type ImageElement,
type ExtractedFrames,
type ExtractionOptions,
type ExtractionResult,
type ExtractionPhaseBreakdown,
type VideoFrameFormat,
VIDEO_FRAME_FORMATS,
isVideoFrameFormat,
} from "./services/videoFrameExtractor.js";
export { createVideoFrameInjector } from "./services/videoFrameInjector.js";
export { parseAudioElements, processCompositionAudio } from "./services/audioMixer.js";
export type {
AudioElement,
AudioTrack,
AudioVolumeKeyframe,
MixResult,
} from "./services/audioMixer.types.js";
// ── Parallel rendering ─────────────────────────────────────────────────────────
export {
calculateOptimalWorkers,
distributeFrames,
distributeFramesInterleaved,
executeParallelCapture,
mergeWorkerFrames,
getSystemResources,
type WorkerTask,
type WorkerResult,
type ParallelProgress,
} from "./services/parallelCoordinator.js";
// ── File server ────────────────────────────────────────────────────────────────
export {
createFileServer,
type FileServerOptions,
type FileServerHandle,
} from "./services/fileServer.js";
// ── Utilities ──────────────────────────────────────────────────────────────────
export { quantizeTimeToFrame, MEDIA_VISUAL_STYLE_PROPERTIES } from "@hyperframes/core";
export {
assertSwiftShader,
readWebGlVendorInfo,
SwiftShaderAssertionError,
BROWSER_GPU_NOT_SOFTWARE,
} from "./utils/assertSwiftShader.js";
export { readWebGlVendorInfoFromCanvas } from "./utils/readWebGlVendorInfoFromCanvas.js";
export {
extractMediaMetadata,
extractVideoMetadata,
extractAudioMetadata,
analyzeKeyframeIntervals,
type VideoMetadata,
type AudioMetadata,
type KeyframeAnalysis,
} from "./utils/ffprobe.js";
export { assertPublicHttpsUrl, downloadToTemp, isHttpUrl } from "./utils/urlDownloader.js";
export {
runFfmpeg,
formatFfmpegError,
type RunFfmpegOptions,
type RunFfmpegResult,
} from "./utils/runFfmpeg.js";
export {
assertConfiguredFfmpegBinariesExist,
getFfmpegBinary,
getFfprobeBinary,
FFMPEG_PATH_ENV,
FFPROBE_PATH_ENV,
} from "./utils/ffmpegBinaries.js";
export { trackChildProcess, killTrackedProcesses } from "./utils/processTracker.js";
export {
decodePng,
decodePngToRgb48le,
blitRgba8OverRgb48le,
blitRgb48leRegion,
blitRgb48leAffine,
parseTransformMatrix,
roundedRectAlpha,
resampleRgb48leObjectFit,
normalizeObjectFit,
type ObjectFit,
} from "./utils/alphaBlit.js";
export { groupIntoLayers, type CompositeLayer } from "./utils/layerCompositor.js";
// ── Shader transitions ────────────────────────────────────────────────────────
export {
type TransitionFn,
TRANSITIONS,
crossfade,
sampleRgb48le,
hdrToLinear,
linearToHdr,
convertTransfer,
} from "./utils/shaderTransitions.js";
export {
initHdrReadback,
uploadAndReadbackHdrFrame,
float16ToPqRgb,
buildHdrChromeArgs,
launchHdrBrowser,
} from "./services/hdrCapture.js";
export { captureScreenshotWithAlpha } from "./services/screenshotService.js";
export {
hideVideoElements,
showVideoElements,
queryVideoElementBounds,
queryElementStacking,
type VideoElementBounds,
type ElementStackingInfo,
} from "./services/videoFrameInjector.js";
export {
isHdrColorSpace,
detectTransfer,
getHdrEncoderColorParams,
analyzeCompositionHdr,
DEFAULT_HDR10_MASTERING,
type HdrTransfer,
type HdrEncoderColorParams,
type CompositionHdrInfo,
type HdrMasteringMetadata,
} from "./utils/hdr.js";
export type { VideoColorSpace } from "./utils/ffprobe.js";