mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-10 22:20:14 +00:00
feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe (#878)
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).
This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.
Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.
The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.
Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.
This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.
* fix(lambda): address PR 878 review feedback
- Verify event.PlanHash against the untarred plan.json at the handler
boundary before invoking the producer primitive. Throws typed
PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
non-retryable; previously the field was schema bloat the handler
ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
but use binary mebibytes); previously mixed units made the 248 MiB
budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
comparison instead of lexicographic sort+reverse — the latter would
silently pick "99.x" over "131.x" once Chrome cached three-digit
majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
imports — file is ESM and the same module is already imported.
* docs(lambda): drop internal plan-doc refs from package README
* ci(windows): fix bun filter UNION bug excluding producer from Windows tests
`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.
Enumerate the packages we DO want to test instead.
This commit is contained in:
@@ -0,0 +1,136 @@
|
||||
/**
|
||||
* Lambda event + result types for the HyperFrames distributed render handler.
|
||||
*
|
||||
* The Step Functions state machine in `examples/aws-lambda/template.yaml`
|
||||
* dispatches on the `Action` field. Each action maps 1:1 onto one of the
|
||||
* three OSS distributed primitives:
|
||||
*
|
||||
* "plan" → `plan(projectDir, config, planDir)` (Activity A)
|
||||
* "renderChunk" → `renderChunk(planDir, chunkIndex, output)` (Activity B)
|
||||
* "assemble" → `assemble(planDir, chunkPaths, audio, out)` (Activity C)
|
||||
*
|
||||
* All file I/O is mediated by S3 — the handler downloads inputs into
|
||||
* `/tmp` (Lambda's only writable filesystem path), invokes the primitive,
|
||||
* uploads outputs back to S3, and returns a small JSON payload that fits
|
||||
* inside Step Functions' history budget (under 200 bytes for chunk
|
||||
* results per §2.4).
|
||||
*/
|
||||
|
||||
import type { DistributedRenderConfig } from "@hyperframes/producer/distributed";
|
||||
|
||||
/** Discriminator for the three roles the one Lambda image fulfills. */
|
||||
export type LambdaAction = "plan" | "renderChunk" | "assemble";
|
||||
|
||||
/**
|
||||
* Top-level shape of any event the handler may receive.
|
||||
*
|
||||
* Step Functions can also invoke with a wrapped payload (e.g. when a Map
|
||||
* state's `ItemSelector` passes through `$$.Map.Item.Value`), so the
|
||||
* handler unwraps both `event.Payload` and `event.Input` before
|
||||
* dispatching.
|
||||
*/
|
||||
export type LambdaEvent =
|
||||
| PlanEvent
|
||||
| RenderChunkEvent
|
||||
| AssembleEvent
|
||||
| { Payload: LambdaEvent }
|
||||
| { Input: LambdaEvent };
|
||||
|
||||
/** Activity A: produce a planDir, upload to S3. */
|
||||
export interface PlanEvent {
|
||||
Action: "plan";
|
||||
/** S3 URI pointing at a `tar -czf`-archived project directory (`s3://bucket/key.tar.gz`). */
|
||||
ProjectS3Uri: string;
|
||||
/** S3 URI prefix where the planDir tar should be uploaded (`s3://bucket/{prefix}/`). */
|
||||
PlanOutputS3Prefix: string;
|
||||
/** `DistributedRenderConfig` minus runtime-only fields (logger, abortSignal). */
|
||||
Config: SerializableDistributedRenderConfig;
|
||||
}
|
||||
|
||||
/** Activity B: fetch planDir, render one chunk, upload result. */
|
||||
export interface RenderChunkEvent {
|
||||
Action: "renderChunk";
|
||||
/** S3 URI of the plan tar produced by a PlanEvent invocation. */
|
||||
PlanS3Uri: string;
|
||||
/**
|
||||
* `PlanResult.planHash` from the Plan invocation. The handler verifies
|
||||
* this against the untarred planDir's `plan.json` before invoking the
|
||||
* producer, throwing a typed `PLAN_HASH_MISMATCH` on divergence so the
|
||||
* state machine routes it as non-retryable. Defense-in-depth — the
|
||||
* producer also re-checks internally.
|
||||
*/
|
||||
PlanHash: string;
|
||||
/** 0-based chunk index this invocation should render. */
|
||||
ChunkIndex: number;
|
||||
/** S3 URI prefix where the chunk output should be uploaded (`s3://bucket/{prefix}/`). */
|
||||
ChunkOutputS3Prefix: string;
|
||||
/** Output container format from the plan's encoder.json; drives file vs frame-dir handling. */
|
||||
Format: "mp4" | "mov" | "png-sequence";
|
||||
}
|
||||
|
||||
/** Activity C: fetch planDir + all chunks + audio, assemble, upload final. */
|
||||
export interface AssembleEvent {
|
||||
Action: "assemble";
|
||||
/** S3 URI of the plan tar produced by a PlanEvent invocation. */
|
||||
PlanS3Uri: string;
|
||||
/** S3 URIs of every chunk, ordered by chunk index. Length must equal `chunkCount`. */
|
||||
ChunkS3Uris: string[];
|
||||
/** S3 URI of the planDir's `audio.aac` if the composition has audio; `null` otherwise. */
|
||||
AudioS3Uri: string | null;
|
||||
/** Final output S3 URI (`s3://bucket/key.mp4`). */
|
||||
OutputS3Uri: string;
|
||||
/** Output container format; drives file vs frame-dir handling. */
|
||||
Format: "mp4" | "mov" | "png-sequence";
|
||||
}
|
||||
|
||||
/**
|
||||
* `DistributedRenderConfig` minus the runtime-only fields (`logger`,
|
||||
* `abortSignal`, `producerConfig`). The Step Functions event JSON cannot
|
||||
* carry function references; the handler reconstitutes the runtime fields
|
||||
* from Lambda environment + the AbortController it owns.
|
||||
*/
|
||||
export type SerializableDistributedRenderConfig = Omit<
|
||||
DistributedRenderConfig,
|
||||
"logger" | "abortSignal" | "producerConfig"
|
||||
>;
|
||||
|
||||
// ── Result types — kept small to fit Step Functions history budgets ─────────
|
||||
|
||||
/** Result of a `plan` invocation. Carries enough to size the Map(N) state. */
|
||||
export interface PlanLambdaResult {
|
||||
Action: "plan";
|
||||
PlanS3Uri: string;
|
||||
PlanHash: string;
|
||||
ChunkCount: number;
|
||||
TotalFrames: number;
|
||||
Fps: 24 | 30 | 60;
|
||||
Width: number;
|
||||
Height: number;
|
||||
Format: "mp4" | "mov" | "png-sequence";
|
||||
HasAudio: boolean;
|
||||
AudioS3Uri: string | null;
|
||||
FfmpegVersion: string;
|
||||
ProducerVersion: string;
|
||||
DurationMs: number;
|
||||
}
|
||||
|
||||
/** Result of a `renderChunk` invocation. Sized ≤200 bytes per §2.4. */
|
||||
export interface RenderChunkLambdaResult {
|
||||
Action: "renderChunk";
|
||||
ChunkS3Uri: string;
|
||||
ChunkIndex: number;
|
||||
Sha256: string;
|
||||
FramesEncoded: number;
|
||||
DurationMs: number;
|
||||
}
|
||||
|
||||
/** Result of an `assemble` invocation. */
|
||||
export interface AssembleLambdaResult {
|
||||
Action: "assemble";
|
||||
OutputS3Uri: string;
|
||||
FramesEncoded: number;
|
||||
FileSize: number;
|
||||
DurationMs: number;
|
||||
}
|
||||
|
||||
export type LambdaResult = PlanLambdaResult | RenderChunkLambdaResult | AssembleLambdaResult;
|
||||
Reference in New Issue
Block a user