* fix(aws-lambda): match the shell to its own pin, ship only hashed files, refuse the shell ZIP in CDK and SAM Review follow-ups to the pinned shell route: - build-zip --source=chrome-headless-shell accepts the folder only when its digest is pinned for this version, not for any pinned build. - It leaves out the files the digest skips (debug.log and chrome_debug.log, which Chrome writes as it runs), so nothing unhashed ships. - The CDK construct throws on chromeSource "chrome-headless-shell" and the SAM template allows only sparticuz, as lambda deploy already refused it: a shell ZIP cannot fit a function. The CHROME_PATH wiring for that ZIP is gone with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(aws-lambda): raise the packed size budget to 82,000 bytes The shipped build-zip script now checks the shell against its own version's pin and skips the files the digest leaves out. That takes the tarball to 81,378 bytes in CI, over the 80,500 budget's 1% tolerance; main was already within about 200 bytes of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
12 KiB
@hyperframes/aws-lambda
AWS Lambda adapter for HyperFrames distributed rendering. Ships three things together:
- The Lambda handler that wraps the OSS
plan/renderChunk/assembleprimitives behind a single dispatch boundary Step Functions can drive (src/handler.ts). - A client-side SDK —
renderToLambda,getRenderProgress,deploySite, plusvalidateDistributedRenderConfigandcomputeRenderCost(src/sdk/). - An
aws-cdk-libL2 construct (HyperframesRenderStack) that provisions the same topology asexamples/aws-lambda/template.yamlinside an adopter's own CDK app (src/cdk/).
The handler ZIP and the SAM template still drive a maintainer-run real-AWS smoke flow; the SDK + CDK are the supported public surface for adopters.
Architecture
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
▼
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
▼
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — Plan v2 manifest/blobs + chunks + output │
└──────────────────────────────────────────────────────────────────┘
The handler downloads inputs from S3 into /tmp, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).
Plan transport
Plan v2 is the only supported distributed plan transport.
renderToLambda sends PlanProtocol: "v2" without requiring a protocol
option:
await renderToLambda({
// ...bucket, state machine, project, and config...
});
V2 never overloads PlanS3Uri. The planner returns
PlanV2ManifestS3Uri and PlanV2ArtifactS3Prefix; chunk workers fetch
only manifest-selected chunk artifacts, while the assembler fetches its
own metadata and audio subset. Blobs are immutable SHA-256-addressed
objects, verified on upload and download, and the manifest is published
last. Unknown protocols and digest mismatches are terminal Step Functions
errors.
Upgrade order
This default changes application behavior and requires a coordinated
infrastructure upgrade. Before upgrading an application that calls
renderToLambda:
- Pause new renders and let existing Step Functions executions drain.
- Redeploy the Lambda handler and SAM template or CDK construct from the same new package version.
- Resume renders, then upgrade the application/SDK dependency.
A site is a sites/<id>/files.json file list that an older stack cannot
read. Before it uploads anything,
renderToLambda reads the state machine definition
(states:DescribeStateMachine) and fails with STACK_TOO_OLD when the
stack predates that format. If infrastructure cannot be redeployed first,
keep the application on its previous package version until the redeploy is
complete. Site ids from earlier versions keep working after the redeploy:
when sites/<id>/files.json is missing, the render uses that site's
project.tar.gz.
Chrome runtime
The package supports two Chromium sources:
| Source | Default | Size | When to pick it |
|---|---|---|---|
@sparticuz/chromium |
yes | ~70 MiB compressed | Lambda. Decompresses into /tmp at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
Bundled chrome-headless-shell |
no | ~296 MB | Does not fit a ZIP function today. Kept for the case where @sparticuz/chromium drops HeadlessExperimental.beginFrame. |
Pick the source at build time:
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
The handler reads HYPERFRAMES_LAMBDA_CHROME_SOURCE at boot, and for
chrome-headless-shell the executable from HYPERFRAMES_LAMBDA_CHROME_PATH.
The build stages the repo's pinned shell build, and only when its folder
matches that build's pinned digest. It stages the whole folder (the
executable needs its ICU data and .pak files beside it), which is over
Lambda's 250 MiB unzipped limit for a ZIP function, so the build fails its
size gate and the x86 ZIP ships @sparticuz/chromium. For the same reason
hyperframes lambda deploy, examples/aws-lambda/template.yaml and the CDK
construct refuse chrome-headless-shell as the ZIP's Chrome source.
The arm64 video image (image/Dockerfile, video target) does not use
@sparticuz/chromium: it installs Chrome for Testing's linux-arm64
chrome-headless-shell folder, pinned by version and sha256, with the shared
libraries it needs, and sets both env vars itself. It keeps @sparticuz/chromium's
fonts and fonts.conf from the handler ZIP, so text resolves to the same fonts as
on the x86 function; CI compares the two on a mixed-script page.
BeginFrame regression guard
HyperFrames' renderer drives Chrome via the CDP
HeadlessExperimental.beginFrame command — same path the K8s deploy uses.
The Lambda adapter assumes that @sparticuz/chromium's
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
The probe boots @sparticuz/chromium inside
public.ecr.aws/lambda/nodejs:22 and asserts CDP beginFrame with
screenshot: true returns a PNG buffer. Exit code 0 = green; non-zero =
@sparticuz/chromium lost BeginFrame. --source=chrome-headless-shell does
not fit a ZIP function, so that function would move to a container image like
the arm64 video function.
Building the ZIP
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
The build script bundles src/handler.ts via esbuild, stages
@sparticuz/chromium and puppeteer-core under node_modules/, copies
ffmpeg-static into bin/, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's /var/task/.
verify:zip-size enforces:
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
The sparticuz ZIP is about 1.5 MiB under the unzipped budget, and the handler
bundles the producer, engine and core. So the Lambda handler ZIP workflow
builds it on every PR that touches them, and fails the PR if either is exceeded.
Running tests
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
Using the SDK
After deploying the stack (via the SAM template, CDK construct below, or your own CFN of choice), drive renders from Node:
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
// One-time upload per project version.
const site = await deploySite({
projectDir: "./my-composition",
bucketName: "hyperframes-render-bucket",
});
// Start a render. Returns immediately — does NOT poll.
const handle = await renderToLambda({
siteHandle: site,
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
config: {
fps: 30,
width: 1920,
height: 1080,
format: "mp4",
chunkSize: 240,
maxParallelChunks: 16,
runtimeCap: "lambda",
},
});
// Poll progress + cost on your own cadence.
const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.overallProgress, progress.costs.displayCost);
if (progress.status === "SUCCEEDED" && progress.outputFile) {
console.log("Render landed at", progress.outputFile.s3Uri);
}
renderToLambda validates the config client-side via
validateDistributedRenderConfig and throws a typed InvalidConfigError
before the Step Functions execution starts, so shape errors surface
synchronously instead of as opaque ExecutionFailed results.
getRenderProgress reports an approximate per-render cost
(accruedSoFarUsd plus a formatted displayCost) derived from Lambda
billed-duration × memory × the us-east-1 on-demand rate plus the Step
Functions transition price. The math is documented in
src/sdk/costAccounting.ts; numbers are best-effort and exclude S3
transfer.
Using the CDK construct
import { App, Stack } from "aws-cdk-lib";
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
const app = new App();
const stack = new Stack(app, "MyApp");
const render = new HyperframesRenderStack(stack, "Render", {
// optional: reservedConcurrency: 8,
// optional: lambdaMemoryMb: 10240,
// optional: chromeSource: "sparticuz",
});
// Re-export so an adopter app can wire dashboards / SNS topics.
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
aws-cdk-lib and constructs are optional peer dependencies: SDK-only
consumers don't pull them at runtime. The construct itself imports from
@hyperframes/aws-lambda/cdk.
What's still ahead
hyperframes lambdaCLI (deploy / sites create / render / progress / destroy) — PR 6.5.- IAM bootstrap subcommand (
policies role | user | validate) — PR 6.9. - Lambda-local regression harness (
--mode=lambda-local) — PR 6.6. - Adopter-facing migration guide — PR 6.8.