* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap The inline Map runs at most 40 chunk renders at once. Both state machine definitions (CDK and SAM) now use a Distributed Map with STANDARD child executions, capped at the function's reserved concurrency so chunks past it queue instead of being throttled. Progress reads each chunk's child execution history so frames, cost and chunk errors still show up. * fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks A throttled task used to burn its four catch-all retries in about 30 s. It now gets its own jittered backoff of several minutes, because another render on the same function can still hold the reserved slots. Plan treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions. * chore(aws-lambda): trim stack comments to keep the comment share flat * fix(aws-lambda): keep render --wait working at hundreds of chunks Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of every child execution's history, take exact chunk frames and cost from the Map output once it exits, and read failed children only when the counts show a failure. The SDK client retries throttled calls up to 8 times with backoff. * feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8 * fix(lambda): keep the concurrency SDK external in published modules * fix(cli): keep an existing stack's reserved concurrency on redeploy Redeploying without --concurrency sent ReservedConcurrency=-1, which removed the cap of 8 that earlier CLI versions reserved by default and stopped recording it in the state file. The override is now left out when the flag is omitted, so SAM keeps the stack's current value (a new stack still gets the template's unreserved default). The state file records the deployed value, and deploy prints the kept reservation. * fix(aws-lambda): keep progress working for callers without DescribeMapRun Callers whose IAM policy came from an earlier `lambda policies user` lack states:DescribeMapRun, so reading a running Distributed Map threw AccessDeniedException out of `lambda progress` and `render --wait` while the render itself went on to succeed. getRenderProgress now catches only that error around the map-run read, returns the parent snapshot (status, plan and assemble progress) and adds a warning to re-apply the policy, which both commands print. Other errors still throw. * fix(cli): let the documented deploy policy drop a reserved concurrency When a redeploy removes a function's reserved concurrency, CloudFormation calls lambda:DeleteFunctionConcurrency. The policy printed by `lambda policies role|user` granted only PutFunctionConcurrency, so the update failed and rolled back. Grant it, and pin it with states:DescribeMapRun in the required-actions test. * fix(aws-lambda): read smoke chunk hashes from the Distributed Map output With chunks in a Distributed Map, their task results live in child executions, so the smoke gate's chunk-hash check found 0 chunks in the parent history and failed every v1/v2 comparison. Read Chunks[] from the RenderChunks / RenderChunksV2 MapStateExited output instead, as getRenderProgress does. The stack this script deploys only has the Distributed Map, so the inline-Map path is gone. * fix(producer): cap a pinned chunk size at the worker concurrency, not 16 With --chunk-size or --target-chunk-frames and no --max-parallel-chunks, the plan still capped the render at 16 chunks however many workers the account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of 113 frames. The cap is now one wave across the worker concurrency, as the --max-parallel-chunks docs say; 16 remains when the concurrency is unknown. * feat(cli): remove a stack's reserved concurrency with --concurrency=none Since an omitted --concurrency now keeps the stack's reservation, there was no CLI way back to the unreserved default. --concurrency=none sends ReservedConcurrency=-1, the state file records the stack as unreserved and deploy says the reservation was removed. Positive N and an omitted flag behave as before; any other value is rejected instead of being silently ignored. * chore(aws-lambda): raise the package size budget for account-sized concurrency Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that.
@hyperframes/aws-lambda
AWS Lambda adapter for HyperFrames distributed rendering. Ships three things together:
- The Lambda handler that wraps the OSS
plan/renderChunk/assembleprimitives behind a single dispatch boundary Step Functions can drive (src/handler.ts). - A client-side SDK —
renderToLambda,getRenderProgress,deploySite, plusvalidateDistributedRenderConfigandcomputeRenderCost(src/sdk/). - An
aws-cdk-libL2 construct (HyperframesRenderStack) that provisions the same topology asexamples/aws-lambda/template.yamlinside an adopter's own CDK app (src/cdk/).
The handler ZIP and the SAM template still drive a maintainer-run real-AWS smoke flow; the SDK + CDK are the supported public surface for adopters.
Architecture
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
▼
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
▼
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — v1 plan tar or v2 manifest/blobs + chunks + output │
└──────────────────────────────────────────────────────────────────┘
The handler downloads inputs from S3 into /tmp, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).
Plan transport selection
Plan v2 is the default for new renders. When planProtocol is omitted,
renderToLambda sends an explicit PlanProtocol: "v2" so the SDK and the
deployed state machine agree:
await renderToLambda({
// ...bucket, state machine, project, and config...
});
V2 never overloads PlanS3Uri. The planner returns
PlanV2ManifestS3Uri and PlanV2ArtifactS3Prefix; chunk workers fetch
only manifest-selected chunk artifacts, while the assembler fetches its
own metadata and audio subset. Blobs are immutable SHA-256-addressed
objects, verified on upload and download, and the manifest is published
last. Unknown protocols and digest mismatches are terminal Step Functions
errors. The monolithic v1 transport remains available as deprecated
compatibility by passing planProtocol: "v1" explicitly.
Upgrade order
This default changes application behavior and requires a coordinated
infrastructure upgrade. Before upgrading an application that calls
renderToLambda:
- Pause new renders and let existing Step Functions executions drain.
- Redeploy the Lambda handler and SAM template or CDK construct from the same new package version.
- Resume renders, then upgrade the application/SDK dependency.
Older state machines can default missing protocol fields to v1 or lack v2
branches, while the new SDK sends explicit v2. If infrastructure cannot be
redeployed first, keep the application on its previous package version or
pass planProtocol: "v1" explicitly until the redeploy is complete.
Chrome runtime
The package supports two Chromium sources:
| Source | Default | Size | When to pick it |
|---|---|---|---|
@sparticuz/chromium |
yes | ~70 MiB compressed | Lambda. Decompresses into /tmp at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
Bundled chrome-headless-shell |
no | ~140 MiB | Fallback. Used if @sparticuz/chromium ever drops HeadlessExperimental.beginFrame support. |
Pick the source at build time:
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
The handler reads HYPERFRAMES_LAMBDA_CHROME_SOURCE at boot. The build
script sets that env var via Lambda function configuration in
examples/aws-lambda/template.yaml.
BeginFrame regression guard
HyperFrames' renderer drives Chrome via the CDP
HeadlessExperimental.beginFrame command — same path the K8s deploy uses.
The Lambda adapter assumes that @sparticuz/chromium's
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
The probe boots @sparticuz/chromium inside
public.ecr.aws/lambda/nodejs:22 and asserts CDP beginFrame with
screenshot: true returns a PNG buffer. Exit code 0 = green; non-zero =
fall back to bundling chrome-headless-shell directly via --source=chrome-headless-shell.
Building the ZIP
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
The build script bundles src/handler.ts via esbuild, stages
@sparticuz/chromium and puppeteer-core under node_modules/, copies
ffmpeg-static into bin/, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's /var/task/.
verify:zip-size enforces:
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
CI fails the PR if either is exceeded.
Running tests
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
Using the SDK
After deploying the stack (via the SAM template, CDK construct below, or your own CFN of choice), drive renders from Node:
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
// One-time upload per project version.
const site = await deploySite({
projectDir: "./my-composition",
bucketName: "hyperframes-render-bucket",
});
// Start a render. Returns immediately — does NOT poll.
const handle = await renderToLambda({
siteHandle: site,
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
config: {
fps: 30,
width: 1920,
height: 1080,
format: "mp4",
chunkSize: 240,
maxParallelChunks: 16,
runtimeCap: "lambda",
},
});
// Poll progress + cost on your own cadence.
const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.overallProgress, progress.costs.displayCost);
if (progress.status === "SUCCEEDED" && progress.outputFile) {
console.log("Render landed at", progress.outputFile.s3Uri);
}
renderToLambda validates the config client-side via
validateDistributedRenderConfig and throws a typed InvalidConfigError
before the Step Functions execution starts, so shape errors surface
synchronously instead of as opaque ExecutionFailed results.
getRenderProgress reports an approximate per-render cost
(accruedSoFarUsd plus a formatted displayCost) derived from Lambda
billed-duration × memory × the us-east-1 on-demand rate plus the Step
Functions transition price. The math is documented in
src/sdk/costAccounting.ts; numbers are best-effort and exclude S3
transfer.
Using the CDK construct
import { App, Stack } from "aws-cdk-lib";
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
const app = new App();
const stack = new Stack(app, "MyApp");
const render = new HyperframesRenderStack(stack, "Render", {
// optional: reservedConcurrency: 8,
// optional: lambdaMemoryMb: 10240,
// optional: chromeSource: "sparticuz",
});
// Re-export so an adopter app can wire dashboards / SNS topics.
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
aws-cdk-lib and constructs are optional peer dependencies: SDK-only
consumers don't pull them at runtime. The construct itself imports from
@hyperframes/aws-lambda/cdk.
What's still ahead
hyperframes lambdaCLI (deploy / sites create / render / progress / destroy) — PR 6.5.- IAM bootstrap subcommand (
policies role | user | validate) — PR 6.9. - Lambda-local regression harness (
--mode=lambda-local) — PR 6.6. - Adopter-facing migration guide — PR 6.8.