Files
hyperframes/packages/aws-lambda/README.md
T
James RussoandClaude Opus 5.5 81d73d2278 fix(aws-lambda): chrome-headless-shell 158 in the arm64 video image, sparticuz 153 in the x86 ZIP (#8155)
* fix(aws-lambda): run chrome-headless-shell 158 in the arm64 video image

The arm64 video image now installs Chrome for Testing's linux-arm64 chrome-headless-shell
158.0.8127.36, pinned by sha256 and fetched with curl -f, as the whole shell folder. The Lambda
Node 22 base has none of the shared libraries the shell links and no fonts, so the image installs
them from the base's repository at the same pinned release. The image sets
HYPERFRAMES_LAMBDA_CHROME_SOURCE and HYPERFRAMES_LAMBDA_CHROME_PATH itself, and the nested video
function no longer passes ChromeSource, which would override them. stage-arm64-image.ts stops
fetching the sparticuz arm64 pack and drops the ZIP's x86 sparticuz Chrome instead.

The x86 handler ZIP stays on @sparticuz/chromium, bumped from 148.0.0 to 153.0.0. It builds at
258,502,222 bytes unzipped, under Lambda's 262,144,000-byte limit and the 248 MiB in-house gate.

Two fixes for --source=chrome-headless-shell:
- build-zip.ts staged only the executable, which exits at launch without its ICU data and .pak
  files. It now stages the whole shell folder at chrome-headless-shell-linux64/.
- Nothing set HYPERFRAMES_LAMBDA_CHROME_PATH, which src/chromium.ts requires for that source.
  The SAM template and the CDK stack now set it to the bundled shell when ChromeSource asks for it.

Drops the unused @types/aws-lambda devDependency, which the dead-code audit reports whenever this
package's manifest changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(publish): build the Lambda arm64 video image on an ARM runner

The video stage now installs Chrome's system libraries with microdnf, so it runs code for arm64.
The pack job built that image on ubuntu-latest with no emulation, which it could do only while the
stage copied files. A new lambda-video-image job builds it natively on ubuntu-24.04-arm, from the
handler zip whose sha256 the zip job reported, and reports the OCI tarball's sha256. Pack downloads
that tarball, checks the sha256 against the job's, and pins it as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(aws-lambda): fit the comment ratchet and the package size budget

chromium.ts loses the two doc lines on BUNDLED_SHELL_DIR and BUNDLED_SHELL_PATH; the names and the
SAM template's comment already say what they are.

@hyperframes/aws-lambda's unpacked budget goes from 290,508 to 298,000 bytes. The package unpacks to
295,324 bytes, up from the shared chrome-headless-shell staging module the ZIP and image scripts now
import and from the README and docs that describe the two Chrome sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 09:07:46 +00:00

241 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# @hyperframes/aws-lambda
AWS Lambda adapter for HyperFrames distributed rendering. Ships three
things together:
1. The **Lambda handler** that wraps the OSS `plan` / `renderChunk` /
`assemble` primitives behind a single dispatch boundary Step Functions
can drive (`src/handler.ts`).
2. A **client-side SDK** — `renderToLambda`, `getRenderProgress`,
`deploySite`, plus `validateDistributedRenderConfig` and
`computeRenderCost` (`src/sdk/`).
3. An **`aws-cdk-lib` L2 construct** (`HyperframesRenderStack`) that
provisions the same topology as `examples/aws-lambda/template.yaml`
inside an adopter's own CDK app (`src/cdk/`).
The handler ZIP and the SAM template still drive a maintainer-run real-AWS
smoke flow; the SDK + CDK are the supported public surface for adopters.
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
▼
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
▼
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — Plan v2 manifest/blobs + chunks + output │
└──────────────────────────────────────────────────────────────────┘
```
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).
### Plan transport
Plan v2 is the only supported distributed plan transport.
`renderToLambda` sends `PlanProtocol: "v2"` without requiring a protocol
option:
```ts
await renderToLambda({
// ...bucket, state machine, project, and config...
});
```
V2 never overloads `PlanS3Uri`. The planner returns
`PlanV2ManifestS3Uri` and `PlanV2ArtifactS3Prefix`; chunk workers fetch
only manifest-selected chunk artifacts, while the assembler fetches its
own metadata and audio subset. Blobs are immutable SHA-256-addressed
objects, verified on upload and download, and the manifest is published
last. Unknown protocols and digest mismatches are terminal Step Functions
errors.
#### Upgrade order
This default changes application behavior and requires a coordinated
infrastructure upgrade. Before upgrading an application that calls
`renderToLambda`:
1. Pause new renders and let existing Step Functions executions drain.
2. Redeploy the Lambda handler and SAM template or CDK construct from the
same new package version.
3. Resume renders, then upgrade the application/SDK dependency.
A site is a `sites/<id>/files.json` file list that an older stack cannot
read. Before it uploads anything,
`renderToLambda` reads the state machine definition
(`states:DescribeStateMachine`) and fails with `STACK_TOO_OLD` when the
stack predates that format. If infrastructure cannot be redeployed first,
keep the application on its previous package version until the redeploy is
complete. Site ids from earlier versions keep working after the redeploy:
when `sites/<id>/files.json` is missing, the render uses that site's
`project.tar.gz`.
## Chrome runtime
The package supports two Chromium sources:
| Source | Default | Size | When to pick it |
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
| Bundled `chrome-headless-shell` | no | ~296 MB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
Pick the source at build time:
```bash
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
```
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot, and for
`chrome-headless-shell` the executable from `HYPERFRAMES_LAMBDA_CHROME_PATH`.
`examples/aws-lambda/template.yaml` and the CDK construct set both from the
Chrome source. The build stages the whole shell folder (the executable needs
its ICU data and `.pak` files beside it), which is over Lambda's 250 MiB
unzipped limit for a ZIP function, so the x86 ZIP ships `@sparticuz/chromium`.
The arm64 video image (`image/Dockerfile`, `video` target) does not use
`@sparticuz/chromium`: it installs Chrome for Testing's linux-arm64
`chrome-headless-shell` folder, pinned by version and sha256, with the shared
libraries and fonts it needs, and sets both env vars itself.
## BeginFrame regression guard
HyperFrames' renderer drives Chrome via the CDP
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
The Lambda adapter assumes that `@sparticuz/chromium`'s
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:
```bash
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
```
The probe boots `@sparticuz/chromium` inside
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
## Building the ZIP
```bash
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
```
The build script bundles `src/handler.ts` via esbuild, stages
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's `/var/task/`.
`verify:zip-size` enforces:
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
CI fails the PR if either is exceeded.
## Running tests
```bash
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
```
## Using the SDK
After deploying the stack (via the SAM template, CDK construct below, or
your own CFN of choice), drive renders from Node:
```ts
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
// One-time upload per project version.
const site = await deploySite({
projectDir: "./my-composition",
bucketName: "hyperframes-render-bucket",
});
// Start a render. Returns immediately — does NOT poll.
const handle = await renderToLambda({
siteHandle: site,
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
config: {
fps: 30,
width: 1920,
height: 1080,
format: "mp4",
chunkSize: 240,
maxParallelChunks: 16,
runtimeCap: "lambda",
},
});
// Poll progress + cost on your own cadence.
const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.overallProgress, progress.costs.displayCost);
if (progress.status === "SUCCEEDED" && progress.outputFile) {
console.log("Render landed at", progress.outputFile.s3Uri);
}
```
`renderToLambda` validates the config client-side via
`validateDistributedRenderConfig` and throws a typed `InvalidConfigError`
before the Step Functions execution starts, so shape errors surface
synchronously instead of as opaque `ExecutionFailed` results.
`getRenderProgress` reports an approximate per-render cost
(`accruedSoFarUsd` plus a formatted `displayCost`) derived from Lambda
billed-duration × memory × the us-east-1 on-demand rate plus the Step
Functions transition price. The math is documented in
`src/sdk/costAccounting.ts`; numbers are best-effort and exclude S3
transfer.
## Using the CDK construct
```ts
import { App, Stack } from "aws-cdk-lib";
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
const app = new App();
const stack = new Stack(app, "MyApp");
const render = new HyperframesRenderStack(stack, "Render", {
// optional: reservedConcurrency: 8,
// optional: lambdaMemoryMb: 10240,
// optional: chromeSource: "sparticuz",
});
// Re-export so an adopter app can wire dashboards / SNS topics.
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
```
`aws-cdk-lib` and `constructs` are **optional peer dependencies**: SDK-only
consumers don't pull them at runtime. The construct itself imports from
`@hyperframes/aws-lambda/cdk`.
## What's still ahead
- `hyperframes lambda` CLI (deploy / sites create / render / progress / destroy) — PR 6.5.
- IAM bootstrap subcommand (`policies role | user | validate`) — PR 6.9.
- Lambda-local regression harness (`--mode=lambda-local`) — PR 6.6.
- Adopter-facing migration guide — PR 6.8.