Files
hyperframes/packages/aws-lambda/README.md
T
James Russo 34d1f0e1d0 feat(lambda): add TypeScript SDK and CDK construct (#909)
* feat(lambda): add TypeScript SDK and CDK construct

Adds the client-side surface on top of the Phase 6a Lambda handler so
adopters can drive a deployed stack from Node without writing AWS-SDK
boilerplate:

- renderToLambda(opts) starts a Step Functions execution and returns a
  handle. Does NOT poll.
- getRenderProgress({ executionArn }) returns a snapshot of progress,
  frames rendered, cost (Lambda GB-seconds + SFN transitions), errors,
  and the final output object once Assemble completes.
- deploySite({ projectDir, bucketName }) content-addresses the project
  tree, tar.gzs it, and uploads to S3 with a HeadObject short-circuit so
  re-renders of the same tree skip the tar+PUT.
- validateDistributedRenderConfig throws a typed InvalidConfigError
  before StartExecution, so shape errors surface synchronously.
- computeRenderCost is exposed for callers who want to format cost out
  of band.

Also ships HyperframesRenderStack, an aws-cdk-lib L2 construct that
emits the same topology as examples/aws-lambda/template.yaml. Lives on
the ./cdk subpath export so SDK-only consumers don't pull aws-cdk-lib
into their runtime graph (declared as an optional peer dependency).

Tests: 24 new unit tests across the SDK plus 9 CDK synth / contract /
snapshot tests. All 83 tests in packages/aws-lambda/src pass.

* refactor(lambda): /simplify pass on the SDK + CDK PR

Pulls shared logic out so the SDK doesn't re-invent things the handler
and the producer already have:

- `formatExtension` extracted to packages/aws-lambda/src/formatExtension.ts.
  handler.ts and renderToLambda.ts both used identical 12-line copies of
  this switch.
- `PLAN_PROJECT_DIR_SKIP_SEGMENTS` is now exported from
  @hyperframes/producer/distributed. deploySite consumes it instead of
  its own duplicate SKIP_TOP_LEVEL set; the two lists were trivially
  identical and would have drifted silently.
- `FakeS3` + `drainBody` factored out of the two SDK test files into
  src/sdk/__fixtures__/fakeS3.ts. Drops ~110 lines of test-file
  duplication and gives future SDK tests a one-line FakeS3 import.
- S3 URI building in deploySite and renderToLambda routes through the
  existing `formatS3Uri` helper instead of inline `s3://...`
  concatenation; matches the convention already in handler.ts.

Net -133 lines across the touched files. All 83 aws-lambda tests still
pass; all 60 producer distributed tests still pass.

* fix(lambda): bump CDK test timeouts for CI cold-start synth

The bun:test default 5s timeout tripped the first CDK snapshot test
in CI when the cold-start `Template.fromStack(stack)` synth took ~5-8s
on the slowest GitHub Actions runner. Locally on a warm shell the
synth measures <1s, so the failure didn't reproduce until PR #909 hit
CI.

Two changes:

  - Both CDK test files cache one synth in `beforeAll(..., 30000)` and
    reuse the result across every test that uses the default props.
    Each individual test now runs in microseconds (pure assertions
    against the already-synthed template), so the 5s timeout no longer
    applies on the hot path.

  - The two contract tests that exercise non-default props
    (reservedConcurrency, projectName) still synth fresh per-test; they
    get a per-test `it(..., 30000)` timeout.

No behavior changes.

* fix(lambda): address PR review on SDK + CDK construct

Three correctness + ergonomics fixes raised in Vai's review:

  - getRenderProgress over-counted SFN transitions by 3-5×. Step
    Functions Standard Workflows bill per state-entry, not per
    history event. Each Task produces ~5-7 history events
    (Scheduled / Started / Succeeded / TaskStateExited / …);
    counting `events.length` reported the runaway. Switch to
    counting `*StateEntered` events explicitly.

  - assembleComplete + outputFile detection was coupled to the
    Lambda payload's `Action` field. Move both signals onto the
    enclosing state name (`StateExited.name === "Assemble"`), which
    is the state-machine identity rather than the Lambda event
    contract. framesRendered increment moves to the same boundary
    (RenderChunk state).

  - SiteHandle now carries `bucketName` directly so README + CLI
    callers don't have to re-parse `projectS3Uri.split("/")[2]`.

Test updates: getRenderProgress tests wrap renderChunk/assemble
events in matching StateEntered + StateExited pairs so the new
state-name-driven dispatch is exercised end-to-end. SiteHandle
fixture in renderToLambda.test.ts gets the new bucketName field.

All 83 aws-lambda tests still pass.
2026-05-17 03:03:51 -04:00

192 lines
8.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# @hyperframes/aws-lambda
AWS Lambda adapter for HyperFrames distributed rendering. Ships three
things together:
1. The **Lambda handler** that wraps the OSS `plan` / `renderChunk` /
`assemble` primitives behind a single dispatch boundary Step Functions
can drive (`src/handler.ts`).
2. A **client-side SDK**`renderToLambda`, `getRenderProgress`,
`deploySite`, plus `validateDistributedRenderConfig` and
`computeRenderCost` (`src/sdk/`).
3. An **`aws-cdk-lib` L2 construct** (`HyperframesRenderStack`) that
provisions the same topology as `examples/aws-lambda/template.yaml`
inside an adopter's own CDK app (`src/cdk/`).
The handler ZIP and the SAM template still drive a maintainer-run real-AWS
smoke flow; the SDK + CDK are the supported public surface for adopters.
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
└──────────────────────────────────────────────────────────────────┘
```
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).
## Chrome runtime
The package supports two Chromium sources:
| Source | Default | Size | When to pick it |
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
Pick the source at build time:
```bash
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
```
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
script sets that env var via Lambda function configuration in
`examples/aws-lambda/template.yaml`.
## BeginFrame regression guard
HyperFrames' renderer drives Chrome via the CDP
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
The Lambda adapter assumes that `@sparticuz/chromium`'s
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:
```bash
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
```
The probe boots `@sparticuz/chromium` inside
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
## Building the ZIP
```bash
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
```
The build script bundles `src/handler.ts` via esbuild, stages
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's `/var/task/`.
`verify:zip-size` enforces:
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
CI fails the PR if either is exceeded.
## Running tests
```bash
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
```
## Using the SDK
After deploying the stack (via the SAM template, CDK construct below, or
your own CFN of choice), drive renders from Node:
```ts
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
// One-time upload per project version.
const site = await deploySite({
projectDir: "./my-composition",
bucketName: "hyperframes-render-bucket",
});
// Start a render. Returns immediately — does NOT poll.
const handle = await renderToLambda({
siteHandle: site,
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
config: {
fps: 30,
width: 1920,
height: 1080,
format: "mp4",
chunkSize: 240,
maxParallelChunks: 16,
runtimeCap: "lambda",
},
});
// Poll progress + cost on your own cadence.
const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.overallProgress, progress.costs.displayCost);
if (progress.status === "SUCCEEDED" && progress.outputFile) {
console.log("Render landed at", progress.outputFile.s3Uri);
}
```
`renderToLambda` validates the config client-side via
`validateDistributedRenderConfig` and throws a typed `InvalidConfigError`
before the Step Functions execution starts, so shape errors surface
synchronously instead of as opaque `ExecutionFailed` results.
`getRenderProgress` reports an approximate per-render cost
(`accruedSoFarUsd` plus a formatted `displayCost`) derived from Lambda
billed-duration × memory × the us-east-1 on-demand rate plus the Step
Functions transition price. The math is documented in
`src/sdk/costAccounting.ts`; numbers are best-effort and exclude S3
transfer.
## Using the CDK construct
```ts
import { App, Stack } from "aws-cdk-lib";
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
const app = new App();
const stack = new Stack(app, "MyApp");
const render = new HyperframesRenderStack(stack, "Render", {
// optional: reservedConcurrency: 8,
// optional: lambdaMemoryMb: 10240,
// optional: chromeSource: "sparticuz",
});
// Re-export so an adopter app can wire dashboards / SNS topics.
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
```
`aws-cdk-lib` and `constructs` are **optional peer dependencies**: SDK-only
consumers don't pull them at runtime. The construct itself imports from
`@hyperframes/aws-lambda/cdk`.
## What's still ahead
- `hyperframes lambda` CLI (deploy / sites create / render / progress / destroy) — PR 6.5.
- IAM bootstrap subcommand (`policies role | user | validate`) — PR 6.9.
- Lambda-local regression harness (`--mode=lambda-local`) — PR 6.6.
- Adopter-facing migration guide — PR 6.8.