mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:54:29 +00:00
James Russo
c50f59a53b
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).
This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.
Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.
The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.
Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.
This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.
* fix(lambda): address PR 878 review feedback
- Verify event.PlanHash against the untarred plan.json at the handler
boundary before invoking the producer primitive. Throws typed
PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
non-retryable; previously the field was schema bloat the handler
ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
but use binary mebibytes); previously mixed units made the 248 MiB
budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
comparison instead of lexicographic sort+reverse — the latter would
silently pick "99.x" over "131.x" once Chrome cached three-digit
majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
imports — file is ESM and the same module is already imported.
* docs(lambda): drop internal plan-doc refs from package README
* ci(windows): fix bun filter UNION bug excluding producer from Windows tests
`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.
Enumerate the packages we DO want to test instead.
114 lines
6.0 KiB
Markdown
114 lines
6.0 KiB
Markdown
# @hyperframes/aws-lambda
|
|
|
|
AWS Lambda adapter for HyperFrames distributed rendering. Wraps the OSS
|
|
`plan` / `renderChunk` / `assemble` primitives into a single Lambda handler
|
|
that Step Functions can dispatch on, plus a build pipeline that bundles
|
|
the handler + Chrome runtime + ffmpeg into a deployable ZIP.
|
|
|
|
The Lambda adapter ships in two parts: the foundation (this package + the
|
|
SAM example) validates the architecture end-to-end on real AWS; the
|
|
user-facing surface (CLI, CDK construct, migration guide) lands in
|
|
follow-up PRs.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ Step Functions state machine │
|
|
│ Plan → Map(N) RenderChunk → Assemble │
|
|
└──────────────────────────────────────────────────────────────────┘
|
|
│ dispatches by event.Action
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ One Lambda function (this package's `dist/handler.zip`) │
|
|
│ handler.mjs │
|
|
│ ├─ Action="plan" → @hyperframes/producer/distributed │
|
|
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
|
|
│ └─ Action="assemble" → @hyperframes/producer/distributed │
|
|
│ bin/ffmpeg — ffmpeg-static │
|
|
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
|
|
└──────────────────────────────────────────────────────────────────┘
|
|
│ pure functions over local paths
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
|
|
└──────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
|
|
uploads outputs back to S3, and returns a small JSON result that fits
|
|
inside Step Functions' history budget (under 200 bytes per chunk).
|
|
|
|
## Chrome runtime
|
|
|
|
The package supports two Chromium sources:
|
|
|
|
| Source | Default | Size | When to pick it |
|
|
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
|
|
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
|
|
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
|
|
|
|
Pick the source at build time:
|
|
|
|
```bash
|
|
bun run --cwd packages/aws-lambda build:zip
|
|
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
|
|
```
|
|
|
|
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
|
|
script sets that env var via Lambda function configuration in
|
|
`examples/aws-lambda/template.yaml`.
|
|
|
|
## BeginFrame regression guard
|
|
|
|
HyperFrames' renderer drives Chrome via the CDP
|
|
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
|
|
The Lambda adapter assumes that `@sparticuz/chromium`'s
|
|
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
|
|
it on every release), the package ships a Docker probe:
|
|
|
|
```bash
|
|
# Build the Lambda-like container and run the probe.
|
|
bun run --cwd packages/aws-lambda probe:beginframe:docker
|
|
```
|
|
|
|
The probe boots `@sparticuz/chromium` inside
|
|
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
|
|
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
|
|
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
|
|
|
|
## Building the ZIP
|
|
|
|
```bash
|
|
bun install # at the monorepo root
|
|
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
|
|
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
|
|
```
|
|
|
|
The build script bundles `src/handler.ts` via esbuild, stages
|
|
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
|
|
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
|
|
designed to extract cleanly into Lambda's `/var/task/`.
|
|
|
|
`verify:zip-size` enforces:
|
|
|
|
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
|
|
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
|
|
|
|
CI fails the PR if either is exceeded.
|
|
|
|
## Running tests
|
|
|
|
```bash
|
|
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
|
|
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
|
|
```
|
|
|
|
## What's NOT in this PR
|
|
|
|
- `examples/aws-lambda/template.yaml` (SAM template — separate PR).
|
|
- Real-AWS deploy + smoke workflow (separate PR).
|
|
- `npx hyperframes lambda deploy` CLI — follow-up.
|
|
- CDK construct (`HyperframesRenderStack`) — follow-up.
|
|
- Migration guide — follow-up.
|