mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 12:52:44 +00:00
* feat(lambda): add TypeScript SDK and CDK construct
Adds the client-side surface on top of the Phase 6a Lambda handler so
adopters can drive a deployed stack from Node without writing AWS-SDK
boilerplate:
- renderToLambda(opts) starts a Step Functions execution and returns a
handle. Does NOT poll.
- getRenderProgress({ executionArn }) returns a snapshot of progress,
frames rendered, cost (Lambda GB-seconds + SFN transitions), errors,
and the final output object once Assemble completes.
- deploySite({ projectDir, bucketName }) content-addresses the project
tree, tar.gzs it, and uploads to S3 with a HeadObject short-circuit so
re-renders of the same tree skip the tar+PUT.
- validateDistributedRenderConfig throws a typed InvalidConfigError
before StartExecution, so shape errors surface synchronously.
- computeRenderCost is exposed for callers who want to format cost out
of band.
Also ships HyperframesRenderStack, an aws-cdk-lib L2 construct that
emits the same topology as examples/aws-lambda/template.yaml. Lives on
the ./cdk subpath export so SDK-only consumers don't pull aws-cdk-lib
into their runtime graph (declared as an optional peer dependency).
Tests: 24 new unit tests across the SDK plus 9 CDK synth / contract /
snapshot tests. All 83 tests in packages/aws-lambda/src pass.
* refactor(lambda): /simplify pass on the SDK + CDK PR
Pulls shared logic out so the SDK doesn't re-invent things the handler
and the producer already have:
- `formatExtension` extracted to packages/aws-lambda/src/formatExtension.ts.
handler.ts and renderToLambda.ts both used identical 12-line copies of
this switch.
- `PLAN_PROJECT_DIR_SKIP_SEGMENTS` is now exported from
@hyperframes/producer/distributed. deploySite consumes it instead of
its own duplicate SKIP_TOP_LEVEL set; the two lists were trivially
identical and would have drifted silently.
- `FakeS3` + `drainBody` factored out of the two SDK test files into
src/sdk/__fixtures__/fakeS3.ts. Drops ~110 lines of test-file
duplication and gives future SDK tests a one-line FakeS3 import.
- S3 URI building in deploySite and renderToLambda routes through the
existing `formatS3Uri` helper instead of inline `s3://...`
concatenation; matches the convention already in handler.ts.
Net -133 lines across the touched files. All 83 aws-lambda tests still
pass; all 60 producer distributed tests still pass.
* fix(lambda): bump CDK test timeouts for CI cold-start synth
The bun:test default 5s timeout tripped the first CDK snapshot test
in CI when the cold-start `Template.fromStack(stack)` synth took ~5-8s
on the slowest GitHub Actions runner. Locally on a warm shell the
synth measures <1s, so the failure didn't reproduce until PR #909 hit
CI.
Two changes:
- Both CDK test files cache one synth in `beforeAll(..., 30000)` and
reuse the result across every test that uses the default props.
Each individual test now runs in microseconds (pure assertions
against the already-synthed template), so the 5s timeout no longer
applies on the hot path.
- The two contract tests that exercise non-default props
(reservedConcurrency, projectName) still synth fresh per-test; they
get a per-test `it(..., 30000)` timeout.
No behavior changes.
* fix(lambda): address PR review on SDK + CDK construct
Three correctness + ergonomics fixes raised in Vai's review:
- getRenderProgress over-counted SFN transitions by 3-5×. Step
Functions Standard Workflows bill per state-entry, not per
history event. Each Task produces ~5-7 history events
(Scheduled / Started / Succeeded / TaskStateExited / …);
counting `events.length` reported the runaway. Switch to
counting `*StateEntered` events explicitly.
- assembleComplete + outputFile detection was coupled to the
Lambda payload's `Action` field. Move both signals onto the
enclosing state name (`StateExited.name === "Assemble"`), which
is the state-machine identity rather than the Lambda event
contract. framesRendered increment moves to the same boundary
(RenderChunk state).
- SiteHandle now carries `bucketName` directly so README + CLI
callers don't have to re-parse `projectS3Uri.split("/")[2]`.
Test updates: getRenderProgress tests wrap renderChunk/assemble
events in matching StateEntered + StateExited pairs so the new
state-name-driven dispatch is exercised end-to-end. SiteHandle
fixture in renderToLambda.test.ts gets the new bucketName field.
All 83 aws-lambda tests still pass.
192 lines
8.8 KiB
Markdown
192 lines
8.8 KiB
Markdown
# @hyperframes/aws-lambda
|
||
|
||
AWS Lambda adapter for HyperFrames distributed rendering. Ships three
|
||
things together:
|
||
|
||
1. The **Lambda handler** that wraps the OSS `plan` / `renderChunk` /
|
||
`assemble` primitives behind a single dispatch boundary Step Functions
|
||
can drive (`src/handler.ts`).
|
||
2. A **client-side SDK** — `renderToLambda`, `getRenderProgress`,
|
||
`deploySite`, plus `validateDistributedRenderConfig` and
|
||
`computeRenderCost` (`src/sdk/`).
|
||
3. An **`aws-cdk-lib` L2 construct** (`HyperframesRenderStack`) that
|
||
provisions the same topology as `examples/aws-lambda/template.yaml`
|
||
inside an adopter's own CDK app (`src/cdk/`).
|
||
|
||
The handler ZIP and the SAM template still drive a maintainer-run real-AWS
|
||
smoke flow; the SDK + CDK are the supported public surface for adopters.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ Step Functions state machine │
|
||
│ Plan → Map(N) RenderChunk → Assemble │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ dispatches by event.Action
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ One Lambda function (this package's `dist/handler.zip`) │
|
||
│ handler.mjs │
|
||
│ ├─ Action="plan" → @hyperframes/producer/distributed │
|
||
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
|
||
│ └─ Action="assemble" → @hyperframes/producer/distributed │
|
||
│ bin/ffmpeg — ffmpeg-static │
|
||
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ pure functions over local paths
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
|
||
uploads outputs back to S3, and returns a small JSON result that fits
|
||
inside Step Functions' history budget (under 200 bytes per chunk).
|
||
|
||
## Chrome runtime
|
||
|
||
The package supports two Chromium sources:
|
||
|
||
| Source | Default | Size | When to pick it |
|
||
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
|
||
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
|
||
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
|
||
|
||
Pick the source at build time:
|
||
|
||
```bash
|
||
bun run --cwd packages/aws-lambda build:zip
|
||
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
|
||
```
|
||
|
||
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
|
||
script sets that env var via Lambda function configuration in
|
||
`examples/aws-lambda/template.yaml`.
|
||
|
||
## BeginFrame regression guard
|
||
|
||
HyperFrames' renderer drives Chrome via the CDP
|
||
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
|
||
The Lambda adapter assumes that `@sparticuz/chromium`'s
|
||
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
|
||
it on every release), the package ships a Docker probe:
|
||
|
||
```bash
|
||
# Build the Lambda-like container and run the probe.
|
||
bun run --cwd packages/aws-lambda probe:beginframe:docker
|
||
```
|
||
|
||
The probe boots `@sparticuz/chromium` inside
|
||
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
|
||
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
|
||
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
|
||
|
||
## Building the ZIP
|
||
|
||
```bash
|
||
bun install # at the monorepo root
|
||
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
|
||
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
|
||
```
|
||
|
||
The build script bundles `src/handler.ts` via esbuild, stages
|
||
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
|
||
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
|
||
designed to extract cleanly into Lambda's `/var/task/`.
|
||
|
||
`verify:zip-size` enforces:
|
||
|
||
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
|
||
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
|
||
|
||
CI fails the PR if either is exceeded.
|
||
|
||
## Running tests
|
||
|
||
```bash
|
||
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
|
||
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
|
||
```
|
||
|
||
## Using the SDK
|
||
|
||
After deploying the stack (via the SAM template, CDK construct below, or
|
||
your own CFN of choice), drive renders from Node:
|
||
|
||
```ts
|
||
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
|
||
|
||
// One-time upload per project version.
|
||
const site = await deploySite({
|
||
projectDir: "./my-composition",
|
||
bucketName: "hyperframes-render-bucket",
|
||
});
|
||
|
||
// Start a render. Returns immediately — does NOT poll.
|
||
const handle = await renderToLambda({
|
||
siteHandle: site,
|
||
bucketName: site.bucketName,
|
||
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
|
||
config: {
|
||
fps: 30,
|
||
width: 1920,
|
||
height: 1080,
|
||
format: "mp4",
|
||
chunkSize: 240,
|
||
maxParallelChunks: 16,
|
||
runtimeCap: "lambda",
|
||
},
|
||
});
|
||
|
||
// Poll progress + cost on your own cadence.
|
||
const progress = await getRenderProgress({ executionArn: handle.executionArn });
|
||
console.log(progress.overallProgress, progress.costs.displayCost);
|
||
if (progress.status === "SUCCEEDED" && progress.outputFile) {
|
||
console.log("Render landed at", progress.outputFile.s3Uri);
|
||
}
|
||
```
|
||
|
||
`renderToLambda` validates the config client-side via
|
||
`validateDistributedRenderConfig` and throws a typed `InvalidConfigError`
|
||
before the Step Functions execution starts, so shape errors surface
|
||
synchronously instead of as opaque `ExecutionFailed` results.
|
||
|
||
`getRenderProgress` reports an approximate per-render cost
|
||
(`accruedSoFarUsd` plus a formatted `displayCost`) derived from Lambda
|
||
billed-duration × memory × the us-east-1 on-demand rate plus the Step
|
||
Functions transition price. The math is documented in
|
||
`src/sdk/costAccounting.ts`; numbers are best-effort and exclude S3
|
||
transfer.
|
||
|
||
## Using the CDK construct
|
||
|
||
```ts
|
||
import { App, Stack } from "aws-cdk-lib";
|
||
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
|
||
|
||
const app = new App();
|
||
const stack = new Stack(app, "MyApp");
|
||
const render = new HyperframesRenderStack(stack, "Render", {
|
||
// optional: reservedConcurrency: 8,
|
||
// optional: lambdaMemoryMb: 10240,
|
||
// optional: chromeSource: "sparticuz",
|
||
});
|
||
|
||
// Re-export so an adopter app can wire dashboards / SNS topics.
|
||
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
|
||
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
|
||
```
|
||
|
||
`aws-cdk-lib` and `constructs` are **optional peer dependencies**: SDK-only
|
||
consumers don't pull them at runtime. The construct itself imports from
|
||
`@hyperframes/aws-lambda/cdk`.
|
||
|
||
## What's still ahead
|
||
|
||
- `hyperframes lambda` CLI (deploy / sites create / render / progress / destroy) — PR 6.5.
|
||
- IAM bootstrap subcommand (`policies role | user | validate`) — PR 6.9.
|
||
- Lambda-local regression harness (`--mode=lambda-local`) — PR 6.6.
|
||
- Adopter-facing migration guide — PR 6.8.
|