Files
hyperframes/packages/aws-lambda
James RussoandClaude Opus 5.5 84799bdc59 fix: take helper programs only from install locations, never the project folder (#7524)
* fix(cli,parsers): take helper programs only from install locations, never the project folder

FFmpeg, FFprobe, whisper-cli, parakeet-mlx, Python and system Chrome are now
found through one shared rule: an absolute PATH entry outside the working
directory, or a fixed install dir. A relative PATH entry and any folder
inside the working directory are skipped, on every platform. Windows walks
PATH itself instead of asking `where`, which lists the working directory
first.

FFmpeg discovery no longer falls back to `.hyperframes/bin` or to an
installer package in the project's node_modules; an installer package
installed next to HyperFrames still works. The env overrides are unchanged.
The CLI skill reference stops suggesting the project-local folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(cli): give the PATH-lookup tests a fixed PATH on Windows

Windows now walks PATH instead of running `where`, so tests that faked
`where` output found the runner's real python and cmake. Both tests now set
PATH to a temp folder holding only the programs they expect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: report a missing helper program instead of running it by bare name

When no trusted FFmpeg or FFprobe is found, the engine, the producer's
probes and Studio's waveform, peak and audio-track helpers now report
"not found" with the HYPERFRAMES_FFMPEG_PATH / HYPERFRAMES_FFPROBE_PATH hint
instead of handing the bare name to the OS lookup. Callers that degraded
on a missing binary still do: runFfmpeg returns a spawn_error result, the
GPU and version probes report none, and Studio's audio helpers fail soft.

The Lambda handler names its bundled ffmpeg and ffprobe by path, since
they live under the task root, which is the working directory.

brew, git and cmake for the whisper.cpp install, and the clipboard tool
for `hyperframes add`, run by the absolute path the trusted lookup
approved. The media-use engine gets a PATH holding only trusted folders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(engine): fail a missing FFmpeg the way a failed spawn always has

Throwing from getFfmpegBinary turned every caller's missing-binary path
into a synchronous throw: the streaming encoder's "failure result, not a
throw" contract broke, and unit lanes that run without FFmpeg crashed at
module load. With no trusted copy the getters now return an absolute path
inside HyperFrames' own install that is never created, named for the
override to set. Spawning it fails with ENOENT exactly as before, and the
OS still never looks the bare name up. The call-site guards and fixture
changes that the throw needed are reverted.

The whisper install tests match git and cmake by name and skip the brew
case on Windows, where the lookup walks the real PATH.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(engine): pin the found FFmpeg before the HDR test looks it up as macOS on Windows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:46:52 +00:00
..
2026-11-11 06:32:08 +00:00

@hyperframes/aws-lambda

AWS Lambda adapter for HyperFrames distributed rendering. Ships three things together:

  1. The Lambda handler that wraps the OSS plan / renderChunk / assemble primitives behind a single dispatch boundary Step Functions can drive (src/handler.ts).
  2. A client-side SDK — renderToLambda, getRenderProgress, deploySite, plus validateDistributedRenderConfig and computeRenderCost (src/sdk/).
  3. An aws-cdk-lib L2 construct (HyperframesRenderStack) that provisions the same topology as examples/aws-lambda/template.yaml inside an adopter's own CDK app (src/cdk/).

The handler ZIP and the SAM template still drive a maintainer-run real-AWS smoke flow; the SDK + CDK are the supported public surface for adopters.

Architecture

┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine                                     │
│   Plan → Map(N) RenderChunk → Assemble                           │
└──────────────────────────────────────────────────────────────────┘
                              │ dispatches by event.Action
                              ▼
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`)          │
│   handler.mjs                                                    │
│     ├─ Action="plan"        → @hyperframes/producer/distributed  │
│     ├─ Action="renderChunk" → @hyperframes/producer/distributed  │
│     └─ Action="assemble"    → @hyperframes/producer/distributed  │
│   bin/ffmpeg                — ffmpeg-static                      │
│   node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium  │
└──────────────────────────────────────────────────────────────────┘
                              │ pure functions over local paths
                              ▼
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — Plan v2 manifest/blobs + chunks + output            │
└──────────────────────────────────────────────────────────────────┘

The handler downloads inputs from S3 into /tmp, calls the OSS primitive, uploads outputs back to S3, and returns a small JSON result that fits inside Step Functions' history budget (under 200 bytes per chunk).

Plan transport

Plan v2 is the only supported distributed plan transport. renderToLambda sends PlanProtocol: "v2" without requiring a protocol option:

await renderToLambda({
  // ...bucket, state machine, project, and config...
});

V2 never overloads PlanS3Uri. The planner returns PlanV2ManifestS3Uri and PlanV2ArtifactS3Prefix; chunk workers fetch only manifest-selected chunk artifacts, while the assembler fetches its own metadata and audio subset. Blobs are immutable SHA-256-addressed objects, verified on upload and download, and the manifest is published last. Unknown protocols and digest mismatches are terminal Step Functions errors.

Upgrade order

This default changes application behavior and requires a coordinated infrastructure upgrade. Before upgrading an application that calls renderToLambda:

  1. Pause new renders and let existing Step Functions executions drain.
  2. Redeploy the Lambda handler and SAM template or CDK construct from the same new package version.
  3. Resume renders, then upgrade the application/SDK dependency.

A site is a sites/<id>/files.json file list that an older stack cannot read. Before it uploads anything, renderToLambda reads the state machine definition (states:DescribeStateMachine) and fails with STACK_TOO_OLD when the stack predates that format. If infrastructure cannot be redeployed first, keep the application on its previous package version until the redeploy is complete. Site ids from earlier versions keep working after the redeploy: when sites/<id>/files.json is missing, the render uses that site's project.tar.gz.

Chrome runtime

The package supports two Chromium sources:

Source Default Size When to pick it
@sparticuz/chromium yes ~70 MiB compressed Lambda. Decompresses into /tmp at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda.
Bundled chrome-headless-shell no ~140 MiB Fallback. Used if @sparticuz/chromium ever drops HeadlessExperimental.beginFrame support.

Pick the source at build time:

bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell

The handler reads HYPERFRAMES_LAMBDA_CHROME_SOURCE at boot. The build script sets that env var via Lambda function configuration in examples/aws-lambda/template.yaml.

BeginFrame regression guard

HyperFrames' renderer drives Chrome via the CDP HeadlessExperimental.beginFrame command — same path the K8s deploy uses. The Lambda adapter assumes that @sparticuz/chromium's chrome-headless-shell build honours BeginFrame. To prove it (and re-prove it on every release), the package ships a Docker probe:

# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker

The probe boots @sparticuz/chromium inside public.ecr.aws/lambda/nodejs:22 and asserts CDP beginFrame with screenshot: true returns a PNG buffer. Exit code 0 = green; non-zero = fall back to bundling chrome-headless-shell directly via --source=chrome-headless-shell.

Building the ZIP

bun install                                          # at the monorepo root
bun run --cwd packages/aws-lambda build:zip          # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size    # CI gate

The build script bundles src/handler.ts via esbuild, stages @sparticuz/chromium and puppeteer-core under node_modules/, copies ffmpeg-static into bin/, and zips the result. The unzipped layout is designed to extract cleanly into Lambda's /var/task/.

verify:zip-size enforces:

  • Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
  • Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)

CI fails the PR if either is exceeded.

Running tests

bun run --cwd packages/aws-lambda test               # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe   # local probe (Linux only)

Using the SDK

After deploying the stack (via the SAM template, CDK construct below, or your own CFN of choice), drive renders from Node:

import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";

// One-time upload per project version.
const site = await deploySite({
  projectDir: "./my-composition",
  bucketName: "hyperframes-render-bucket",
});

// Start a render. Returns immediately — does NOT poll.
const handle = await renderToLambda({
  siteHandle: site,
  bucketName: site.bucketName,
  stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
  config: {
    fps: 30,
    width: 1920,
    height: 1080,
    format: "mp4",
    chunkSize: 240,
    maxParallelChunks: 16,
    runtimeCap: "lambda",
  },
});

// Poll progress + cost on your own cadence.
const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.overallProgress, progress.costs.displayCost);
if (progress.status === "SUCCEEDED" && progress.outputFile) {
  console.log("Render landed at", progress.outputFile.s3Uri);
}

renderToLambda validates the config client-side via validateDistributedRenderConfig and throws a typed InvalidConfigError before the Step Functions execution starts, so shape errors surface synchronously instead of as opaque ExecutionFailed results.

getRenderProgress reports an approximate per-render cost (accruedSoFarUsd plus a formatted displayCost) derived from Lambda billed-duration × memory × the us-east-1 on-demand rate plus the Step Functions transition price. The math is documented in src/sdk/costAccounting.ts; numbers are best-effort and exclude S3 transfer.

Using the CDK construct

import { App, Stack } from "aws-cdk-lib";
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";

const app = new App();
const stack = new Stack(app, "MyApp");
const render = new HyperframesRenderStack(stack, "Render", {
  // optional: reservedConcurrency: 8,
  // optional: lambdaMemoryMb: 10240,
  // optional: chromeSource: "sparticuz",
});

// Re-export so an adopter app can wire dashboards / SNS topics.
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });

aws-cdk-lib and constructs are optional peer dependencies: SDK-only consumers don't pull them at runtime. The construct itself imports from @hyperframes/aws-lambda/cdk.

What's still ahead

  • hyperframes lambda CLI (deploy / sites create / render / progress / destroy) — PR 6.5.
  • IAM bootstrap subcommand (policies role | user | validate) — PR 6.9.
  • Lambda-local regression harness (--mode=lambda-local) — PR 6.6.
  • Adopter-facing migration guide — PR 6.8.