* feat(aws-lambda): attach the arm64 video image to releases and pin it
The release builds the arm64 video function image from the handler ZIP it
attaches, attaches it as an OCI layout tarball with its sha256, and pins it
in prebuilt/video-image.json, read through prebuiltVideoImage(). A retried
release reuses the image already attached.
* fix(aws-lambda): keep the release pin script out of the published package
pin-handler-zip.ts is run from a checkout by publish.yml; nothing in dist or the CLI imports it. Leaving it out keeps aws-lambda under its size budget with the video image reader added.
* feat(aws-lambda): arm64 video function image from the handler ZIP
The image is the release handler ZIP with its x86 ffmpeg and Chrome swapped for
the native ffmpeg tree and the sha256-pinned Sparticuz arm64 Chrome pack, so the
x86 and arm64 functions run the same handler code. The image stage only copies,
so any host builds it. CI builds it and runs ffmpeg, a real Chrome render and the
Lambda runtime on an ARM runner.
* fix(aws-lambda): launch Chrome in the image smoke the way Lambda does
@sparticuz/chromium inflates its AL2023 libraries only when AWS_EXECUTION_ENV
names a Node runtime; Lambda sets it, a bare docker run does not. The workflow
also runs on @sparticuz/chromium bumps and handler ZIP layout changes, which
re-stage the image.
* fix(aws-lambda): smoke the arm64 image through its own Lambda runtime
The smoke is now a handler invoked through the image's entrypoint, whose
bootstrap sets AWS_EXECUTION_ENV the way Lambda does, so @sparticuz/chromium
takes its Lambda branch with nothing set by hand. It returns the value it saw.
* fix(aws-lambda): agree the move of the ffmpeg build workflow into the image workflow
* fix(aws-lambda): build the Chrome pack URL from the pin and hand the checked pack to tar in memory
* feat(aws-lambda): run chunks on an optional arm64 image function
A new VideoImageUri parameter adds an arm64 container-image function that only
renders chunks. Plan, audio and assemble stay on the x86 ZIP function, so audio
encodes are unchanged. Empty (the default) deploys exactly as before. The
reserved concurrency moves to whichever function renders the chunks.
* docs(aws-lambda): the template tags both render functions
* fix(aws-lambda): nest the arm64 video function so sam deploy needs no image repository
sam deploy refuses a template with any top-level image function unless it is given an image
repository, even when that function's Condition is false, so every deploy of the template failed.
The video function now lives in video-function.yaml, a nested stack created only when
VideoImageUri is set. The release ships it beside template.yaml and the CLI stages both.
A new workflow runs the CLI's sam deploy argv against the template with fake credentials and
requires it to reach the first upload.
* fix(aws-lambda): run the plan on the arm64 video function with the chunks it plans
* refactor(cli): name the staged template directory instead of commenting it
* docs(aws-lambda): say the arm64 image runs the plan as well as the chunks
* feat(cli): lambda deploy works from the npm install, no repo checkout
Each release builds the Lambda handler zip on Linux, attaches it to the
GitHub release, and pins its URL and sha256 (plus the SAM template) in
@hyperframes/aws-lambda before npm publish. Outside a checkout, lambda
deploy downloads that zip, refuses it on a sha256 mismatch, and deploys
it with the packaged template. With neither a checkout nor a pinned zip
it fails up front with one message naming every way out.
* fix(cli): lambda deploy test mocks the pinned-handler reader, comments fit the checks
* test(cli): sam tests deploy from a SAM directory, not the repo root
* chore(aws-lambda): raise the package size budget for the pinned handler template
Measured with pnpm@10.17.1 pack on Linux. Release state (prebuilt/ written by pin-handler-zip): 77,307 packed / 285,654 unpacked. CI state (no prebuilt/): 70,104 / 253,390. Packed is the check's 5% headroom over the release size. Unpacked is 292,000 (2.2% over release) because a 5% budget would put the CI pack under the 85% ratchet.
* test(cli): anchor the docs link check in the lambda deploy fallback test
* fix(cli): lambda deploy logs the verified handler sha256 and keeps a presigned query out of errors
* fix(cli): a redirect-limit download error keeps a signed query out of its message
* test(cli): the redirect-limit leak test types its rejection
* docs(aws-lambda): header comments cut to what the code cannot say
* chore(aws-lambda): size budget fits both the CI and the release package
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap
The inline Map runs at most 40 chunk renders at once. Both state machine
definitions (CDK and SAM) now use a Distributed Map with STANDARD child
executions, capped at the function's reserved concurrency so chunks past it
queue instead of being throttled. Progress reads each chunk's child
execution history so frames, cost and chunk errors still show up.
* fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks
A throttled task used to burn its four catch-all retries in about 30 s.
It now gets its own jittered backoff of several minutes, because another
render on the same function can still hold the reserved slots. Plan
treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions.
* chore(aws-lambda): trim stack comments to keep the comment share flat
* fix(aws-lambda): keep render --wait working at hundreds of chunks
Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of
every child execution's history, take exact chunk frames and cost from the Map
output once it exits, and read failed children only when the counts show a
failure. The SDK client retries throttled calls up to 8 times with backoff.
* feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8
* fix(lambda): keep the concurrency SDK external in published modules
* fix(cli): keep an existing stack's reserved concurrency on redeploy
Redeploying without --concurrency sent ReservedConcurrency=-1, which
removed the cap of 8 that earlier CLI versions reserved by default and
stopped recording it in the state file. The override is now left out
when the flag is omitted, so SAM keeps the stack's current value (a new
stack still gets the template's unreserved default). The state file
records the deployed value, and deploy prints the kept reservation.
* fix(aws-lambda): keep progress working for callers without DescribeMapRun
Callers whose IAM policy came from an earlier `lambda policies user`
lack states:DescribeMapRun, so reading a running Distributed Map threw
AccessDeniedException out of `lambda progress` and `render --wait`
while the render itself went on to succeed. getRenderProgress now
catches only that error around the map-run read, returns the parent
snapshot (status, plan and assemble progress) and adds a warning to
re-apply the policy, which both commands print. Other errors still
throw.
* fix(cli): let the documented deploy policy drop a reserved concurrency
When a redeploy removes a function's reserved concurrency, CloudFormation
calls lambda:DeleteFunctionConcurrency. The policy printed by
`lambda policies role|user` granted only PutFunctionConcurrency, so the
update failed and rolled back. Grant it, and pin it with
states:DescribeMapRun in the required-actions test.
* fix(aws-lambda): read smoke chunk hashes from the Distributed Map output
With chunks in a Distributed Map, their task results live in child
executions, so the smoke gate's chunk-hash check found 0 chunks in the
parent history and failed every v1/v2 comparison. Read Chunks[] from the
RenderChunks / RenderChunksV2 MapStateExited output instead, as
getRenderProgress does. The stack this script deploys only has the
Distributed Map, so the inline-Map path is gone.
* fix(producer): cap a pinned chunk size at the worker concurrency, not 16
With --chunk-size or --target-chunk-frames and no --max-parallel-chunks,
the plan still capped the render at 16 chunks however many workers the
account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of
113 frames. The cap is now one wave across the worker concurrency, as
the --max-parallel-chunks docs say; 16 remains when the concurrency is
unknown.
* feat(cli): remove a stack's reserved concurrency with --concurrency=none
Since an omitted --concurrency now keeps the stack's reservation, there
was no CLI way back to the unreserved default. --concurrency=none sends
ReservedConcurrency=-1, the state file records the stack as unreserved
and deploy says the reservation was removed. Positive N and an omitted
flag behave as before; any other value is rejected instead of being
silently ignored.
* chore(aws-lambda): raise the package size budget for account-sized concurrency
Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that.
* feat(core): the renderer's font pipeline is importable from @hyperframes/core/fonts/embed
* style(core): format the font pipeline and state the bundled-subset invariant
* test(producer): repin the test lane map after the font tests moved to core
* fix(core): images build fonts once, CLI bundles one pipeline, font tests pass on Windows
* test(core): font retry tests get a budget no slow machine can exhaust
* test(core): the Retry-After budget test fails when the budget is ignored
* fix(capture): never silently render a webgpu composition blank
A composition declaring data-requires-webgpu only got the Chrome WebGPU
flag when hardware GPU auto-detection succeeded. On a software-GPU host
it launched without the flag and rendered blank with no error.
compositionRequiresWebGpu (static declaration check) and
assertWebGpuAdapterAvailable (live per-page adapter probe) now live once
in packages/engine's browserManager, the shared choke point every launch
site already goes through. All six launch sites (frame capture, the four
CLI diagnostic commands, the studio thumbnail server, and the render
pipeline's capture-options builder) route through those two functions;
the old CLI-only static-heuristic guard is deleted, not kept alongside
the new one. A software host that cannot obtain any WebGPU adapter now
fails loudly naming the requirement instead of producing a blank frame.
* fix(capture): thread requiresWebGpu through the distributed chunk-render path
The distributed renderChunk() path built its CaptureOptions without
requiresWebGpu, so it fell through to frameCapture's own fetch-based
detection, whose failure path silently treats "requires webgpu" as
"doesn't" if that fetch ever fails. Every other launch site (frame
capture, all four CLI diagnostic commands, the studio thumbnail server,
and the in-process render pipeline) already reads the value from HTML
already in memory or on disk instead of relying on that fallback.
Reads the compiled composition's index.html the same way probeStage
already does and derives requiresWebGpu from it before building
CaptureOptions. Adds a regression test that pins the exact path
renderChunk() reads at render time, since the composition doesn't
actually exercise WebGPU so an end-to-end render can't distinguish the
two cases on this host.
* test(producer): observe the real CaptureOptions renderChunk() builds
The previous regression test called plan() and compositionRequiresWebGpu()
directly, bypassing renderChunk() entirely, so it could not detect a
regression to the wiring it was meant to protect (confirmed: it stayed
green with the fix reverted).
Exports the existing createCaptureSession dependency seam so a test can
substitute a spy that delegates to the real implementation while
capturing the CaptureOptions object renderChunk() actually constructs.
Verified non-vacuous by reverting the fix and confirming the test fails.
* fix(gcp-cloud-run): resolve puppeteer-core from a real file in the webgpu probe
The image build's WebGPU adapter probe imported puppeteer-core from an
inline `bun -e` eval string. The eval's synthetic /app/[eval] location
can't resolve the workspace-hoisted dependency, so every build of this
stage failed closed with "Cannot find package 'puppeteer-core'" before
the probe itself ever ran.
Moves the probe into a real script file next to probe-beginframe.ts,
which already imports puppeteer-core dynamically the same way and
resolves fine from its real workspace location. Verified locally against
a cached chrome-headless-shell binary.
* fix(engine): bound the webgpu adapter probe so a stuck driver can't hang the caller
requestAdapter() has no native timeout. A broken software GPU driver
that never settles it would hang assertWebGpuAdapterAvailable
indefinitely, including inside the GCP image build's probe RUN step,
producing an opaque build timeout instead of a clean probe failure.
Races the in-page request against a bounded timeout so a stuck adapter
reads as unavailable and fails the same way a genuinely absent one does.
* test(cli): mock the engine's webgpu exports in the browser-fallback test
* test(producer): run the typegpu adapter fixture only on hosts with a hardware WebGPU adapter
The fixture declares data-requires-webgpu, so a software-only host now refuses it
instead of silently rendering the shader blank. The old golden had no shader in it.
Tag it webgpu, skip that tag in the CI image and the test scripts, and re-record the
golden from a Metal render with the shader visible.
* fix(engine): bound the composition fetch that decides the webgpu flag
createCaptureSession fetched the served page with no timeout before launching the
browser, so a hung server stalled a render forever and hung a fake-timer test. Give it a
five second limit, and let that test say it has no WebGPU requirement the way the
producer does.
* fix(capture): refuse a software fallback WebGPU adapter instead of rendering on it
swiftshader reports a WebGPU adapter but cannot render canvas WebGPU, so a
data-requires-webgpu composition on a software-only host crashed on its first frame.
Refuse fallback adapters with the same clear WebGPU-unavailable error. The GCP image
probe now serves its page over http (file:// hides navigator.gpu) and reports a
software-only host instead of failing the image build.
* fix(engine): parse the composition root tag without a backtracking regex
* fix(engine): keep quoted attribute values whole when reading the composition root tag
Miguel's fix (f999b40d) added the require/__filename/__dirname shims to the
Lambda handler bundle after #1932 crashed every render at import with
"__dirname is not defined in ES module scope" (wawoff2's emscripten build
reads __dirname at module scope; it's inlined via producer -> fontCompression).
The accompanying test only grepped build-zip.ts for the banner literals, so it
passes even if the shim is renamed, reordered into a broken form, or if a new
inlined CJS dep needs a global the banner doesn't provide.
Replace it with a behavioral test: extract the banner to _handlerBanner.ts
(build-zip.ts self-executes on import, so it can't be imported directly),
bundle a fixture that touches __dirname/__filename/require with the real
banner, and import the output under real Node -- not the bun test runtime,
which defines __dirname in ESM and would mask a missing shim. The import
faithfully reproduces Lambda's Node ESM environment and fails with the exact
#1932 error when any shim is dropped.
Handler bundle output is unchanged (identical banner string).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`ffmpeg-static` materializes a single platform-selected binary at install
time. By default that follows the install host, so a default macOS/arm64
or linux/arm64 install can stage a binary that Lambda's x86-64 runtime
cannot execute.
Verify the ELF header (ELFCLASS64, EM_X86_64) before copying the binary
into `bin/ffmpeg`. When the check fails, surface the canonical workarounds:
run the build inside a linux/amd64 container, or pre-install with
`npm_config_platform=linux npm_config_arch=x64`.
In the distributed pipeline this mismatch can fail as early as Plan's
`ffmpeg -version` probe after browser probing and before chunks are
scheduled. Later encode/assemble spawns would fail for the same reason.
ffprobe already goes through the platform-segmented
`ffprobe-static/bin/linux/x64` path, so that side doesn't need the same
check.
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).
This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.
Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.
The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.
Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.
This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.
* fix(lambda): address PR 878 review feedback
- Verify event.PlanHash against the untarred plan.json at the handler
boundary before invoking the producer primitive. Throws typed
PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
non-retryable; previously the field was schema bloat the handler
ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
but use binary mebibytes); previously mixed units made the 248 MiB
budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
comparison instead of lexicographic sort+reverse — the latter would
silently pick "99.x" over "131.x" once Chrome cached three-digit
majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
imports — file is ESM and the same module is already imported.
* docs(lambda): drop internal plan-doc refs from package README
* ci(windows): fix bun filter UNION bug excluding producer from Windows tests
`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.
Enumerate the packages we DO want to test instead.
Miguel Ángel