Commit Graph
4 Commits
Author SHA1 Message Date
Miguel Ángel b501dc4e1b feat(cli): lambda deploy works from the npm install, no repo checkout (#5866)
* feat(cli): lambda deploy works from the npm install, no repo checkout

Each release builds the Lambda handler zip on Linux, attaches it to the
GitHub release, and pins its URL and sha256 (plus the SAM template) in
@hyperframes/aws-lambda before npm publish. Outside a checkout, lambda
deploy downloads that zip, refuses it on a sha256 mismatch, and deploys
it with the packaged template. With neither a checkout nor a pinned zip
it fails up front with one message naming every way out.

* fix(cli): lambda deploy test mocks the pinned-handler reader, comments fit the checks

* test(cli): sam tests deploy from a SAM directory, not the repo root

* chore(aws-lambda): raise the package size budget for the pinned handler template

Measured with pnpm@10.17.1 pack on Linux. Release state (prebuilt/ written by pin-handler-zip): 77,307 packed / 285,654 unpacked. CI state (no prebuilt/): 70,104 / 253,390. Packed is the check's 5% headroom over the release size. Unpacked is 292,000 (2.2% over release) because a 5% budget would put the CI pack under the 85% ratchet.

* test(cli): anchor the docs link check in the lambda deploy fallback test

* fix(cli): lambda deploy logs the verified handler sha256 and keeps a presigned query out of errors

* fix(cli): a redirect-limit download error keeps a signed query out of its message

* test(cli): the redirect-limit leak test types its rejection

* docs(aws-lambda): header comments cut to what the code cannot say

* chore(aws-lambda): size budget fits both the CI and the release package
2026-10-19 21:41:50 +00:00
Miguel Ángel 2a14cbc924 perf(aws-lambda): size distributed renders from account concurrency limits (#5813)
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap

The inline Map runs at most 40 chunk renders at once. Both state machine
definitions (CDK and SAM) now use a Distributed Map with STANDARD child
executions, capped at the function's reserved concurrency so chunks past it
queue instead of being throttled. Progress reads each chunk's child
execution history so frames, cost and chunk errors still show up.

* fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks

A throttled task used to burn its four catch-all retries in about 30 s.
It now gets its own jittered backoff of several minutes, because another
render on the same function can still hold the reserved slots. Plan
treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions.

* chore(aws-lambda): trim stack comments to keep the comment share flat

* fix(aws-lambda): keep render --wait working at hundreds of chunks

Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of
every child execution's history, take exact chunk frames and cost from the Map
output once it exits, and read failed children only when the counts show a
failure. The SDK client retries throttled calls up to 8 times with backoff.

* feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8

* fix(lambda): keep the concurrency SDK external in published modules

* fix(cli): keep an existing stack's reserved concurrency on redeploy

Redeploying without --concurrency sent ReservedConcurrency=-1, which
removed the cap of 8 that earlier CLI versions reserved by default and
stopped recording it in the state file. The override is now left out
when the flag is omitted, so SAM keeps the stack's current value (a new
stack still gets the template's unreserved default). The state file
records the deployed value, and deploy prints the kept reservation.

* fix(aws-lambda): keep progress working for callers without DescribeMapRun

Callers whose IAM policy came from an earlier `lambda policies user`
lack states:DescribeMapRun, so reading a running Distributed Map threw
AccessDeniedException out of `lambda progress` and `render --wait`
while the render itself went on to succeed. getRenderProgress now
catches only that error around the map-run read, returns the parent
snapshot (status, plan and assemble progress) and adds a warning to
re-apply the policy, which both commands print. Other errors still
throw.

* fix(cli): let the documented deploy policy drop a reserved concurrency

When a redeploy removes a function's reserved concurrency, CloudFormation
calls lambda:DeleteFunctionConcurrency. The policy printed by
`lambda policies role|user` granted only PutFunctionConcurrency, so the
update failed and rolled back. Grant it, and pin it with
states:DescribeMapRun in the required-actions test.

* fix(aws-lambda): read smoke chunk hashes from the Distributed Map output

With chunks in a Distributed Map, their task results live in child
executions, so the smoke gate's chunk-hash check found 0 chunks in the
parent history and failed every v1/v2 comparison. Read Chunks[] from the
RenderChunks / RenderChunksV2 MapStateExited output instead, as
getRenderProgress does. The stack this script deploys only has the
Distributed Map, so the inline-Map path is gone.

* fix(producer): cap a pinned chunk size at the worker concurrency, not 16

With --chunk-size or --target-chunk-frames and no --max-parallel-chunks,
the plan still capped the render at 16 chunks however many workers the
account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of
113 frames. The cap is now one wave across the worker concurrency, as
the --max-parallel-chunks docs say; 16 remains when the concurrency is
unknown.

* feat(cli): remove a stack's reserved concurrency with --concurrency=none

Since an omitted --concurrency now keeps the stack's reservation, there
was no CLI way back to the unreserved default. --concurrency=none sends
ReservedConcurrency=-1, the state file records the stack as unreserved
and deploy says the reservation was removed. Positive N and an omitted
flag behave as before; any other value is rejected instead of being
silently ignored.

* chore(aws-lambda): raise the package size budget for account-sized concurrency

Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that.
2026-10-19 00:57:30 +00:00
Miguel Ángel c69fd2c6ec perf(producer): size distributed chunks to the worker limit and concurrency (#5765)
* perf(producer): size distributed chunks to the worker limit and concurrency

* chore(cli): drop a comment the field name already says
2026-10-17 08:45:47 +00:00
James Russo e90ad2da61 feat(cli): add hyperframes lambda deploy/render/progress/destroy (#910)
* feat(cli): add hyperframes lambda deploy/render/progress/destroy

Wraps the @hyperframes/aws-lambda SDK + the Phase 6a SAM template behind
a single CLI surface so an end-to-end render is three commands instead
of the ~8 manual bun+sam+aws steps the smoke script does today:

  hyperframes lambda deploy
  hyperframes lambda render ./my-project --width 1920 --height 1080 --wait
  hyperframes lambda destroy

Subcommands:
  - deploy:        build handler.zip + sam-deploy + persist stack outputs
                   to <cwd>/.hyperframes/lambda-stack-<name>.json
  - sites create:  pre-upload a project to S3 with a stable content hash
                   so re-renders skip the tar+PUT pass
  - render:        start a Step Functions execution; --wait blocks and
                   streams per-chunk progress + accrued cost
  - progress:      one-shot snapshot — status, frames, cost breakdown,
                   errors. Accepts renderId or executionArn
  - destroy:       sam-delete + drop the local state file (S3 bucket
                   is Retain'd by the template; documented in --help
                   and in docs/packages/cli.mdx)

To keep @sparticuz/chromium out of the CLI's transitive deps, this also
adds a dedicated ./sdk subpath export to @hyperframes/aws-lambda; the
CLI imports from @hyperframes/aws-lambda/sdk exclusively. The existing
. barrel still re-exports both handler + SDK for adopters who want one
entry point.

Defaults are deliberately cost-conservative for first-time users:
--concurrency=8 (low enough to never surprise) and --memory=10240 (the
common case; documented for adopters who want to tune down).

Tests: 5 unit tests on the state-file round-trip. CLI integration
against sam local invoke is part of the upcoming PR 6.6 (lambda-local
regression harness).

* refactor(cli): /simplify pass on the lambda command group

Two small cleanups on top of the lambda CLI:

  - Replace parseFormat / parseCodec / parseQuality / parseChromeSource
    (four near-identical helpers) with a single generic parseEnum() +
    typed const-tuple lookups. The four callers now read as one-line
    arrow functions that lift the allowed values out of the function
    body so they're easy to extend.

  - DEFAULT_STACK_NAME was const-declared then re-exported at the
    bottom of state.ts; just mark the const export inline.

No behavior changes. All CLI tests still pass.

* fix(cli): keep @hyperframes/aws-lambda external in the tsup bundle

esbuild can't bundle @hyperframes/aws-lambda's transitive AWS SDK
deps (@aws-sdk/* + @smithy/*) cleanly into a node binary — the
SDK's .browser.js conditional re-exports break the resolver:

  ESM Build failed
    No matching export in "splitStream.browser.js" for import
    "splitStream" (and ~10 similar errors)

Mark aws-lambda as `external` so esbuild doesn't follow it, and
move it from devDependencies to dependencies so the published CLI
can resolve it from node_modules at runtime. The lambda subverb
files dynamic-import only on `hyperframes lambda *` invocation, so
the CLI cold-start cost is unchanged.

The install-size hit (AWS SDK + @sparticuz/chromium ≈ 200 MiB) is
documented as a v1 tradeoff; a future split into a lambda-sdk-only
subpackage can pare this back.

* fix(cli): address PR review on lambda CLI

Two blockers + four important items from Vai's review:

  - `--memory` was parsed and recorded in the local state file but
    never forwarded to `sam deploy` as a parameter override. Worse,
    `progress.ts` then read the *recorded* value for cost math, so
    `--memory 5120` produced wrong cost numbers downstream. Thread
    `LambdaMemoryMb` through samDeploy's --parameter-overrides.

  - `--profile` was only consumed by deploy / destroy. render and
    progress fell back to the default credentials chain — a user
    with `--profile prod` would silently render against their
    default account (wrong-account billing footgun). Set
    `process.env.AWS_PROFILE` (and `AWS_REGION`) in the dispatcher
    before any subverb runs; the AWS SDK reads them natively, so
    render / progress / sites all benefit without each subverb
    threading the flag through the SDK call.

  - `--profile` + destroy now also reads `process.env.AWS_PROFILE`
    as a fallback (matching deploy's existing env fallback).

  - `--wait --json` printed both the start handle AND the final
    progress snapshot, producing two concatenated JSON blobs that
    `jq` rejected. Now emits a single document: handle (without
    --wait) OR final progress (with --wait).

  - Negative integers on `--width` / `--height` / `--chunk-size` /
    `--max-parallel-chunks` / `--memory` / `--concurrency` now fail
    loudly via a new `parsePositiveInt` wrapper instead of flowing
    into the SDK and producing opaque AWS validation errors mid-
    render.

  - `DEFAULT_STACK_NAME` is now centralized to the literal
    `"hyperframes-default"` and consumed from one place. Previously
    the value was assembled as `hyperframes-${"default"}` in three
    sites and hardcoded as `"hyperframes-default"` in a fourth.
    `requireStack`'s hint now matches the dispatcher's default.

The faked `SiteHandle` for `--site-id` keeps the documented
placeholder fields but also surfaces `bucketName` (from PR 909's
extended SiteHandle interface), matching the SDK contract.

All CLI unit tests + the full bundler build still pass.

* fix(cli): keep aws-lambda out of CLI runtime deps

The "Smoke: global install" CI step packs the CLI via `npm pack` and
installs it globally via `npm install -g <tgz>`. npm doesn't understand
the workspace: protocol, so a runtime `dependencies` entry of
`@hyperframes/aws-lambda: workspace:*` blows up with:

  npm error code EUNSUPPORTEDPROTOCOL
  npm error Unsupported URL Type "workspace:": workspace:*

(pnpm rewrites workspace:* on publish; npm pack doesn't.)

Three changes to unblock the smoke + keep the published CLI install
small for users who don't deploy to Lambda:

  - Move `@hyperframes/aws-lambda` from CLI's `dependencies` back to
    `devDependencies`. It's already external in tsup.config.ts; the
    bundle references it via runtime resolution only.

  - Convert the static `import { … } from "@hyperframes/aws-lambda/sdk"`
    in sites.ts / render.ts / progress.ts to `await import()` inside
    each function. tsup with `splitting: false` was inlining those
    static imports at the top of the bundle, which made Node eagerly
    resolve them at CLI startup (MODULE_NOT_FOUND before any lambda
    subcommand even runs). Dynamic imports stay dynamic in the bundle.

  - Add a friendly missing-module check in the lambda dispatcher.
    When a user runs `hyperframes lambda deploy / render / sites /
    progress / destroy` without aws-lambda installed, they now see:

      @hyperframes/aws-lambda is not installed.
      The `hyperframes lambda deploy` command needs it at runtime.
      Install it alongside the CLI:
        npm install -g @hyperframes/aws-lambda

Verified locally: pack + global install + `hyperframes init --example
blank` now succeeds end-to-end (was the same scenario the CI smoke job
runs).
2026-05-17 13:06:00 -04:00