feat(lambda): add TypeScript SDK and CDK construct (#909)

* feat(lambda): add TypeScript SDK and CDK construct

Adds the client-side surface on top of the Phase 6a Lambda handler so
adopters can drive a deployed stack from Node without writing AWS-SDK
boilerplate:

- renderToLambda(opts) starts a Step Functions execution and returns a
  handle. Does NOT poll.
- getRenderProgress({ executionArn }) returns a snapshot of progress,
  frames rendered, cost (Lambda GB-seconds + SFN transitions), errors,
  and the final output object once Assemble completes.
- deploySite({ projectDir, bucketName }) content-addresses the project
  tree, tar.gzs it, and uploads to S3 with a HeadObject short-circuit so
  re-renders of the same tree skip the tar+PUT.
- validateDistributedRenderConfig throws a typed InvalidConfigError
  before StartExecution, so shape errors surface synchronously.
- computeRenderCost is exposed for callers who want to format cost out
  of band.

Also ships HyperframesRenderStack, an aws-cdk-lib L2 construct that
emits the same topology as examples/aws-lambda/template.yaml. Lives on
the ./cdk subpath export so SDK-only consumers don't pull aws-cdk-lib
into their runtime graph (declared as an optional peer dependency).

Tests: 24 new unit tests across the SDK plus 9 CDK synth / contract /
snapshot tests. All 83 tests in packages/aws-lambda/src pass.

* refactor(lambda): /simplify pass on the SDK + CDK PR

Pulls shared logic out so the SDK doesn't re-invent things the handler
and the producer already have:

- `formatExtension` extracted to packages/aws-lambda/src/formatExtension.ts.
  handler.ts and renderToLambda.ts both used identical 12-line copies of
  this switch.
- `PLAN_PROJECT_DIR_SKIP_SEGMENTS` is now exported from
  @hyperframes/producer/distributed. deploySite consumes it instead of
  its own duplicate SKIP_TOP_LEVEL set; the two lists were trivially
  identical and would have drifted silently.
- `FakeS3` + `drainBody` factored out of the two SDK test files into
  src/sdk/__fixtures__/fakeS3.ts. Drops ~110 lines of test-file
  duplication and gives future SDK tests a one-line FakeS3 import.
- S3 URI building in deploySite and renderToLambda routes through the
  existing `formatS3Uri` helper instead of inline `s3://...`
  concatenation; matches the convention already in handler.ts.

Net -133 lines across the touched files. All 83 aws-lambda tests still
pass; all 60 producer distributed tests still pass.

* fix(lambda): bump CDK test timeouts for CI cold-start synth

The bun:test default 5s timeout tripped the first CDK snapshot test
in CI when the cold-start `Template.fromStack(stack)` synth took ~5-8s
on the slowest GitHub Actions runner. Locally on a warm shell the
synth measures <1s, so the failure didn't reproduce until PR #909 hit
CI.

Two changes:

  - Both CDK test files cache one synth in `beforeAll(..., 30000)` and
    reuse the result across every test that uses the default props.
    Each individual test now runs in microseconds (pure assertions
    against the already-synthed template), so the 5s timeout no longer
    applies on the hot path.

  - The two contract tests that exercise non-default props
    (reservedConcurrency, projectName) still synth fresh per-test; they
    get a per-test `it(..., 30000)` timeout.

No behavior changes.

* fix(lambda): address PR review on SDK + CDK construct

Three correctness + ergonomics fixes raised in Vai's review:

  - getRenderProgress over-counted SFN transitions by 3-5×. Step
    Functions Standard Workflows bill per state-entry, not per
    history event. Each Task produces ~5-7 history events
    (Scheduled / Started / Succeeded / TaskStateExited / …);
    counting `events.length` reported the runaway. Switch to
    counting `*StateEntered` events explicitly.

  - assembleComplete + outputFile detection was coupled to the
    Lambda payload's `Action` field. Move both signals onto the
    enclosing state name (`StateExited.name === "Assemble"`), which
    is the state-machine identity rather than the Lambda event
    contract. framesRendered increment moves to the same boundary
    (RenderChunk state).

  - SiteHandle now carries `bucketName` directly so README + CLI
    callers don't have to re-parse `projectS3Uri.split("/")[2]`.

Test updates: getRenderProgress tests wrap renderChunk/assemble
events in matching StateEntered + StateExited pairs so the new
state-name-driven dispatch is exercised end-to-end. SiteHandle
fixture in renderToLambda.test.ts gets the new bucketName field.

All 83 aws-lambda tests still pass.
This commit is contained in:
James Russo
2026-05-17 03:03:51 -04:00
committed by GitHub
parent 0d726ae010
commit 34d1f0e1d0
23 changed files with 2662 additions and 47 deletions
@@ -0,0 +1,339 @@
/**
* `getRenderProgress` — read-only progress + cost snapshot for a single
* render started by {@link renderToLambda}.
*
* Pulls one `DescribeExecution` + one `GetExecutionHistory` per call. The
* history is paginated server-side; the helper loops until exhausted so a
* 1,000-event Step Functions execution still produces a single
* `RenderProgress` snapshot.
*
* Progress math:
* - 0 before Plan completes (no frame count is known yet)
* - 0.1 once Plan completes (we know `totalFrames`)
* - 0.1 + 0.8 × framesEncoded / totalFrames during chunk render
* - 1.0 after Assemble completes
*
* Frame counts come from the parsed Lambda result payloads on each
* `TaskSucceeded` event — Plan reports `TotalFrames`, RenderChunk reports
* `FramesEncoded`. The shape mirrors what the handler produces in
* `events.ts`, so the parser doesn't need to know anything beyond
* "JSON.parse this string and grab two fields."
*/
import {
DescribeExecutionCommand,
GetExecutionHistoryCommand,
type HistoryEvent,
SFNClient,
} from "@aws-sdk/client-sfn";
import {
type BilledLambdaInvocation,
computeRenderCost,
type RenderCost,
} from "./costAccounting.js";
/** Options for {@link getRenderProgress}. */
export interface GetRenderProgressOptions {
/** Execution ARN from a {@link renderToLambda} call. */
executionArn: string;
/**
* Default memory size in MB to assume for Lambda invocations when the
* history event payload doesn't carry it explicitly. Matches the
* `LambdaMemoryMb` parameter the stack was deployed with.
*/
defaultMemorySizeMb?: number;
region?: string;
/** Test injection seam. */
sfn?: SFNClient;
}
/** Render-status discriminant; mirrors Step Functions execution states. */
export type RenderStatus =
| "RUNNING"
| "SUCCEEDED"
| "FAILED"
| "TIMED_OUT"
| "ABORTED"
| "PENDING_REDRIVE";
export interface RenderError {
/** State name where the failure surfaced (`Plan`, `RenderChunk`, `Assemble`, or `<unknown>`). */
state: string;
/** Error class / type as Step Functions reports it. */
error: string;
/** Cause string Step Functions surfaces (often a stringified JSON payload from the handler). */
cause: string;
}
/** Snapshot of a single render's progress + cost + errors at one point in time. */
export interface RenderProgress {
status: RenderStatus;
/** `[0, 1]`; see module doc for the math. */
overallProgress: number;
framesRendered: number;
/** `null` until Plan completes. */
totalFrames: number | null;
/** Count of `LambdaFunctionScheduled` events seen in the history so far. */
lambdasInvoked: number;
costs: RenderCost;
/** Final output object if Assemble succeeded; `null` otherwise. */
outputFile: { s3Uri: string; bytes: number | null } | null;
errors: RenderError[];
/** `true` once the execution has terminated in a non-`SUCCEEDED` state. */
fatalErrorEncountered: boolean;
startedAt: string;
endedAt: string | null;
}
const DEFAULT_MEMORY_MB = 10240;
/** Pull a current progress snapshot for one render. */
export async function getRenderProgress(opts: GetRenderProgressOptions): Promise<RenderProgress> {
if (!opts.executionArn) {
throw new Error("[getRenderProgress] executionArn is required");
}
const sfn = opts.sfn ?? new SFNClient({ region: opts.region });
const memoryMb = opts.defaultMemorySizeMb ?? DEFAULT_MEMORY_MB;
const describe = await sfn.send(
new DescribeExecutionCommand({ executionArn: opts.executionArn }),
);
const status = (describe.status ?? "RUNNING") as RenderStatus;
const startedAt = describe.startDate?.toISOString() ?? new Date(0).toISOString();
const endedAt = describe.stopDate?.toISOString() ?? null;
const history = await loadFullHistory(sfn, opts.executionArn);
const summary = summarizeHistory(history, memoryMb);
const costs = computeRenderCost(summary.lambdaInvocations, summary.stateTransitions);
const overallProgress = computeOverallProgress({
status,
totalFrames: summary.totalFrames,
framesRendered: summary.framesRendered,
assembleComplete: summary.assembleComplete,
});
return {
status,
overallProgress,
framesRendered: summary.framesRendered,
totalFrames: summary.totalFrames,
lambdasInvoked: summary.lambdasInvoked,
costs,
outputFile: summary.outputFile,
errors: summary.errors,
fatalErrorEncountered: isTerminalFailure(status),
startedAt,
endedAt,
};
}
async function loadFullHistory(sfn: SFNClient, executionArn: string): Promise<HistoryEvent[]> {
const events: HistoryEvent[] = [];
let nextToken: string | undefined;
for (let page = 0; page < 50; page++) {
const res = await sfn.send(
new GetExecutionHistoryCommand({
executionArn,
maxResults: 1000,
nextToken,
reverseOrder: false,
}),
);
if (res.events) events.push(...res.events);
nextToken = res.nextToken;
if (!nextToken) break;
}
return events;
}
interface HistorySummary {
lambdaInvocations: BilledLambdaInvocation[];
stateTransitions: number;
framesRendered: number;
totalFrames: number | null;
lambdasInvoked: number;
assembleComplete: boolean;
outputFile: { s3Uri: string; bytes: number | null } | null;
errors: RenderError[];
}
/**
* One pass over the history events that pulls every number {@link getRenderProgress}
* needs. State transitions = the count of events that advance the state
* machine (entering/exiting states + map iteration completions). Lambda
* invocations = `LambdaFunctionScheduled` count. Frame totals come from
* the success-payload of each Lambda invocation.
*/
function summarizeHistory(events: HistoryEvent[], memoryMb: number): HistorySummary {
let framesRendered = 0;
let totalFrames: number | null = null;
let lambdasInvoked = 0;
let assembleComplete = false;
let outputFile: HistorySummary["outputFile"] = null;
let stateTransitions = 0;
const errors: RenderError[] = [];
const lambdaInvocations: BilledLambdaInvocation[] = [];
// Track the state name we most recently entered, so we can:
// - attach the enclosing state to LambdaFunctionFailed errors, and
// - identify when the Assemble state finished (StateExited.Assemble)
// without relying on the inner Lambda payload's `Action` field.
let currentLambdaState: string | null = null;
for (const ev of events) {
switch (ev.type) {
case "TaskStateEntered":
case "MapStateEntered":
case "PassStateEntered":
case "ChoiceStateEntered":
case "SucceedStateEntered":
case "FailStateEntered":
case "WaitStateEntered":
case "ParallelStateEntered":
// Step Functions Standard Workflows bill per *state entry*, not per
// history event. Lambda invocations produce ~5-7 history events
// each (Scheduled / Started / Succeeded / TaskStateExited / …);
// counting every event as a transition over-reports cost by 3-5×.
stateTransitions++;
currentLambdaState = ev.stateEnteredEventDetails?.name ?? currentLambdaState;
break;
case "LambdaFunctionScheduled":
lambdasInvoked++;
break;
case "LambdaFunctionSucceeded": {
const payload = parseJson(ev.lambdaFunctionSucceededEventDetails?.output);
const billedDurationMs = inferBilledMs(payload);
lambdaInvocations.push({
billedDurationMs,
memorySizeMb: memoryMb,
estimated: billedDurationMs === 0,
});
if (payload && typeof payload === "object") {
const obj = payload as Record<string, unknown>;
if (typeof obj.TotalFrames === "number") totalFrames = obj.TotalFrames;
if (typeof obj.FramesEncoded === "number") {
// Plan and Assemble also return FramesEncoded; count framesRendered
// only inside the RenderChunk state so we don't double-count
// it on the Assemble pass. Keyed off the enclosing state name
// (set by the matching StateEntered) rather than the payload's
// `Action` field — `Action` is part of the Lambda event
// contract and not load-bearing for state-machine identity.
if (currentLambdaState === "RenderChunk") {
framesRendered += obj.FramesEncoded;
}
}
}
break;
}
case "TaskStateExited":
case "MapStateExited":
// Mark the assemble step complete on its state-exit, independent
// of the inner Lambda payload shape. The Assemble state's
// ResultSelector pulls FileSize + OutputS3Uri from the Lambda
// result, so we re-extract them here from the state exit's
// own output rather than relying on the Lambda payload.
if (ev.stateExitedEventDetails?.name === "Assemble") {
assembleComplete = true;
const exitPayload = parseJson(ev.stateExitedEventDetails?.output);
if (exitPayload && typeof exitPayload === "object") {
const obj = exitPayload as Record<string, unknown>;
const out = obj.Output as Record<string, unknown> | undefined;
const outputS3Uri = typeof out?.OutputS3Uri === "string" ? out.OutputS3Uri : null;
const bytes = typeof out?.FileSize === "number" ? out.FileSize : null;
outputFile = outputS3Uri ? { s3Uri: outputS3Uri, bytes } : outputFile;
}
}
break;
case "LambdaFunctionFailed":
errors.push({
state: currentLambdaState ?? "<unknown>",
error: ev.lambdaFunctionFailedEventDetails?.error ?? "UNKNOWN",
cause: ev.lambdaFunctionFailedEventDetails?.cause ?? "",
});
break;
case "ExecutionFailed":
errors.push({
state: "<execution>",
error: ev.executionFailedEventDetails?.error ?? "UNKNOWN",
cause: ev.executionFailedEventDetails?.cause ?? "",
});
break;
case "ExecutionAborted":
errors.push({
state: "<execution>",
error: ev.executionAbortedEventDetails?.error ?? "ABORTED",
cause: ev.executionAbortedEventDetails?.cause ?? "",
});
break;
case "ExecutionTimedOut":
errors.push({
state: "<execution>",
error: "TIMEOUT",
cause: ev.executionTimedOutEventDetails?.cause ?? "",
});
break;
default:
break;
}
}
return {
lambdaInvocations,
stateTransitions,
framesRendered,
totalFrames,
lambdasInvoked,
assembleComplete,
outputFile,
errors,
};
}
function parseJson(s: string | undefined): unknown {
if (!s) return null;
try {
return JSON.parse(s);
} catch {
return null;
}
}
/**
* Lambda success payloads from our handler include `DurationMs` — the
* wall-clock the handler observed. We use it as a best-effort proxy
* for `BilledDuration` when SFN doesn't expose the latter directly
* on `LambdaFunctionSucceeded` (the dedicated `BilledDuration` field
* is in CloudWatch Metrics, not the SFN history payload).
*/
function inferBilledMs(payload: unknown): number {
if (!payload || typeof payload !== "object") return 0;
const obj = payload as Record<string, unknown>;
if (typeof obj.DurationMs === "number") return obj.DurationMs;
return 0;
}
interface ComputeProgressArgs {
status: RenderStatus;
totalFrames: number | null;
framesRendered: number;
assembleComplete: boolean;
}
function computeOverallProgress({
status,
totalFrames,
framesRendered,
assembleComplete,
}: ComputeProgressArgs): number {
if (status === "SUCCEEDED") return 1;
if (assembleComplete) return 1;
if (totalFrames === null) return 0;
// 10 % Plan + 80 % chunk render + 10 % Assemble.
const chunkProgress = Math.min(1, framesRendered / totalFrames);
return 0.1 + 0.8 * chunkProgress;
}
function isTerminalFailure(status: RenderStatus): boolean {
return status === "FAILED" || status === "TIMED_OUT" || status === "ABORTED";
}