ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-04 16:42:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(cloud): default distributed plans to v2 (#3311)

Browse Source 
* feat(cloud): default distributed plans to v2

* fix(cloud): address plan v2 review feedback

* fix(examples): document explicit v2 samples
This commit is contained in:
James Russo
2026-08-17 17:24:31 -04:00
committed by GitHub
parent 6b17c24f98
commit 17a2a00ed5
36 changed files with 403 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/packages/aws-lambda.mdx
+11 -5
View File
@@ -70,7 +70,6 @@ const site = await deploySite({
const handle = await renderToLambda({
siteHandle: site,
planProtocol: "v2",
bucketName: site.bucketName,
stateMachineArn: "arn:aws:states:us-east-1:123456789012:stateMachine:hyperframes-render",
config: {
@@ -88,10 +87,17 @@ const progress = await getRenderProgress({ executionArn: handle.executionArn });
console.log(progress.status, progress.overallProgress, progress.costs.displayCost);
```
Plan v2 is recommended for new integrations because workers fetch
manifest-selected content-addressed artifacts. For backwards compatibility,
omitting `planProtocol` still selects v1; existing callers do not change
behavior until they opt in.
Plan v2 is the default because workers fetch manifest-selected
content-addressed artifacts. The SDK sends explicit v2 when `planProtocol` is
omitted. Deprecated v1 compatibility remains available by passing
`planProtocol: "v1"`.
<Warning>
For an existing installation, pause new renders and drain active Step
Functions executions. Redeploy the Lambda handler and SAM/CDK state machine
from the same package version before upgrading the application SDK. Older
infrastructure may default omission to v1 or lack v2 support.
</Warning>
`renderToLambda()` validates the distributed render config before starting the Step Functions execution, so invalid dimensions, formats, chunk sizes, or payload sizes fail synchronously.
docs/packages/gcp-cloud-run.mdx
+11 -5
View File
@@ -79,7 +79,6 @@ import { getRenderProgress, renderToCloudRun } from "@hyperframes/gcp-cloud-run/
const handle = await renderToCloudRun({
projectDir: "./my-composition",
planProtocol: "v2",
config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
bucketName: "hyperframes-render-my-project",
projectId: "my-project",
@@ -97,10 +96,17 @@ while (progress.status === "running") {
console.log(progress.status, progress.outputFile, progress.costs.displayCost);
```
Plan v2 is recommended for new integrations because workers fetch
manifest-selected content-addressed artifacts. For backwards compatibility,
omitting `planProtocol` still selects v1; existing callers do not change
behavior until they opt in.
Plan v2 is the default because workers fetch manifest-selected
content-addressed artifacts. The SDK sends explicit v2 when `planProtocol` is
omitted. Deprecated v1 compatibility remains available by passing
`planProtocol: "v1"`.
<Warning>
For an existing installation, pause new renders and drain active workflow
executions. Redeploy the Cloud Run image and Cloud Workflows definition from
the same package version before upgrading the application SDK. Older
workflows may default omission to v1 or lack v2 support.
</Warning>
Pass `projectDir` for one-shot uploads, or call `deploySite()` separately and reuse the returned site handle across many renders.
examples/aws-lambda/README.md
+22 -10
View File
@@ -75,7 +75,6 @@ aws stepfunctions start-execution \
"ProjectS3Uri": "s3://${RENDER_BUCKET}/projects/my-project.tar.gz",
"PlanOutputS3Prefix": "s3://${RENDER_BUCKET}/renders/$(date +%s)/",
"OutputS3Uri": "s3://${RENDER_BUCKET}/output.mp4",
"PlanProtocol": "v2",
"Config": {
"fps": 30,
"width": 1920,
@@ -92,10 +91,20 @@ EOF
The Step Functions execution kicks off Plan, fans out RenderChunk via
the Map state, and finally Assemble. Final mp4 lands at `OutputS3Uri`.
Plan v2 is recommended for new integrations. `PlanProtocol` may be `"v1"` or
`"v2"`; absent still defaults to v1 for backwards compatibility. V2 uses
separate manifest and content-addressed artifact locators throughout the
workflow and never places a v2 object in `PlanS3Uri`.
Plan v2 is the default when `PlanProtocol` is absent. V2 uses separate
manifest and content-addressed artifact locators throughout the workflow and
never places a v2 object in `PlanS3Uri`. The deprecated v1 transport remains
available by sending `"PlanProtocol": "v1"` explicitly.
### Upgrading an existing stack
Pause new renders and let active Step Functions executions drain before the
upgrade. Redeploy the Lambda handler and this state machine (or the matching
CDK construct) from the same package version before upgrading the application
that calls `renderToLambda`. The new SDK sends explicit v2 by default, while
older infrastructure may default omission to v1 or lack v2 support. Keep
passing `planProtocol: "v1"` until the infrastructure redeploy completes if
you need a staged migration.
## Local invocation
@@ -112,10 +121,13 @@ sam validate
sam local invoke RenderFunction --event sample-events/plan.json
```
The `sample-events/` directory ships small JSON payloads for each of the
three actions. They reference fake S3 URIs — useful for sanity-checking
the handler's dispatch logic; not for full end-to-end testing (real S3
calls require credentials and a project zip to actually exist).
The `sample-events/` directory ships three tiers for each action:
`*.json` demonstrates default v2 with `PlanProtocol` omitted, `*-v1.json`
demonstrates deprecated explicit-v1 compatibility, and `*-v2.json`
demonstrates callers that stamp v2 explicitly. They reference fake S3 URIs —
useful for sanity-checking the handler's dispatch logic; not for full
end-to-end testing (real S3 calls require credentials and a project zip to
actually exist).
## End-to-end smoke + benchmark
@@ -124,7 +136,7 @@ the architecture works on a deployed Lambda — use the local smoke
script:
```bash
# Defaults use the fixture's meta.json minPsnr (30 dB for mp4-h264-sdr).
# Defaults use Plan v2 and the fixture's meta.json minPsnr (30 dB for mp4-h264-sdr).
./scripts/smoke.sh
# Customised:
examples/aws-lambda/sample-events/assemble-v1.json
+12
View File
@@ -0,0 +1,12 @@
{
"Action": "assemble",
"PlanProtocol": "v1",
"PlanS3Uri": "s3://example-bucket/renders/sample/plan.tar.gz",
"ChunkS3Uris": [
"s3://example-bucket/renders/sample/chunks/0000.mp4",
"s3://example-bucket/renders/sample/chunks/0001.mp4"
],
"AudioS3Uri": null,
"OutputS3Uri": "s3://example-bucket/renders/sample/output.mp4",
"Format": "mp4"
}
examples/aws-lambda/sample-events/assemble.json
+3 -1
View File
@@ -1,6 +1,8 @@
{
"Action": "assemble",
"PlanS3Uri": "s3://example-bucket/renders/sample/plan.tar.gz",
"PlanV2ManifestS3Uri": "s3://example-bucket/renders/sample/v2/manifest.json",
"PlanV2ArtifactS3Prefix": "s3://example-bucket/renders/sample/v2/artifacts/sha256",
"PlanHash": "0000000000000000000000000000000000000000000000000000000000000000",
"ChunkS3Uris": [
"s3://example-bucket/renders/sample/chunks/0000.mp4",
"s3://example-bucket/renders/sample/chunks/0001.mp4",
examples/aws-lambda/sample-events/plan-v1.json
+15
View File
@@ -0,0 +1,15 @@
{
"Action": "plan",
"PlanProtocol": "v1",
"ProjectS3Uri": "s3://example-bucket/projects/sample.tar.gz",
"PlanOutputS3Prefix": "s3://example-bucket/renders/sample/",
"Config": {
"fps": 30,
"width": 1920,
"height": 1080,
"format": "mp4",
"chunkSize": 240,
"maxParallelChunks": 8,
"runtimeCap": "lambda"
}
}
examples/aws-lambda/sample-events/plan.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"Action": "plan",
"ProjectS3Uri": "s3://example-bucket/projects/sample-composition.tar.gz",
"ProjectS3Uri": "s3://example-bucket/projects/sample.tar.gz",
"PlanOutputS3Prefix": "s3://example-bucket/renders/sample/",
"Config": {
"fps": 30,
examples/aws-lambda/sample-events/render-chunk-v1.json
+9
View File
@@ -0,0 +1,9 @@
{
"Action": "renderChunk",
"PlanProtocol": "v1",
"PlanS3Uri": "s3://example-bucket/renders/sample/plan.tar.gz",
"PlanHash": "0000000000000000000000000000000000000000000000000000000000000000",
"ChunkIndex": 0,
"ChunkOutputS3Prefix": "s3://example-bucket/renders/sample/",
"Format": "mp4"
}
examples/aws-lambda/sample-events/render-chunk.json
+2 -1
View File
@@ -1,6 +1,7 @@
{
"Action": "renderChunk",
"PlanS3Uri": "s3://example-bucket/renders/sample/plan.tar.gz",
"PlanV2ManifestS3Uri": "s3://example-bucket/renders/sample/v2/manifest.json",
"PlanV2ArtifactS3Prefix": "s3://example-bucket/renders/sample/v2/artifacts/sha256",
"PlanHash": "0000000000000000000000000000000000000000000000000000000000000000",
"ChunkIndex": 0,
"ChunkOutputS3Prefix": "s3://example-bucket/renders/sample/",
examples/aws-lambda/scripts/smoke.sh
+3 -3
View File
@@ -31,7 +31,7 @@
# --region <region> (default: $AWS_REGION or us-east-1)
# --profile <name> (default: $AWS_PROFILE, otherwise the AWS
# default profile resolution chain)
# --plan-protocol <v1|v2|both> (default: v1)
# --plan-protocol <v1|v2|both> (default: v2)
# --keep-stack (skip `sam delete` at the end)
# --skip-build (skip the ZIP rebuild; use the existing one)
#
@@ -82,7 +82,7 @@ SMOKE_RUN_ID="${HYPERFRAMES_SMOKE_RUN_ID:-$(hf_new_smoke_run_id)}"
STACK_NAME="${STACK_NAME:-hyperframes-lambda-smoke-${SMOKE_RUN_ID}}"
AWS_REGION="${AWS_REGION:-us-east-1}"
AWS_PROFILE="${AWS_PROFILE:-}"
PLAN_PROTOCOL="${PLAN_PROTOCOL:-v1}"
PLAN_PROTOCOL="${PLAN_PROTOCOL:-v2}"
KEEP_STACK="false"
SKIP_BUILD="false"
REQUIRE_ENCODED_SHA_EQUAL="${REQUIRE_ENCODED_SHA_EQUAL:-false}"
@@ -110,7 +110,7 @@ Flags:
--stack-name <name> SAM stack name (default: hyperframes-lambda-smoke-<unique-run-id>)
--region <region> AWS region (default: $AWS_REGION or us-east-1)
--profile <name> AWS profile (default: $AWS_PROFILE)
--plan-protocol <v1|v2|both> plan transport(s) to compare (default: v1)
--plan-protocol <v1|v2|both> plan transport(s) to compare (default: v2)
--reserved-concurrency <N> Lambda Map MaxConcurrency cap (default: 16)
--keep-stack skip `sam delete` at the end (manual teardown later)
--require-encoded-sha-equal also gate byte-identical encoded MP4 output
examples/aws-lambda/template.yaml
+5 -2
View File
@@ -222,12 +222,12 @@ Resources:
- Variable: $.PlanProtocol
IsPresent: true
Next: UnsupportedPlanProtocol
Default: Plan
Default: PlanV2
UnsupportedPlanProtocol:
Type: Fail
Error: PLAN_PROTOCOL_UNSUPPORTED
Cause: PlanProtocol must be "v1", "v2", or absent (defaults to v1).
Cause: PlanProtocol must be "v1", "v2", or absent (defaults to v2).
Plan:
Type: Task
@@ -236,6 +236,7 @@ Resources:
FunctionName: !GetAtt RenderFunction.Arn
Payload:
Action: plan
PlanProtocol: v1
ProjectS3Uri.$: "$.ProjectS3Uri"
PlanOutputS3Prefix.$: "$.PlanOutputS3Prefix"
Config.$: "$.Config"
@@ -391,6 +392,7 @@ Resources:
FunctionName: !GetAtt RenderFunction.Arn
Payload:
Action: renderChunk
PlanProtocol: v1
ChunkIndex.$: "$.ChunkIndex"
PlanS3Uri.$: "$.PlanS3Uri"
PlanHash.$: "$.PlanHash"
@@ -427,6 +429,7 @@ Resources:
FunctionName: !GetAtt RenderFunction.Arn
Payload:
Action: assemble
PlanProtocol: v1
PlanS3Uri.$: "$.Plan.PlanS3Uri"
ChunkS3Uris.$: "$.Chunks[*].ChunkS3Uri"
AudioS3Uri.$: "$.Plan.AudioS3Uri"
examples/gcp-cloud-run/README.md
+21 -9
View File
@@ -8,7 +8,7 @@ Cloud Workflows adapter for HyperFrames distributed rendering.
```text
scripts/smoke.sh Owner-isolated real-GCP deploy, render, parity, cleanup
sample-events/ v1 and v2 handler request examples
sample-events/ Default-v2, explicit-v1, and explicit-v2 request examples
```
The Terraform module and Cloud Workflows definition live in
@@ -16,8 +16,9 @@ The Terraform module and Cloud Workflows definition live in
## Protocol rollout
The workflow defaults to plan protocol v1 when `PlanProtocol` is absent. V2 is
accepted only when the caller explicitly sends `PlanProtocol: "v2"`.
The workflow defaults to Plan v2 when `PlanProtocol` is absent. Deprecated v1
compatibility remains available only when the caller explicitly sends
`PlanProtocol: "v1"`.
V1 and v2 use disjoint plan locators:
@@ -26,9 +27,15 @@ V1 and v2 use disjoint plan locators:
The workflow validates that the plan response matches the selected protocol
before starting chunk fan-out. It never silently falls back from v2 to v1.
Deploy the v2 workflow only with a Cloud Run image whose handler implements
the matching v2 request/response contract. An older v1-only handler will keep
serving default v1 requests, but explicit v2 smoke executions will fail closed.
Deploy the workflow only with a Cloud Run image whose handler implements the
matching v2 request/response contract.
For an existing installation, pause new renders and drain active workflow
executions. Redeploy the Cloud Run image and workflow from the same package
version before upgrading the application SDK. The new SDK sends explicit v2;
older workflows may still default omission to v1 or lack v2 support. Keep
passing `planProtocol: "v1"` until the infrastructure redeploy completes if
you need a staged migration.
## Prerequisites
@@ -40,7 +47,7 @@ serving default v1 requests, but explicit v2 smoke executions will fail closed.
## Run the smoke
V1 remains the safe default:
Plan v2 is the normal smoke path:
```bash
./scripts/smoke.sh \
@@ -120,12 +127,17 @@ old invocation's state or image implicitly.
The sample events mirror the request bodies sent by Cloud Workflows:
```bash
# V1
# Default v2 (PlanProtocol omitted)
curl -sX POST localhost:8080/ \
-H 'content-type: application/json' \
--data @sample-events/plan.json | jq .
# Explicit v2
# Deprecated explicit v1 compatibility
curl -sX POST localhost:8080/ \
-H 'content-type: application/json' \
--data @sample-events/plan-v1.json | jq .
# Explicit v2 for callers that always stamp the protocol
curl -sX POST localhost:8080/ \
-H 'content-type: application/json' \
--data @sample-events/plan-v2.json | jq .
examples/gcp-cloud-run/sample-events/assemble-v1.json
+12
View File
@@ -0,0 +1,12 @@
{
"Action": "assemble",
"PlanProtocol": "v1",
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
"ChunkGcsUris": [
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0000.mp4",
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0001.mp4"
],
"AudioGcsUri": null,
"OutputGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/output.mp4",
"Format": "mp4"
}
examples/gcp-cloud-run/sample-events/assemble.json
+3 -2
View File
@@ -1,7 +1,8 @@
{
"Action": "assemble",
"PlanProtocol": "v1",
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
"PlanV2ManifestGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/manifest.json",
"PlanV2ArtifactGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/artifacts/sha256",
"PlanHash": "REPLACE_WITH_PLAN_HASH",
"ChunkGcsUris": [
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0000.mp4",
"gs://hyperframes-render-PROJECT/renders/hf-render-demo/chunks/0001.mp4"
examples/gcp-cloud-run/sample-events/plan-v1.json
+7
View File
@@ -0,0 +1,7 @@
{
"Action": "plan",
"PlanProtocol": "v1",
"ProjectGcsUri": "gs://hyperframes-render-PROJECT/sites/abc123/project.tar.gz",
"PlanOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
"Config": { "fps": 30, "width": 1920, "height": 1080, "format": "mp4" }
}
examples/gcp-cloud-run/sample-events/plan.json
-1
View File
@@ -1,6 +1,5 @@
{
"Action": "plan",
"PlanProtocol": "v1",
"ProjectGcsUri": "gs://hyperframes-render-PROJECT/sites/abc123/project.tar.gz",
"PlanOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
"Config": { "fps": 30, "width": 1920, "height": 1080, "format": "mp4" }
examples/gcp-cloud-run/sample-events/render-chunk-v1.json
+9
View File
@@ -0,0 +1,9 @@
{
"Action": "renderChunk",
"PlanProtocol": "v1",
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
"PlanHash": "REPLACE_WITH_PLAN_HASH",
"ChunkIndex": 0,
"ChunkOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
"Format": "mp4"
}
examples/gcp-cloud-run/sample-events/render-chunk.json
+2 -2
View File
@@ -1,7 +1,7 @@
{
"Action": "renderChunk",
"PlanProtocol": "v1",
"PlanGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/plan.tar.gz",
"PlanV2ManifestGcsUri": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/manifest.json",
"PlanV2ArtifactGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/v2/artifacts/sha256",
"PlanHash": "REPLACE_WITH_PLAN_HASH",
"ChunkIndex": 0,
"ChunkOutputGcsPrefix": "gs://hyperframes-render-PROJECT/renders/hf-render-demo/",
examples/gcp-cloud-run/scripts/smoke.sh
+6 -6
View File
@@ -2,11 +2,11 @@
# Owner-isolated real-GCP smoke + v1/v2 parity test for the HyperFrames
# Cloud Run adapter.
#
# The default is intentionally v1-only. Plan protocol v2 must be opted into
# explicitly with --protocols v1,v2. Every invocation derives a unique,
# length-safe resource prefix and uses an isolated Terraform working directory
# and state file. Cleanup verifies every owned resource is absent and fails
# closed on API/authentication errors.
# The default exercises Plan v2. Deprecated v1 compatibility can be selected
# explicitly with --protocols v1 or compared with --protocols v1,v2. Every
# invocation derives a unique, length-safe resource prefix and uses an isolated
# Terraform working directory and state file. Cleanup verifies every owned
# resource is absent and fails closed on API/authentication errors.
#
# Usage:
# ./smoke.sh --project <gcp-project>
@@ -33,7 +33,7 @@ REGION="${GCP_REGION:-us-central1}"
FIXTURE="${FIXTURE:-mp4-h264-sdr}"
CHUNK_SIZES="${CHUNK_SIZES:-}"
PSNR_THRESHOLD="${PSNR_THRESHOLD:-35}"
PROTOCOLS="${PROTOCOLS:-v1}"
PROTOCOLS="${PROTOCOLS:-v2}"
OWNER="${HYPERFRAMES_SMOKE_OWNER:-}"
AR_REPO="${AR_REPO:-}"
AR_REPO_WAS_EXPLICIT=0
packages/aws-lambda/README.md
+21 -5
View File
@@ -47,14 +47,13 @@ inside Step Functions' history budget (under 200 bytes per chunk).
### Plan transport selection
Plan v2 is recommended for new integrations. `renderToLambda` still defaults
an omitted `planProtocol` to the existing monolithic v1 transport for
backwards compatibility, so select v2 explicitly:
Plan v2 is the default for new renders. When `planProtocol` is omitted,
`renderToLambda` sends an explicit `PlanProtocol: "v2"` so the SDK and the
deployed state machine agree:
```ts
await renderToLambda({
// ...bucket, state machine, project, and config...
planProtocol: "v2",
});
```
@@ -64,7 +63,24 @@ only manifest-selected chunk artifacts, while the assembler fetches its
own metadata and audio subset. Blobs are immutable SHA-256-addressed
objects, verified on upload and download, and the manifest is published
last. Unknown protocols and digest mismatches are terminal Step Functions
errors. Omit the selector—or use `"v1"`—to retain the prior wire contract.
errors. The monolithic v1 transport remains available as deprecated
compatibility by passing `planProtocol: "v1"` explicitly.
#### Upgrade order
This default changes application behavior and requires a coordinated
infrastructure upgrade. Before upgrading an application that calls
`renderToLambda`:
1. Pause new renders and let existing Step Functions executions drain.
2. Redeploy the Lambda handler and SAM template or CDK construct from the
same new package version.
3. Resume renders, then upgrade the application/SDK dependency.
Older state machines can default missing protocol fields to v1 or lack v2
branches, while the new SDK sends explicit v2. If infrastructure cannot be
redeployed first, keep the application on its previous package version or
pass `planProtocol: "v1"` explicitly until the redeploy is complete.
## Chrome runtime
packages/aws-lambda/src/cdk/HyperframesRenderStack.snapshot.test.ts
+12
View File
@@ -151,6 +151,18 @@ describe("HyperframesRenderStack — snapshot", () => {
expect(actualStates.sort()).toEqual([...EXPECTED_STATE_NAMES].sort());
});
it("defaults omitted plan protocol to v2 and preserves the explicit v1 branch", () => {
for (const definition of [SYNTHED.definition, readSamDefinition()]) {
const selection = requireRecord(
definition.States.SelectPlanProtocol,
"SelectPlanProtocol state",
);
expect(selection.Default).toBe("PlanV2");
expect(JSON.stringify(selection)).toContain('"StringEquals":"v1"');
expect(JSON.stringify(definition.States.Plan)).toContain('"PlanProtocol":"v1"');
}
});
it("preserves every typed non-retryable error name across the three Lambda tasks", () => {
const { definition } = SYNTHED;
const collected = new Set<string>();
packages/aws-lambda/src/cdk/HyperframesRenderStack.ts
+5 -2
View File
@@ -247,6 +247,7 @@ export class HyperframesRenderStack extends Construct {
lambdaFunction: this.renderFunction,
payload: sfn.TaskInput.fromObject({
Action: "plan",
PlanProtocol: "v1",
"ProjectS3Uri.$": "$.ProjectS3Uri",
"PlanOutputS3Prefix.$": "$.PlanOutputS3Prefix",
"Config.$": "$.Config",
@@ -319,6 +320,7 @@ export class HyperframesRenderStack extends Construct {
lambdaFunction: this.renderFunction,
payload: sfn.TaskInput.fromObject({
Action: "renderChunk",
PlanProtocol: "v1",
"ChunkIndex.$": "$.ChunkIndex",
"PlanS3Uri.$": "$.PlanS3Uri",
"PlanHash.$": "$.PlanHash",
@@ -361,6 +363,7 @@ export class HyperframesRenderStack extends Construct {
lambdaFunction: this.renderFunction,
payload: sfn.TaskInput.fromObject({
Action: "assemble",
PlanProtocol: "v1",
"PlanS3Uri.$": "$.Plan.PlanS3Uri",
"ChunkS3Uris.$": "$.Chunks[*].ChunkS3Uri",
"AudioS3Uri.$": "$.Plan.AudioS3Uri",
@@ -473,13 +476,13 @@ export class HyperframesRenderStack extends Construct {
const unsupportedPlanProtocol = new sfn.Fail(this, "UnsupportedPlanProtocol", {
error: "PLAN_PROTOCOL_UNSUPPORTED",
cause: 'PlanProtocol must be "v1", "v2", or absent (defaults to v1).',
cause: 'PlanProtocol must be "v1", "v2", or absent (defaults to v2).',
});
return new sfn.Choice(this, "SelectPlanProtocol")
.when(sfn.Condition.stringEquals("$.PlanProtocol", "v2"), planV2)
.when(sfn.Condition.stringEquals("$.PlanProtocol", "v1"), plan)
.when(sfn.Condition.isPresent("$.PlanProtocol"), unsupportedPlanProtocol)
.otherwise(plan);
.otherwise(planV2);
}
}
packages/aws-lambda/src/events.ts
+10 -10
View File
@@ -55,17 +55,17 @@ interface PlanEventBase {
}
/**
* Legacy/default plan transport. Absence is deliberately interpreted as v1.
* Legacy plan transport. Callers must select it explicitly.
*
* @deprecated Use {@link PlanV2Event} for new integrations.
*/
export interface PlanV1Event extends PlanEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
}
/** Explicit opt-in to the content-addressed v2 plan transport. */
/** Default content-addressed v2 plan transport. */
export interface PlanV2Event extends PlanEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
}
export type PlanEvent = PlanV1Event | PlanV2Event;
@@ -90,12 +90,12 @@ interface RenderChunkEventBase {
}
/**
* Legacy/default chunk event.
* Legacy chunk event. Callers must select it explicitly.
*
* @deprecated Use {@link RenderChunkV2Event} for new integrations.
*/
export interface RenderChunkV1Event extends RenderChunkEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
/** S3 URI of the v1 plan tar produced by a PlanEvent invocation. */
PlanS3Uri: string;
}
@@ -105,7 +105,7 @@ export interface RenderChunkV1Event extends RenderChunkEventBase {
* describes the exact content-addressed artifacts needed by this chunk.
*/
export interface RenderChunkV2Event extends RenderChunkEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
PlanV2ManifestS3Uri: string;
PlanV2ArtifactS3Prefix: string;
}
@@ -136,19 +136,19 @@ interface AssembleEventBase {
}
/**
* Legacy/default assemble event.
* Legacy assemble event. Callers must select it explicitly.
*
* @deprecated Use {@link AssembleV2Event} for new integrations.
*/
export interface AssembleV1Event extends AssembleEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
/** S3 URI of the v1 plan tar produced by a PlanEvent invocation. */
PlanS3Uri: string;
}
/** V2 assemble event, scoped to manifest-declared assembler artifacts. */
export interface AssembleV2Event extends AssembleEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
PlanV2ManifestS3Uri: string;
PlanV2ArtifactS3Prefix: string;
PlanHash: string;
packages/aws-lambda/src/handler.test.ts
+61 -5
View File
@@ -145,6 +145,7 @@ describe("unwrapEvent", () => {
it("unwraps a Step Functions { Payload } envelope", () => {
const inner: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanS3Uri: "s3://bucket/plan.tar.gz",
PlanHash: "deadbeef",
ChunkIndex: 3,
@@ -158,6 +159,7 @@ describe("unwrapEvent", () => {
it("unwraps multiple levels of envelopes", () => {
const inner: AssembleEvent = {
Action: "assemble",
PlanProtocol: "v1",
PlanS3Uri: "s3://bucket/plan.tar.gz",
ChunkS3Uris: ["s3://bucket/chunks/0001.mp4"],
AudioS3Uri: null,
@@ -176,7 +178,7 @@ describe("unwrapEvent", () => {
});
describe("handler dispatch", () => {
it("routes Action='plan' to the plan primitive", async () => {
it("preserves explicit v1 plan compatibility", async () => {
const tmpRoot = makeTmpRoot();
const s3 = new FakeS3Client();
// Seed a fake project tarball so the untar step has something to chew on.
@@ -213,6 +215,7 @@ describe("handler dispatch", () => {
const event: PlanEvent = {
Action: "plan",
PlanProtocol: "v1",
ProjectS3Uri: "s3://bucket/project.tar.gz",
PlanOutputS3Prefix: "s3://bucket/renders/abc/",
Config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
@@ -270,6 +273,7 @@ describe("handler dispatch", () => {
handler(
{
Action: "plan",
PlanProtocol: "v1",
ProjectS3Uri: "s3://bucket/project.tar.gz",
PlanOutputS3Prefix: "s3://bucket/renders/terminal/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
@@ -332,6 +336,7 @@ describe("handler dispatch", () => {
const event: PlanEvent = {
Action: "plan",
PlanProtocol: "v1",
ProjectS3Uri: "s3://bucket/project.tar.gz",
PlanOutputS3Prefix: "s3://bucket/renders/abc/",
Config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
@@ -406,6 +411,7 @@ describe("handler dispatch", () => {
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanS3Uri: "s3://bucket/plan.tar.gz",
PlanHash: "fakehash",
ChunkIndex: 2,
@@ -455,6 +461,7 @@ describe("handler dispatch", () => {
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanS3Uri: "s3://bucket/plan.tar.gz",
PlanHash: "not-the-real-hash",
ChunkIndex: 0,
@@ -511,6 +518,7 @@ describe("handler dispatch", () => {
const event: AssembleEvent = {
Action: "assemble",
PlanProtocol: "v1",
PlanS3Uri: "s3://bucket/plan.tar.gz",
ChunkS3Uris: ["s3://bucket/chunks/0001.mp4", "s3://bucket/chunks/0002.mp4"],
AudioS3Uri: null,
@@ -541,7 +549,7 @@ describe("handler dispatch", () => {
expect(assembleMock).toHaveBeenCalledTimes(1);
});
it("runs v2 plan → target-scoped chunk → assemble without a PlanS3Uri", async () => {
it("defaults omitted plan protocol to v2 across plan → chunk → assemble", async () => {
const tmpRoot = makeTmpRoot();
const s3 = new FakeS3Client();
s3.objects.set("s3://bucket/project.tar.gz", await makeMinimalProjectTar());
@@ -614,7 +622,6 @@ describe("handler dispatch", () => {
const planned = await handler(
{
Action: "plan",
PlanProtocol: "v2",
ProjectS3Uri: "s3://bucket/project.tar.gz",
PlanOutputS3Prefix: "s3://bucket/renders/v2/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
@@ -642,7 +649,6 @@ describe("handler dispatch", () => {
const chunk = await handler(
{
Action: "renderChunk",
PlanProtocol: "v2",
PlanV2ManifestS3Uri: planned.PlanV2ManifestS3Uri,
PlanV2ArtifactS3Prefix: planned.PlanV2ArtifactS3Prefix,
PlanHash: planned.PlanHash,
@@ -661,7 +667,6 @@ describe("handler dispatch", () => {
await handler(
{
Action: "assemble",
PlanProtocol: "v2",
PlanV2ManifestS3Uri: planned.PlanV2ManifestS3Uri,
PlanV2ArtifactS3Prefix: planned.PlanV2ArtifactS3Prefix,
PlanHash: planned.PlanHash,
@@ -677,6 +682,56 @@ describe("handler dispatch", () => {
).toBe(true);
});
it("rejects unknown plan protocol values", async () => {
const tmpRoot = makeTmpRoot();
const s3 = new FakeS3Client();
await expect(
handler(
{
Action: "plan",
PlanProtocol: "v3",
ProjectS3Uri: "s3://bucket/project.tar.gz",
PlanOutputS3Prefix: "s3://bucket/renders/invalid/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
} as unknown as LambdaEvent,
{
s3: s3 as unknown as import("@aws-sdk/client-s3").S3Client,
tmpRoot,
skipChromeResolution: true,
},
),
).rejects.toMatchObject({ name: "PLAN_PROTOCOL_UNSUPPORTED" });
expect(s3.ops).toHaveLength(0);
});
it("rejects mixed v1/v2 plan locators at runtime", async () => {
const tmpRoot = makeTmpRoot();
const s3 = new FakeS3Client();
await expect(
handler(
{
Action: "renderChunk",
PlanS3Uri: "s3://bucket/plan.tar.gz",
PlanHash: "fakehash",
ChunkIndex: 0,
ChunkOutputS3Prefix: "s3://bucket/renders/mixed/",
Format: "mp4",
} as unknown as LambdaEvent,
{
s3: s3 as unknown as import("@aws-sdk/client-s3").S3Client,
tmpRoot,
skipChromeResolution: true,
},
),
).rejects.toMatchObject({
name: "PLAN_PROTOCOL_UNSUPPORTED",
message: expect.stringContaining("mixed or missing plan locators"),
});
expect(s3.ops).toHaveLength(0);
});
it("rejects unknown actions", async () => {
const tmpRoot = makeTmpRoot();
await expect(
@@ -735,6 +790,7 @@ describe("handler — S3 URI allowlist (security: F-004)", () => {
const event: AssembleEvent = {
Action: "assemble",
PlanProtocol: "v1",
PlanS3Uri: "s3://good-bucket/plan.tar.gz",
ChunkS3Uris: ["s3://good-bucket/chunks/0001.mp4", "s3://evil-bucket/chunks/0002.mp4"],
AudioS3Uri: null,
packages/aws-lambda/src/handler.ts
+54 -13
View File
@@ -44,9 +44,12 @@ import type {
LambdaEvent,
LambdaResult,
PlanEvent,
PlanV2Event,
PlanLambdaResult,
RenderChunkEvent,
RenderChunkV2Event,
RenderChunkLambdaResult,
AssembleV2Event,
} from "./events.js";
import {
downloadS3ObjectToFile,
@@ -97,6 +100,7 @@ export interface HandlerDeps {
*/
export async function handler(event: LambdaEvent, deps?: HandlerDeps): Promise<LambdaResult> {
const unwrapped = unwrapEvent(event);
validatePlanProtocolShape(unwrapped);
validateEventS3Uris(unwrapped);
primeRuntimeEnv();
// Single structured boot log line — CloudWatch Logs Insights queries
@@ -201,6 +205,43 @@ function isLambdaAction(value: string): value is LambdaAction {
return value === "plan" || value === "renderChunk" || value === "assemble";
}
// This is the single fail-closed boundary for the wire union. Keeping all
// forbidden locator combinations together makes mixed-protocol input auditable.
// fallow-ignore-next-line complexity
function validatePlanProtocolShape(event: PlanEvent | RenderChunkEvent | AssembleEvent): void {
const raw = event as unknown as Record<string, unknown>;
const protocol = raw.PlanProtocol;
if (protocol !== undefined && protocol !== "v1" && protocol !== "v2") {
const error = new Error(
`[handler] unsupported PlanProtocol ${JSON.stringify(protocol)}; expected "v1", "v2", or absent`,
);
error.name = "PLAN_PROTOCOL_UNSUPPORTED";
throw error;
}
if (event.Action === "plan") return;
const effectiveProtocol = protocol ?? "v2";
const hasV1Locator = typeof raw.PlanS3Uri === "string";
const hasV2Manifest = typeof raw.PlanV2ManifestS3Uri === "string";
const hasV2Prefix = typeof raw.PlanV2ArtifactS3Prefix === "string";
const valid =
effectiveProtocol === "v2"
? !hasV1Locator && hasV2Manifest && hasV2Prefix
: hasV1Locator && !hasV2Manifest && !hasV2Prefix;
if (!valid) {
const error = new Error(
`[handler] ${effectiveProtocol} ${event.Action} event has mixed or missing plan locators`,
);
error.name = "PLAN_PROTOCOL_UNSUPPORTED";
throw error;
}
if (effectiveProtocol === "v2" && event.Action === "assemble" && event.AudioS3Uri !== null) {
const error = new Error("[handler] v2 assemble audio must be materialized from the manifest");
error.name = "PLAN_PROTOCOL_UNSUPPORTED";
throw error;
}
}
/**
* Emit a single JSON line to stdout. CloudWatch ingests each line as a
* structured event; Logs Insights queries can `filter event="..."` and
@@ -229,14 +270,14 @@ function summarizeEvent(
return {
projectS3Uri: event.ProjectS3Uri,
planOutputS3Prefix: event.PlanOutputS3Prefix,
planProtocol: event.PlanProtocol ?? "v1",
planProtocol: event.PlanProtocol ?? "v2",
format: event.Config.format,
fps: event.Config.fps,
};
case "renderChunk":
return {
planProtocol: event.PlanProtocol ?? "v1",
...(event.PlanProtocol === "v2"
planProtocol: event.PlanProtocol ?? "v2",
...(event.PlanProtocol !== "v1"
? { planV2ManifestS3Uri: event.PlanV2ManifestS3Uri }
: { planS3Uri: event.PlanS3Uri }),
chunkIndex: event.ChunkIndex,
@@ -244,8 +285,8 @@ function summarizeEvent(
};
case "assemble":
return {
planProtocol: event.PlanProtocol ?? "v1",
...(event.PlanProtocol === "v2"
planProtocol: event.PlanProtocol ?? "v2",
...(event.PlanProtocol !== "v1"
? { planV2ManifestS3Uri: event.PlanV2ManifestS3Uri }
: { planS3Uri: event.PlanS3Uri }),
chunkCount: event.ChunkS3Uris.length,
@@ -278,7 +319,7 @@ function primeRuntimeEnv(): void {
// The v1 handler owns one transactional download, plan, archive, upload, and cleanup lifecycle.
// fallow-ignore-next-line complexity
async function handlePlan(event: PlanEvent, deps?: HandlerDeps): Promise<PlanLambdaResult> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handlePlanV2(event, deps);
}
const started = Date.now();
@@ -358,7 +399,7 @@ async function handlePlan(event: PlanEvent, deps?: HandlerDeps): Promise<PlanLam
// and manifest-last publication must stay ordered and fail together.
// fallow-ignore-next-line complexity
async function handlePlanV2(
event: Extract<PlanEvent, { PlanProtocol: "v2" }>,
event: PlanV2Event,
deps?: HandlerDeps,
): Promise<Extract<PlanLambdaResult, { PlanProtocol: "v2" }>> {
const started = Date.now();
@@ -412,7 +453,7 @@ async function handleRenderChunk(
event: RenderChunkEvent,
deps?: HandlerDeps,
): Promise<RenderChunkLambdaResult> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handleRenderChunkV2(event, deps);
}
const started = Date.now();
@@ -481,7 +522,7 @@ async function handleRenderChunk(
// render, and upload in one lifecycle so cleanup and errors remain atomic.
// fallow-ignore-next-line complexity
async function handleRenderChunkV2(
event: Extract<RenderChunkEvent, { PlanProtocol: "v2" }>,
event: RenderChunkV2Event,
deps?: HandlerDeps,
): Promise<RenderChunkLambdaResult> {
const started = Date.now();
@@ -554,7 +595,7 @@ async function handleAssemble(
event: AssembleEvent,
deps?: HandlerDeps,
): Promise<AssembleLambdaResult> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handleAssembleV2(event, deps);
}
const started = Date.now();
@@ -610,7 +651,7 @@ async function handleAssemble(
// keeping the steps local makes its temporary-storage ownership explicit.
// fallow-ignore-next-line complexity
async function handleAssembleV2(
event: Extract<AssembleEvent, { PlanProtocol: "v2" }>,
event: AssembleV2Event,
deps?: HandlerDeps,
): Promise<AssembleLambdaResult> {
const started = Date.now();
@@ -776,12 +817,12 @@ function getEventS3Uris(event: PlanEvent | RenderChunkEvent | AssembleEvent): st
case "plan":
return [event.ProjectS3Uri, event.PlanOutputS3Prefix];
case "renderChunk":
return event.PlanProtocol === "v2"
return event.PlanProtocol !== "v1"
? [event.PlanV2ManifestS3Uri, event.PlanV2ArtifactS3Prefix, event.ChunkOutputS3Prefix]
: [event.PlanS3Uri, event.ChunkOutputS3Prefix];
case "assemble":
return [
...(event.PlanProtocol === "v2"
...(event.PlanProtocol !== "v1"
? [event.PlanV2ManifestS3Uri, event.PlanV2ArtifactS3Prefix]
: [event.PlanS3Uri]),
...event.ChunkS3Uris,
packages/aws-lambda/src/sdk/renderToLambda.test.ts
+7 -7
View File
@@ -98,11 +98,11 @@ describe("renderToLambda", () => {
PlanOutputS3Prefix: "s3://test-bucket/renders/smoke-1/",
OutputS3Uri: "s3://test-bucket/renders/smoke-1/output.mp4",
Config: baseConfig,
PlanProtocol: "v1",
PlanProtocol: "v2",
});
});
it("opts the complete execution into plan protocol v2 explicitly", async () => {
it("preserves explicit plan protocol v1 compatibility", async () => {
const sfn = new FakeSFN();
const s3 = new FakeS3();
const handle = await renderToLambda({
@@ -110,18 +110,18 @@ describe("renderToLambda", () => {
bucketName: "test-bucket",
stateMachineArn: "arn:aws:states:us-east-1:1234:stateMachine:hf",
config: baseConfig,
executionName: "smoke-v2",
planProtocol: "v2",
executionName: "smoke-v1",
planProtocol: "v1",
sfn: asSFNClient(sfn),
s3: asS3Client(s3),
});
expect(sfn.starts[0]?.input).toEqual({
ProjectS3Uri: handle.projectS3Uri,
PlanOutputS3Prefix: "s3://test-bucket/renders/smoke-v2/",
OutputS3Uri: "s3://test-bucket/renders/smoke-v2/output.mp4",
PlanOutputS3Prefix: "s3://test-bucket/renders/smoke-v1/",
OutputS3Uri: "s3://test-bucket/renders/smoke-v1/output.mp4",
Config: baseConfig,
PlanProtocol: "v2",
PlanProtocol: "v1",
});
});
packages/aws-lambda/src/sdk/renderToLambda.ts
+3 -3
View File
@@ -38,8 +38,8 @@ export interface RenderToLambdaOptions {
/** Validated `SerializableDistributedRenderConfig` (no logger / abortSignal). */
config: SerializableDistributedRenderConfig;
/**
* Distributed plan transport. Defaults to `"v1"` for backwards
* compatibility. New integrations should explicitly select `"v2"`.
* Distributed plan transport. Defaults to `"v2"`. Select `"v1"`
* explicitly only for deprecated compatibility with the monolithic plan.
*/
planProtocol?: LambdaPlanProtocol;
/** S3 bucket from the SAM stack output (`RenderBucketName`). */
@@ -115,7 +115,7 @@ export async function renderToLambda(opts: RenderToLambdaOptions): Promise<Rende
PlanOutputS3Prefix: planOutputS3Prefix,
OutputS3Uri: outputS3Uri,
Config: opts.config,
PlanProtocol: opts.planProtocol ?? "v1",
PlanProtocol: opts.planProtocol ?? "v2",
};
// Reject oversize input client-side. Step Functions Standard caps the
packages/gcp-cloud-run/README.md
+17 -4
View File
@@ -45,14 +45,13 @@ per-step durations on success.
### Plan transport selection
Plan v2 is recommended for new integrations. `renderToCloudRun` still
interprets an omitted `planProtocol` as `"v1"` for backwards compatibility,
so new callers should select v2 explicitly:
Plan v2 is the default for new renders. When `planProtocol` is omitted,
`renderToCloudRun` sends an explicit `PlanProtocol: "v2"` so the SDK and
the deployed workflow agree:
```ts
await renderToCloudRun({
// ...project, bucket, workflow, service, and config...
planProtocol: "v2",
});
```
@@ -60,6 +59,20 @@ V2 uses separate manifest and content-addressed artifact locators throughout
the workflow. Unknown protocols and integrity failures fail closed; a render
never mixes v1 and v2 artifacts.
The monolithic v1 transport remains available as deprecated compatibility by
passing `planProtocol: "v1"` explicitly.
#### Upgrade order
Redeploy the Cloud Run image and Cloud Workflows definition from the same new
package version before upgrading an application that calls
`renderToCloudRun`. Pause new renders and drain active workflow executions
during the infrastructure update. Older workflows can default omission to v1
or lack the v2 branch, while the new SDK sends explicit v2. If infrastructure
cannot be redeployed first, keep the previous SDK version or pass
`planProtocol: "v1"` explicitly until the Terraform/workflow redeploy is
complete.
## Chrome runtime
Unlike the Lambda adapter — which fights a 250 MB ZIP ceiling and
packages/gcp-cloud-run/src/events.ts
+10 -10
View File
@@ -61,17 +61,17 @@ interface PlanEventBase {
}
/**
* Legacy/default plan transport. Absence is deliberately interpreted as v1.
* Legacy plan transport. Callers must select it explicitly.
*
* @deprecated Use {@link PlanV2Event} for new integrations.
*/
export interface PlanV1Event extends PlanEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
}
/** Explicit opt-in to the content-addressed v2 plan transport. */
/** Default content-addressed v2 plan transport. */
export interface PlanV2Event extends PlanEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
}
export type PlanEvent = PlanV1Event | PlanV2Event;
@@ -96,12 +96,12 @@ interface RenderChunkEventBase {
}
/**
* Legacy/default chunk event.
* Legacy chunk event. Callers must select it explicitly.
*
* @deprecated Use {@link RenderChunkV2Event} for new integrations.
*/
export interface RenderChunkV1Event extends RenderChunkEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
/** GCS URI of the v1 plan tar produced by a PlanEvent invocation. */
PlanGcsUri: string;
PlanV2ManifestGcsUri?: never;
@@ -113,7 +113,7 @@ export interface RenderChunkV1Event extends RenderChunkEventBase {
* describes the exact content-addressed artifacts needed by this chunk.
*/
export interface RenderChunkV2Event extends RenderChunkEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
PlanV2ManifestGcsUri: string;
PlanV2ArtifactGcsPrefix: string;
PlanGcsUri?: never;
@@ -143,12 +143,12 @@ interface AssembleEventBase {
}
/**
* Legacy/default assemble event.
* Legacy assemble event. Callers must select it explicitly.
*
* @deprecated Use {@link AssembleV2Event} for new integrations.
*/
export interface AssembleV1Event extends AssembleEventBase {
PlanProtocol?: "v1";
PlanProtocol: "v1";
/** GCS URI of the v1 plan tar produced by a PlanEvent invocation. */
PlanGcsUri: string;
/** Legacy standalone audio locator; `null` when audio is embedded in the v1 plan tar. */
@@ -159,7 +159,7 @@ export interface AssembleV1Event extends AssembleEventBase {
/** V2 assemble event, scoped to manifest-declared assembler artifacts. */
export interface AssembleV2Event extends AssembleEventBase {
PlanProtocol: "v2";
PlanProtocol?: "v2";
PlanV2ManifestGcsUri: string;
PlanV2ArtifactGcsPrefix: string;
PlanHash: string;
packages/gcp-cloud-run/src/sdk/renderToCloudRun.test.ts
+4 -4
View File
@@ -88,16 +88,16 @@ describe("renderToCloudRun", () => {
OutputGcsUri: "gs://b/renders/hf-render-fixed/output.mp4",
ServiceUrl: "https://render-abc.run.app",
Config: config,
PlanProtocol: "v1",
PlanProtocol: "v2",
});
expect(fake.lastParent).toBe(
"projects/proj/locations/us-central1/workflows/hyperframes-render",
);
});
it("forwards an explicit v2 whole-render opt-in", async () => {
it("preserves explicit plan protocol v1 compatibility", async () => {
const fake = new FakeExecutions();
await renderToCloudRun({ ...opts(fake), planProtocol: "v2" });
await renderToCloudRun({ ...opts(fake), planProtocol: "v1" });
const arg = JSON.parse(fake.lastArgument ?? "{}");
expect(arg).toEqual({
RenderId: "hf-render-fixed",
@@ -106,7 +106,7 @@ describe("renderToCloudRun", () => {
OutputGcsUri: "gs://b/renders/hf-render-fixed/output.mp4",
ServiceUrl: "https://render-abc.run.app",
Config: config,
PlanProtocol: "v2",
PlanProtocol: "v1",
});
});
packages/gcp-cloud-run/src/sdk/renderToCloudRun.ts
+3 -3
View File
@@ -53,8 +53,8 @@ export interface RenderToCloudRunOptions {
/** Validated `SerializableDistributedRenderConfig` (no logger / abortSignal). */
config: SerializableDistributedRenderConfig;
/**
* Distributed plan transport. Defaults to `"v1"` for backwards
* compatibility. New integrations should explicitly select `"v2"`.
* Distributed plan transport. Defaults to `"v2"`. Select `"v1"`
* explicitly only for deprecated compatibility with the monolithic plan.
*/
planProtocol?: CloudRunPlanProtocol;
/** GCS bucket from the Terraform output (`render_bucket_name`). */
@@ -149,7 +149,7 @@ export async function renderToCloudRun(opts: RenderToCloudRunOptions): Promise<R
OutputGcsUri: outputGcsUri,
ServiceUrl: opts.serviceUrl,
Config: opts.config,
PlanProtocol: opts.planProtocol ?? "v1",
PlanProtocol: opts.planProtocol ?? "v2",
};
// Reject oversize input client-side. Cloud Workflows caps the execution
packages/gcp-cloud-run/src/server.test.ts
+15 -6
View File
@@ -178,11 +178,12 @@ describe("unwrapEvent", () => {
});
describe("dispatch", () => {
it("routes plan, uploads the plan tarball", async () => {
it("preserves explicit v1 plan compatibility", async () => {
const gcs = new FakeGcs();
await seedProjectTar(gcs, "gs://b/sites/x/project.tar.gz");
const event: PlanEvent = {
Action: "plan",
PlanProtocol: "v1",
ProjectGcsUri: "gs://b/sites/x/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/r1/",
Config: { fps: 30, width: 1920, height: 1080, format: "mp4" } as PlanEvent["Config"],
@@ -201,6 +202,7 @@ describe("dispatch", () => {
await seedPlanTar(gcs, "gs://b/renders/r1/plan.tar.gz", PLAN_HASH);
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 2,
@@ -220,6 +222,7 @@ describe("dispatch", () => {
await seedPlanTar(gcs, "gs://b/renders/r1/plan.tar.gz", PLAN_HASH);
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: "WRONG_HASH",
ChunkIndex: 0,
@@ -236,6 +239,7 @@ describe("dispatch", () => {
gcs.seed("gs://b/renders/r1/chunks/0001.mp4", Buffer.from("c1"));
const event: AssembleEvent = {
Action: "assemble",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
ChunkGcsUris: ["gs://b/renders/r1/chunks/0000.mp4", "gs://b/renders/r1/chunks/0001.mp4"],
AudioGcsUri: null,
@@ -251,7 +255,7 @@ describe("dispatch", () => {
// This end-to-end adapter contract is intentionally one narrative test: it
// verifies ordering and target isolation across all three handler roles.
// fallow-ignore-next-line complexity
it("runs v2 plan → target-scoped chunk → assemble with manifest-last CAS", async () => {
it("defaults omitted plan protocol to v2 across plan → chunk → assemble", async () => {
const gcs = new FakeGcs();
await seedProjectTar(gcs, "gs://b/sites/v2/project.tar.gz");
const root = mkTmp("hf-v2-e2e-");
@@ -306,7 +310,6 @@ describe("dispatch", () => {
const planned = await dispatch(
{
Action: "plan",
PlanProtocol: "v2",
ProjectGcsUri: "gs://b/sites/v2/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/v2/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
@@ -326,7 +329,6 @@ describe("dispatch", () => {
await dispatch(
{
Action: "plan",
PlanProtocol: "v2",
ProjectGcsUri: "gs://b/sites/v2/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/v2/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
@@ -342,7 +344,6 @@ describe("dispatch", () => {
const chunk = await dispatch(
{
Action: "renderChunk",
PlanProtocol: "v2",
PlanV2ManifestGcsUri: planned.PlanV2ManifestGcsUri,
PlanV2ArtifactGcsPrefix: planned.PlanV2ArtifactGcsPrefix,
PlanHash: planned.PlanHash,
@@ -360,7 +361,6 @@ describe("dispatch", () => {
await dispatch(
{
Action: "assemble",
PlanProtocol: "v2",
PlanV2ManifestGcsUri: planned.PlanV2ManifestGcsUri,
PlanV2ArtifactGcsPrefix: planned.PlanV2ArtifactGcsPrefix,
PlanHash: planned.PlanHash,
@@ -409,6 +409,7 @@ describe("bucket allowlist guard", () => {
try {
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://evil-bucket/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -451,6 +452,7 @@ describe("bucket allowlist guard", () => {
try {
const event: RenderChunkEvent = {
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://any-bucket/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -476,6 +478,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -497,6 +500,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: "WRONG",
ChunkIndex: 0,
@@ -524,6 +528,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -561,6 +566,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -598,6 +604,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/plan.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
@@ -626,6 +633,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "plan",
PlanProtocol: "v1",
ProjectGcsUri: "gs://b/sites/invalid-video-metadata/project.tar.gz",
PlanOutputGcsPrefix: "gs://b/renders/invalid-video-metadata/",
Config: { fps: 30, width: 640, height: 360, format: "mp4" },
@@ -645,6 +653,7 @@ describe("createApp HTTP mapping", () => {
headers: { "content-type": "application/json" },
body: JSON.stringify({
Action: "renderChunk",
PlanProtocol: "v1",
PlanGcsUri: "gs://b/renders/r1/missing.tar.gz",
PlanHash: PLAN_HASH,
ChunkIndex: 0,
packages/gcp-cloud-run/src/server.ts
+20 -16
View File
@@ -46,13 +46,16 @@ import {
import { resolveChromeExecutablePath } from "./chromium.js";
import type {
AssembleEvent,
AssembleV2Event,
AssembleResultBody,
CloudRunAction,
CloudRunEvent,
CloudRunResult,
PlanEvent,
PlanV2Event,
PlanResultBody,
RenderChunkEvent,
RenderChunkV2Event,
RenderChunkResultBody,
} from "./events.js";
import { type DistributedFormat, formatExtension } from "./formatExtension.js";
@@ -155,21 +158,22 @@ function validatePlanProtocolShape(event: PlanEvent | RenderChunkEvent | Assembl
}
if (event.Action === "plan") return;
const effectiveProtocol = protocol ?? "v2";
const hasV1Locator = typeof raw.PlanGcsUri === "string";
const hasV2Manifest = typeof raw.PlanV2ManifestGcsUri === "string";
const hasV2Prefix = typeof raw.PlanV2ArtifactGcsPrefix === "string";
const valid =
protocol === "v2"
effectiveProtocol === "v2"
? !hasV1Locator && hasV2Manifest && hasV2Prefix
: hasV1Locator && !hasV2Manifest && !hasV2Prefix;
if (!valid) {
const error = new Error(
`[handler] ${protocol === "v2" ? "v2" : "v1"} ${event.Action} event has mixed or missing plan locators`,
`[handler] ${effectiveProtocol} ${event.Action} event has mixed or missing plan locators`,
);
error.name = "PLAN_PROTOCOL_UNSUPPORTED";
throw error;
}
if (protocol === "v2" && event.Action === "assemble" && event.AudioGcsUri !== null) {
if (effectiveProtocol === "v2" && event.Action === "assemble" && event.AudioGcsUri !== null) {
const error = new Error("[handler] v2 assemble audio must be materialized from the manifest");
error.name = "PLAN_PROTOCOL_UNSUPPORTED";
throw error;
@@ -254,14 +258,14 @@ function summarizeEvent(
return {
projectGcsUri: event.ProjectGcsUri,
planOutputGcsPrefix: event.PlanOutputGcsPrefix,
planProtocol: event.PlanProtocol ?? "v1",
planProtocol: event.PlanProtocol ?? "v2",
format: event.Config.format,
fps: event.Config.fps,
};
case "renderChunk":
return {
planProtocol: event.PlanProtocol ?? "v1",
...(event.PlanProtocol === "v2"
planProtocol: event.PlanProtocol ?? "v2",
...(event.PlanProtocol !== "v1"
? { planV2ManifestGcsUri: event.PlanV2ManifestGcsUri }
: { planGcsUri: event.PlanGcsUri }),
chunkIndex: event.ChunkIndex,
@@ -269,8 +273,8 @@ function summarizeEvent(
};
case "assemble":
return {
planProtocol: event.PlanProtocol ?? "v1",
...(event.PlanProtocol === "v2"
planProtocol: event.PlanProtocol ?? "v2",
...(event.PlanProtocol !== "v1"
? { planV2ManifestGcsUri: event.PlanV2ManifestGcsUri }
: { planGcsUri: event.PlanGcsUri }),
chunkCount: event.ChunkGcsUris.length,
@@ -297,7 +301,7 @@ function primeChrome(deps?: HandlerDeps): void {
// fallow-ignore-next-line complexity
async function handlePlan(event: PlanEvent, deps?: HandlerDeps): Promise<PlanResultBody> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handlePlanV2(event, deps);
}
const started = Date.now();
@@ -365,7 +369,7 @@ async function handlePlan(event: PlanEvent, deps?: HandlerDeps): Promise<PlanRes
*/
// fallow-ignore-next-line complexity
async function handlePlanV2(
event: Extract<PlanEvent, { PlanProtocol: "v2" }>,
event: PlanV2Event,
deps?: HandlerDeps,
): Promise<Extract<PlanResultBody, { PlanProtocol: "v2" }>> {
const started = Date.now();
@@ -418,7 +422,7 @@ async function handleRenderChunk(
event: RenderChunkEvent,
deps?: HandlerDeps,
): Promise<RenderChunkResultBody> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handleRenderChunkV2(event, deps);
}
const started = Date.now();
@@ -475,7 +479,7 @@ async function handleRenderChunk(
/** Materialize only this chunk's verified v2 dependencies before rendering. */
// fallow-ignore-next-line complexity
async function handleRenderChunkV2(
event: Extract<RenderChunkEvent, { PlanProtocol: "v2" }>,
event: RenderChunkV2Event,
deps?: HandlerDeps,
): Promise<RenderChunkResultBody> {
const started = Date.now();
@@ -548,7 +552,7 @@ async function handleAssemble(
event: AssembleEvent,
deps?: HandlerDeps,
): Promise<AssembleResultBody> {
if (event.PlanProtocol === "v2") {
if (event.PlanProtocol !== "v1") {
return handleAssembleV2(event, deps);
}
const started = Date.now();
@@ -613,7 +617,7 @@ async function handleAssemble(
*/
// fallow-ignore-next-line complexity
async function handleAssembleV2(
event: Extract<AssembleEvent, { PlanProtocol: "v2" }>,
event: AssembleV2Event,
deps?: HandlerDeps,
): Promise<AssembleResultBody> {
const started = Date.now();
@@ -781,12 +785,12 @@ function getEventGcsUris(event: PlanEvent | RenderChunkEvent | AssembleEvent): s
case "plan":
return [event.ProjectGcsUri, event.PlanOutputGcsPrefix];
case "renderChunk":
return event.PlanProtocol === "v2"
return event.PlanProtocol !== "v1"
? [event.PlanV2ManifestGcsUri, event.PlanV2ArtifactGcsPrefix, event.ChunkOutputGcsPrefix]
: [event.PlanGcsUri, event.ChunkOutputGcsPrefix];
case "assemble":
return [
...(event.PlanProtocol === "v2"
...(event.PlanProtocol !== "v1"
? [event.PlanV2ManifestGcsUri, event.PlanV2ArtifactGcsPrefix]
: [event.PlanGcsUri]),
...event.ChunkGcsUris,
packages/gcp-cloud-run/terraform/smoke-safety.test.ts
+2 -2
View File
@@ -7,8 +7,8 @@ const smoke = readFileSync(smokePath, "utf-8");
const dockerfile = readFileSync(join(import.meta.dir, "../Dockerfile"), "utf-8");
describe("GCP smoke ownership and protocol safety", () => {
it("defaults to v1 and requires an explicit v2 protocol argument", () => {
expect(smoke).toContain('PROTOCOLS="${PROTOCOLS:-v1}"');
it("defaults to v2 and retains explicit v1/v2 protocol arguments", () => {
expect(smoke).toContain('PROTOCOLS="${PROTOCOLS:-v2}"');
expect(smoke).toContain("--protocols)");
expect(smoke).toContain("PlanProtocol: $protocol");
expect(smoke).toContain("decodedFramesEqual");
packages/gcp-cloud-run/terraform/workflow.test.ts
+2 -2
View File
@@ -77,8 +77,8 @@ describe("Cloud Workflows plan protocol routing", () => {
expect(source.match(/max_retries: 4/g)).toHaveLength(4);
});
it("keeps v1 as the default and rejects unknown protocols before plan", () => {
expect(source).toContain('default(map.get(args, "PlanProtocol"), "v1")');
it("defaults omitted protocol to v2 and rejects unknown protocols before plan", () => {
expect(source).toContain('default(map.get(args, "PlanProtocol"), "v2")');
expect(namedStep("selectPlanProtocol")).toMatchObject({
next: "unsupportedPlanProtocol",
});
packages/gcp-cloud-run/terraform/workflow.yaml
+3 -3
View File
@@ -26,9 +26,9 @@ main:
- planOutputGcsPrefix: ${args.PlanOutputGcsPrefix}
- outputGcsUri: ${args.OutputGcsUri}
- config: ${args.Config}
# Backward-compatible default. v2 is accepted only through an
# explicit top-level PlanProtocol opt-in.
- planProtocol: ${default(map.get(args, "PlanProtocol"), "v1")}
# Plan v2 is the default. Explicit v1 remains available during the
# deprecated monolithic-plan compatibility window.
- planProtocol: ${default(map.get(args, "PlanProtocol"), "v2")}
# ── Plan (Activity A) ────────────────────────────────────────────────────
- selectPlanProtocol: