Files
hyperframes/packages/gcp-cloud-run/src/events.ts
T
James RussoandClaude Opus 4.8 4da567df22 feat(gcp-cloud-run): Google Cloud Run + Workflows distributed render adapter (#1253)
* feat(gcp-cloud-run): add Google Cloud Run + Workflows distributed render adapter

Adds @hyperframes/gcp-cloud-run, the GCP counterpart to @hyperframes/aws-lambda
(issue #932). The OSS distributed primitives (plan, renderChunk x N, assemble)
are unchanged; this package is the storage/compute/orchestration glue.

Package: Cloud Run handler (one image, three actions), runs under bun; GCS
transport; in-image chrome-headless-shell resolver; client SDK
(renderToCloudRun, getRenderProgress, deploySite, computeRenderCost); Dockerfile;
Cloud Workflows definition; Terraform module; CLI cloudrun
deploy|sites|render|render-batch|progress|destroy with --output-resolution and
--strict-variables; 62 unit tests + docs + live smoke script.

Shared extraction (removes ~640 lines of adapter duplication): move the
cloud-agnostic config validator + content-hash into producer/distributed; both
adapters import them. Validated end-to-end on GCP at 37.4 dB PSNR vs baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): resolve @hyperframes/gcp-cloud-run in the CLI build + root build

The CLI bundle (esbuild) couldn't resolve `@hyperframes/gcp-cloud-run/sdk`,
failing Build/Typecheck/CLI-smoke (and the perf/windows/regression jobs that
build first). Mirror the aws-lambda handling: mark the gcp adapter + its /sdk
subpath external in tsup.config.ts with a source alias, and add gcp-cloud-run
to the root `build` filter so its dist exists for publish + runtime.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): copy gcp-cloud-run manifest in Dockerfile.test for frozen install

The regression test image runs `bun install --frozen-lockfile` after copying
each workspace package.json individually. The CLI now depends on
@hyperframes/gcp-cloud-run (workspace:*), so the frozen install fails to
resolve it unless its manifest is present. Add the COPY line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): add machine-sizing flags to `cloudrun deploy`

Closes the parity gap with `lambda deploy` (which exposes --memory etc.).
`cloudrun deploy` now threads --cpu, --memory, --max-instances, and --timeout
into the Terraform apply; omitted flags keep the module defaults
(4 vCPU / 16Gi / 100 instances / 3600s). For finer control, apply the module
directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gcp-cloud-run): address PR review (security, waste, limits, alerts)

- server.ts: bucket-allowlist guard no longer fails open silently. Unset env
  logs a one-time WARNING; "*" is an explicit opt-out; otherwise it enforces.
- server.ts: stop double-shipping audio.aac. It already rides in the plan
  tarball every consumer downloads, so drop the redundant standalone upload
  (plan) + re-download/overwrite (assemble); assemble reads it from the untar,
  falling back to a supplied AudioGcsUri for compat.
- server.ts: chunk extension via path.extname() instead of slice(lastIndexOf).
- workflow.yaml: clamp parallel concurrency_limit to math.min(chunkCount, 20)
  — Cloud Workflows hard-caps concurrent iterations at 20.
- Dockerfile: pin bun (bun-v1.3.9) so an interop change can't silently break
  the image rebuild.
- terraform: add min_instances var (default 0); add a workflow-failure alert
  (finished_execution_count status=FAILED) alongside the request-count one.
- costAccounting: document that displayCost excludes GCS storage/egress.

Verified against the actual APIs: @google-cloud/workflows@4.4.0
ICreateExecutionRequest has no executionId (so the idempotency-token suggestion
isn't available in this client); Workflows concurrency cap is 20; failure
metric is workflows.googleapis.com/finished_execution_count (status label).
174 adapter tests pass, fallow/oxlint/oxfmt/terraform clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gcp-cloud-run): address round-2 review — error code + CFR forwarding

- workflow.yaml: rename the zero-chunk failure code PLAN_TOO_LARGE →
  PLAN_PRODUCED_ZERO_CHUNKS. The old code implied a size-ceiling breach (the
  opposite cause), misleading anyone triaging the alert.
- workflow.yaml: forward Config.cfr to the assemble step
  (`Cfr: ${("cfr" in config) and config.cfr}`). It was read by the handler
  but never sent, so exact-CFR was silently off for every Cloud Run render.
  Uses the same `in`-operator guard already proven in the retryable predicate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(release): include gcp-cloud-run in set-version PACKAGES list

set-version.ts (driven by release:prepare) bumps an explicit package list to
the shared version on each release. gcp-cloud-run was wired into the build +
publish.yml but missing here, so a release would leave it at a stale version
and publish.yml would push the wrong version. Add it so the new package
version-bumps + publishes in lockstep with the others.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 14:43:38 -07:00

147 lines
5.6 KiB
TypeScript

/**
* Request + result types for the HyperFrames distributed render handler
* running on Cloud Run.
*
* The Cloud Workflows definition in `packages/gcp-cloud-run/terraform/workflow.yaml`
* dispatches on the `Action` field of the JSON request body. Each action
* maps 1:1 onto one of the three OSS distributed primitives:
*
* "plan" → `plan(projectDir, config, planDir)` (Activity A)
* "renderChunk" → `renderChunk(planDir, chunkIndex, output)` (Activity B)
* "assemble" → `assemble(planDir, chunkPaths, audio, out)` (Activity C)
*
* All file I/O is mediated by GCS — the handler downloads inputs into a
* per-request workdir under the container's writable `/tmp`, invokes the
* primitive, uploads outputs back to GCS, and returns a small JSON payload
* that fits inside a Cloud Workflows step variable (Workflows caps a single
* step's memory; chunk results stay well under 1 KB so the orchestration
* can hold one per Map iteration).
*
* These shapes are intentionally identical to `@hyperframes/aws-lambda`'s
* `events.ts` apart from the URI scheme (`gs://` vs `s3://`): the wire
* contract is the adapter's, the primitives underneath are shared.
*/
import type {
DistributedFormat,
SerializableDistributedRenderConfig,
} from "@hyperframes/producer/distributed";
export type { SerializableDistributedRenderConfig } from "@hyperframes/producer/distributed";
/** Discriminator for the three roles the one Cloud Run image fulfills. */
export type CloudRunAction = "plan" | "renderChunk" | "assemble";
/**
* Top-level shape of any request body the handler may receive.
*
* Cloud Workflows passes the step's `body` through verbatim, but a caller
* driving the service directly (or a Workflows definition that wraps the
* payload) may nest it under `Payload` / `Input`; the handler unwraps both
* before dispatching, matching the Lambda adapter's envelope tolerance.
*/
export type CloudRunEvent =
| PlanEvent
| RenderChunkEvent
| AssembleEvent
| { Payload: CloudRunEvent }
| { Input: CloudRunEvent };
/** Activity A: produce a planDir, upload to GCS. */
export interface PlanEvent {
Action: "plan";
/** GCS URI pointing at a `tar -czf`-archived project directory (`gs://bucket/key.tar.gz`). */
ProjectGcsUri: string;
/** GCS URI prefix where the planDir tar should be uploaded (`gs://bucket/{prefix}/`). */
PlanOutputGcsPrefix: string;
/** `DistributedRenderConfig` minus runtime-only fields (logger, abortSignal). */
Config: SerializableDistributedRenderConfig;
}
/** Activity B: fetch planDir, render one chunk, upload result. */
export interface RenderChunkEvent {
Action: "renderChunk";
/** GCS URI of the plan tar produced by a PlanEvent invocation. */
PlanGcsUri: string;
/**
* `PlanResult.planHash` from the Plan invocation. The handler verifies
* this against the untarred planDir's `plan.json` before invoking the
* producer, throwing a typed `PLAN_HASH_MISMATCH` on divergence so the
* workflow routes it as non-retryable. Defense-in-depth — the producer
* also re-checks internally.
*/
PlanHash: string;
/** 0-based chunk index this invocation should render. */
ChunkIndex: number;
/** GCS URI prefix where the chunk output should be uploaded (`gs://bucket/{prefix}/`). */
ChunkOutputGcsPrefix: string;
/** Output container format from the plan's encoder.json; drives file vs frame-dir handling. */
Format: DistributedFormat;
}
/** Activity C: fetch planDir + all chunks + audio, assemble, upload final. */
export interface AssembleEvent {
Action: "assemble";
/** GCS URI of the plan tar produced by a PlanEvent invocation. */
PlanGcsUri: string;
/** GCS URIs of every chunk, ordered by chunk index. Length must equal `chunkCount`. */
ChunkGcsUris: string[];
/** GCS URI of the planDir's `audio.aac` if the composition has audio; `null` otherwise. */
AudioGcsUri: string | null;
/** Final output GCS URI (`gs://bucket/key.mp4`). */
OutputGcsUri: string;
/** Output container format; drives file vs frame-dir handling. */
Format: DistributedFormat;
/**
* Optional exact-CFR re-encode at assemble time. When `true`, the final
* assembled video is re-encoded with `-fps_mode cfr -r <fps>` so the
* stream's `avg_frame_rate` matches the container's `r_frame_rate`
* exactly (and the file's duration is exact, not PTS-derived). Trade-off
* is ~2-5x the assemble wall-clock. mp4 only — webm / mov stream-copy
* paths already produce exact avg_frame_rate. Default `false` /
* unset preserves current `-c copy` behavior.
*/
Cfr?: boolean;
}
// ── Result types — kept small to fit Cloud Workflows step budgets ────────────
/** Result of a `plan` invocation. Carries enough to size the Map(N) state. */
export interface PlanResultBody {
Action: "plan";
PlanGcsUri: string;
PlanHash: string;
ChunkCount: number;
TotalFrames: number;
Fps: 24 | 30 | 60;
Width: number;
Height: number;
Format: DistributedFormat;
HasAudio: boolean;
AudioGcsUri: string | null;
FfmpegVersion: string;
ProducerVersion: string;
DurationMs: number;
}
/** Result of a `renderChunk` invocation. Sized ≤200 bytes. */
export interface RenderChunkResultBody {
Action: "renderChunk";
ChunkGcsUri: string;
ChunkIndex: number;
Sha256: string;
FramesEncoded: number;
DurationMs: number;
}
/** Result of an `assemble` invocation. */
export interface AssembleResultBody {
Action: "assemble";
OutputGcsUri: string;
FramesEncoded: number;
FileSize: number;
DurationMs: number;
}
export type CloudRunResult = PlanResultBody | RenderChunkResultBody | AssembleResultBody;