feat(gcp-cloud-run): Google Cloud Run + Workflows distributed render adapter (#1253)

* feat(gcp-cloud-run): add Google Cloud Run + Workflows distributed render adapter Adds @hyperframes/gcp-cloud-run, the GCP counterpart to @hyperframes/aws-lambda (issue #932). The OSS distributed primitives (plan, renderChunk x N, assemble) are unchanged; this package is the storage/compute/orchestration glue. Package: Cloud Run handler (one image, three actions), runs under bun; GCS transport; in-image chrome-headless-shell resolver; client SDK (renderToCloudRun, getRenderProgress, deploySite, computeRenderCost); Dockerfile; Cloud Workflows definition; Terraform module; CLI cloudrun deploy|sites|render|render-batch|progress|destroy with --output-resolution and --strict-variables; 62 unit tests + docs + live smoke script. Shared extraction (removes ~640 lines of adapter duplication): move the cloud-agnostic config validator + content-hash into producer/distributed; both adapters import them. Validated end-to-end on GCP at 37.4 dB PSNR vs baseline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): resolve @hyperframes/gcp-cloud-run in the CLI build + root build The CLI bundle (esbuild) couldn't resolve `@hyperframes/gcp-cloud-run/sdk`, failing Build/Typecheck/CLI-smoke (and the perf/windows/regression jobs that build first). Mirror the aws-lambda handling: mark the gcp adapter + its /sdk subpath external in tsup.config.ts with a source alias, and add gcp-cloud-run to the root `build` filter so its dist exists for publish + runtime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): copy gcp-cloud-run manifest in Dockerfile.test for frozen install The regression test image runs `bun install --frozen-lockfile` after copying each workspace package.json individually. The CLI now depends on @hyperframes/gcp-cloud-run (workspace:*), so the frozen install fails to resolve it unless its manifest is present. Add the COPY line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): add machine-sizing flags to `cloudrun deploy` Closes the parity gap with `lambda deploy` (which exposes --memory etc.). `cloudrun deploy` now threads --cpu, --memory, --max-instances, and --timeout into the Terraform apply; omitted flags keep the module defaults (4 vCPU / 16Gi / 100 instances / 3600s). For finer control, apply the module directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address PR review (security, waste, limits, alerts) - server.ts: bucket-allowlist guard no longer fails open silently. Unset env logs a one-time WARNING; "*" is an explicit opt-out; otherwise it enforces. - server.ts: stop double-shipping audio.aac. It already rides in the plan tarball every consumer downloads, so drop the redundant standalone upload (plan) + re-download/overwrite (assemble); assemble reads it from the untar, falling back to a supplied AudioGcsUri for compat. - server.ts: chunk extension via path.extname() instead of slice(lastIndexOf). - workflow.yaml: clamp parallel concurrency_limit to math.min(chunkCount, 20) — Cloud Workflows hard-caps concurrent iterations at 20. - Dockerfile: pin bun (bun-v1.3.9) so an interop change can't silently break the image rebuild. - terraform: add min_instances var (default 0); add a workflow-failure alert (finished_execution_count status=FAILED) alongside the request-count one. - costAccounting: document that displayCost excludes GCS storage/egress. Verified against the actual APIs: @google-cloud/workflows@4.4.0 ICreateExecutionRequest has no executionId (so the idempotency-token suggestion isn't available in this client); Workflows concurrency cap is 20; failure metric is workflows.googleapis.com/finished_execution_count (status label). 174 adapter tests pass, fallow/oxlint/oxfmt/terraform clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address round-2 review — error code + CFR forwarding - workflow.yaml: rename the zero-chunk failure code PLAN_TOO_LARGE → PLAN_PRODUCED_ZERO_CHUNKS. The old code implied a size-ceiling breach (the opposite cause), misleading anyone triaging the alert. - workflow.yaml: forward Config.cfr to the assemble step (`Cfr: ${("cfr" in config) and config.cfr}`). It was read by the handler but never sent, so exact-CFR was silently off for every Cloud Run render. Uses the same `in`-operator guard already proven in the retryable predicate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(release): include gcp-cloud-run in set-version PACKAGES list set-version.ts (driven by release:prepare) bumps an explicit package list to the shared version on each release. gcp-cloud-run was wired into the build + publish.yml but missing here, so a release would leave it at a stale version and publish.yml would push the wrong version. Add it so the new package version-bumps + publishes in lockstep with the others. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-11 06:30:03 +00:00 · 2026-06-07 14:43:38 -07:00
parent 806b226b34
commit 4da567df22
60 changed files with 5782 additions and 362 deletions
@@ -0,0 +1,127 @@
+/**
+ * `renderToCloudRun` unit tests — argument assembly, required-field
+ * validation, and the CreateExecution call over a fake ExecutionsClient.
+ */
+
+import { describe, expect, it } from "bun:test";
+import type { SerializableDistributedRenderConfig } from "../events.js";
+import { type ExecutionsClientLike, renderToCloudRun } from "./renderToCloudRun.js";
+import type { SiteHandle } from "./deploySite.js";
+
+const config = {
+  fps: 30,
+  width: 1920,
+  height: 1080,
+  format: "mp4",
+} as SerializableDistributedRenderConfig;
+
+const site: SiteHandle = {
+  siteId: "abc",
+  bucketName: "b",
+  projectGcsUri: "gs://b/sites/abc/project.tar.gz",
+  bytes: 100,
+  uploadedAt: "2026-06-06T00:00:00Z",
+  uploaded: true,
+};
+
+class FakeExecutions implements ExecutionsClientLike {
+  lastArgument: string | null = null;
+  lastParent: string | null = null;
+
+  workflowPath(project: string, location: string, workflow: string): string {
+    return `projects/${project}/locations/${location}/workflows/${workflow}`;
+  }
+
+  async createExecution(req: {
+    parent: string;
+    execution: { argument: string };
+  }): Promise<[{ name?: string | null; state?: string | null }]> {
+    this.lastParent = req.parent;
+    this.lastArgument = req.execution.argument;
+    return [{ name: `${req.parent}/executions/exec-123`, state: "ACTIVE" }];
+  }
+}
+
+function opts(executions: ExecutionsClientLike) {
+  return {
+    siteHandle: site,
+    config,
+    bucketName: "b",
+    projectId: "proj",
+    location: "us-central1",
+    workflowId: "hyperframes-render",
+    serviceUrl: "https://render-abc.run.app",
+    renderId: "hf-render-fixed",
+    executions,
+  };
+}
+
+describe("renderToCloudRun", () => {
+  it("starts an execution and returns a handle", async () => {
+    const fake = new FakeExecutions();
+    const handle = await renderToCloudRun(opts(fake));
+    expect(handle.renderId).toBe("hf-render-fixed");
+    expect(handle.executionName).toBe(
+      "projects/proj/locations/us-central1/workflows/hyperframes-render/executions/exec-123",
+    );
+    expect(handle.outputGcsUri).toBe("gs://b/renders/hf-render-fixed/output.mp4");
+    expect(handle.projectGcsUri).toBe("gs://b/sites/abc/project.tar.gz");
+  });
+
+  it("builds the workflow argument the YAML expects", async () => {
+    const fake = new FakeExecutions();
+    await renderToCloudRun(opts(fake));
+    const arg = JSON.parse(fake.lastArgument ?? "{}");
+    expect(arg.RenderId).toBe("hf-render-fixed");
+    expect(arg.ProjectGcsUri).toBe("gs://b/sites/abc/project.tar.gz");
+    expect(arg.PlanOutputGcsPrefix).toBe("gs://b/renders/hf-render-fixed/");
+    expect(arg.OutputGcsUri).toBe("gs://b/renders/hf-render-fixed/output.mp4");
+    expect(arg.ServiceUrl).toBe("https://render-abc.run.app");
+    expect(arg.Config.format).toBe("mp4");
+    expect(fake.lastParent).toBe(
+      "projects/proj/locations/us-central1/workflows/hyperframes-render",
+    );
+  });
+
+  it("derives the output extension from the format", async () => {
+    const fake = new FakeExecutions();
+    const handle = await renderToCloudRun({
+      ...opts(fake),
+      config: { ...config, format: "webm" } as SerializableDistributedRenderConfig,
+    });
+    expect(handle.outputGcsUri).toBe("gs://b/renders/hf-render-fixed/output.webm");
+  });
+
+  it("requires serviceUrl", async () => {
+    const fake = new FakeExecutions();
+    await expect(renderToCloudRun({ ...opts(fake), serviceUrl: "" })).rejects.toThrow(
+      /serviceUrl is required/,
+    );
+  });
+
+  it("requires a siteHandle or projectDir", async () => {
+    const fake = new FakeExecutions();
+    const { siteHandle, ...rest } = opts(fake);
+    void siteHandle;
+    await expect(renderToCloudRun(rest)).rejects.toThrow(/siteHandle or projectDir/);
+  });
+
+  it("validates the config before any GCP call", async () => {
+    const fake = new FakeExecutions();
+    await expect(
+      renderToCloudRun({ ...opts(fake), config: { ...config, fps: 25 } as never }),
+    ).rejects.toThrow(/config\.fps/);
+    expect(fake.lastArgument).toBeNull();
+  });
+
+  it("rejects a renderId that could escape the GCS key prefix", async () => {
+    const fake = new FakeExecutions();
+    await expect(renderToCloudRun({ ...opts(fake), renderId: "../escape" })).rejects.toThrow(
+      /renderId must match/,
+    );
+    await expect(renderToCloudRun({ ...opts(fake), renderId: "has/slash" })).rejects.toThrow(
+      /renderId must match/,
+    );
+    expect(fake.lastArgument).toBeNull();
+  });
+});