mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-03 20:37:30 +00:00
* feat(gcp-cloud-run): add Google Cloud Run + Workflows distributed render adapter Adds @hyperframes/gcp-cloud-run, the GCP counterpart to @hyperframes/aws-lambda (issue #932). The OSS distributed primitives (plan, renderChunk x N, assemble) are unchanged; this package is the storage/compute/orchestration glue. Package: Cloud Run handler (one image, three actions), runs under bun; GCS transport; in-image chrome-headless-shell resolver; client SDK (renderToCloudRun, getRenderProgress, deploySite, computeRenderCost); Dockerfile; Cloud Workflows definition; Terraform module; CLI cloudrun deploy|sites|render|render-batch|progress|destroy with --output-resolution and --strict-variables; 62 unit tests + docs + live smoke script. Shared extraction (removes ~640 lines of adapter duplication): move the cloud-agnostic config validator + content-hash into producer/distributed; both adapters import them. Validated end-to-end on GCP at 37.4 dB PSNR vs baseline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): resolve @hyperframes/gcp-cloud-run in the CLI build + root build The CLI bundle (esbuild) couldn't resolve `@hyperframes/gcp-cloud-run/sdk`, failing Build/Typecheck/CLI-smoke (and the perf/windows/regression jobs that build first). Mirror the aws-lambda handling: mark the gcp adapter + its /sdk subpath external in tsup.config.ts with a source alias, and add gcp-cloud-run to the root `build` filter so its dist exists for publish + runtime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): copy gcp-cloud-run manifest in Dockerfile.test for frozen install The regression test image runs `bun install --frozen-lockfile` after copying each workspace package.json individually. The CLI now depends on @hyperframes/gcp-cloud-run (workspace:*), so the frozen install fails to resolve it unless its manifest is present. Add the COPY line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): add machine-sizing flags to `cloudrun deploy` Closes the parity gap with `lambda deploy` (which exposes --memory etc.). `cloudrun deploy` now threads --cpu, --memory, --max-instances, and --timeout into the Terraform apply; omitted flags keep the module defaults (4 vCPU / 16Gi / 100 instances / 3600s). For finer control, apply the module directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address PR review (security, waste, limits, alerts) - server.ts: bucket-allowlist guard no longer fails open silently. Unset env logs a one-time WARNING; "*" is an explicit opt-out; otherwise it enforces. - server.ts: stop double-shipping audio.aac. It already rides in the plan tarball every consumer downloads, so drop the redundant standalone upload (plan) + re-download/overwrite (assemble); assemble reads it from the untar, falling back to a supplied AudioGcsUri for compat. - server.ts: chunk extension via path.extname() instead of slice(lastIndexOf). - workflow.yaml: clamp parallel concurrency_limit to math.min(chunkCount, 20) — Cloud Workflows hard-caps concurrent iterations at 20. - Dockerfile: pin bun (bun-v1.3.9) so an interop change can't silently break the image rebuild. - terraform: add min_instances var (default 0); add a workflow-failure alert (finished_execution_count status=FAILED) alongside the request-count one. - costAccounting: document that displayCost excludes GCS storage/egress. Verified against the actual APIs: @google-cloud/workflows@4.4.0 ICreateExecutionRequest has no executionId (so the idempotency-token suggestion isn't available in this client); Workflows concurrency cap is 20; failure metric is workflows.googleapis.com/finished_execution_count (status label). 174 adapter tests pass, fallow/oxlint/oxfmt/terraform clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address round-2 review — error code + CFR forwarding - workflow.yaml: rename the zero-chunk failure code PLAN_TOO_LARGE → PLAN_PRODUCED_ZERO_CHUNKS. The old code implied a size-ceiling breach (the opposite cause), misleading anyone triaging the alert. - workflow.yaml: forward Config.cfr to the assemble step (`Cfr: ${("cfr" in config) and config.cfr}`). It was read by the handler but never sent, so exact-CFR was silently off for every Cloud Run render. Uses the same `in`-operator guard already proven in the retryable predicate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(release): include gcp-cloud-run in set-version PACKAGES list set-version.ts (driven by release:prepare) bumps an explicit package list to the shared version on each release. gcp-cloud-run was wired into the build + publish.yml but missing here, so a release would leave it at a stale version and publish.yml would push the wrong version. Add it so the new package version-bumps + publishes in lockstep with the others. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
80 lines
3.4 KiB
TypeScript
80 lines
3.4 KiB
TypeScript
/**
|
|
* Client-side validation for the AWS Lambda adapter.
|
|
*
|
|
* The cloud-agnostic config-shape validation (`validateDistributedRenderConfig`,
|
|
* `validateVariablesPayload`, `InvalidConfigError`) lives in
|
|
* `@hyperframes/producer/distributed` and is shared with the other adapters.
|
|
* This module re-exports those and adds the one piece specific to Step
|
|
* Functions: the 256 KiB Standard-workflow execution-input size cap.
|
|
*/
|
|
|
|
import { InvalidConfigError } from "@hyperframes/producer/distributed";
|
|
|
|
export {
|
|
InvalidConfigError,
|
|
validateDistributedRenderConfig,
|
|
validateVariablesPayload,
|
|
} from "@hyperframes/producer/distributed";
|
|
|
|
/**
|
|
* Hard cap on Step Functions Standard workflow execution input — 256 KiB per
|
|
* the AWS limits page. Express workflows cap at 32 KiB; the render stack runs
|
|
* Standard for execution-history visibility, so the larger limit applies. The
|
|
* cap is on the entire serialized input, not just the variables, because
|
|
* users hit it at the wire boundary regardless of which field caused the
|
|
* bloat.
|
|
*
|
|
* Specific to Step Functions Standard. Other workflow runtimes (Temporal,
|
|
* Express SFN, Cloud Workflows, raw Lambda invoke) have different caps; don't
|
|
* reuse this constant for those without confirming the limit.
|
|
*/
|
|
export const MAX_STEP_FUNCTIONS_INPUT_BYTES = 256 * 1024;
|
|
|
|
/** Pointer to the docs section that explains the URL-your-assets convention. */
|
|
const LARGE_VARIABLES_DOCS_URL =
|
|
"https://hyperframes.heygen.com/deploy/templates-on-lambda#working-with-large-variables";
|
|
|
|
/**
|
|
* Validate that the serialized Step Functions execution input fits inside the
|
|
* 256 KiB Standard-workflow cap. Measured in UTF-8 bytes (the format Step
|
|
* Functions uses on the wire) — JS strings count UTF-16 code units, which
|
|
* under-reports for any multi-byte character.
|
|
*
|
|
* Throws {@link InvalidConfigError} with a clear message naming the actual
|
|
* byte count, the cap, and a pointer to the "working with large variables"
|
|
* docs section, so users hit the limit at the SDK boundary with actionable
|
|
* guidance instead of as a `States.DataLimitExceeded` 50 ms into the
|
|
* execution.
|
|
*/
|
|
// fallow-ignore-next-line complexity
|
|
export function validateStepFunctionsInputSize(input: unknown): void {
|
|
let serialized: string | undefined;
|
|
try {
|
|
serialized = JSON.stringify(input);
|
|
} catch (err) {
|
|
throw new InvalidConfigError(
|
|
"config",
|
|
`Step Functions execution input is not JSON-serializable: ${err instanceof Error ? err.message : String(err)}`,
|
|
);
|
|
}
|
|
if (serialized === undefined) {
|
|
throw new InvalidConfigError(
|
|
"config",
|
|
"Step Functions execution input is not JSON-serializable (JSON.stringify returned undefined). " +
|
|
"Check that all fields, including config.variables, are plain JSON values.",
|
|
);
|
|
}
|
|
const byteLength = Buffer.byteLength(serialized, "utf8");
|
|
if (byteLength > MAX_STEP_FUNCTIONS_INPUT_BYTES) {
|
|
throw new InvalidConfigError(
|
|
"config",
|
|
`Step Functions execution input is ${byteLength} bytes, which exceeds the ` +
|
|
`${MAX_STEP_FUNCTIONS_INPUT_BYTES}-byte (256 KiB) limit for Standard workflows. ` +
|
|
`Variables are for typed data (strings, numbers, structured records); media assets ` +
|
|
`(images, audio, video) should be passed as URL references the composition resolves ` +
|
|
`at render time, not inlined as base64. See ${LARGE_VARIABLES_DOCS_URL} for the ` +
|
|
`URL-your-assets convention.`,
|
|
);
|
|
}
|
|
}
|