mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
main
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
51f9fa7c54 |
perf(aws-lambda): small-chunk cloud renders stitch their output in parallel (#6635)
* perf(aws-lambda): small-chunk cloud renders stitch their output in parallel When every chunk's payload is under S3's 5 MiB part minimum (typical for short 4K chunks), the assemble step could not copy any part server-side and pulled the whole output through one Lambda's network link. Those range-only parts now go to parallel invocations of the warm function that just rendered the chunks, about 32 MiB each and never more at once than the stack's reserved concurrency allows. The output bytes are unchanged. * fix(aws-lambda): a throttled stitch helper builds its parts in the assemble instead Another render can hold the stack's reserved concurrency while this one assembles, so a helper invocation may be refused. The assemble now builds that batch itself, logs it, and counts it in assemble_stages; the helper's error type is kept as the error name. Comments trimmed back to each file's share at the base. * chore(aws-lambda): raise the package size budget to the measured size plus headroom The parallel stitch helpers add about 1.7 KB packed and 9.9 KB unpacked; the new budget is the measured size times the checker's own 1.05 headroom. |
||
|
Miguel Ángel
|
fb2f7c1a80 |
perf(aws-lambda): warm the audio function while the plan runs (#6148)
* perf(aws-lambda): warm the audio function while the plan runs The audio task runs on the x86 render function, which is cold when the plan ran on the arm64 video function. A fire-and-forget invoke at the start of a render now wakes it, so the audio no longer outlasts the chunks. * perf(aws-lambda): warm the render function only on a stack with a video function An x86-only stack has no cold function to warm, so it takes the old path with no extra step. The CDK stack has no video function and is unchanged. * perf(aws-lambda): wrap a template comment * style(aws-lambda): format the warm-up template test |
||
|
Miguel Ángel
|
03ed14615a |
feat(aws-lambda): run chunks on an optional arm64 image function (#6065)
* feat(aws-lambda): run chunks on an optional arm64 image function A new VideoImageUri parameter adds an arm64 container-image function that only renders chunks. Plan, audio and assemble stay on the x86 ZIP function, so audio encodes are unchanged. Empty (the default) deploys exactly as before. The reserved concurrency moves to whichever function renders the chunks. * docs(aws-lambda): the template tags both render functions * fix(aws-lambda): nest the arm64 video function so sam deploy needs no image repository sam deploy refuses a template with any top-level image function unless it is given an image repository, even when that function's Condition is false, so every deploy of the template failed. The video function now lives in video-function.yaml, a nested stack created only when VideoImageUri is set. The release ships it beside template.yaml and the CLI stages both. A new workflow runs the CLI's sam deploy argv against the template with fake credentials and requires it to reach the first upload. * fix(aws-lambda): run the plan on the arm64 video function with the chunks it plans * refactor(cli): name the staged template directory instead of commenting it * docs(aws-lambda): say the arm64 image runs the plan as well as the chunks |
||
|
Miguel Ángel
|
dee8380d22 |
feat(aws-lambda): a cloud render mixes its audio beside its chunks (#5997)
* feat(aws-lambda): a cloud render mixes its audio beside its chunks * fix(aws-lambda): the audio task keeps a worker free and runs only when the plan left audio out * test(aws-lambda): the audio task's free worker holds for a reserved pool of 1 and 2 * refactor(aws-lambda): one helper points every handler at the bundled Chrome |
||
|
Miguel Ángel
|
992df8b203 |
perf(cloud): move project bytes once through shared content storage (#5772)
* perf(producer): distributed chunks download only the videos they show A project video under compiled/ that chunks extract their own frames from now goes only to the chunks that show it, like video-sources/ files. A chunk skips the readiness wait for a deferred video it does not show, so it no longer logs media_load_failed for a file it was never given. * fix(producer): cloud site uploads pack the files their id hashes, symlinks included * perf(aws-lambda): chunks read only the byte ranges of a video they need A chunk that extracts its own frames from a deferred project video now reads it from S3 by byte range through a short-lived pre-signed URL, instead of downloading and hashing the whole object first. S3 attests the object's stored SHA-256 before the URL is signed; every other artifact is still downloaded and verified as before. * perf(aws-lambda): upload project files once to a shared content-addressed area * chore(producer): keep planV2 comment share at the base level * fix(engine): verify the TLS certificate of https video inputs and never surface their signed query * fix(aws-lambda): renders re-upload a site's expired files instead of failing * perf(aws-lambda): the plan step probes project videos in place instead of downloading them * fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause. * fix(aws-lambda): share entry bundles and ratchet package budget * fix(aws-lambda): site ids uploaded before files.json still render A site id from an earlier sites create has only sites/<id>/project.tar.gz. renderToLambda now falls back to that tarball, which the plan step still reads, and when neither key exists it fails before starting with both keys named and the fix (run sites create again). * fix(aws-lambda): renders fail before upload on a stack too old to read sites A stack deployed before files.json sites untars the site file in its plan step and fails after four retries. renderToLambda now reads the state machine definition first and stops with STACK_TOO_OLD and the redeploy command when the plan task does not list SITE_MANIFEST_INVALID, which new SAM and CDK stacks now treat as terminal. The README's upgrade section drops the planProtocol v1 fallback, which hits the same failure. * docs(aws-lambda): document shared content storage and the upgrade path The deploy guide still described the tarball HeadObject short-circuit. It now covers the cas/sha256/ store and its 30-day lifecycle rule, the STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site ids keep working, and that every project uploads once more after the upgrade. A test keeps a troubleshooting entry for each pre-submit error. * fix(producer): chunks log each video they read by URL, without its query A chunk reading a video straight from the bucket left no trace in its log. It now logs one line per remote source with the path and the URL minus its query string, so signatures never reach the log. * fix(engine): ffmpeg errors never keep part of a signed URL's query A process's stderr tail was trimmed before its URLs were redacted, so a cut inside a signed URL left a query fragment the URL pattern no longer matched, and the fragment reached the error. The stderr tail is now redacted before every trim, and the redactor also strips a signing parameter cut off from its URL. * fix(aws-lambda): comments name the site manifest instead of a file path * fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it The stderr tail was redacted on each chunk and then cut, so once a cut severed a long URL's scheme, the rest of its signature in the next chunk matched neither pattern. The tail is now kept raw with 2x slack and redacted only when the text is produced, then cut to size. The unused producer audioExtractor (its own spawn, unredacted stderr) is deleted and the parity harness's PCM decode goes through ManagedChildProcess; a test fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that does not. * chore(producer): record the audioExtractor deletion as intended * fix(engine): drop a signature fragment the stderr tail cut from its URL The raw tail is kept at twice the cap, but that cut can itself land inside a signature, and when redaction shrinks the rest below the cap the fragment survived. After a cut, the severed token is dropped before redaction. * refactor(engine): name the severed-token drop instead of commenting it * test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured * fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured * chore(producer): size budget at the measured size, no headroom * chore(producer): size budget back to measured x1.05, the check's own rule |
||
|
Miguel Ángel
|
2a14cbc924 |
perf(aws-lambda): size distributed renders from account concurrency limits (#5813)
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap The inline Map runs at most 40 chunk renders at once. Both state machine definitions (CDK and SAM) now use a Distributed Map with STANDARD child executions, capped at the function's reserved concurrency so chunks past it queue instead of being throttled. Progress reads each chunk's child execution history so frames, cost and chunk errors still show up. * fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks A throttled task used to burn its four catch-all retries in about 30 s. It now gets its own jittered backoff of several minutes, because another render on the same function can still hold the reserved slots. Plan treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions. * chore(aws-lambda): trim stack comments to keep the comment share flat * fix(aws-lambda): keep render --wait working at hundreds of chunks Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of every child execution's history, take exact chunk frames and cost from the Map output once it exits, and read failed children only when the counts show a failure. The SDK client retries throttled calls up to 8 times with backoff. * feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8 * fix(lambda): keep the concurrency SDK external in published modules * fix(cli): keep an existing stack's reserved concurrency on redeploy Redeploying without --concurrency sent ReservedConcurrency=-1, which removed the cap of 8 that earlier CLI versions reserved by default and stopped recording it in the state file. The override is now left out when the flag is omitted, so SAM keeps the stack's current value (a new stack still gets the template's unreserved default). The state file records the deployed value, and deploy prints the kept reservation. * fix(aws-lambda): keep progress working for callers without DescribeMapRun Callers whose IAM policy came from an earlier `lambda policies user` lack states:DescribeMapRun, so reading a running Distributed Map threw AccessDeniedException out of `lambda progress` and `render --wait` while the render itself went on to succeed. getRenderProgress now catches only that error around the map-run read, returns the parent snapshot (status, plan and assemble progress) and adds a warning to re-apply the policy, which both commands print. Other errors still throw. * fix(cli): let the documented deploy policy drop a reserved concurrency When a redeploy removes a function's reserved concurrency, CloudFormation calls lambda:DeleteFunctionConcurrency. The policy printed by `lambda policies role|user` granted only PutFunctionConcurrency, so the update failed and rolled back. Grant it, and pin it with states:DescribeMapRun in the required-actions test. * fix(aws-lambda): read smoke chunk hashes from the Distributed Map output With chunks in a Distributed Map, their task results live in child executions, so the smoke gate's chunk-hash check found 0 chunks in the parent history and failed every v1/v2 comparison. Read Chunks[] from the RenderChunks / RenderChunksV2 MapStateExited output instead, as getRenderProgress does. The stack this script deploys only has the Distributed Map, so the inline-Map path is gone. * fix(producer): cap a pinned chunk size at the worker concurrency, not 16 With --chunk-size or --target-chunk-frames and no --max-parallel-chunks, the plan still capped the render at 16 chunks however many workers the account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of 113 frames. The cap is now one wave across the worker concurrency, as the --max-parallel-chunks docs say; 16 remains when the concurrency is unknown. * feat(cli): remove a stack's reserved concurrency with --concurrency=none Since an omitted --concurrency now keeps the stack's reservation, there was no CLI way back to the unreserved default. --concurrency=none sends ReservedConcurrency=-1, the state file records the stack as unreserved and deploy says the reservation was removed. Positive N and an omitted flag behave as before; any other value is rejected instead of being silently ignored. * chore(aws-lambda): raise the package size budget for account-sized concurrency Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that. |
||
|
Miguel Ángel
|
70fbe32f8d |
perf(cloud): assemble chunk output with a single faststart index (#5812)
* perf(producer): assemble chunks that carry audio under one faststart index Reads each chunk's moov, merges the sample tables into one moov at the front, and writes the chunks' mdat payloads after it unchanged: the same bytes storage can copy into place. The assembler no longer needs the plan audio for such plans. * perf(aws-lambda): build the final mp4 inside S3 with its index up front The Lambda plan asks chunks to carry their audio. Assemble writes the faststart header as part 1 of a multipart upload and copies each chunk's mdat payload with UploadPartCopy, so the video bytes never pass through the function; any failure aborts the upload. * fix(engine): a chunk's first frame shows its own time, not the composition's opening frame * fix(producer): stitch silent AAC tails and clean failed outputs * fix(producer): open stitched chunks read-only with an explicit mode * chore(aws-lambda): raise the package size budget for S3-side MP4 assembly handler.js and index.js each bundle the new S3 stitch code (+5,550 bytes apiece) and s3Stitch.d.ts adds 970: +3,479 packed and +12,070 unpacked over main, measured with the CI pack. The budget rises by exactly that delta. * fix(aws-lambda): v1 plans give chunks their audio too, so both protocols match * perf(producer): plans whose chunks carry audio encode it once, at the frames' exact length * test(producer): terminate the plan audio ffprobe argv like every other probe |
||
|
James Russo
|
17a2a00ed5 |
feat(cloud): default distributed plans to v2 (#3311)
* feat(cloud): default distributed plans to v2 * fix(cloud): address plan v2 review feedback * fix(examples): document explicit v2 samples |
||
|
Vance IngallsandClaude Opus 5
|
349c066a83 |
fix(producer): classify JSON error bodies as non-media sources too
A source that answers with a JSON error body still reached ffprobe and
produced `moov atom not found`. Replicate returns
`{"detail": "requested file not found"}` for a dead asset, and a gateway
in front of it can relay that body with a success status.
The sniff now treats `<`, `{`, or `[` as the opening byte of a text
document. No supported container starts with any of them, so this is the
same trade as before: three bytes instead of an allowlist that grows one
entry per payload shape observed in production.
Renamed accordingly, since the class now covers JSON as well as markup:
MARKUP_NOT_MEDIA -> NOT_MEDIA_PAYLOAD, MarkupNotMediaError ->
NotMediaPayloadError, markupPayload.ts -> notMediaPayload.ts. Registry
entries in the Lambda name map, the CDK and SAM plan lists, the Cloud Run
set, and SAFE_RENDER_ERROR_CODES move with it.
Also documents the reachability boundary on the error class: only a 2xx
response gets here. `downloadToTemp` rejects 404/410 as `http_not_found`
before writing a byte, and every ffprobe input is local because
videoFrameExtractor downloads http srcs first. So the shapes this
classifies are soft-404 and interstitial HTML, S3/CloudFront error
documents, and JSON API error bodies -- each served with a success
status. A genuine 404 surfaces as a download failure, not as this error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
f3689c1481 |
fix(producer): scope and harden the markup-payload sniff
Review follow-up on the STUDIO-5433 defense.
Correctness
- The sniff ran above the documented video/audio failure split, so an
<audio> src that resolved to an HTML payload aborted the whole render
instead of degrading to duration 0. It now runs inside the same try, so
video surfaces the typed error while audio still drops out, with a
warning naming the element.
- Raw fs errors (EISDIR on a directory src, EACCES, the existsSync->open
ENOENT race, EMFILE) escaped and failed the compile with an unclassified
error carrying an unredacted temp path. The sniff is now a classifier that
never throws: an unreadable file reports "not markup" and the real probe
produces the real error.
- Elements whose duration the compiler never resolves (a data-end video, a
looping audio) skipped the sniff entirely, so the original ffprobe error
still escaped, and looping audio was reported as owner "system" after
every frame had been captured. Video is now caught in the asset preflight,
which sees every local src regardless of authored timing; audio is
classified per-element in audioMixer as source/invalid_media/owner "user",
keeping audio failures non-fatal as they already were.
- Detection is a byte-level check for a leading "<" (BOM-, whitespace- and
NUL-tolerant, looped read) instead of a <!doctype|<html|<?xml string
prefix, which missed a NUL-prefixed payload, >256B of leading whitespace,
UTF-16-encoded HTML, and a prolog-less <svg. No supported container starts
with "<", so the allowlist no longer grows per payload shape.
- finally { await fh.close() } could replace the in-flight typed error with
the close error.
Routing and privacy
- MARKUP_NOT_MEDIA is now in SAFE_RENDER_ERROR_CODES, the Lambda terminal
name map, the CDK and SAM non-retryable plan lists, and the Cloud Run
non-retryable set, and the class carries owner/retryable. Previously the
API emitted errorCode: undefined and a deterministic authoring bug burned
the full distributed retry budget.
- The message no longer carries 32 raw payload bytes or the src.
redactTelemetryString preserves host and path for HTTP srcs, so
per-tenant CDN paths reached a message the server forwards to clients.
Correlation is a sha256 element fingerprint, matching
AssetMediaTypeMismatchError.
- The message names both causes (unresolved nested-composition URL, or an
HTML/XML error page served as 200) rather than misdiagnosing an S3 403
body as an authoring bug.
Tests
- Byte-level detection is unit-tested in engine: markup shapes, BOMs,
UTF-16, nine container signatures, unreadable inputs.
- Replaced the tautological assertions. The old checks for "html" in and
"moov" absent from a fixed message template could not fail for any input.
- New coverage for audio degradation, the audioMixer classification, the
preflight video/image/audio split, and the API error metadata.
- The sibling htmlCompiler.mediaType failure was a vitest-under-bun runner
mismatch, not a missing ffmpeg binary. It passes, including the 4-wide
probe-semaphore invariant the sniff now runs inside.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
James Russo
|
1d636f603c |
refactor(producer): share plan execution builder (#2906)
## What Refactor distributed planning around one shared local execution-plan builder: - `buildLocalExecutionPlan()` now owns compile/probe/extract/audio/freeze. - Legacy `plan()` remains a deprecated v1 transport wrapper. - Plan v2 calls the shared builder directly and publishes through the existing manifest/CAS contract. - Add neutral `createPlanV2FromExecutionPlan()`, `publishPlanV2FromExecutionPlan()`, `getPlanV2ExecutionPlanHash()`, and `PLAN_PROTOCOL_V1` names. - Retain deprecated v1-named exports and wire aliases. - Recommend explicit Plan v2 opt-in for new producer, Lambda, and Cloud Run integrations. ## Why Plan v2 previously looked like it invoked a v1 planner even though v1 and v2 share the same frozen local execution representation. This removes that migration-era coupling while preserving the public minor-version compatibility contract. ## How The shared builder returns neutral internal execution-plan fields. The v1 wrapper maps those fields back to the existing `PlanResult`; the v2 publisher consumes them directly. Compatibility is intentional and covered by exact shape tests: - omitted `planProtocol` still serializes/selects `"v1"`; - v1 layouts, descriptor-less decoding, event unions, workflow branches, and exports remain; - the v1 descriptor JSON is byte-identical and `CURRENT_PLAN_PROTOCOL` is an identity-preserving alias; - v2 manifest bytes, key order, hash framing, and `sourcePlanV1Hash` wire key remain unchanged; - no enumerable neutral hash field was added to manifests or returned result objects; - v1/v2 result objects, cloud event payloads, and SDK handle key sets remain unchanged. ## Test plan - Focused Plan v1/v2/protocol/export/size compatibility: 141 passed - `@hyperframes/core`: 1,419 passed - `@hyperframes/producer` unit lane: 990 passed - `@hyperframes/aws-lambda`: 140 passed - `@hyperframes/gcp-cloud-run`: 101 passed - Producer, Lambda, and Cloud Run typechecks - Repository-wide lint, format check, workspace/package-subpath checks - Full workspace build - `git diff --check` - [x] Unit tests added/updated - [ ] Manual testing performed - [x] Documentation updated (if applicable) |
||
|
James Russo
|
557d82b6a9 |
fix(producer): validate distributed video metadata (#2839)
## What - enforce a finite, validated `meta/videos.json` contract shared by Plan v1 and Plan v2 - preserve authored finite ends and source-derived trim-aware ends; bound any still-open end at the validated composition end - fail distributed planning when any declared video source did not extract instead of publishing a blank-capable plan - make the v1 chunk reader reject malformed/null video timing before frame injection - route deterministic video-source/metadata failures as non-retryable in AWS and GCP while retaining retries for transient extraction failures ## Why An open-ended video whose remote source could not be resolved retained `Infinity` through planning. Plan v2 correctly rejected that value, while Plan v1 serialized it as `null`; the v1 frame lookup could then suppress injected frames and silently produce incorrect output. The invariant belongs at the shared metadata boundary. Both protocols must receive identical finite timing, and unavailable sources must fail closed before plan publication. ## Test plan - [x] producer distributed planning, metadata, v1 chunk boundary, Plan v2 conversion/materialization, and public exports - [x] core runtime media semantics (authored slots, natural duration, looping, non-looping hold) - [x] engine video extraction and frame lookup - [x] AWS Lambda/CDK/SAM and GCP Cloud Run error normalization/retry classification - [x] producer, core, engine, AWS, and GCP typechecks/builds - [x] formatting, oxlint, tracked-artifact, fallow, and commit hooks - [x] exact incident composition replayed through the AWS Lambda handler's Lambda-local path in a Lambda-like container; Plan v1 and Plan v2 both fail closed as `VIDEO_SOURCE_UNRENDERABLE` during planning, before plan publication - [x] full PR CI, including all nine regression shards and Windows render/tests No production flags or deployment/release workflows are changed. |
||
|
James Russo
|
5bf61d6df0 |
feat(aws-lambda): support plan protocol v2 (#2789)
* feat(aws-lambda): support plan protocol v2 * fix(aws-lambda): align SAM v2 terminal errors |
||
|
James Russo
|
aa58ebc048 |
feat(lambda): add real-AWS smoke + benchmark script (#908)
* feat(lambda): add real-AWS smoke + benchmark script
Phase 6.3 of the distributed rendering plan
(DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6a). Local bash script that
deploys the PR 6.2 SAM template to your own AWS account, renders a
fixture composition through the Step Functions state machine at several
chunk counts, PSNR-compares each output against the in-process
baseline, and tears the stack down.
This is the gate that proves the architecture works on real Lambda
infrastructure. After it runs green and the numbers are good, Phase 6b
(CLI, CDK, docs) can proceed with confidence.
Script lives at examples/aws-lambda/scripts/smoke.sh. Defaults:
- fixture: mp4-h264-sdr
- chunk_counts: 2,4,8
- psnr-threshold: 50 dB
- region: us-east-1
- stack-name: hyperframes-lambda-smoke-<timestamp>
AWS credentials come from the standard resolution chain (env vars,
~/.aws/credentials, SSO, IMDS). Pin a specific profile via --profile or
AWS_PROFILE; the script doesn't ship a default.
Workflow:
1. Pre-flight: verify aws/sam/bun/ffmpeg/jq/zip on PATH; check
credentials via sts:GetCallerIdentity.
2. Build the PR 6.1 ZIP and run verify:zip-size.
3. sam validate --lint + sam deploy under a per-run stack name.
4. Zip the fixture's src/ and upload to the render bucket.
5. For each chunk count, start a Step Functions execution, poll for
completion (25-min cap), download the output mp4 and the execution
history JSON, ffmpeg-psnr against the LFS-tracked in-process
baseline, and append to results.json.
6. Gate on the PSNR threshold.
7. Empty the bucket + sam delete (unless --keep-stack).
Outputs land under ./lambda-smoke-artifacts/:
- results.json (chunkCount x wallClockMs x psnrAvgDb)
- renders/N<N>-output.mp4
- renders/N<N>-history.json (full Step Functions execution history)
Per-run stack name with concurrency-safe AWS resource isolation. Run
multiple smokes in parallel without races; teardown guards against
stale stacks via cleanup_and_exit on every failure path.
Distinction from CI: this is a maintainer-run gate, not part of regular
CI. The architecture's per-PR safety net is the local Docker-based
BeginFrame probe (PR 6.1) and the upcoming Lambda RIE smoke mode (PR
6.6). No GitHub Actions / OIDC / cross-account secrets required.
This is part of the 8-PR Phase 6 stack; PR 6.3 of 8 — the last PR of
Phase 6a (validation). Phase 6b (CLI + CDK + docs) starts once 6.3's
benchmark numbers come back.
* fix(lambda): address PR 880 review feedback
- Document wall-clock methodology bias inline (eval.sh header + README):
local timing includes bun + tsx + harness scaffolding while Lambda
timing measures pure SFN execution, so "speedup" is end-to-end CLI
experience, not renderer-vs-renderer.
- Add --iterations N (default 1) with median wall-clock reporting via
awk-side median. Cold-start variance is ±5-10s per chunk; single-
sample readings made the PR-body speedup table not ground truth.
- Add --reserved-concurrency flag to both scripts; default still 16 but
no longer hardcoded. Pass-through to ReservedConcurrency CFN param.
- README: cost-per-pass estimate for both scripts.
- Replace `sed -n '2,30p' "$0"` help with usage() heredoc in both
scripts — fragile to header reflows and didn't survive the comment
expansion this commit adds anyway.
- eval.sh RMS-level parser: add a third fallback (`RMS level:` with no
`dB` suffix) for older ffmpeg builds where astats predates the unit
tag. Word-boundary guards keep `RMS peak level` from being eaten.
* docs(lambda): drop internal plan-doc + Rio refs from smoke/eval scripts
|
||
|
James Russo
|
6664e54053 |
feat(lambda): add SAM template and sample events for AWS deployment (#907)
* feat(lambda): add SAM template and sample events for AWS deployment Phase 6.2 of the distributed rendering plan (DISTRIBUTED-RENDERING-PLAN.md §15). Reference SAM template for deploying HyperFrames distributed rendering on AWS — one Lambda function in three roles, choreographed by a Step Functions standard workflow with a Map state for parallel chunk rendering. Resources created by the template: - Lambda function pointing at the Phase 6.1 ZIP - Step Functions state machine: Plan -> Map(N) RenderChunk -> Assemble - S3 bucket for plan tarballs, chunk outputs, final mp4 - IAM role for the state machine - CloudWatch alarm guarding against runaway chunk invocations Retry policy: 4 attempts, 2s initial, 2x backoff, max 60s, with the typed non-retryable error codes from plan §9.3 explicitly opted out. CodeUri points at packages/aws-lambda/dist/handler.zip; sam deploy resolves the local path and uploads to a SAM-managed bucket on first deploy. Validated: sam validate --lint passes against the template. This is part of the 8-PR Phase 6 stack; PR 6.2 of 8. * fix(lambda): address PR 879 review feedback - Add CloudWatch alarms for Lambda Errors metric (5min window, threshold 1) and Step Functions ExecutionsFailed metric. The existing runaway- invocations alarm catches too-many-calls but missed silent per-chunk failures and retry-exhaustion. - Document VersioningConfiguration: Suspended tradeoff inline. Adopters treating the final mp4 as user-keepable should bump to Enabled. - Cost-allocation Tags on RenderBucket + Lambda Globals. - Lambda Tracing: Active so X-Ray spans don't terminate at the SF→Lambda boundary (the state machine already had tracing). - State-machine top-level TimeoutSeconds: 3600 as defensive ceiling on the whole choreography — catches Plan-retry storms before they hit individual task budgets. - AssertChunkCount Choice state: if Plan ever returns ChunkCount=0 the Map would silently iterate zero times and Assemble would receive an empty ChunkS3Uris[] producing an empty output. Fail-fast with typed PLAN_TOO_LARGE error instead. - Architecture comment: explicit x86_64-only constraint from @sparticuz/chromium so adopters trying Graviton don't get bitten. * docs(lambda): drop internal plan-doc refs from SAM example + template |