mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-19 11:26:02 +00:00
579feb02eefc0b50988634d9d9c44cc02ef14dea
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
ed3f05a05b |
fix(aws-lambda): pin the shell ZIP route, gate ZIP headroom, match arm64 fonts to x86 (#8170)
- build-zip --source=chrome-headless-shell stages only the repo's pinned build (managedChromeVersion) and only when its folder matches the pinned digest (isPinnedHeadlessShell), instead of the newest folder in the cache. The folder cannot fit a ZIP function, so lambda deploy refuses --chrome-source chrome-headless-shell up front on either arch, and the docs say the arm64 video image is where the shell runs. - build-zip prints the headroom under the unzipped gate, and a new "Lambda handler ZIP" workflow builds the ZIP on PRs that touch the code it bundles, so growth fails before merge rather than in the release job. - The arm64 image keeps @sparticuz/chromium's fonts and fonts.conf from the handler ZIP (FONTCONFIG_PATH=/var/task/fonts) instead of liberation-fonts, so text falls back to the same fonts as on x86. The image workflow renders a mixed-script page on both arches and fails when any line resolves to different platform fonts. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
81d73d2278 |
fix(aws-lambda): chrome-headless-shell 158 in the arm64 video image, sparticuz 153 in the x86 ZIP (#8155)
* fix(aws-lambda): run chrome-headless-shell 158 in the arm64 video image The arm64 video image now installs Chrome for Testing's linux-arm64 chrome-headless-shell 158.0.8127.36, pinned by sha256 and fetched with curl -f, as the whole shell folder. The Lambda Node 22 base has none of the shared libraries the shell links and no fonts, so the image installs them from the base's repository at the same pinned release. The image sets HYPERFRAMES_LAMBDA_CHROME_SOURCE and HYPERFRAMES_LAMBDA_CHROME_PATH itself, and the nested video function no longer passes ChromeSource, which would override them. stage-arm64-image.ts stops fetching the sparticuz arm64 pack and drops the ZIP's x86 sparticuz Chrome instead. The x86 handler ZIP stays on @sparticuz/chromium, bumped from 148.0.0 to 153.0.0. It builds at 258,502,222 bytes unzipped, under Lambda's 262,144,000-byte limit and the 248 MiB in-house gate. Two fixes for --source=chrome-headless-shell: - build-zip.ts staged only the executable, which exits at launch without its ICU data and .pak files. It now stages the whole shell folder at chrome-headless-shell-linux64/. - Nothing set HYPERFRAMES_LAMBDA_CHROME_PATH, which src/chromium.ts requires for that source. The SAM template and the CDK stack now set it to the bundled shell when ChromeSource asks for it. Drops the unused @types/aws-lambda devDependency, which the dead-code audit reports whenever this package's manifest changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(publish): build the Lambda arm64 video image on an ARM runner The video stage now installs Chrome's system libraries with microdnf, so it runs code for arm64. The pack job built that image on ubuntu-latest with no emulation, which it could do only while the stage copied files. A new lambda-video-image job builds it natively on ubuntu-24.04-arm, from the handler zip whose sha256 the zip job reported, and reports the OCI tarball's sha256. Pack downloads that tarball, checks the sha256 against the job's, and pins it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(aws-lambda): fit the comment ratchet and the package size budget chromium.ts loses the two doc lines on BUNDLED_SHELL_DIR and BUNDLED_SHELL_PATH; the names and the SAM template's comment already say what they are. @hyperframes/aws-lambda's unpacked budget goes from 290,508 to 298,000 bytes. The package unpacks to 295,324 bytes, up from the shared chrome-headless-shell staging module the ZIP and image scripts now import and from the README and docs that describe the two Chrome sources. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7930525024 |
build(chrome): pin chrome-headless-shell 158.0.8127.36 wherever the repo installs it (#8145)
The engine ceiling moves from Beta 152.0.7977.30 to stable 158.0.8127.36, with its Chrome for Testing digests. The places that installed `@stable` or an older pin follow it: the CI test workspace, the regression test image, the k8s and Cloud Run examples, the CLI render image (amd64) and the Lambda build hint. The motion-blur nightly keeps its pinned-versus-stable comparison: the pin leg now uses the workspace's pinned shell, and a stable leg fetches stable. The chat regression golden is regenerated on 158. Chrome 157 renders it pixel-identically to 158, so the previous golden recorded Chrome 152's paint of a background-clip:text phrase whose word spans start at opacity 0. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f9255e39a5 |
fix(aws-lambda): the handler ZIP installs its runtime from a committed lockfile with scripts off (#7485)
* fix(aws-lambda): the handler ZIP installs its runtime from a committed lockfile with scripts off build-zip.ts ran `npm install --no-package-lock` in the publish job, so transitive versions floated at release time and any dependency's install script ran beside the release credentials. The ZIP now runs `npm ci --ignore-scripts` from runtime-lock/<source>/package-lock.json (versions and integrity for every package), and the build fails if that lock disagrees with bun.lock instead of falling back to a range or "latest". `bun packages/aws-lambda/scripts/lock-runtime.ts` regenerates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(aws-lambda): check only direct runtime versions against bun.lock; Renovate refreshes the rest The runtime lock's transitive tree is npm's own resolution, not bun.lock's (modern-tar 0.8.6 there vs 0.8.4 in bun.lock), so the parity claim now covers puppeteer-core and @sparticuz/chromium only. Renovate's weekly lockFileMaintenance refreshes the transitive tree; its direct-version bumps are off for these dirs because those follow bun.lock. — Rames Jusso Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(aws-lambda): keep the maintainer-only lock-runtime script out of the package Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b501dc4e1b |
feat(cli): lambda deploy works from the npm install, no repo checkout (#5866)
* feat(cli): lambda deploy works from the npm install, no repo checkout Each release builds the Lambda handler zip on Linux, attaches it to the GitHub release, and pins its URL and sha256 (plus the SAM template) in @hyperframes/aws-lambda before npm publish. Outside a checkout, lambda deploy downloads that zip, refuses it on a sha256 mismatch, and deploys it with the packaged template. With neither a checkout nor a pinned zip it fails up front with one message naming every way out. * fix(cli): lambda deploy test mocks the pinned-handler reader, comments fit the checks * test(cli): sam tests deploy from a SAM directory, not the repo root * chore(aws-lambda): raise the package size budget for the pinned handler template Measured with pnpm@10.17.1 pack on Linux. Release state (prebuilt/ written by pin-handler-zip): 77,307 packed / 285,654 unpacked. CI state (no prebuilt/): 70,104 / 253,390. Packed is the check's 5% headroom over the release size. Unpacked is 292,000 (2.2% over release) because a 5% budget would put the CI pack under the 85% ratchet. * test(cli): anchor the docs link check in the lambda deploy fallback test * fix(cli): lambda deploy logs the verified handler sha256 and keeps a presigned query out of errors * fix(cli): a redirect-limit download error keeps a signed query out of its message * test(cli): the redirect-limit leak test types its rejection * docs(aws-lambda): header comments cut to what the code cannot say * chore(aws-lambda): size budget fits both the CI and the release package |
||
|
|
2a14cbc924 |
perf(aws-lambda): size distributed renders from account concurrency limits (#5813)
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap The inline Map runs at most 40 chunk renders at once. Both state machine definitions (CDK and SAM) now use a Distributed Map with STANDARD child executions, capped at the function's reserved concurrency so chunks past it queue instead of being throttled. Progress reads each chunk's child execution history so frames, cost and chunk errors still show up. * fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks A throttled task used to burn its four catch-all retries in about 30 s. It now gets its own jittered backoff of several minutes, because another render on the same function can still hold the reserved slots. Plan treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions. * chore(aws-lambda): trim stack comments to keep the comment share flat * fix(aws-lambda): keep render --wait working at hundreds of chunks Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of every child execution's history, take exact chunk frames and cost from the Map output once it exits, and read failed children only when the counts show a failure. The SDK client retries throttled calls up to 8 times with backoff. * feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8 * fix(lambda): keep the concurrency SDK external in published modules * fix(cli): keep an existing stack's reserved concurrency on redeploy Redeploying without --concurrency sent ReservedConcurrency=-1, which removed the cap of 8 that earlier CLI versions reserved by default and stopped recording it in the state file. The override is now left out when the flag is omitted, so SAM keeps the stack's current value (a new stack still gets the template's unreserved default). The state file records the deployed value, and deploy prints the kept reservation. * fix(aws-lambda): keep progress working for callers without DescribeMapRun Callers whose IAM policy came from an earlier `lambda policies user` lack states:DescribeMapRun, so reading a running Distributed Map threw AccessDeniedException out of `lambda progress` and `render --wait` while the render itself went on to succeed. getRenderProgress now catches only that error around the map-run read, returns the parent snapshot (status, plan and assemble progress) and adds a warning to re-apply the policy, which both commands print. Other errors still throw. * fix(cli): let the documented deploy policy drop a reserved concurrency When a redeploy removes a function's reserved concurrency, CloudFormation calls lambda:DeleteFunctionConcurrency. The policy printed by `lambda policies role|user` granted only PutFunctionConcurrency, so the update failed and rolled back. Grant it, and pin it with states:DescribeMapRun in the required-actions test. * fix(aws-lambda): read smoke chunk hashes from the Distributed Map output With chunks in a Distributed Map, their task results live in child executions, so the smoke gate's chunk-hash check found 0 chunks in the parent history and failed every v1/v2 comparison. Read Chunks[] from the RenderChunks / RenderChunksV2 MapStateExited output instead, as getRenderProgress does. The stack this script deploys only has the Distributed Map, so the inline-Map path is gone. * fix(producer): cap a pinned chunk size at the worker concurrency, not 16 With --chunk-size or --target-chunk-frames and no --max-parallel-chunks, the plan still capped the render at 16 chunks however many workers the account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of 113 frames. The cap is now one wave across the worker concurrency, as the --max-parallel-chunks docs say; 16 remains when the concurrency is unknown. * feat(cli): remove a stack's reserved concurrency with --concurrency=none Since an omitted --concurrency now keeps the stack's reservation, there was no CLI way back to the unreserved default. --concurrency=none sends ReservedConcurrency=-1, the state file records the stack as unreserved and deploy says the reservation was removed. Positive N and an omitted flag behave as before; any other value is rejected instead of being silently ignored. * chore(aws-lambda): raise the package size budget for account-sized concurrency Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that. |
||
|
|
2b1109db4c |
test(lambda): behavioral guard for handler CJS banner shims (#1977)
Miguel's fix (
|
||
|
|
f999b40d73 | fix(lambda): define dirname in handler bundle | ||
|
|
9b34114dd1 |
fix(aws-lambda): fail build-zip when ffmpeg-static binary is not Linux x86-64
`ffmpeg-static` materializes a single platform-selected binary at install time. By default that follows the install host, so a default macOS/arm64 or linux/arm64 install can stage a binary that Lambda's x86-64 runtime cannot execute. Verify the ELF header (ELFCLASS64, EM_X86_64) before copying the binary into `bin/ffmpeg`. When the check fails, surface the canonical workarounds: run the build inside a linux/amd64 container, or pre-install with `npm_config_platform=linux npm_config_arch=x64`. In the distributed pipeline this mismatch can fail as early as Plan's `ffmpeg -version` probe after browser probing and before chunks are scheduled. Later encode/assemble spawns would fail for the same reason. ffprobe already goes through the platform-segmented `ffprobe-static/bin/linux/x64` path, so that side doesn't need the same check. |
||
|
|
c50f59a53b |
feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe (#878)
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).
This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.
Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.
The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.
Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.
This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.
* fix(lambda): address PR 878 review feedback
- Verify event.PlanHash against the untarred plan.json at the handler
boundary before invoking the producer primitive. Throws typed
PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
non-retryable; previously the field was schema bloat the handler
ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
but use binary mebibytes); previously mixed units made the 248 MiB
budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
comparison instead of lexicographic sort+reverse — the latter would
silently pick "99.x" over "131.x" once Chrome cached three-digit
majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
imports — file is ESM and the same module is already imported.
* docs(lambda): drop internal plan-doc refs from package README
* ci(windows): fix bun filter UNION bug excluding producer from Windows tests
`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.
Enumerate the packages we DO want to test instead.
|