mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-02 03:36:52 +00:00
v0.8.213
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
0df088926e |
feat(aws-lambda): push a prebuilt image to ECR without Docker (#6063)
* feat(aws-lambda): push a prebuilt image to ECR without Docker Adds pushImageToEcr and ensureEcrRepository to the SDK subpath. The push reads a single-platform OCI image layout, verifies every blob against its descriptor before any ECR call, uploads only the blobs ECR lacks through the layer-upload API, and puts the manifest under a tag, returning the digest-pinned image URI. * fix(aws-lambda): keep the ECR client out of the SDK bundle Like the other AWS SDK clients, @aws-sdk/client-ecr is a dependency the consumer resolves. Bundled into the ESM SDK it hits a dynamic require of node:https and the packed package fails to import. * feat(aws-lambda): build the ECR client from a region like the other SDK calls * build(aws-lambda): add only the ECR client to the lockfile * fix(aws-lambda): adopt an existing ECR repository only when it carries the caller's tags * chore(aws-lambda): raise the size budget for pushing an image to ECR pushImageToEcr ships in dist (unminified chunk plus its .d.ts); no source maps, tests or release-only files are involved. Built dist, npm pack --dry-run, main versus main plus this branch: +2486 B packed, +8846 B unpacked. The merge-group run measured 73369 packed and 266805 unpacked, which are the new budgets (were 70874 and 258166). |
||
|
Miguel Ángel
|
ccc13c3b5a |
fix(aws-lambda): the size check on every PR measures the package a release publishes (#6017)
* fix(aws-lambda): the size check on every PR measures the package a release publishes * style(aws-lambda): format the prebuilt template helper |
||
|
Miguel Ángel
|
992df8b203 |
perf(cloud): move project bytes once through shared content storage (#5772)
* perf(producer): distributed chunks download only the videos they show A project video under compiled/ that chunks extract their own frames from now goes only to the chunks that show it, like video-sources/ files. A chunk skips the readiness wait for a deferred video it does not show, so it no longer logs media_load_failed for a file it was never given. * fix(producer): cloud site uploads pack the files their id hashes, symlinks included * perf(aws-lambda): chunks read only the byte ranges of a video they need A chunk that extracts its own frames from a deferred project video now reads it from S3 by byte range through a short-lived pre-signed URL, instead of downloading and hashing the whole object first. S3 attests the object's stored SHA-256 before the URL is signed; every other artifact is still downloaded and verified as before. * perf(aws-lambda): upload project files once to a shared content-addressed area * chore(producer): keep planV2 comment share at the base level * fix(engine): verify the TLS certificate of https video inputs and never surface their signed query * fix(aws-lambda): renders re-upload a site's expired files instead of failing * perf(aws-lambda): the plan step probes project videos in place instead of downloading them * fix(engine): https video reads reconnect and time out, and redaction covers wrapped causes Signed-URL reads now pass -rw_timeout, -reconnect and -reconnect_delay_max (both Lambda builds accept them) and ffmpeg also -reconnect_on_network_error, so a transient S3 error or a stalled read fails fast or recovers instead of waiting out the process timeout. Query redaction now descends into error.cause. * fix(aws-lambda): share entry bundles and ratchet package budget * fix(aws-lambda): site ids uploaded before files.json still render A site id from an earlier sites create has only sites/<id>/project.tar.gz. renderToLambda now falls back to that tarball, which the plan step still reads, and when neither key exists it fails before starting with both keys named and the fix (run sites create again). * fix(aws-lambda): renders fail before upload on a stack too old to read sites A stack deployed before files.json sites untars the site file in its plan step and fails after four retries. renderToLambda now reads the state machine definition first and stops with STACK_TOO_OLD and the redeploy command when the plan task does not list SITE_MANIFEST_INVALID, which new SAM and CDK stacks now treat as terminal. The README's upgrade section drops the planProtocol v1 fallback, which hits the same failure. * docs(aws-lambda): document shared content storage and the upgrade path The deploy guide still described the tarball HeadObject short-circuit. It now covers the cas/sha256/ store and its 30-day lifecycle rule, the STACK_TOO_OLD, SITE_FILES_MISSING and SITE_NOT_FOUND errors, that old site ids keep working, and that every project uploads once more after the upgrade. A test keeps a troubleshooting entry for each pre-submit error. * fix(producer): chunks log each video they read by URL, without its query A chunk reading a video straight from the bucket left no trace in its log. It now logs one line per remote source with the path and the URL minus its query string, so signatures never reach the log. * fix(engine): ffmpeg errors never keep part of a signed URL's query A process's stderr tail was trimmed before its URLs were redacted, so a cut inside a signed URL left a query fragment the URL pattern no longer matched, and the fragment reached the error. The stderr tail is now redacted before every trim, and the redactor also strips a signing parameter cut off from its URL. * fix(aws-lambda): comments name the site manifest instead of a file path * fix(engine): stderr redaction survives chunk splits, and every ffmpeg spawn uses it The stderr tail was redacted on each chunk and then cut, so once a cut severed a long URL's scheme, the rest of its signature in the next chunk matched neither pattern. The tail is now kept raw with 2x slack and redacted only when the text is produced, then cut to size. The unused producer audioExtractor (its own spawn, unredacted stderr) is deleted and the parity harness's PCM decode goes through ManagedChildProcess; a test fails on any ffmpeg/ffprobe spawn in engine, producer or aws-lambda that does not. * chore(producer): record the audioExtractor deletion as intended * fix(engine): drop a signature fragment the stderr tail cut from its URL The raw tail is kept at twice the cap, but that cut can itself land inside a signature, and when redaction shrinks the rest below the cap the fragment survived. After a cut, the severed token is dropped before redaction. * refactor(engine): name the severed-token drop instead of commenting it * test(engine): a cut-off URL leaves nothing behind; aws-lambda size budget to measured * fix(producer): chunk URL-read tests use main's frame list; producer size budget to measured * chore(producer): size budget at the measured size, no headroom * chore(producer): size budget back to measured x1.05, the check's own rule |
||
|
Miguel Ángel
|
2a14cbc924 |
perf(aws-lambda): size distributed renders from account concurrency limits (#5813)
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap The inline Map runs at most 40 chunk renders at once. Both state machine definitions (CDK and SAM) now use a Distributed Map with STANDARD child executions, capped at the function's reserved concurrency so chunks past it queue instead of being throttled. Progress reads each chunk's child execution history so frames, cost and chunk errors still show up. * fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks A throttled task used to burn its four catch-all retries in about 30 s. It now gets its own jittered backoff of several minutes, because another render on the same function can still hold the reserved slots. Plan treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions. * chore(aws-lambda): trim stack comments to keep the comment share flat * fix(aws-lambda): keep render --wait working at hundreds of chunks Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of every child execution's history, take exact chunk frames and cost from the Map output once it exits, and read failed children only when the counts show a failure. The SDK client retries throttled calls up to 8 times with backoff. * feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8 * fix(lambda): keep the concurrency SDK external in published modules * fix(cli): keep an existing stack's reserved concurrency on redeploy Redeploying without --concurrency sent ReservedConcurrency=-1, which removed the cap of 8 that earlier CLI versions reserved by default and stopped recording it in the state file. The override is now left out when the flag is omitted, so SAM keeps the stack's current value (a new stack still gets the template's unreserved default). The state file records the deployed value, and deploy prints the kept reservation. * fix(aws-lambda): keep progress working for callers without DescribeMapRun Callers whose IAM policy came from an earlier `lambda policies user` lack states:DescribeMapRun, so reading a running Distributed Map threw AccessDeniedException out of `lambda progress` and `render --wait` while the render itself went on to succeed. getRenderProgress now catches only that error around the map-run read, returns the parent snapshot (status, plan and assemble progress) and adds a warning to re-apply the policy, which both commands print. Other errors still throw. * fix(cli): let the documented deploy policy drop a reserved concurrency When a redeploy removes a function's reserved concurrency, CloudFormation calls lambda:DeleteFunctionConcurrency. The policy printed by `lambda policies role|user` granted only PutFunctionConcurrency, so the update failed and rolled back. Grant it, and pin it with states:DescribeMapRun in the required-actions test. * fix(aws-lambda): read smoke chunk hashes from the Distributed Map output With chunks in a Distributed Map, their task results live in child executions, so the smoke gate's chunk-hash check found 0 chunks in the parent history and failed every v1/v2 comparison. Read Chunks[] from the RenderChunks / RenderChunksV2 MapStateExited output instead, as getRenderProgress does. The stack this script deploys only has the Distributed Map, so the inline-Map path is gone. * fix(producer): cap a pinned chunk size at the worker concurrency, not 16 With --chunk-size or --target-chunk-frames and no --max-parallel-chunks, the plan still capped the render at 16 chunks however many workers the account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of 113 frames. The cap is now one wave across the worker concurrency, as the --max-parallel-chunks docs say; 16 remains when the concurrency is unknown. * feat(cli): remove a stack's reserved concurrency with --concurrency=none Since an omitted --concurrency now keeps the stack's reservation, there was no CLI way back to the unreserved default. --concurrency=none sends ReservedConcurrency=-1, the state file records the stack as unreserved and deploy says the reservation was removed. Positive N and an omitted flag behave as before; any other value is rejected instead of being silently ignored. * chore(aws-lambda): raise the package size budget for account-sized concurrency Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that. |
||
|
James RussoandClaude Opus 5.5
|
f55b366175 |
fix(build): ship NodeNext-resolvable declarations from every package (#5532)
* fix(build): ship NodeNext-resolvable declarations from every package tsc copies relative specifiers verbatim, so packages whose sources use bundler-style extensionless imports (core, inspector) published .d.ts files that a module/moduleResolution nodenext consumer cannot resolve. Core already rewrote its emitted JavaScript for this, but not its declarations. Move that rewrite to scripts/rewrite-relative-import-extensions.mjs, make it cover .d.ts too (adding .js or /index.js, and failing the build when a declaration import has no declaration), and run it after every tsc emit: core, engine, sdk, producer, aws-lambda, gcp-cloud-run and inspector. verify:packed-manifests now typechecks the clean packed consumer with skipLibCheck false and type-only imports of every typed subpath, so an unresolvable import inside a published declaration fails the gate and is attributed to its package. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * chore(ci): allow the moved core rewrite-esm-extensions script deletion Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: rebuild the edit-accuracy bench when the import-extension rewriter changes The package builds now run scripts/rewrite-relative-import-extensions.mjs, so main's edit_accuracy paths filter has to include it (check-edit-accuracy-filter test). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
bb9e507e84 |
perf(packages): ship npm packages without source maps and guard their size in CI (#5403)
* perf(packages): ship npm packages without source maps and guard their size in CI No package or consumer reads the published source maps, so they are no longer built. The CLI copies only the studio app it serves instead of the whole studio dist. verify:packed-manifests now holds every published package to a size budget that ratchets down, and CI rejects pull requests that add a large binary outside LFS. * fix(ci): large-binary check reads every filename and fails on git errors * ci: run only Build when a skill bundled into the CLI changes * fix(ci): size guards also catch type changes, odd file names, unreadable blobs and bad budgets * fix(ci): large-binary check reads blobs by id and skips submodules * fix(ci): exempt only exact registry copies under docs/public/catalog * test(ci): cover the staged catalog-copy exemption; drop the local-only override hint in CI * fix(ci): producer budget follows its slimmed package; size checks name their real cause |
||
|
James
|
4051899533 |
chore(lambda): publish-readiness for @hyperframes/aws-lambda
The Lambda adapter has been on `main` since PR #909 but its package manifest still shipped TypeScript source (`main: ./src/index.ts`, `build: tsc --noEmit`, `version: 0.0.1`) and the publish workflow didn't list it. This wires it up to publish alongside the other `@hyperframes/*` packages on the next `v*` tag. Changes: - **packages/aws-lambda/build.mjs (new)** — mirrors `packages/producer/build.mjs`: esbuild bundles four entry points (`src/index.ts`, `src/handler.ts`, `src/sdk/index.ts`, `src/cdk/index.ts`) → `dist/`, then `tsc --emitDeclarationOnly` emits .d.ts via `tsconfig.build.json`. All runtime/peer deps (@aws-sdk/*, @hyperframes/producer*, @sparticuz/chromium, aws-cdk-lib, constructs, ffmpeg-static, ffprobe-static, puppeteer-core, tar) are external so consumers resolve them through their own node_modules. - **packages/aws-lambda/tsconfig.build.json (new)** — drops the workspace `paths` overrides so `@hyperframes/producer*` resolves through node_modules to producer's already-built `dist/` types instead of pulling its full source tree into emit (which would violate `rootDir`). - **packages/aws-lambda/tsconfig.json** — keeps `noEmit: true` + workspace `paths` for fast in-place typechecks; also excludes `src/**/__fixtures__/**` so test-only helpers (fakeS3) don't leak into emitted declarations. - **packages/aws-lambda/package.json**: * version bumped 0.0.1 → 0.6.18 (matches the repo's lockstep release cadence) * main / types / exports map points at `dist/...` * files: ["dist/", "scripts/", "README.md"] (scripts/ kept whole because build-zip.ts and verify-zip-size.ts both import scripts/_formatBytes.ts) * scripts.build = `node build.mjs` - **package.json** — root `build` filter includes `aws-lambda` so `bun run build` builds it in topological order after producer. - **.github/workflows/publish.yml** — one new `publish_pkg "@hyperframes/aws-lambda" "@hyperframes/aws-lambda"` line. First publish is automatic via the `--access public` flag in `publish_pkg`; the @hyperframes scope already owns the name. Verification: bun run build # full root build green bun run verify:packed-manifests # aws-lambda passes pnpm pack packages/cli # @hyperframes/aws-lambda # rewrites workspace:* → 0.6.18 npm install -g <cli-tgz> # smoke-install still works hyperframes lambda deploy # friendly missing-package # error still fires when # aws-lambda isn't installed |