mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-18 16:08:56 +00:00
39347da9de71923d4bd9f1acd3c8b6f28368a4d7
24
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
39347da9de |
fix(cli): leave files from outside the project out of Lambda and Cloud Run site uploads (#8060)
* fix(cli): leave files from outside the project out of Lambda and Cloud Run site uploads A site upload for `lambda` or `cloudrun` followed every symlink in the project and shipped what it pointed at, wherever that was on the machine. `publish` and `cloud render` already ask first. Site uploads now leave a file or folder linked in from outside the project out and name each one; `--include-outside-files` (or `includeOutsideFiles` in the SDK's deploySite, renderToLambda and renderToCloudRun) uploads them. publishProjectArchive called without a prepared archive now leaves outside files out too, since nothing approved them. — Rames Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(producer): drop the new option comments the comment ratchet refuses Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2d3033c719 |
fix(cli): exit 2 for remaining flag validation errors (#6639)
* fix(cli): exit 2 for remaining flag validation errors Flag and argument checks that commands make themselves still exited 1 and reported runtime_error under --json. They now take the existing usage-error path (failUsage / CliUsageError): exit 2 and a usage_error envelope. This covers lambda and cloudrun render/render-batch (--width/--height, --fps, positive-number, enum and --output-resolution flags, unknown subcommands), capture, feedback, benchmark --runs, tts, init, snapshot, compare, grade-compare, check, normalize-audio, media-treatment, play/present --port, models, docs, browser and telemetry unknown subcommands, cloud render conflicting inputs, cloud delete without --no-confirm, and inline --variables. A few checks now run before project or stack lookup so a bad flag is reported first. Runtime failures (missing files, network, render errors) still exit 1. An unattended spinner now writes its start and progress text as plain stderr lines, not only the outcome, so tools watching init can see "Transcoding". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): reject a bad snapshot --frames, quiet spinner progress, drop added comments - `snapshot --frames abc` (or 0, -3, 2.5) is now a usage error: exit 2, and a usage_error envelope under --json. It used to fall back to 5 frames silently. - An unattended spinner prints its start line once and its outcome, but no longer one line per progress update, which flooded stderr during remove-background and the Chrome download. - Removes the comment lines this branch added, so the comment ratchet passes. - The render --json test now expects exit 2 and usage_error for bad inline --variables. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
22e6da0233 |
feat(cli): define every command through one factory and test them all as an agent (#6616)
* feat(cli): define every command through one factory and test them all as an agent Every command now comes from defineCommand in utils/defineCommand.ts. A lint rule bans importing defineCommand from citty anywhere else, and a new contract test walks the command registry in cli.ts, checks each command was built by the factory, and runs it the way a coding agent does: no terminal, stdin closed, CI=1, CLAUDECODE=1 and a home folder of its own. Usage errors now exit 2: unknown flags, extra positionals, and citty's own parse errors (missing argument, bad value, unknown or missing subcommand). An unknown flag names the help command instead of printing nothing else. isUnattended() in utils/attendedTerminal.ts is the one answer to "can a person respond here". It is true without a TTY on stdin or stdout, under CI (any value except "false" or "0"), when the existing agent detection finds an agent, or with HYPERFRAMES_AGENT=1. Color is off when it is true. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(cli): spawn preview in the agent-mode test only when the studio packages are built From source, preview refuses to load until the studio workspace packages are built, and the CLI test jobs do not build them. preview is still checked for being defined with the shared factory. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): count only agent-run shells as unattended, and exit 2 for missing arguments isUnattended() used the telemetry fingerprint detectAgentRuntime(), which also matches a person typing in Cursor's or Windsurf's terminal (TERM_PROGRAM) and any Replit shell (REPL_ID, REPLIT_USER). It now uses isAgentDriven(), which keeps only markers an agent sets on the shells it runs: CURSOR_AGENT counts, TERM_PROGRAM=cursor, TERM_PROGRAM=windsurf and the Replit variables do not. The hand-written argument checks in lambda, cloudrun, lambda policies, capture, remove-background, init (--resolution), tts (--lang) and cloud render (--idempotency-key, --dry-run) exited 1. They are usage errors and now exit 2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
54d0af57eb |
feat(engine): render sets a keyframe interval and a peak bitrate cap (#6444)
* feat(engine): encoders cap peak bitrate and place keyframes on a set interval * style(engine): shorten the rate-cap comment * style(engine): drop the doc lines the comment ratchet flagged * feat(producer): mp4 renders honour a keyframe interval and a peak bitrate cap on every encoder path * style(producer): shorten the delivery encoder file headers * style(producer): drop a comment line from the segment manifest * style(producer): shorten the integration test header * test: delivery encoder tests follow the ffprobe argv contract and run in the integration lane * feat: the render request, server and Lambda plan carry the keyframe interval and peak bitrate cap * fix(studio): the Studio render route passes the keyframe interval and bitrate cap to the encoder * chore(studio-server): drop a comment the helper's name already says * feat(cli): render and lambda render take --keyframe-interval, --max-bitrate and --buffer-size |
||
|
|
a11280eef0 |
perf(cli): lambda deploy warms the video function so the first render does not wait for its image (#6376)
* perf(cli): lambda deploy warms the video function so the first render does not wait for its image A freshly deployed container-image function loads its image on first use, which added about 12 s to the first render's plan on some deploys. Deploy now invokes the function's warm action once after the stack is up (default on, --no-warm skips it). A failed warm-up prints a warning and the deploy still succeeds. * chore(cli): move the deploy warm-up to its own file, regenerate the skills manifest Regenerates skills-manifest.json for the lambda reference change, keeps runDeploy under the complexity gate by pulling the warm step into a helper, and leaves sam.ts untouched. * fix(cli): the deploy warm-up asks the aws CLI for JSON output whatever the profile sets A profile configured for text or yaml output made the reply unparseable, which printed a spurious warm-up warning. * refactor(cli): split the reserved-concurrency notice and warm-up gate out of runDeploy |
||
|
|
8d773fc68a | feat(cli): lambda render --width/--height at a whole-number multiple renders at that scale (#6390) | ||
|
|
504ec2b307 |
feat(cli): lambda deploy renders on arm64 by default; --video-arch x86_64 keeps x86 (#6073)
* feat(aws-lambda): price arm64 chunk invocations at the arm64 rate getRenderProgress takes chunkArchitecture, the architecture of the function that renders chunks. RenderChunk invocations are priced at its per GB-second rate, both inline and from a Distributed Map's output; plan, audio and assemble stay at the x86 rate. * feat(cli): render chunks on an arm64 function with lambda deploy --video-arch --video-arch arm64 pushes the arm64 video function image to the stack's own ECR repository and deploys the template with VideoImageUri set to its digest. Outside a checkout the image is the one the installed adapter pins; inside a checkout it must come from HYPERFRAMES_LAMBDA_VIDEO_IMAGE, built from that checkout. The default stays x86_64, which always sends an empty VideoImageUri so a redeploy drops a video function rather than keeping it. destroy deletes the repository, render and progress price chunks at the arm64 rate, and policies lists the ECR actions. * fix(cli): push the video image only to an ECR repository this stack tagged * fix(cli): price the plan at the video function's rate now that it runs there * feat(cli): lambda deploy renders on arm64 by default; --video-arch x86_64 keeps x86 * refactor(cli): drop two doc comments the names already carry |
||
|
|
b501dc4e1b |
feat(cli): lambda deploy works from the npm install, no repo checkout (#5866)
* feat(cli): lambda deploy works from the npm install, no repo checkout Each release builds the Lambda handler zip on Linux, attaches it to the GitHub release, and pins its URL and sha256 (plus the SAM template) in @hyperframes/aws-lambda before npm publish. Outside a checkout, lambda deploy downloads that zip, refuses it on a sha256 mismatch, and deploys it with the packaged template. With neither a checkout nor a pinned zip it fails up front with one message naming every way out. * fix(cli): lambda deploy test mocks the pinned-handler reader, comments fit the checks * test(cli): sam tests deploy from a SAM directory, not the repo root * chore(aws-lambda): raise the package size budget for the pinned handler template Measured with pnpm@10.17.1 pack on Linux. Release state (prebuilt/ written by pin-handler-zip): 77,307 packed / 285,654 unpacked. CI state (no prebuilt/): 70,104 / 253,390. Packed is the check's 5% headroom over the release size. Unpacked is 292,000 (2.2% over release) because a 5% budget would put the CI pack under the 85% ratchet. * test(cli): anchor the docs link check in the lambda deploy fallback test * fix(cli): lambda deploy logs the verified handler sha256 and keeps a presigned query out of errors * fix(cli): a redirect-limit download error keeps a signed query out of its message * test(cli): the redirect-limit leak test types its rejection * docs(aws-lambda): header comments cut to what the code cannot say * chore(aws-lambda): size budget fits both the CI and the release package |
||
|
|
2a14cbc924 |
perf(aws-lambda): size distributed renders from account concurrency limits (#5813)
* perf(aws-lambda): run chunks through a Distributed Map to lift the 40-chunk cap The inline Map runs at most 40 chunk renders at once. Both state machine definitions (CDK and SAM) now use a Distributed Map with STANDARD child executions, capped at the function's reserved concurrency so chunks past it queue instead of being throttled. Progress reads each chunk's child execution history so frames, cost and chunk errors still show up. * fix(aws-lambda): retry Lambda throttles for minutes and stop on oversized chunks A throttled task used to burn its four catch-all retries in about 30 s. It now gets its own jittered backoff of several minutes, because another render on the same function can still hold the reserved slots. Plan treats CHUNK_EXCEEDS_WORKER_TIMEOUT as terminal in both definitions. * chore(aws-lambda): trim stack comments to keep the comment share flat * fix(aws-lambda): keep render --wait working at hundreds of chunks Progress polls read the chunk Map Run's counts (one DescribeMapRun) instead of every child execution's history, take exact chunk frames and cost from the Map output once it exits, and read failed children only when the counts show a failure. The SDK client retries throttled calls up to 8 times with backoff. * feat(aws-lambda): size each render's concurrency to the account instead of a fixed 8 * fix(lambda): keep the concurrency SDK external in published modules * fix(cli): keep an existing stack's reserved concurrency on redeploy Redeploying without --concurrency sent ReservedConcurrency=-1, which removed the cap of 8 that earlier CLI versions reserved by default and stopped recording it in the state file. The override is now left out when the flag is omitted, so SAM keeps the stack's current value (a new stack still gets the template's unreserved default). The state file records the deployed value, and deploy prints the kept reservation. * fix(aws-lambda): keep progress working for callers without DescribeMapRun Callers whose IAM policy came from an earlier `lambda policies user` lack states:DescribeMapRun, so reading a running Distributed Map threw AccessDeniedException out of `lambda progress` and `render --wait` while the render itself went on to succeed. getRenderProgress now catches only that error around the map-run read, returns the parent snapshot (status, plan and assemble progress) and adds a warning to re-apply the policy, which both commands print. Other errors still throw. * fix(cli): let the documented deploy policy drop a reserved concurrency When a redeploy removes a function's reserved concurrency, CloudFormation calls lambda:DeleteFunctionConcurrency. The policy printed by `lambda policies role|user` granted only PutFunctionConcurrency, so the update failed and rolled back. Grant it, and pin it with states:DescribeMapRun in the required-actions test. * fix(aws-lambda): read smoke chunk hashes from the Distributed Map output With chunks in a Distributed Map, their task results live in child executions, so the smoke gate's chunk-hash check found 0 chunks in the parent history and failed every v1/v2 comparison. Read Chunks[] from the RenderChunks / RenderChunksV2 MapStateExited output instead, as getRenderProgress does. The stack this script deploys only has the Distributed Map, so the inline-Map path is gone. * fix(producer): cap a pinned chunk size at the worker concurrency, not 16 With --chunk-size or --target-chunk-frames and no --max-parallel-chunks, the plan still capped the render at 16 chunks however many workers the account allowed, so --chunk-size 30 on a 60 s render gave 16 chunks of 113 frames. The cap is now one wave across the worker concurrency, as the --max-parallel-chunks docs say; 16 remains when the concurrency is unknown. * feat(cli): remove a stack's reserved concurrency with --concurrency=none Since an omitted --concurrency now keeps the stack's reservation, there was no CLI way back to the unreserved default. --concurrency=none sends ReservedConcurrency=-1, the state file records the stack as unreserved and deploy says the reservation was removed. Positive N and an omitted flag behave as before; any other value is rejected instead of being silently ignored. * chore(aws-lambda): raise the package size budget for account-sized concurrency Measured with pnpm@10.17.1 pack after rebasing on main with the S3 assembly change: 67,763 packed / 244,685 unpacked; the budget is the check's own 5% headroom over that. |
||
|
|
d0da232ec9 |
fix(cli): show the real import error when a cloud adapter is installed (#5763)
lambda reported any ERR_MODULE_NOT_FOUND as '@hyperframes/aws-lambda is not installed', and cloudrun matched its package name anywhere in the message, which also names the adapter when one of its own dependencies is missing. Both now use one helper that matches only Node's "Cannot find package '<pkg>'" and rethrow anything else. |
||
|
|
18a8191700 |
fix(cli): warn when a cloud render ignores the composition's fps (#5757)
lambda and cloudrun render at 30 fps when data-fps is outside the cloud set (for example 50 or 30000/1001) and --fps is not passed. They now say so and point at --fps. The 30 fps default has one owner, defaultCloudFps. |
||
|
|
c69fd2c6ec |
perf(producer): size distributed chunks to the worker limit and concurrency (#5765)
* perf(producer): size distributed chunks to the worker limit and concurrency * chore(cli): drop a comment the field name already says |
||
|
|
d5162beb50 |
refactor(core): one owner for the distributed fps set (#5756)
The integer rates Lambda and Cloud Run accept were written out by hand in about 25 places across core, engine, producer, the adapters and the CLI. They now come from DISTRIBUTED_FPS / DistributedFps / isDistributedFps in @hyperframes/core. Engine config snapshots accept 25 like the rest. |
||
|
|
cc9e1bb33a |
feat(producer): distributed renders accept 25 fps (#5423)
Lambda and Cloud Run renders refused 25 fps although it is an integer
rate like 24, 30 and 60, and PAL footage is 25 fps. The distributed fps
set is now {24, 25, 30, 60} in the producer's config validation, the
plan v2 manifest, the render request mapping, the Lambda and Cloud Run
event types, the CLI's lambda and cloudrun commands and the regression
harnesses.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
|
3aa2404747 | refactor(cli): centralize process lifecycle | ||
|
|
2d398ed274 |
fix(cli): wire aspect-agnostic resolution through cloudrun/lambda/batch + preflight recompute
Addresses R2 CHANGES_REQUESTED from Miga + Rames on PR #2529: 1. Sibling-surface gap (blocker): `hyperframes cloudrun render{,-batch}`, `hyperframes lambda render{,-batch}` all advertised the same tier-only aliases (`1080p` / `hd` / `4k` / `uhd`) but normalized them to `landscape` and never set `outputResolutionAspectAgnostic`. The distributed plumbing PR #2529 added received `undefined` from those callers, so portrait `1080p` still hit the original aspect-mismatch on Cloud Run / Lambda. Fix: introduce `resolveResolutionFlagPair` in `@hyperframes/parsers` (the single source of truth for the two-step normalize + aspect-agnostic detect) and route every distributed entrypoint through a shared `parseOutputResolutionFlag` CLI util so the alias signal now reaches `SerializableDistributedRenderConfig`. Studio Server keeps its canonical-only HTTP contract; that intent is now pinned in tests. 2. Preflight recompute (hardening): the earlier "downgrade aspect-mismatch" preflight cleared un-remapped mismatches, so IG 4:5 (non-preset aspect, no sibling) and portrait-4K comp + `--resolution 1080p` (remap + downsample) both slipped through to fail late in `resolveDeviceScaleFactor`. Now `checkRenderResolutionPreflight` computes the effective preset via `suggestMatchingPreset` (mirroring the compile stage's `adaptAspectAgnosticResolution`) and re-checks against that — only genuinely-fixable mismatches clear early. New tests pin both regressed input classes. 3. Docker forwarding boundary test (Miga's important #2): pinned `1080p` survives verbatim as `--resolution 1080p` in the Docker args so the in-container CLI can re-run `isAspectAgnosticResolutionAlias`. 4. Doc-nit (Miga): parsers/src/types.ts no longer references the nonexistent `resolveResolutionForComposition` — points at the actual remap helpers. Fallow: cloudrun.ts / lambda.ts share 390 lines of pre-existing structural symmetry (parallel AWS + GCP dispatchers), and lambda/render.ts + render-batch.ts declare parallel RenderArgs interfaces. Both re-flagged after threading the aspect-agnostic field through each surface; ignored with justification in .fallowrc.jsonc. lambda.ts's `run` and lambda/render.ts's `waitForCompletion` are pre-existing CRAP-score hotspots untouched by this PR — added under health.ignore. Co-Authored-By: Claude <noreply@anthropic.com> — Via |
||
|
|
de27b46680 |
fix(cli): default render fps to the composition's data-fps
* fix(cli): default render fps to the composition's data-fps
hyperframes render hard-coded fps to 30 when --fps was omitted, ignoring a
data-fps declared on the composition root — so a composition authored at
data-fps="24" silently rendered at 30fps unless the user knew to pass --fps 24.
The runtime already honors data-fps; the CLI now matches it.
Precedence: explicit --fps > composition root data-fps > 30. New pure
readCompositionFps() extracts the root data-fps via linkedom (mirrors the
runtime's root resolution: [data-composition-id][data-root=true], else the
outermost [data-composition-id]); render validates it through parseFps and
falls back to 30 on an absent/invalid value. Unit-tested.
* fix(cli): honor composition data-fps on cloud renders and --composition targets
The local render command read data-fps from project.dir/index.html even when
--composition rendered a different file, and the lambda/cloudrun render paths
ignored data-fps entirely (hardcoded ?? 30). Both are the same silently-wrong-
fps bug on other render entry points:
- render.ts resolves the entry file first, then reads data-fps from the file
actually being rendered (falling back to index.html).
- lambda render/render-batch and cloudrun render/render-batch default fps from
the composition's data-fps, accepted only when it is one of the cloud-allowed
values {24,30,60}, else the existing 30 default. Explicit --fps still wins.
* fix(cli): drop citty fps default so data-fps resolution actually runs
The fps arg had default: "30", so citty set args.fps="30" on omission and
resolveDefaultFpsArg short-circuited (explicitFps never null) — reverting the
command to always-30 and making the whole data-fps feature a no-op (caught in
review). Remove the arg default; the "30" fallback already lives at
parseFps(fpsArg ?? "30"). Adds a regression guard asserting the arg has no
default.
* test(cli): read citty args through a plain record in the fps-default guard
The regression guard accessed cmd.args.fps directly, but citty types args as
Resolvable<ArgsDef> so .fps failed typecheck in CI. Read it through a plain
record cast.
|
||
|
|
9b18fadccd |
feat(producer): optional targetChunkFrames to bound per-chunk frames (#1332)
* feat(producer): optional targetChunkFrames to bound per-chunk frames * feat(cli): expose --target-chunk-frames on lambda + cloudrun render; document it |
||
|
|
6d1236a0cc |
feat(cli): add --output-resolution to lambda render
Allows authored-at-1080p compositions to render at 4K/2K via Chrome deviceScaleFactor supersampling without re-laying-out the composition. Plain --width 3840 silently lays out at 1920×1080 because data-width/ data-height attrs override Config.width — this flag is the supported way to ask the renderer to supersample. Accepts canonical CanvasResolution names (landscape, landscape-4k, portrait, portrait-4k, square, square-4k) and aliases (1080p, 4k, uhd, hd, 1080p-portrait, 4k-portrait, 1080p-square, 4k-square). Wired through render + render-batch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
f0a2740f6e |
feat(cli): hyperframes lambda render-batch verb
New subcommand for automated template-rendering pipelines. Given a
project dir + a JSONL batch file, fans out N personalised renders by
calling renderToLambda once per batch row with per-entry variables and
outputKey:
hyperframes lambda render-batch ./my-template \
--batch ./users.jsonl \
--width 1920 --height 1080 \
--max-concurrent 10
JSONL format (one JSON object per line):
{"outputKey": "renders/alice.mp4", "variables": {"name": "Alice"}}
{"outputKey": "renders/bob.mp4", "variables": {"name": "Bob"}}
The verb deploys the site once and reuses it across renders (--site-id
skips the deploy when the project was pre-uploaded). Concurrent Step
Functions starts are capped at --max-concurrent (default 50) via a
semaphore so a 10 000-entry batch doesn't try to spawn 10 000
executions simultaneously and trip the AWS account's concurrent-
execution quota.
Per-entry results land in a manifest (one row per input line) with
executionArn + status. --json emits the manifest as machine-readable
JSON. --dry-run prints the manifest with status: "would-invoke" for
every entry without calling AWS, so callers can lint their batch file
before paying for N executions.
Variables in each batch entry pre-validate against the composition's
data-composition-variables declaration (mirroring the local
hyperframes render UX). --strict-variables aborts the run on the first
failing entry before any AWS call. The reportVariableIssues helper from
PR 9.3 is reused so the warning format matches the single-render path
exactly.
Distinction from --max-parallel-chunks: --max-concurrent caps
ORCHESTRATOR-side fan-out (how many StartExecution calls run at once);
--max-parallel-chunks caps chunks PER render. AWS account-level Lambda
concurrent-execution limits live one level up and render-batch can't
enforce those; pick --max-concurrent based on your account quota +
the reserved concurrency you provisioned via lambda deploy.
Tests cover the concurrency-cap semaphore (preserve-order,
peak-in-flight, empty-input, limit > inputs.length, propagate
rejection) and the JSONL parser (blank-line handling, malformed JSON,
missing outputKey, non-object variables).
Phase 9 PR 9.4 of the distributed rendering plan.
|
||
|
|
cb948d5fcf |
feat(cli): hyperframes lambda render --variables / --variables-file / --strict-variables
Mirror the local hyperframes render variables UX on the Lambda CLI: - --variables '<json>' inline JSON object of variable values - --variables-file <path> path to a JSON file with variable values - --strict-variables fail on type/declared-mismatch (warn by default) Resolution + validation logic is hoisted to packages/cli/src/utils/variables.ts so both surfaces share one parser. The new reportVariableIssues helper formats the warning block + handles --strict-variables exit, deduping the per-CLI issue-handling block. Variables flow into SerializableDistributedRenderConfig.variables and reach every chunk worker via the path PR 9.1 + 9.2 wired up (plan() → meta/encoder.json → renderChunk() → window.__hfVariables). Pre-validation against the composition's data-composition-variables declaration runs only when the project's index.html is on disk — --site-id pointing at a pre-uploaded site that was packaged elsewhere skips the check, matching how the local CLI treats unreadable index files. The render.ts re-exports of parseVariablesArg / resolveVariablesArg / validateVariablesAgainstProject are dropped; the matching tests move to packages/cli/src/utils/variables.test.ts where the implementations now live. Docs: docs/packages/cli.mdx adds a section on --variables / --variables-file / --strict-variables for lambda render, including the 256 KiB Step Functions execution-input cap and a pointer to the upcoming templates-on-lambda guide (PR 9.5). Phase 9 PR 9.3 of the distributed rendering plan. |
||
|
|
5d264e146c |
docs(lambda): document webm support + simplify-review fixes (#953)
* docs(lambda): document webm support in distributed mode PR 8.4 of the WebM distributed-rendering plan (v1.5 backlog #1; see DISTRIBUTED-RENDERING-PLAN.md §7.2). User-facing docs catch up with the shipped capability. Updates docs/deploy/migrating-to-hyperframes-lambda.mdx: - "Output format" row in the migration table now lists `webm` alongside mp4 / mov / png-sequence with a note that webm uses libvpx-vp9 + closed-GOP concat-copy. HDR mp4 remains the only refused format. - "No webm distributed" caveat replaced with "webm uses closed-GOP VP9" explainer covering the encoder args (`-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, `-cpu-used 2`), why alt-ref disable is load-bearing, and that the output preserves alpha via yuva420p with Opus audio. - Migration checklist no longer asks adopters to filter out webm compositions; only HDR-dependent renders need to stay on the previous framework. aws-lambda.mdx doesn't currently call out webm as unsupported (only HDR in the v1 surface list), so it gets no copy edits beyond the migration guide. The internal planning doc (DISTRIBUTED-RENDERING-PLAN.md §7.2, §8, §12 — kept outside the repo) gets matching updates: format support matrix flipped ✓, v1.5 backlog #1 marked shipped, HDR promoted to the new top item, and the rev-12 → rev-13 status line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: address simplify-review findings on webm stack Folds in cleanups identified by a multi-agent code-review pass over the 4-PR webm-distributed stack: - plan.ts: `resolveEncoderTriple()` webm case now calls `getEncoderPreset(quality, "webm")` for its preset string instead of hardcoding "good". The hardcode was wrong for `quality: "draft"` (`getEncoderPreset` returns "realtime" for that tier) — would have silently overridden the draft → realtime mapping for distributed webm renders. - chunkEncoder.ts: trim the new VP9 closed-GOP comment block from ~18 lines of WHY narration down to the 6 lines that actually explain why (alt-ref + cpu-used drift). Match the alpha branch's idempotent-push comment to the same standard. - chunkEncoder.test.ts: drop the duplicate WHY comment that restated the implementation comment in plain words. - webm-concat-copy.test.ts: rewrite the file-header docstring to describe the contract being tested instead of the PR-8.1-gating history; strip "PR 8.2 / Path A / Path B" references from error messages (they belong in PR bodies, not in test output). Consolidate the yuva420p alpha smoke into a single `it()` block (was a full 4-test describe with duplicated setup) — the yuv420p block already covers the probe/decode/frame-count contract; the alpha smoke only needs to prove the alpha args don't break concat-copy. - plan.test.ts: drop the "PR 8.1 proved the contract" comment. - webm-vp9 fixture: drop the aspirational "Other webm-with-audio fixtures cover the mux path separately when added" sentence (no other fixtures exist). Regenerated the baseline via `docker:test:update webm-vp9` to reflect the updated comment. - migrating-to-hyperframes-lambda.mdx: add a paragraph about distributed webm's perf cost — ~10-25% larger files at constant CRF due to forced keyframes, and slower per-chunk encode due to `-cpu-used 2` being more conservative than the libvpx default. All unit tests + the webm-vp9 distributed-simulated regression still pass after these changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): accept --format=webm in `hyperframes lambda render` The CLI's `lambda render` subcommand's FORMATS allowlist and the `RenderArgs.format` type still narrowed to `mp4 | mov | png-sequence`, so even though the producer + aws-lambda packages now support webm end-to-end, the CLI surface rejected it with `--format must be mp4|mov| png-sequence`. Add webm to both spots and update the --help description. Surfaced during real-AWS deploy prep — the local lambda-local / distributed-simulated tests didn't go through the CLI so the gap went unnoticed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(producer): font cache writes to /tmp on Lambda (read-only \$HOME) The deterministic Google Fonts cache was rooted at `\$HOME/.cache/hyperframes/fonts`, which fails on AWS Lambda — the runtime's `\$HOME` resolves to a `/home/sbx_*` directory tree that's read-only. `mkdirSync(..., { recursive: true })` can't create that path and the plan stage trips with `ENOENT: no such file or directory, mkdir '/home/sbx_user1051/.cache/hyperframes/fonts/space-mono'` on every Lambda render that pulls a Google Font (i.e. every distributed fixture using `@import url("https://fonts.googleapis.com/...")`). Detect Lambda via `\$AWS_LAMBDA_FUNCTION_NAME` and route the cache to `tmpdir()/hyperframes/fonts` in that case. Lambda's `/tmp` survives across invocations on a warm container, so cache hit rate is the same as non-Lambda runs. Also honor an explicit `\$HYPERFRAMES_FONT_CACHE_DIR` override for adopters who want a different location regardless of the runtime. Surfaced while verifying webm distributed end-to-end on real AWS — the same bug affects mp4 fixtures using Google Fonts; webm just happened to be the one I tried first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract DistributedFormat type + trim font-cache resolver Second simplify-review pass on the webm stack flagged two cleanups: 1. **`DistributedFormat` type duplicated 10 times.** Every file in the distributed pipeline carried its own copy of `"mp4" | "mov" | "png-sequence" | "webm"` — adding a new format meant a 10-place edit with no compile-time guarantee they stayed in sync. Extract a single source of truth in `packages/producer/src/services/distributed/shared.ts`, re-export from `@hyperframes/producer/distributed` and `@hyperframes/aws-lambda/sdk`, and have all callers pull from there. The aws-lambda `ALLOWED_FORMATS` runtime tuple and the CLI's `FORMATS` tuple now both use `satisfies readonly DistributedFormat[]` so the compiler enforces the runtime allowlist stays in sync with the type. 2. **`deterministicFonts.ts` font-cache resolver was over-commented.** Trim the 7-line block to 4 lines (drop the aspirational "and other read-only-FS execution environments" — only Lambda is detected — and the warm-container `/tmp` persistence narration — anyone reading already knows Lambda /tmp semantics). Collapse the two-step `if (explicit && explicit.length > 0)` into a single nullish-coalesce expression now that the empty-string defensive check is gone (`process.env.X` is `string | undefined`, no third shape to guard against). Out-of-scope skips (called out by the agents, deferred): - In-process `RenderConfig.format` and the in-process CLI's `render.ts` format union still carry their own inline copies. The union happens to coincide today but they're separate concerns — leaving them alone limits this PR's blast radius. - `fontCacheDir(slug)` / `resolveFontCacheRoot()` naming asymmetry flagged as taste; skipping. - Pre-existing redundant `existsSync` before `mkdirSync({ recursive: true })` in `fontCacheDir` — out of scope. All tests + typecheck still pass. Lambda render still works end-to-end (no functional changes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(lambda): drop plan-doc reference from migration checklist PR review feedback: source/docs should not mention the distributed-rendering planning doc. Tighten the migration checklist sentence to describe the webm path directly rather than referencing the doc's version label. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(producer): split resolveEncoderTriple into mp4 + non-mp4 helpers CI Fallow audit on PR #953 flagged `resolveEncoderTriple` at CRAP 31.6 — the function interleaved (a) mp4 codec validation + dispatch, (b) the non-mp4 codec-rejection throw, and (c) per-format dispatch. Splitting into `resolveMp4EncoderTriple` + `resolveNonMp4EncoderTriple` drops the top-level function's cyclomatic complexity below the threshold while preserving every error message and code path. Behavior unchanged. Also extracts an `EncoderTriple` type alias so the three functions share the return shape declaratively rather than repeating it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
62ddd29b74 |
feat(cli): add hyperframes lambda policies role/user/validate (#912)
* feat(cli): add hyperframes lambda policies role/user/validate
IAM bootstrap subcommand for the lambda CLI. Closes the "first run hits
'User is not authorized to perform iam:CreateRole'" gap that adopters
otherwise have to figure out by hand.
hyperframes lambda policies user
→ prints an inline-policy doc to attach to the IAM user that runs
the CLI
hyperframes lambda policies role --principal=cloudformation
→ prints { TrustRelationship, InlinePolicy } for a service role
cloudformation can assume
hyperframes lambda policies validate ./infra/policy.json
→ diffs a checked-in policy against the CLI's required action set,
expanding s3:* / s3:Get* / * wildcards, exits non-zero on missing
actions (wire it into CI to catch drift before deploys fail)
The required-actions list is derived from what the SAM template at
examples/aws-lambda/template.yaml needs to create plus what
renderToLambda/getRenderProgress call against S3 + Step Functions at
runtime. Sorted alphabetically per-service so diffs stay readable.
Resource is "*" by design — CloudFormation creates new function /
state-machine / bucket ARNs on every adopter's first deploy. The
generated policy is documented as a starting point; adopters with
stricter postures narrow Resource to the deployed ARNs after the
first successful run.
Tests: 10 unit tests covering the action set, doc shape, trust policy
service principal, and validate() against valid / missing / wildcard /
single-Statement / Deny-statement inputs.
* refactor(cli): /simplify pass on lambda policies
Adds a typed TrustPolicyDocument / TrustPolicyStatement pair so
buildRoleTrustPolicy can return a real type instead of unknown. The
trust-policy shape has a Principal field that the generic
PolicyStatement doesn't model, but it was previously punted via a
return unknown rather than a parallel type.
Test cleanup: drop the `as {...}` casts that the previous return-
unknown signature forced.
* fix(cli): address PR review on lambda policies
One blocker + four importants from Vai's review:
- REQUIRED_ACTIONS was missing `s3:ListAllMyBuckets` (called by
`sam deploy --resolve-s3` on first run to discover/create the
`aws-sam-cli-managed-default-*` artifact bucket) and
`cloudformation:ValidateTemplate` (CFN template validation
during change-set creation). Without these, a first-deploy
adopter with the generated policy hits AccessDenied on the
very call the PR was meant to unblock. Added both.
- `policies role --principal=lambda` was a footgun — it produced
a `lambda.amazonaws.com` trust paired with the full deploy
superset, i.e. a confusingly-overscoped Lambda execution role
no human should attach (the SAM template creates its own
scoped execution role automatically). Dropped `lambda` as a
principal option; `policies role` now always emits a
CloudFormation service-role doc.
- `validatePolicy` silently misreported NotAction/NotResource
statements (treating them as zero grants), producing false
negatives. Detect both shapes and surface them via a new
`warnings: string[]` field; NotAction statements are skipped
(rather than producing a false negative), NotResource is
treated as full action grant + a warning.
- Mid-string wildcards (`s3:Get*Object`, `?`) silently failed
the matcher. End-anchored wildcards still work; mid-string
patterns now warn so users know the validator can't expand
them.
- Dropped the dead `samArtifactBucket` action group (fully
subsumed by `s3Bucket` + `s3Object`).
- `validate --json` now wraps errors in a friendly envelope
(`{ ok: false, error: "..." }`) so CI consumers have one
parse shape regardless of failure mode.
- lambda.ts subcommand description and examples updated to
include `policies`.
Tests: 5 new negative-path tests cover NotAction warning,
NotResource warning, mid-string wildcard warning, missing file
(ENOENT), malformed JSON (SyntaxError), and absent Statement
field. All 21 policies tests pass.
|
||
|
|
e90ad2da61 |
feat(cli): add hyperframes lambda deploy/render/progress/destroy (#910)
* feat(cli): add hyperframes lambda deploy/render/progress/destroy
Wraps the @hyperframes/aws-lambda SDK + the Phase 6a SAM template behind
a single CLI surface so an end-to-end render is three commands instead
of the ~8 manual bun+sam+aws steps the smoke script does today:
hyperframes lambda deploy
hyperframes lambda render ./my-project --width 1920 --height 1080 --wait
hyperframes lambda destroy
Subcommands:
- deploy: build handler.zip + sam-deploy + persist stack outputs
to <cwd>/.hyperframes/lambda-stack-<name>.json
- sites create: pre-upload a project to S3 with a stable content hash
so re-renders skip the tar+PUT pass
- render: start a Step Functions execution; --wait blocks and
streams per-chunk progress + accrued cost
- progress: one-shot snapshot — status, frames, cost breakdown,
errors. Accepts renderId or executionArn
- destroy: sam-delete + drop the local state file (S3 bucket
is Retain'd by the template; documented in --help
and in docs/packages/cli.mdx)
To keep @sparticuz/chromium out of the CLI's transitive deps, this also
adds a dedicated ./sdk subpath export to @hyperframes/aws-lambda; the
CLI imports from @hyperframes/aws-lambda/sdk exclusively. The existing
. barrel still re-exports both handler + SDK for adopters who want one
entry point.
Defaults are deliberately cost-conservative for first-time users:
--concurrency=8 (low enough to never surprise) and --memory=10240 (the
common case; documented for adopters who want to tune down).
Tests: 5 unit tests on the state-file round-trip. CLI integration
against sam local invoke is part of the upcoming PR 6.6 (lambda-local
regression harness).
* refactor(cli): /simplify pass on the lambda command group
Two small cleanups on top of the lambda CLI:
- Replace parseFormat / parseCodec / parseQuality / parseChromeSource
(four near-identical helpers) with a single generic parseEnum() +
typed const-tuple lookups. The four callers now read as one-line
arrow functions that lift the allowed values out of the function
body so they're easy to extend.
- DEFAULT_STACK_NAME was const-declared then re-exported at the
bottom of state.ts; just mark the const export inline.
No behavior changes. All CLI tests still pass.
* fix(cli): keep @hyperframes/aws-lambda external in the tsup bundle
esbuild can't bundle @hyperframes/aws-lambda's transitive AWS SDK
deps (@aws-sdk/* + @smithy/*) cleanly into a node binary — the
SDK's .browser.js conditional re-exports break the resolver:
ESM Build failed
No matching export in "splitStream.browser.js" for import
"splitStream" (and ~10 similar errors)
Mark aws-lambda as `external` so esbuild doesn't follow it, and
move it from devDependencies to dependencies so the published CLI
can resolve it from node_modules at runtime. The lambda subverb
files dynamic-import only on `hyperframes lambda *` invocation, so
the CLI cold-start cost is unchanged.
The install-size hit (AWS SDK + @sparticuz/chromium ≈ 200 MiB) is
documented as a v1 tradeoff; a future split into a lambda-sdk-only
subpackage can pare this back.
* fix(cli): address PR review on lambda CLI
Two blockers + four important items from Vai's review:
- `--memory` was parsed and recorded in the local state file but
never forwarded to `sam deploy` as a parameter override. Worse,
`progress.ts` then read the *recorded* value for cost math, so
`--memory 5120` produced wrong cost numbers downstream. Thread
`LambdaMemoryMb` through samDeploy's --parameter-overrides.
- `--profile` was only consumed by deploy / destroy. render and
progress fell back to the default credentials chain — a user
with `--profile prod` would silently render against their
default account (wrong-account billing footgun). Set
`process.env.AWS_PROFILE` (and `AWS_REGION`) in the dispatcher
before any subverb runs; the AWS SDK reads them natively, so
render / progress / sites all benefit without each subverb
threading the flag through the SDK call.
- `--profile` + destroy now also reads `process.env.AWS_PROFILE`
as a fallback (matching deploy's existing env fallback).
- `--wait --json` printed both the start handle AND the final
progress snapshot, producing two concatenated JSON blobs that
`jq` rejected. Now emits a single document: handle (without
--wait) OR final progress (with --wait).
- Negative integers on `--width` / `--height` / `--chunk-size` /
`--max-parallel-chunks` / `--memory` / `--concurrency` now fail
loudly via a new `parsePositiveInt` wrapper instead of flowing
into the SDK and producing opaque AWS validation errors mid-
render.
- `DEFAULT_STACK_NAME` is now centralized to the literal
`"hyperframes-default"` and consumed from one place. Previously
the value was assembled as `hyperframes-${"default"}` in three
sites and hardcoded as `"hyperframes-default"` in a fourth.
`requireStack`'s hint now matches the dispatcher's default.
The faked `SiteHandle` for `--site-id` keeps the documented
placeholder fields but also surfaces `bucketName` (from PR 909's
extended SiteHandle interface), matching the SDK contract.
All CLI unit tests + the full bundler build still pass.
* fix(cli): keep aws-lambda out of CLI runtime deps
The "Smoke: global install" CI step packs the CLI via `npm pack` and
installs it globally via `npm install -g <tgz>`. npm doesn't understand
the workspace: protocol, so a runtime `dependencies` entry of
`@hyperframes/aws-lambda: workspace:*` blows up with:
npm error code EUNSUPPORTEDPROTOCOL
npm error Unsupported URL Type "workspace:": workspace:*
(pnpm rewrites workspace:* on publish; npm pack doesn't.)
Three changes to unblock the smoke + keep the published CLI install
small for users who don't deploy to Lambda:
- Move `@hyperframes/aws-lambda` from CLI's `dependencies` back to
`devDependencies`. It's already external in tsup.config.ts; the
bundle references it via runtime resolution only.
- Convert the static `import { … } from "@hyperframes/aws-lambda/sdk"`
in sites.ts / render.ts / progress.ts to `await import()` inside
each function. tsup with `splitting: false` was inlining those
static imports at the top of the bundle, which made Node eagerly
resolve them at CLI startup (MODULE_NOT_FOUND before any lambda
subcommand even runs). Dynamic imports stay dynamic in the bundle.
- Add a friendly missing-module check in the lambda dispatcher.
When a user runs `hyperframes lambda deploy / render / sites /
progress / destroy` without aws-lambda installed, they now see:
@hyperframes/aws-lambda is not installed.
The `hyperframes lambda deploy` command needs it at runtime.
Install it alongside the CLI:
npm install -g @hyperframes/aws-lambda
Verified locally: pack + global install + `hyperframes init --example
blank` now succeeds end-to-end (was the same scenario the CI smoke job
runs).
|