mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 18:56:42 +00:00
stack/canvas-selection
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vance Ingalls
|
c03cc2c52c |
Merge branch 'main' into via/studio-5433-html-sniff-defense
Both conflicts were import/export unions in the engine package, resolved by keeping both sides: - packages/engine/src/index.ts — main widened the urlDownloader re-export (fetchPublicHttpsText, safeDownloadUrlIdentity, writeUrlDownloadTelemetry and their types) while this branch added the notMediaPayload exports. - packages/engine/src/services/audioMixer.ts — main added UrlDownloadError and writeUrlDownloadTelemetry to the urlDownloader import; this branch added isNotMediaPayload. In audioMixer's prepare path both intents compose in order: main's download telemetry and typed download failure, then the STUDIO-5433 non-media sniff before the probe. |
||
|
Vance IngallsandClaude Opus 5
|
349c066a83 |
fix(producer): classify JSON error bodies as non-media sources too
A source that answers with a JSON error body still reached ffprobe and
produced `moov atom not found`. Replicate returns
`{"detail": "requested file not found"}` for a dead asset, and a gateway
in front of it can relay that body with a success status.
The sniff now treats `<`, `{`, or `[` as the opening byte of a text
document. No supported container starts with any of them, so this is the
same trade as before: three bytes instead of an allowlist that grows one
entry per payload shape observed in production.
Renamed accordingly, since the class now covers JSON as well as markup:
MARKUP_NOT_MEDIA -> NOT_MEDIA_PAYLOAD, MarkupNotMediaError ->
NotMediaPayloadError, markupPayload.ts -> notMediaPayload.ts. Registry
entries in the Lambda name map, the CDK and SAM plan lists, the Cloud Run
set, and SAFE_RENDER_ERROR_CODES move with it.
Also documents the reachability boundary on the error class: only a 2xx
response gets here. `downloadToTemp` rejects 404/410 as `http_not_found`
before writing a byte, and every ffprobe input is local because
videoFrameExtractor downloads http srcs first. So the shapes this
classifies are soft-404 and interstitial HTML, S3/CloudFront error
documents, and JSON API error bodies -- each served with a success
status. A genuine 404 surfaces as a download failure, not as this error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Vance IngallsandClaude Opus 5
|
f3689c1481 |
fix(producer): scope and harden the markup-payload sniff
Review follow-up on the STUDIO-5433 defense.
Correctness
- The sniff ran above the documented video/audio failure split, so an
<audio> src that resolved to an HTML payload aborted the whole render
instead of degrading to duration 0. It now runs inside the same try, so
video surfaces the typed error while audio still drops out, with a
warning naming the element.
- Raw fs errors (EISDIR on a directory src, EACCES, the existsSync->open
ENOENT race, EMFILE) escaped and failed the compile with an unclassified
error carrying an unredacted temp path. The sniff is now a classifier that
never throws: an unreadable file reports "not markup" and the real probe
produces the real error.
- Elements whose duration the compiler never resolves (a data-end video, a
looping audio) skipped the sniff entirely, so the original ffprobe error
still escaped, and looping audio was reported as owner "system" after
every frame had been captured. Video is now caught in the asset preflight,
which sees every local src regardless of authored timing; audio is
classified per-element in audioMixer as source/invalid_media/owner "user",
keeping audio failures non-fatal as they already were.
- Detection is a byte-level check for a leading "<" (BOM-, whitespace- and
NUL-tolerant, looped read) instead of a <!doctype|<html|<?xml string
prefix, which missed a NUL-prefixed payload, >256B of leading whitespace,
UTF-16-encoded HTML, and a prolog-less <svg. No supported container starts
with "<", so the allowlist no longer grows per payload shape.
- finally { await fh.close() } could replace the in-flight typed error with
the close error.
Routing and privacy
- MARKUP_NOT_MEDIA is now in SAFE_RENDER_ERROR_CODES, the Lambda terminal
name map, the CDK and SAM non-retryable plan lists, and the Cloud Run
non-retryable set, and the class carries owner/retryable. Previously the
API emitted errorCode: undefined and a deterministic authoring bug burned
the full distributed retry budget.
- The message no longer carries 32 raw payload bytes or the src.
redactTelemetryString preserves host and path for HTTP srcs, so
per-tenant CDN paths reached a message the server forwards to clients.
Correlation is a sha256 element fingerprint, matching
AssetMediaTypeMismatchError.
- The message names both causes (unresolved nested-composition URL, or an
HTML/XML error page served as 200) rather than misdiagnosing an S3 403
body as an authoring bug.
Tests
- Byte-level detection is unit-tested in engine: markup shapes, BOMs,
UTF-16, nine container signatures, unreadable inputs.
- Replaced the tautological assertions. The old checks for "html" in and
"moov" absent from a fixed message template could not fail for any input.
- New coverage for audio degradation, the audioMixer classification, the
preflight video/image/audio split, and the API error metadata.
- The sibling htmlCompiler.mediaType failure was a vitest-under-bun runner
mismatch, not a missing ffmpeg binary. It passes, including the 4-wide
probe-semaphore invariant the sniff now runs inside.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
ukimsanov
|
b5509602fa |
docs: remove the Reference Project — nobody wants to be sent to GitHub
The "One project, open end to end" section on Examples was four cards linking into a GitHub folder. A reader on the docs site does not want to leave for a repo tree to read a BRIEF.md. The section is gone, and so is the examples/docs-reference-project folder it pointed at. Also removed the prose references that leaned on it: the Reference Project paragraph on Go further, the two GitHub-inspect links on Developers, and the mention on the Studio landing. Each was reworded to talk about "a project" generally rather than that specific folder. Examples is now purely the nineteen finished films plus Start from a template. The changelog entry recording #2977 is left as history. 0 broken links. |
||
|
Ular Kimsanov
|
edfe66a953 |
docs: add the shared page components and the Reference Project (#2977)
* docs: add the shared page components Adds the six React snippets the rebuilt documentation pages compose against, plus the styles they need. Nothing imports them yet, so this lands with no user-visible change and no navigation churn. - DocsVideo / ShowcaseWall — the film player and the Showcase grid - LiveReferenceProject — embeds the Reference Project via <hyperframes-player> - WorkflowChooser, AgentAction, and the two grid snippets The scrub indicator is a timecode bubble rather than a thumbnail. Mounting a second <video> with the same src to drive a preview frame made every page carrying a film download the whole file twice, which is not worth a thumbnail. * docs: add the Reference Project example One real 10-second project the documentation can point at instead of describing a hypothetical one: a live capture of example.com, synthesised narration, and caption timings measured from that narration. It passes its own gates — `hyperframes lint` clean, `hyperframes check` passed, 28/28 text checks WCAG AA. No page imports it yet, so this lands without touching navigation. Only the two WAV masters exceed the repository's 500 KB non-LFS limit, so only those go through LFS. The MP3 stings and the capture PNG stay plain, which keeps the example usable after a clone without `git lfs pull`. `bun run docs:bundle-reference` regenerates the single-file embed the Introduction page loads from the CDN. * docs: keep the Reference Project verification report The Examples page links this file twice — as "What changed after review" and as "The real verification report" — in the section that makes the project's brief, source, revision notes, and checks public end to end. It is a published artifact, not leftover scaffolding. * docs: state the Reference Project embed's isolation contract The composition is fetched from the CDN and handed to the player as a blob: URL, which inherits the docs origin, and <hyperframes-player> sandboxes its iframe with allow-scripts + allow-same-origin. So the embedded composition runs with script access to this origin. That is a consequence of how the player works — it drives seeking through the iframe's document, which a cross-origin frame does not expose — not something this component can fix. Serving the CDN URL directly would isolate the frame and break playback. The guard is therefore the source, so the comment says so out loud: src must stay a first-party path we publish, never user- or community-supplied HTML. * fix(docs): resolve reduced-motion on the first render, and the embed's dep gap Both defects from Rames Jusso's review on #2977. Neither is visible today because nothing imports these files yet, which is what makes them cheap now. **Reduced motion resolved one paint too late, in all three grids.** `useState(false)` plus a `matchMedia` read in an effect meant the first committed render always emitted `<video src autoPlay loop>`; a reduce-motion visitor had 6 + 8 + 4 tiles already fetching before the attributes came off. `autoPlay` also overrides `preload="metadata"`, so those were the files, not metadata probes — and dropping `src` with no following `load()` is not a reliable abort. A lazy initializer knows the answer on the first render. **LiveReferenceProject never sent the initial variables.** The sending effect read `playerRef.current`, assigned by the effect above it on the commit where `compositionSrc` lands — a commit with nothing in the sending effect's dep array. So it ran once against a null ref and never again. It looked correct only because the three defaults match what the composition already renders. Also from the same review: - The object URL could outlive its revoke: once the body resolves, `abort()` no longer stops the chain, so the blob could be minted after cleanup ran with `objectUrl` still undefined. Same `cancelled` guard the effect above uses. - `postMessage` targeted `"*"` while the isolation comment argues the frame is same-origin. Naming `window.location.origin` turns that prose guard into an enforced one. - Nothing reached a terminal state when the player script never arrived: `whenDefined()` does not reject, and a later mount reuses the tag without its error listener. A CSP rule or content blocker never fires `error` at all. A deadline covers every path instead of sitting on "Loading…" forever. - `loadFailed` was never cleared, so one transient failure stuck. - The README claimed a clone works without `git lfs pull`. It does for the visuals; both WAVs are pointers and they are the bed and the voiceover, so the captions would play over silence. Says so now. - The bundler stripped trailing whitespace document-wide while inlining the runtime, which reaches inside script template literals where those spaces are data. It also assumed a literal `<head>` and would silently ship an embed with no `<base>`. Strip removed, anchor asserted. Copilot's five "missing hook imports" comments are wrong — Mintlify pre-injects the hooks, and `TemplateCard.jsx`, cited as the counter-example, uses the `export function` form the same page says is unsupported. * fix(docs): stop preview loops when Reduce Motion is turned on mid-session Miguel's changes-requested on #2977. He is right about the mechanism: dropping `src` and `autoPlay` through React props neither pauses a playing element nor aborts its selected resource, so a visitor who turned Reduce Motion on with the page already open kept every tile running. Measured in a browser rather than argued from the spec, same clip, same sequence: playing paused=false t=2.90 readyState=4 networkState=1 React props only paused=false t=3.90 readyState=4 networkState=1 + pause/removeAttr/load paused=true t=0 readyState=0 networkState=0 The middle row is the bug: time still advancing, resource still held. Rames' follow-up asked for a remount-to-poster instead, because a video that ends with `src` removed holds its last frame and `poster` only paints before playback begins. `load()` covers that too — it drops readyState to HAVE_NOTHING, which is precisely the state that paints the poster. Confirmed side by side on screen: the React-props-only tile sits on an arbitrary mid-clip frame, the pause/load tile shows the poster again. So no remount is needed. The guard cannot be shared as code — Mintlify compiles each snippet in isolation and forbids one importing another — so it is copy-pasted into all three grids. A duplicated invariant is the kind that rots, and a rendering test would mean adding React to a repo that only carries it inside packages/studio, plus mocking Mintlify's hook-injection contract with a mock that can stay green while the page breaks. `scripts/check-docs-snippet-motion.mjs` asserts the source instead, wired into `bun run lint`, with unit tests covering both edges. That gate immediately found `docs/snippets/TemplateCard.jsx`: autoplays with no reduced-motion handling at all. It is imported by zero pages, and it uses the `export function` form Mintlify's constraints page says is unsupported, so it would not work if it were. Deleted rather than fixed. * refactor(scripts): split the motion guard into named predicates fallow flagged findMotionGuardViolations at CRAP 42 — a finding this branch introduced, so it gets fixed rather than suppressed, same as the catalog generator earlier in the stack. The two conditions are now their own predicates behind a small requirements table, which drops the branch count under the threshold and makes each rule readable on its own line. Same output, same tests. * fix(docs): move the stop effect above ShowcaseWall's early return Rames' changes-requested on `e1a03c63`. The effect I added in the previous commit landed below `if (open) return`, so `ShowcaseWall` called five hooks on the grid render and four once a tile was open. That is a conditional hook: clicking a tile — the component's primary interaction — threw "Rendered fewer hooks than expected". Worth naming why it landed in one of three. `workflow-chooser` and `advanced-path-grid` have no early return, so the same paste position was fine there. `ShowcaseWall` is the only one with a conditional return and it got the same copy. That is the duplication cost this script's own header warns about, showing up in the commit that added the script. **The bespoke gate could not have caught it, and now the generic one does.** `.oxlintrc.json` already loaded the `react` plugin and never excluded `docs/` — only `.prettierignore` does, which is why formatting is not a finding here but linting reaches these files. Naming the two hook rules in an override scoped to `docs/snippets/**` reports this bug directly, and also reports the `compositionSrc` dependency gap from round one that was found by reading. Verified both ways: reintroducing the conditional hook produces `react-hooks(rules-of-hooks)`, and `bunx oxlint .` is clean repo-wide, so nothing lit up in `packages/studio`. **Two holes in the script itself, both from the same review.** It matched whole files while the invariant is per component, so a second unguarded grid in `docs-video.jsx` would have ridden in on `ShowcaseWall`'s guard. It now splits by component. That immediately surfaced the distinction between a component that decides to autoplay and one that forwards its caller's `autoPlay` prop — `DocsVideo` only ever plays because a reader clicked, so it does not owe a preference check. And `readsPreferenceLazily` never tied its halves: any lazy initializer plus the media-query string anywhere in the file passed, which is the original bug satisfying the check written to prevent it. The query now has to sit inside the initializer's own expression. Both holes have tests. fallow is clean at 0 introduced. * fix(scripts): close the two silent gaps in the motion gate Both from Rames' approval pass on #2977, and both found by running these functions rather than reading them. Both fail the same quiet way: a component `autoplays` misses is filtered out before any requirement runs, so the gate reports zero problems instead of a violation. `autoplays` had become narrower than the version it replaced. Excluding the `autoPlay={autoPlay}` passthrough was right, but the replacement only matched `autoPlay={` or `autoPlay` alone on a line, so `<video autoPlay muted />` on one line slipped through. Restored the old breadth. Two things are stripped first rather than one — the passthrough, and the prop's own default in the signature, which is a declaration and not a use. Without the second strip, `DocsVideo` is asked to own a decision it only forwards. `splitComponents` anchored on `^export`, so anything not exported folded into the previous exported component and inherited its guard. Same hole as the whole-file match, narrowed from file scope to non-export scope. The anchor no longer requires `export`. Ten tests now, including his exact examples for both. * docs: remove the live-composition embed and its build apparatus The Introduction no longer carries the embed (removed in #2979), and nothing else used any of this: the 200-line snippet, 26 CSS rules, the bundler that built the single-file HTML for the CDN, its npm script, and the README section explaining how to regenerate it. The Reference Project itself stays — Examples, Developers, and Go further all link to it as the worked example; only the interactive embed of it is gone. This also retires the isolation contract I documented two rounds ago. That comment existed because the embed handed CDN HTML to a same-origin blob; with the embed gone there is no such surface to reason about, which is a better outcome than a comment explaining why it was acceptable. * docs: remove the AgentAction snippet Its only consumer is gone. The Quickstart now shows the agent instruction in a plain fence instead, because this component rendered a Copy button and never displayed the request — a reader copied text they could not read, which is the wrong shape for the one affordance a non-technical visitor depends on. Mintlify fences already carry a copy button and show their contents. |
||
|
James Russo
|
1d636f603c |
refactor(producer): share plan execution builder (#2906)
## What Refactor distributed planning around one shared local execution-plan builder: - `buildLocalExecutionPlan()` now owns compile/probe/extract/audio/freeze. - Legacy `plan()` remains a deprecated v1 transport wrapper. - Plan v2 calls the shared builder directly and publishes through the existing manifest/CAS contract. - Add neutral `createPlanV2FromExecutionPlan()`, `publishPlanV2FromExecutionPlan()`, `getPlanV2ExecutionPlanHash()`, and `PLAN_PROTOCOL_V1` names. - Retain deprecated v1-named exports and wire aliases. - Recommend explicit Plan v2 opt-in for new producer, Lambda, and Cloud Run integrations. ## Why Plan v2 previously looked like it invoked a v1 planner even though v1 and v2 share the same frozen local execution representation. This removes that migration-era coupling while preserving the public minor-version compatibility contract. ## How The shared builder returns neutral internal execution-plan fields. The v1 wrapper maps those fields back to the existing `PlanResult`; the v2 publisher consumes them directly. Compatibility is intentional and covered by exact shape tests: - omitted `planProtocol` still serializes/selects `"v1"`; - v1 layouts, descriptor-less decoding, event unions, workflow branches, and exports remain; - the v1 descriptor JSON is byte-identical and `CURRENT_PLAN_PROTOCOL` is an identity-preserving alias; - v2 manifest bytes, key order, hash framing, and `sourcePlanV1Hash` wire key remain unchanged; - no enumerable neutral hash field was added to manifests or returned result objects; - v1/v2 result objects, cloud event payloads, and SDK handle key sets remain unchanged. ## Test plan - Focused Plan v1/v2/protocol/export/size compatibility: 141 passed - `@hyperframes/core`: 1,419 passed - `@hyperframes/producer` unit lane: 990 passed - `@hyperframes/aws-lambda`: 140 passed - `@hyperframes/gcp-cloud-run`: 101 passed - Producer, Lambda, and Cloud Run typechecks - Repository-wide lint, format check, workspace/package-subpath checks - Full workspace build - `git diff --check` - [x] Unit tests added/updated - [ ] Manual testing performed - [x] Documentation updated (if applicable) |
||
|
James Russo
|
557d82b6a9 |
fix(producer): validate distributed video metadata (#2839)
## What - enforce a finite, validated `meta/videos.json` contract shared by Plan v1 and Plan v2 - preserve authored finite ends and source-derived trim-aware ends; bound any still-open end at the validated composition end - fail distributed planning when any declared video source did not extract instead of publishing a blank-capable plan - make the v1 chunk reader reject malformed/null video timing before frame injection - route deterministic video-source/metadata failures as non-retryable in AWS and GCP while retaining retries for transient extraction failures ## Why An open-ended video whose remote source could not be resolved retained `Infinity` through planning. Plan v2 correctly rejected that value, while Plan v1 serialized it as `null`; the v1 frame lookup could then suppress injected frames and silently produce incorrect output. The invariant belongs at the shared metadata boundary. Both protocols must receive identical finite timing, and unavailable sources must fail closed before plan publication. ## Test plan - [x] producer distributed planning, metadata, v1 chunk boundary, Plan v2 conversion/materialization, and public exports - [x] core runtime media semantics (authored slots, natural duration, looping, non-looping hold) - [x] engine video extraction and frame lookup - [x] AWS Lambda/CDK/SAM and GCP Cloud Run error normalization/retry classification - [x] producer, core, engine, AWS, and GCP typechecks/builds - [x] formatting, oxlint, tracked-artifact, fallow, and commit hooks - [x] exact incident composition replayed through the AWS Lambda handler's Lambda-local path in a Lambda-like container; Plan v1 and Plan v2 both fail closed as `VIDEO_SOURCE_UNRENDERABLE` during planning, before plan publication - [x] full PR CI, including all nine regression shards and Windows render/tests No production flags or deployment/release workflows are changed. |
||
|
James
|
2a284a8e3a | fix(gcp): enforce effective BeginFrame capture | ||
|
James Russo
|
0499a5cbcb | fix(gcp-cloud-run): normalize v2 integrity codes (#2790) | ||
|
James Russo
|
5bf61d6df0 |
feat(aws-lambda): support plan protocol v2 (#2789)
* feat(aws-lambda): support plan protocol v2 * fix(aws-lambda): align SAM v2 terminal errors |
||
|
James RussoandClaude Opus 4.8
|
4da567df22 |
feat(gcp-cloud-run): Google Cloud Run + Workflows distributed render adapter (#1253)
* feat(gcp-cloud-run): add Google Cloud Run + Workflows distributed render adapter Adds @hyperframes/gcp-cloud-run, the GCP counterpart to @hyperframes/aws-lambda (issue #932). The OSS distributed primitives (plan, renderChunk x N, assemble) are unchanged; this package is the storage/compute/orchestration glue. Package: Cloud Run handler (one image, three actions), runs under bun; GCS transport; in-image chrome-headless-shell resolver; client SDK (renderToCloudRun, getRenderProgress, deploySite, computeRenderCost); Dockerfile; Cloud Workflows definition; Terraform module; CLI cloudrun deploy|sites|render|render-batch|progress|destroy with --output-resolution and --strict-variables; 62 unit tests + docs + live smoke script. Shared extraction (removes ~640 lines of adapter duplication): move the cloud-agnostic config validator + content-hash into producer/distributed; both adapters import them. Validated end-to-end on GCP at 37.4 dB PSNR vs baseline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): resolve @hyperframes/gcp-cloud-run in the CLI build + root build The CLI bundle (esbuild) couldn't resolve `@hyperframes/gcp-cloud-run/sdk`, failing Build/Typecheck/CLI-smoke (and the perf/windows/regression jobs that build first). Mirror the aws-lambda handling: mark the gcp adapter + its /sdk subpath external in tsup.config.ts with a source alias, and add gcp-cloud-run to the root `build` filter so its dist exists for publish + runtime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): copy gcp-cloud-run manifest in Dockerfile.test for frozen install The regression test image runs `bun install --frozen-lockfile` after copying each workspace package.json individually. The CLI now depends on @hyperframes/gcp-cloud-run (workspace:*), so the frozen install fails to resolve it unless its manifest is present. Add the COPY line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): add machine-sizing flags to `cloudrun deploy` Closes the parity gap with `lambda deploy` (which exposes --memory etc.). `cloudrun deploy` now threads --cpu, --memory, --max-instances, and --timeout into the Terraform apply; omitted flags keep the module defaults (4 vCPU / 16Gi / 100 instances / 3600s). For finer control, apply the module directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address PR review (security, waste, limits, alerts) - server.ts: bucket-allowlist guard no longer fails open silently. Unset env logs a one-time WARNING; "*" is an explicit opt-out; otherwise it enforces. - server.ts: stop double-shipping audio.aac. It already rides in the plan tarball every consumer downloads, so drop the redundant standalone upload (plan) + re-download/overwrite (assemble); assemble reads it from the untar, falling back to a supplied AudioGcsUri for compat. - server.ts: chunk extension via path.extname() instead of slice(lastIndexOf). - workflow.yaml: clamp parallel concurrency_limit to math.min(chunkCount, 20) — Cloud Workflows hard-caps concurrent iterations at 20. - Dockerfile: pin bun (bun-v1.3.9) so an interop change can't silently break the image rebuild. - terraform: add min_instances var (default 0); add a workflow-failure alert (finished_execution_count status=FAILED) alongside the request-count one. - costAccounting: document that displayCost excludes GCS storage/egress. Verified against the actual APIs: @google-cloud/workflows@4.4.0 ICreateExecutionRequest has no executionId (so the idempotency-token suggestion isn't available in this client); Workflows concurrency cap is 20; failure metric is workflows.googleapis.com/finished_execution_count (status label). 174 adapter tests pass, fallow/oxlint/oxfmt/terraform clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gcp-cloud-run): address round-2 review — error code + CFR forwarding - workflow.yaml: rename the zero-chunk failure code PLAN_TOO_LARGE → PLAN_PRODUCED_ZERO_CHUNKS. The old code implied a size-ceiling breach (the opposite cause), misleading anyone triaging the alert. - workflow.yaml: forward Config.cfr to the assemble step (`Cfr: ${("cfr" in config) and config.cfr}`). It was read by the handler but never sent, so exact-CFR was silently off for every Cloud Run render. Uses the same `in`-operator guard already proven in the retryable predicate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(release): include gcp-cloud-run in set-version PACKAGES list set-version.ts (driven by release:prepare) bumps an explicit package list to the shared version on each release. gcp-cloud-run was wired into the build + publish.yml but missing here, so a release would leave it at a stale version and publish.yml would push the wrong version. Add it so the new package version-bumps + publishes in lockstep with the others. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
James Russo
|
b6b6b8ed3b |
docs(lambda): add migration guide + non-Lambda Dockerfile example (#915)
Two adopter-facing artifacts that close out Phase 6b's user-facing
surface:
- docs/deploy/migrating-to-hyperframes-lambda.mdx — side-by-side
concept mapping for users coming from another one-command-deploy
video renderer. Covers the verb mapping (deploy/render/progress/
destroy/sites/policies), composition format (plain HTML vs JSX),
render config, and a handful of intentional differences (no HDR
in distributed mode, no webm, gpu-mode=software requirement,
fail-closed font fetch, local stack-state files, narrow-after-
first-deploy IAM pattern). Closes with a migration checklist.
Per repo convention, no competitor framework is named anywhere
in the source — adopters self-identify.
- examples/k8s-jobs/Dockerfile.example + README.md — reference
Dockerfile for adopters who want to run distributed renders
outside AWS Lambda. Bakes Node 22 + chrome-headless-shell +
ffmpeg + the producer source. Deliberately not published to
a registry; adopters build it themselves so Chrome / ffmpeg /
producer versions stay pinned to the checkout they audited.
The README documents the typical K8s Jobs orchestration shape
that points adopters at packages/aws-lambda/src/handler.ts as
the reference adapter.
Migration guide registered under the existing Deploy group in
docs.json. .gitignore extended to negate the new examples/k8s-jobs/
path the same way examples/aws-lambda/ is negated.
No source code changes.
|
||
|
James Russo
|
aa58ebc048 |
feat(lambda): add real-AWS smoke + benchmark script (#908)
* feat(lambda): add real-AWS smoke + benchmark script
Phase 6.3 of the distributed rendering plan
(DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6a). Local bash script that
deploys the PR 6.2 SAM template to your own AWS account, renders a
fixture composition through the Step Functions state machine at several
chunk counts, PSNR-compares each output against the in-process
baseline, and tears the stack down.
This is the gate that proves the architecture works on real Lambda
infrastructure. After it runs green and the numbers are good, Phase 6b
(CLI, CDK, docs) can proceed with confidence.
Script lives at examples/aws-lambda/scripts/smoke.sh. Defaults:
- fixture: mp4-h264-sdr
- chunk_counts: 2,4,8
- psnr-threshold: 50 dB
- region: us-east-1
- stack-name: hyperframes-lambda-smoke-<timestamp>
AWS credentials come from the standard resolution chain (env vars,
~/.aws/credentials, SSO, IMDS). Pin a specific profile via --profile or
AWS_PROFILE; the script doesn't ship a default.
Workflow:
1. Pre-flight: verify aws/sam/bun/ffmpeg/jq/zip on PATH; check
credentials via sts:GetCallerIdentity.
2. Build the PR 6.1 ZIP and run verify:zip-size.
3. sam validate --lint + sam deploy under a per-run stack name.
4. Zip the fixture's src/ and upload to the render bucket.
5. For each chunk count, start a Step Functions execution, poll for
completion (25-min cap), download the output mp4 and the execution
history JSON, ffmpeg-psnr against the LFS-tracked in-process
baseline, and append to results.json.
6. Gate on the PSNR threshold.
7. Empty the bucket + sam delete (unless --keep-stack).
Outputs land under ./lambda-smoke-artifacts/:
- results.json (chunkCount x wallClockMs x psnrAvgDb)
- renders/N<N>-output.mp4
- renders/N<N>-history.json (full Step Functions execution history)
Per-run stack name with concurrency-safe AWS resource isolation. Run
multiple smokes in parallel without races; teardown guards against
stale stacks via cleanup_and_exit on every failure path.
Distinction from CI: this is a maintainer-run gate, not part of regular
CI. The architecture's per-PR safety net is the local Docker-based
BeginFrame probe (PR 6.1) and the upcoming Lambda RIE smoke mode (PR
6.6). No GitHub Actions / OIDC / cross-account secrets required.
This is part of the 8-PR Phase 6 stack; PR 6.3 of 8 — the last PR of
Phase 6a (validation). Phase 6b (CLI + CDK + docs) starts once 6.3's
benchmark numbers come back.
* fix(lambda): address PR 880 review feedback
- Document wall-clock methodology bias inline (eval.sh header + README):
local timing includes bun + tsx + harness scaffolding while Lambda
timing measures pure SFN execution, so "speedup" is end-to-end CLI
experience, not renderer-vs-renderer.
- Add --iterations N (default 1) with median wall-clock reporting via
awk-side median. Cold-start variance is ±5-10s per chunk; single-
sample readings made the PR-body speedup table not ground truth.
- Add --reserved-concurrency flag to both scripts; default still 16 but
no longer hardcoded. Pass-through to ReservedConcurrency CFN param.
- README: cost-per-pass estimate for both scripts.
- Replace `sed -n '2,30p' "$0"` help with usage() heredoc in both
scripts — fragile to header reflows and didn't survive the comment
expansion this commit adds anyway.
- eval.sh RMS-level parser: add a third fallback (`RMS level:` with no
`dB` suffix) for older ffmpeg builds where astats predates the unit
tag. Word-boundary guards keep `RMS peak level` from being eaten.
* docs(lambda): drop internal plan-doc + Rio refs from smoke/eval scripts
|
||
|
James Russo
|
6664e54053 |
feat(lambda): add SAM template and sample events for AWS deployment (#907)
* feat(lambda): add SAM template and sample events for AWS deployment Phase 6.2 of the distributed rendering plan (DISTRIBUTED-RENDERING-PLAN.md §15). Reference SAM template for deploying HyperFrames distributed rendering on AWS — one Lambda function in three roles, choreographed by a Step Functions standard workflow with a Map state for parallel chunk rendering. Resources created by the template: - Lambda function pointing at the Phase 6.1 ZIP - Step Functions state machine: Plan -> Map(N) RenderChunk -> Assemble - S3 bucket for plan tarballs, chunk outputs, final mp4 - IAM role for the state machine - CloudWatch alarm guarding against runaway chunk invocations Retry policy: 4 attempts, 2s initial, 2x backoff, max 60s, with the typed non-retryable error codes from plan §9.3 explicitly opted out. CodeUri points at packages/aws-lambda/dist/handler.zip; sam deploy resolves the local path and uploads to a SAM-managed bucket on first deploy. Validated: sam validate --lint passes against the template. This is part of the 8-PR Phase 6 stack; PR 6.2 of 8. * fix(lambda): address PR 879 review feedback - Add CloudWatch alarms for Lambda Errors metric (5min window, threshold 1) and Step Functions ExecutionsFailed metric. The existing runaway- invocations alarm catches too-many-calls but missed silent per-chunk failures and retry-exhaustion. - Document VersioningConfiguration: Suspended tradeoff inline. Adopters treating the final mp4 as user-keepable should bump to Enabled. - Cost-allocation Tags on RenderBucket + Lambda Globals. - Lambda Tracing: Active so X-Ray spans don't terminate at the SF→Lambda boundary (the state machine already had tracing). - State-machine top-level TimeoutSeconds: 3600 as defensive ceiling on the whole choreography — catches Plan-retry storms before they hit individual task budgets. - AssertChunkCount Choice state: if Plan ever returns ChunkCount=0 the Map would silently iterate zero times and Assemble would receive an empty ChunkS3Uris[] producing an empty output. Fail-fast with typed PLAN_TOO_LARGE error instead. - Architecture comment: explicit x86_64-only constraint from @sparticuz/chromium so adopters trying Graviton don't get bitten. * docs(lambda): drop internal plan-doc refs from SAM example + template |