ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 04:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,192 Commits 856 Branches 404 Tags
ac8d34fb9db1442e6d4ea1e7f0bbdf9c91df2b79
Commit Graph
29 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
James Russo b7b855845a feat(cli): add hyperframes auth OAuth (PKCE + loopback + refresh) (#1084) 
## What

Adds OAuth 2.0 + PKCE login as the default for `hyperframes auth login`,
plus refresh-token + 401 auto-retry + `auth refresh`. Stacks on top of
PR #1081 (the API-key + shared store work).

- `hyperframes auth login` (no flags) — opens the user's browser to
  `/v1/oauth/authorize`, captures the code on an ephemeral
  `127.0.0.1:<port>/oauth/callback`, exchanges it for tokens with
  PKCE S256, and persists. `--api-key` opts back into the legacy
  long-lived-key path from PR #1081.
- `hyperframes auth refresh` — force-refresh the OAuth access token
  using the stored refresh_token. Mostly useful for testing the path.
- `hyperframes auth logout` — best-effort revokes via
  `POST /v1/oauth/revoke` (RFC 7009) before wiping local state.
- `AuthClient` now refreshes-and-retries once on a 401 when the
  caller wires `onUnauthenticatedRefresh`. `auth status` wires it.

Internals added in `packages/cli/src/auth/`:
- `pkce.ts` — RFC 7636 code_verifier + S256 code_challenge.
- `loopback.ts` — ephemeral 127.0.0.1 HTTP server; state validation,
  120s timeout, styled success/error page.
- `browser.ts` — wraps `open` with a `BROWSER=none` /
  `HF_NO_BROWSER=1` fallback that prints the URL.
- `oauth.ts` — `startAuthorizationCodeFlow`, `refreshTokens`,
  `revokeTokens`, `requireOAuthConfigured`, `parseTokenResponse`.

## Why

This is the foundation OAuth flow that lets free-tier users authenticate
without managing a long-lived key. Refresh + auto-retry means CLI
commands keep working past the access_token lifetime without bugging
the user.

The OAuth client_id (`q2A2QRSke2LrFTPJhoDbHtXh`) is the one James
created in the `oauth2_client` table. Baked in as a build-time default;
override via `HYPERFRAMES_OAUTH_CLIENT_ID` for dev/test.

## How

- Public client: PKCE only, no `client_secret`. Backend already
  requires PKCE (`movio/logic/oauth2.py:638`).
- Loopback port is ephemeral (`server.listen(0)`) — the backend
  wildcards localhost ports for public clients
  (`movio/model/oauth2.py:check_redirect_uri`), so the registered
  redirect URI's port is a placeholder.
- State parameter is generated per-flow + validated on callback to
  prevent CSRF.
- Token-response parsing is permissive on `expires_in` type (some
  servers return it as a string) but strict on `access_token` presence.
- 401 retry happens at the `AuthClient.fetchUser` layer, not the
  command layer — so future endpoints inherit it for free.
- `persistOAuth` merges into the existing store (preserves co-located
  `api_key`). `auth login` (API-key path) does the symmetric thing.

## Test plan

- [x] 80 unit tests, all green. `vitest run src/auth/`.
- [x] PKCE: verifier within 43-128 chars, challenge = SHA-256, S256
      method, distinct outputs each call.
- [x] Loopback: state mismatch / IdP error / missing-code / timeout /
      404 non-callback paths all rejected; success path captures `code`.
- [x] OAuth: `refreshTokens` posts correct body, persists, throws
      `REFRESH_FAILED` on 400/401 and `API_ERROR` on 5xx. Existing
      api_key preserved on refresh.
- [x] AuthClient: 401 retries with refreshed bearer on OAuth, does
      NOT retry for api_key, returns 401 if refresh hook fails.
- [x] `bunx oxlint` / `bunx oxfmt --check` / `bunx tsc` clean.
- [x] `bunx fallow audit --base origin/main --fail-on-issues` — only
      inherited `help.ts:showUsage` finding (from main, not this PR).
- [ ] Smoke test against dev API:
      `HEYGEN_API_URL=https://api.dev.heygen.com hyperframes auth login`
      then `hyperframes auth status` then `hyperframes auth refresh`.

## Out of scope

- Cloud render commands — separate plan.
- PR 4 (heygen-cli read-side JSON support) — independent, ships after.
 2026-05-28 02:13:24 -04:00
James Russo b9dbafdf6a feat(cli): add hyperframes auth login --api-key, status, logout (#1081) 
## What

Introduces the `hyperframes auth` command group + a shared credential
store library that hyperframes-CLI and heygen-cli will both read from.

- `hyperframes auth login --api-key` saves a HeyGen API key to
  `~/.heygen/credentials.json` (stdin pipe or hidden-input prompt).
- `hyperframes auth status` resolves the active credential (env vars
  → file) and verifies it against `GET /v3/users/me`, printing
  identity + billing.
- `hyperframes auth logout` removes the credential (`--keep-api-key`
  drops only the OAuth block).

Internals (`packages/cli/src/auth/`):
- `paths.ts` — `~/.heygen` layout, `HEYGEN_CONFIG_DIR` override.
- `store.ts` — read/write `credentials.json` (file 0600, dir 0700)
  with legacy single-line plaintext fallback so existing heygen-cli
  users don't lose their session.
- `resolver.ts` — chain: `HEYGEN_API_KEY` → `HYPERFRAMES_API_KEY` →
  file (unexpired OAuth wins over api_key).
- `client.ts` — hand-written typed wrapper for `GET /v3/users/me`
  (intentionally not OpenAPI codegen — single endpoint).
- `errors.ts` — typed `AuthError` with discriminating `code`.

## Why

This is the foundation for `hyperframes cloud render`. Splitting it
out keeps the cloud-render PR small and lets users sign in today.

The plan originally called for a library-only PR followed by a
commands PR. The `fallow` dead-code gate flagged the library-only
shape as unused exports, so I bundled them — the library and its
first consumers ship together. PR 3 (OAuth PKCE) and PR 4
(heygen-cli read-side JSON support) follow.

## How

- Credential file format: JSON with optional `api_key` + `oauth`
  blocks. Both CLIs read it; the resolver picks the freshest valid
  credential.
- Auth header selection happens in the HTTP client: OAuth →
  `Authorization: Bearer ...`, API key → `x-api-key: ...`.
- `HEYGEN_API_URL` lets dev testing target `api.dev.heygen.com`
  without rebuilding.
- The new `auth` command lazy-loads its subverbs (same pattern as
  `lambda`).

## Test plan

- [x] Unit tests added (`vitest`) for paths, store, resolver,
      client, and errors — 45 tests, all green.
- [x] `bunx tsc --noEmit -p packages/cli/tsconfig.json` clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` clean.
- [x] `bunx fallow audit --base origin/main --fail-on-issues` —
      zero new findings.
- [ ] Smoke test against dev API:
      `HEYGEN_API_URL=https://api.dev.heygen.com hyperframes auth login --api-key`
      then `hyperframes auth status`.
 2026-05-28 01:48:25 -04:00
James Russo 4729254b7e Merge pull request #1053 from heygen-com/fix/distributed-single-chunk-fps-flag 
fix(distributed): apply -r <fps> to single-chunk pass-through path
 2026-05-23 23:50:08 -04:00
James Russo 4f274d3605 Merge pull request #920 from heygen-com/chore/aws-lambda-publish-ready 
chore(lambda): publish-readiness for @hyperframes/aws-lambda
 2026-05-17 19:41:04 -04:00
James Russo cc6bc2e019 test(producer): add chunk-boundary fixtures per first-party adapter (#852) 
## Description

Phase 4 of the distributed rendering plan: test fixtures (see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 4 + §10 test strategy). This is PR 4.7 of the Phase 4 remainder — the final fixture PR.

This PR adds six per-adapter chunk-boundary fixtures under `tests/distributed/{gsap,anime,three,lottie,css,waapi}-boundary/` plus a single `bun:test` driver (`chunkBoundary.test.ts`) that exercises each fixture's seek-determinism contract. Each fixture is a 60-frame composition (2s @ 30fps, 320×180) that drives the named adapter through the HyperFrames runtime's seek hook. The test renders each at `chunkSize=60` (N=1 chunk, no seams) and `chunkSize=15` (N=4 chunks, three seams at frames 15/30/45), then asserts every PNG frame is byte-identical across the two runs.

**Why png-sequence**: mp4 bitstreams encode keyframe placement directly. At `chunkSize=60` libx264 emits 1 IDR; at `chunkSize=15` it emits 4 IDRs at frames 0/15/30/45. Those are legitimately different bytes even when the captured pixels are identical. png-sequence's assemble path merges chunk frame directories with no re-encode, so per-frame byte equality is exactly pixel equality — the strongest contract a distributed render can satisfy.

**Fixture design** (each is ~60 lines of HTML):
- `gsap-boundary` — single GSAP `tl.to(...)` driving translateX + rotation linearly across 2s.
- `anime-boundary` — anime.js v4 timeline registered via `window.__hfAnime`.
- `three-boundary` — minimal Three.js scene; cube rotation derived from `window.__hfThreeTime`.
- `lottie-boundary` — inline Lottie JSON (rectangle layer animating position+rotation) loaded via `lottie-web` and registered via `window.__hfLottie`.
- `css-boundary` — pure `@keyframes` animation; the HyperFrames CSS adapter seeks via `animation-delay`.
- `waapi-boundary` — `element.animate()` with linear keyframes; runtime sets `currentTime` per frame.

The fixtures intentionally omit `meta.json` so the regression-harness discovery skips them with a clear `missing meta.json` log (they're driven exclusively by `chunkBoundary.test.ts`). The test passes `rejectOnSystemFonts: false` because some adapter bundles (notably anime.js's IIFE) embed CSS-shaped strings inside their JS source — `font-family: ui-monospace, monospace` for internal devtools styling — which `validateNoSystemFonts`'s document-wide regex would otherwise false-positive on every adapter fixture that loads such a bundle. The fixtures display no text, so the relaxed font validation doesn't affect the contract under test.

The 7th test case is a layout sanity check that asserts every expected `*-boundary` fixture directory exists.

## Testing

- `bun test packages/producer/src/services/distributed/chunkBoundary.test.ts` — 7 tests pass on host (6 adapters × byte-identical N=1 vs N=4 + the layout check, 41.6s)
- `bun test packages/producer/src/services/distributed/` — all 49 distributed unit tests pass (43.6s)
- `bun run --cwd packages/producer docker:test:distributed font-variant-numeric many-cuts gsap-letters-render-compat style-1-prod sub-composition-video mp4-h264-sdr png-sequence mov-prores mp4-h265-sdr -- --sequential` — full smoke set + all four prior stacked fixtures pass (9/9)
- `bunx oxlint` + `bunx oxfmt --check` clean
- `bunx tsc --noEmit` (producer package) clean

## After this stack lands

The Phase 4 fixture set is complete: PR 4.6 (#844) pins cross-worker idempotency; 4.2/4.3/4.4/4.5 (#845/#851/#847/#848) prove each format produces correct chunked output at the fixture's `minPsnr`; 4.3-pre (#850) added the codec knob H.265 needed; and this PR proves each first-party adapter's seek-determinism survives chunk seams. Phase 5 (CLI surface for `hyperframes plan/chunk/assemble`) and Phase 6 (AWS Lambda turnkey) are unblocked.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-14 23:09:56 -04:00
James Russo 7bd4be13c1 test(producer): add mp4 H.265 SDR distributed fixture (#851) 
## Description

Phase 4 of the distributed rendering plan: test fixtures (see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 4 + §10 test strategy). This is PR 4.3 of the Phase 4 remainder, stacked on PR 4.3-pre (#850) which added the codec knob to `DistributedRenderConfig`.

This PR adds the mp4 H.265 SDR fixture (`tests/distributed/mp4-h265-sdr/`) plus the small harness plumbing that exposes the codec knob through `meta.json`. Composition mirrors the H.264 fixture: 2 seconds (60 frames) at 30fps with text, a crossfade transition straddling the frame-30 chunk seam, and a continuously rotating SVG icon. `renderConfig.format: "mp4"` + `renderConfig.codec: "h265"` + `chunkSize: 15` routes the distributed pipeline through libx265 with closed-GOP keyint params (`min-keyint=N:scenecut=0:open-gop=0:repeat-headers=1`) so concat-copy at assemble time round-trips losslessly.

**Cross-codec PSNR assertion**: the in-process renderer doesn't expose a codec hint (its RenderConfig only switches codec via HDR mode), so the in-process baseline for this fixture is rendered as h264. The harness's PSNR comparison therefore measures "libx265 chunked + concat" against "libx264 single-pass" on the same source frames. At "high" quality both encoders are near-lossless on simple vector+text content; observed PSNR is ~48dB across all 100 checkpoints — well above the 30dB threshold. This catches gross codec/encoder failures (e.g. libx265 emitting wrong bit depth or losing the IDR-at-chunk-seam contract) while accepting normal cross-codec PSNR drift. The in-process arm renders h264 vs the h264 baseline byte-identically.

Harness extensions:

1. **`TestMetadata.renderConfig.codec`** field accepted by `validateMetadata`. Rejected with format ∉ {mp4} for symmetry with the `DistributedRenderConfig` runtime check from 4.3-pre.
2. **`RunDistributedSimulatedInput.codec`** plumbed through to `plan()`. The non-mp4 plan-config branch keeps the field structurally absent (so byte-identical to pre-codec planDirs for mov/png-sequence) rather than passing `undefined`, which would surface in JSON.

## Testing

- `bun run --cwd packages/producer docker:test:update mp4-h265-sdr` — baseline rendered inside `Dockerfile.test` (h264 mp4 from in-process, used as the cross-codec reference)
- `bun run --cwd packages/producer docker:test mp4-h265-sdr` — in-process passes (renders h264, byte-identical against h264 baseline)
- `bun run --cwd packages/producer docker:test:distributed mp4-h265-sdr` — distributed-simulated passes (h265 mp4, ~48dB PSNR across all 100 checkpoints vs h264 baseline)
- `bun run --cwd packages/producer docker:test:distributed font-variant-numeric many-cuts gsap-letters-render-compat style-1-prod sub-composition-video mp4-h264-sdr png-sequence mov-prores mp4-h265-sdr -- --sequential` — full smoke set + all 4 stacked fixtures pass (9/9)
- `bunx oxlint` + `bunx oxfmt --check` clean
- `bunx tsc --noEmit` (producer package) clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-14 22:46:35 -04:00
James Russo 2886c2b24d feat(producer): add codec knob to DistributedRenderConfig (#850) 
## Description

Phase 4 prerequisite for PR 4.3 (the mp4 H.265 SDR distributed fixture). Splits the codec selection out of `DistributedRenderConfig.format` so callers can ask for libx265 without changing the format.

**Surface change** (`@hyperframes/producer/distributed`):

- `DistributedRenderConfig.codec?: "h264" | "h265"` — defaults to `"h264"`, ignored for non-mp4 formats. Passing `codec` with `format !== "mp4"` throws at plan time with a clear error so caller mistakes surface immediately rather than producing a silently-wrong planDir.
- `FORMAT_ENCODER_TABLE` is replaced by `resolveEncoderTriple(config)` — a small function that switches on `(format, codec)`. mp4 + h265 → `{encoder: "libx265-software", pixelFormat: "yuv420p"}`. mov and png-sequence are unchanged.

**Plumbing through `renderChunk`**:

The chunk worker reads `LockedRenderConfig.encoder` from `meta/encoder.json`. When that's `"libx265-software"`, the worker overrides `getEncoderPreset(quality, "mp4")`'s default `codec: "h264"` with `"h265"` so `runEncodeStage` invokes libx265 with the closed-GOP keyint params (`min-keyint=N:scenecut=0:open-gop=0:repeat-headers=1`) that survive concat-copy at assemble time. The engine layer (`packages/engine/src/services/chunkEncoder.ts`) already supports both codecs — this PR is purely the distributed config surface.

**Bit depth**: SDR-only, 8-bit yuv420p for both codecs. h265 + 10-bit yuv420p10le is HDR territory and lives in v1.5 (see plan §12).

## Testing

- `bun test packages/producer/src/services/distributed/plan.test.ts` — 14 tests pass including 3 new codec cases (`codec` defaults to h264, `codec: "h265"` maps to libx265-software, non-mp4 + codec throws)
- `bun test packages/producer/src/services/distributed/` — all 42 distributed unit tests pass
- `bunx oxlint` + `bunx oxfmt --check` clean
- `bunx tsc --noEmit` (producer package) clean

The H.265 fixture that exercises this end-to-end inside `Dockerfile.test` lands in the follow-up PR 4.3.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-14 22:21:30 -04:00
James Russo b47a3e798b feat(producer): refuse distributed-unsupported formats (webm + HDR mp4) (#815) 
## What

Phase 3 of the distributed rendering plan: §11 PR 3.5 format banlist. Extends `plan()` to refuse two v1-unsupported formats up front with a typed non-retryable `FormatNotSupportedInDistributedError` (`code === "FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED"`).

## Why

Both webm and HDR mp4 are documented as deferred to v1.5 (§7.2 + §12), but until this PR the only signal at the runtime layer is the in-process pipeline silently producing wrong output (chunk concat-copy doesn't round-trip VP9; HDR signaling gets stripped at the chunk boundary). Failing fast at `plan()` time keeps adopters from spending fan-out compute on a render that can't succeed and gives them a typed error code their workflow adapter can route on.

## How

- New exports in `services/distributed/plan.ts`:
  - `FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED` — non-retryable error code matching §11's wording.
  - `FormatNotSupportedInDistributedError` — typed error class with `code`, `format`, and `reason` fields. Message names the rejected format and tells adopters to fall back to the in-process renderer (`executeRenderJob`) which has full format support.
  - `rejectUnsupportedDistributedFormat(config)` — pure helper exported separately so adapters can run the same gate at their input layer (Step Functions input validation, Temporal workflow start) before the activity even runs.
- `plan()` calls `rejectUnsupportedDistributedFormat(config)` as the first line of the function — BEFORE `mkdirSync(planDir)` so a banned input never produces a partial planDir.
- Replaced the previous ad-hoc `if (hdrMode === "force-hdr") throw new Error(...)` with the typed error class.

### What did NOT change

`executeRenderJob`, the in-process orchestrator, the `hyperframes render` CLI, producer HTTP routes — all unchanged. The in-process renderer continues to accept webm + HDR (its existing functionality).

## Test plan

- [x] Unit tests added — `packages/producer/src/services/distributed/planFormatBanlist.test.ts`. 5 cases:
  - `rejectUnsupportedDistributedFormat` accepts the v1-supported formats (mp4, mov, png-sequence) with both `auto` and `force-sdr` hdrMode.
  - Rejects webm — error has `code === FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED`, `format === "webm"`, message mentions in-process renderer.
  - Rejects HDR mp4 (`hdrMode === "force-hdr"`) — error has `format === "mp4-hdr"`, message mentions HDR.
  - End-to-end via `plan()`: webm throws with no planDir leaking to disk.
  - End-to-end via `plan()`: HDR mp4 throws with no planDir leaking to disk.
- [x] `bun test packages/producer/src/services/distributed/` — 30 pass.
- [x] `bun run --filter @hyperframes/producer typecheck` — clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files.
- [ ] Producer Docker regression harness — pending CI. `executeRenderJob` is unchanged; PSNR baselines should hold.

This is PR 5 of a 6-PR Phase 3 stack:

- 3.1 — `services/distributed/plan.ts` (#808)
- 3.2 — `services/distributed/renderChunk.ts` (#809)
- 3.3 — `services/distributed/assemble.ts` (#813)
- 3.4 — `planDir` size cap (`PLAN_TOO_LARGE`) (#814)
- **3.5 (this PR)** — distributed format banlist (webm + HDR mp4)
- 3.6 — public exports + `@hyperframes/producer/distributed` subpath

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-13 23:38:29 -04:00
James Russo 3eb7ad26ad feat(producer): enforce planDir size cap (PLAN_TOO_LARGE) (#814) 
## What

Phase 3 of the distributed rendering plan: §6.4 / §9.3 size cap. Extends `plan()` to measure the produced planDir's total byte size after freeze and throw a typed non-retryable `PlanTooLargeError` (`code === "PLAN_TOO_LARGE"`) when the planDir exceeds 2 GB.

## Why

Distributed chunk workers ship the entire planDir to whatever ephemeral storage they're running on — `/tmp` on AWS Lambda (10 GB), the container filesystem on Cloud Run Jobs, etc. A planDir that doesn't fit can't be rendered. v1.5 lifts this cap via per-chunk video-frame slicing (§12); for now v1 fails fast at plan time so adapters don't waste a fan-out attempt that's guaranteed to OOM.

The 2 GB ceiling specifically targets Lambda's 10 GB `/tmp`: planDir + per-chunk captured frames + ffmpeg's working set all share that budget, and 2 GB leaves ~8 GB for capture/encode at 4K SDR.

## How

- New exports in `services/distributed/plan.ts`:
  - `PLAN_DIR_SIZE_LIMIT_BYTES` — the 2 GB constant.
  - `PLAN_TOO_LARGE` — the non-retryable error code (matches §9.3).
  - `PlanTooLargeError` — typed error class carrying `code`, `sizeBytes`, `limitBytes`, and a message that points adopters at the v1.5 slicing roadmap + the in-process renderer escape hatch.
  - `measurePlanDirBytes(planDir)` — recursive on-disk size walker. Symlinks skipped intentionally.
- `DistributedRenderConfig.planDirSizeLimitBytes?: number` — optional override. Defaults to `PLAN_DIR_SIZE_LIMIT_BYTES`. Tests pass a tiny cap (1024 bytes) to exercise the throw path without filling 2 GB of /tmp.
- The check runs in `plan()` AFTER the temp work tree is removed (so `.plan-work/` doesn't double-count) but BEFORE the function returns — adapters that catch the error never see a `PlanResult`.

### What did NOT change

`executeRenderJob`, the in-process orchestrator, the `hyperframes render` CLI, producer HTTP routes — all unchanged. Only `plan()` (which is itself opt-in) enforces the cap.

## Test plan

- [x] Unit tests added — `packages/producer/src/services/distributed/planSizeCap.test.ts`. 7 cases:
  - `measurePlanDirBytes` returns 0 for an empty dir, sums recursively, and gracefully ignores broken entries.
  - `PLAN_DIR_SIZE_LIMIT_BYTES` is `2 * 1024 * 1024 * 1024` (§6.4 pin).
  - `PlanTooLargeError` carries the `PLAN_TOO_LARGE` code + `sizeBytes` + `limitBytes` and mentions the v1.5 escape hatch.
  - `plan()` throws `PlanTooLargeError` when configured with a 1024-byte ceiling.
  - `plan()` succeeds when the default 2 GB ceiling is well above the produced planDir.
- [x] `bun test packages/producer/src/services/distributed/` — 25 pass (PRs 3.1 + 3.2 + 3.3 + 3.4).
- [x] `bun run --filter @hyperframes/producer typecheck` — clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files.
- [ ] Producer Docker regression harness — pending CI. `executeRenderJob` is unchanged; PSNR baselines should hold.

This is PR 4 of a 6-PR Phase 3 stack:

- 3.1 — `services/distributed/plan.ts` (#808)
- 3.2 — `services/distributed/renderChunk.ts` (#809)
- 3.3 — `services/distributed/assemble.ts` (#813)
- **3.4 (this PR)** — `planDir` size cap (`PLAN_TOO_LARGE`)
- 3.5 — distributed format banlist (webm + HDR mp4)
- 3.6 — public exports + `@hyperframes/producer/distributed` subpath

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-13 23:08:25 -04:00
James Russo 3468db6f37 feat(producer): add services/distributed/assemble.ts (#813) 
## What

Phase 3 of the distributed rendering plan: the third public primitive. Adds `assemble(planDir, chunkPaths, audioPath, outputPath)` and its supporting types at `packages/producer/src/services/distributed/assemble.ts`. See `DISTRIBUTED-RENDERING-PLAN.md` §11 Phase 3.

## Why

`plan()` (#808) and `renderChunk()` (#809) produce the planDir and the per-chunk outputs respectively. `assemble()` is what every distributed fan-out workflow runs last: it stitches the chunks into the final deliverable. Without it, the planDir → chunks chain stops at a list of files; nothing produces the user-facing mp4/mov/png-sequence.

## How

`assemble()` branches on the planDir's encoder format:

**mp4 / mov**: ffmpeg `-f concat -c copy` over the ordered chunk paths. Each chunk's first frame is an IDR keyframe (PR 3.2 set `lockGopForChunkConcat: true`), so concat-copy round-trips losslessly. The concatenated output is then:
1. Passed through `padOrTrimAudioToVideoFrameCount` (PR 2.7 surface) when `audioPath` is non-null, so audio length is exactly `frameCount / fps` rather than the audio mixer's original-duration output.
2. Muxed with the normalized audio via the engine's `muxVideoWithAudio` (same helper the in-process renderer's `assembleStage` uses).
3. Passed through `applyFaststart` so the `moov` atom moves to the file's start.

When no audio is present, the concat output skips mux and goes straight to `applyFaststart`.

**png-sequence**: chunks are directories of `frame_NNNNNN.png` files numbered locally per chunk. `assemble()` merges them with a continuous global index so chunk 0's `frame_000000.png` lands at `frame_000001.png` in the output, chunk 1's first frame becomes `frame_(N+1)`, etc. When `audioPath` is non-null we copy it alongside as `audio.aac` so callers who need to re-mux later have it.

### Validation

Both branches assert `chunkPaths.length === chunks.length` (the value read from `meta/chunks.json`) and that each chunk path exists. A missing or mismatched manifest trips a typed error before any ffmpeg invocation.

### What did NOT change

No engine helpers, no Phase 1 stages, no in-process orchestrator. `assemble()` reuses `muxVideoWithAudio` / `applyFaststart` / `runFfmpeg` / `padOrTrimAudioToVideoFrameCount` exactly as they exist — the in-process `runAssembleStage` is intentionally not called because it operates on a `RenderJob` and emits `updateJobStatus` payloads, neither of which the distributed activity has.

## Test plan

- [x] Unit tests added — `packages/producer/src/services/distributed/assemble.test.ts`. 5 cases:
  - Concat-copies two mp4 chunks and applies faststart (ffprobe asserts codec, frame count, atom order).
  - Muxes audio with `frame-count-derived` duration when `audio.aac` is present (ffprobe asserts audio duration within 50ms of `totalFrames / fps`).
  - Merges png-sequence chunk directories with continuous global numbering (asserts filenames `frame_000001..frame_000007`).
  - Rejects mismatched `chunkPaths.length` vs `chunks.json.length`.
  - Rejects a planDir missing `plan.json`.
- [x] `bun test packages/producer/src/services/distributed/` — 18 pass (PRs 3.1 + 3.2 + 3.3).
- [x] `bun run --filter @hyperframes/producer typecheck` — clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files.
- [ ] Producer Docker regression harness — pending CI. `executeRenderJob` is unchanged; PSNR baselines should hold.

The mp4 fixture pre-renders test inputs via raw ffmpeg (`testsrc` filter + closed-GOP libx264) rather than going through the Chrome capture pipeline. This isolates concat-copy + mux + faststart from the renderChunk path that PRs 3.1/3.2 already cover, and avoids the chrome-headless-shell smoke-test gating that PR 3.2 needed.

This is PR 3 of a 6-PR Phase 3 stack:

- 3.1 — `services/distributed/plan.ts` (#808)
- 3.2 — `services/distributed/renderChunk.ts` (#809)
- **3.3 (this PR)** — `services/distributed/assemble.ts`
- 3.4 — `planDir` size cap (`PLAN_TOO_LARGE`)
- 3.5 — distributed format banlist (webm + HDR mp4)
- 3.6 — public exports + `@hyperframes/producer/distributed` subpath

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-13 22:38:46 -04:00
James Russo 602ef8779c feat(producer): add services/distributed/renderChunk.ts (#809) 
## What

Phase 3 of the distributed rendering plan: the second public primitive. Adds `renderChunk(planDir, chunkIndex, outputChunkPath)` and its supporting types as `packages/producer/src/services/distributed/renderChunk.ts`. See `DISTRIBUTED-RENDERING-PLAN.md` §11 Phase 3.

## Why

`plan()` (PR #808) produces the planDir. `renderChunk()` is what every chunk worker actually runs — Temporal activity, Lambda Step Functions Map task, Cloud Run Job invocation. Phase 1 extracted capture/encode stages; Phase 2 added the determinism-hardening flags those stages need. This PR is the first caller that flips them `true` for the capture + encode path (PR #808 was the first caller for `compileForRender`'s flag).

## How

`renderChunk` composes the pipeline:

1. Read + validate `plan.json`, `meta/{composition,encoder,chunks}.json`. Out-of-range `chunkIndex`, missing artifacts, or `browserGpuMode !== "software"` trip a typed `RenderChunkValidationError` with `code === PLAN_HASH_MISMATCH` / `BROWSER_GPU_NOT_SOFTWARE`.
2. `readFfmpegVersion()` matches against `plan.ffmpegVersion` — any drift trips a non-retryable `FFMPEG_VERSION_MISMATCH` per §9.3.
3. `applyRuntimeEnvSnapshot(encoder.runtimeEnv)` BEFORE the file server is created — `RENDER_MODE_SCRIPT` bakes those env vars into served HTML at module load.
4. File server points at `<planDir>/compiled/` with `buildVirtualTimeShim({ seedRandomFromFrame: true })` (PR 2.4 surface).
5. `createCaptureSession` with `lockWarmupTicks: true` (PR 2.3 surface).
6. `assertSwiftShader(session.page, readWebGlVendorInfoFromCanvas)` BEFORE `initializeSession`. The default `assertSwiftShader` reader navigates to `chrome://gpu`, which `chrome-headless-shell` serves as an empty document on multiple builds we've tested; this PR threads a canvas + `WEBGL_debug_renderer_info` reader through the helper's existing `readInfo` override so the assertion works on both regular Chrome and `chrome-headless-shell`. No fork of the Phase 2 helper.
7. `discardWarmupCapture(session, startFrame, startTime)` once before the chunk's first real frame (PR 2.6 surface).
8. `runCaptureStage` with `frameRange: { startFrame, endFrame }` and `workerCount: 1`. The new optional `frameRange` field on `CaptureStageInput` extends the sequential capture branch: per-frame TIMES use absolute composition frame indices (so virtual time matches an in-process render at that frame), file NAMES normalize to zero (so the encoder reads them without an `-start_number` override). In-process callers omit `frameRange` and get the existing `[0, totalFrames)` behavior verbatim.
9. `runEncodeStage` with `lockGopForChunkConcat: true` + `gopSize: framesInChunk` + `hasAudio: false`. Closed-GOP per §7.1 so concat-copy at assemble time round-trips losslessly. The two new optional fields are pass-through to `EncoderOptions`; in-process callers omit them.
10. Hash output via SHA-256 (file for mp4/mov; sorted-frame-fingerprint for png-sequence), write a perf sidecar, return `ChunkResult`.

### Changes outside the new module

| File | Change | Backwards compat |
|---|---|---|
| `render/stages/captureStage.ts` | Optional `frameRange?: { startFrame, endFrame }` on `CaptureStageInput`. Sequential branch only; rejects `workerCount > 1 && frameRange`. | `frameRange === undefined` → identical behavior to `[0, totalFrames)`. |
| `render/stages/encodeStage.ts` | Optional `lockGopForChunkConcat?` + `gopSize?` on `EncodeStageInput`, pass-through to `EncoderOptions`. | Defaults are pass-through; in-process call site omits both. |

`renderOrchestrator.ts`, the `hyperframes render` CLI, and the producer HTTP routes are untouched.

## Test plan

- [x] Unit tests added — `packages/producer/src/services/distributed/renderChunk.test.ts`. 3 cases:
  - Byte-identical retry contract (§5.1) — renders chunk 0 twice on the same planDir and asserts the `ChunkResult.sha256` matches.
  - OOB `chunkIndex` rejected before Chrome init.
  - Missing `plan.json` rejected before Chrome init.
- [x] `bun test packages/producer/src/services/distributed/` — 13 pass (PR 3.1 + 3.2 combined).
- [x] `bun test packages/producer/src/` — 315 pass, 1 fail. The one failure is the pre-existing `writeCompiledArtifacts — external assets on Windows drive-letter paths` flake that also fails on a clean checkout of `origin/main`.
- [x] `bun run --filter @hyperframes/producer typecheck` — clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files.
- [ ] Producer Docker regression harness — pending CI. `executeRenderJob` is unchanged here; PSNR baselines should hold.

### Known limitation surfaced during testing

`chrome-headless-shell` on some dev/CI hosts can't initialize SwiftShader (the GL stack errors out before `BeginFrame` can run). The byte-identical retry test soft-skips on those hosts and the Docker harness (where the chrome-headless-shell build is matched to the planDir's ffmpegVersion) is the source of truth for the determinism contract. To exercise the test locally on a host with a working `chrome-headless-shell`, the test fixture uses `format: "png-sequence"` which forces screenshot mode and avoids the BeginFrame dependency.

This is PR 2 of a 6-PR Phase 3 stack:

- 3.1 — `services/distributed/plan.ts` (#808)
- **3.2 (this PR)** — `services/distributed/renderChunk.ts`
- 3.3 — `services/distributed/assemble.ts`
- 3.4 — `planDir` size cap (`PLAN_TOO_LARGE`)
- 3.5 — distributed format banlist (webm + HDR mp4)
- 3.6 — public exports + `@hyperframes/producer/distributed` subpath

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-13 22:09:40 -04:00
James Russo 7585f79dc1 feat(producer): add services/distributed/plan.ts (#808) 
## What

Phase 3 of the distributed rendering plan: the first half of the public distributed primitives. Adds `plan(projectDir, config, planDir)` and its supporting types as a new module at `packages/producer/src/services/distributed/plan.ts`. See `DISTRIBUTED-RENDERING-PLAN.md` §11 Phase 3.

## Why

Phase 1 extracted the in-process renderer's six pipeline phases into individually-callable stage functions; Phase 2 added the determinism-hardening utilities and flags those stages needed. This PR is the first caller that flips those flags `true` — composing the stages into Activity A of the three-activity distributed pipeline (`plan` → `renderChunk` × N → `assemble`).

Output is a self-contained `<planDir>/` with the documented §4.1 layout plus a content-addressed `planHash` (§4.2). Adapter authors (Temporal, AWS Lambda + Step Functions, etc.) consume the directory + hash; the OSS library never touches transport.

## How

`plan()` composes (in order):

1. `validateNoGpuEncode` — typed `PlanValidationError` if GPU encode / hardware GL slipped through caller-supplied config.
2. `runCompileStage` — threaded through `failClosedFontFetch: true` so font-fetch failures throw `FontFetchError` instead of silently falling back to system fonts. Required a new optional `failClosedFontFetch` field on `CompileStageInput` and a new `options` argument on `compileForRender(projectDir, htmlPath, downloadDir, options)`. Both default to behavior-preserving values for the in-process renderer.
3. `validateNoSystemFonts(compiled.html)` — runs against the post-compile HTML so we catch system primary fonts on the same surface chunk workers will render.
4. `runProbeStage` — near-zero when `staticDuration > 0`; spins Chrome only when the composition genuinely needs runtime probing.
5. `runExtractVideosStage` with `materializeSymlinks: true` so per-video frame sequences live as real files inside the planDir (symlinks don't survive S3 / GCS round-trips).
6. `runAudioStage` — produces `<planDir>/audio.aac` if the composition has audio.
7. Materialize the `<planDir>/{compiled,video-frames,audio.aac,meta}/...` layout from the staged work tree.
8. `freezePlan` — writes `meta/{composition,encoder,chunks}.json` + `plan.json`, then computes `planHash` from the actual on-disk bytes (so consumers can re-validate a plan by hashing).

`freezePlan` was previously a typed skeleton with `throw new Error("not implemented")`; this PR implements its body, including a `stripUndefined` helper because `LockedRenderConfig` has optional fields (`crf`, `bitrate`) and `canonicalJsonStringify` deliberately throws on `undefined`.

Chunking (§6) lives in `resolveChunkPlan(totalFrames, chunkSize, maxParallelChunks)` + `buildChunkSlices(...)` — exported from `plan.ts` so PR 3.2 (renderChunk) and adapter code can import them directly.

### What did NOT change

`executeRenderJob`, the `hyperframes render` CLI, the producer HTTP `/render` routes, and every existing stage signature are untouched. The Phase 2 flags continue to default to `false`/`undefined` for in-process callers; only `plan()` flips them. PSNR baselines for the regression harness should be unchanged.

## Test plan

- [x] Unit tests added — `packages/producer/src/services/distributed/plan.test.ts`. 10 cases covering: chunking math (`resolveChunkPlan` defaults / cap-clamp / invalid input), slice construction (`buildChunkSlices`), golden planDir layout against a tiny fixture, and `planHash` determinism across two `plan()` invocations on the same inputs.
- [x] `bun test packages/producer/src/services/distributed/` — 10 pass.
- [x] `bun test packages/producer/src/` — 312 pass, 1 fail. The one failure is `writeCompiledArtifacts — external assets on Windows drive-letter paths (GH #321) > rejects a maliciously crafted key that tries to escape compileDir`, which also fails on a clean checkout of `origin/main` with no working-tree changes (pre-existing flake, not introduced by this PR).
- [x] `bun run --filter @hyperframes/producer typecheck` — clean.
- [x] `bun run --filter @hyperframes/producer build` — clean.
- [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files.
- [ ] Producer Docker regression harness — pending CI run. `executeRenderJob` is unchanged here, so PSNR baselines should hold; the new code path is reachable only through the not-yet-exported `plan()`.

This is PR 1 of a 6-PR Phase 3 stack:

- **3.1 (this PR)** — `services/distributed/plan.ts`
- 3.2 — `services/distributed/renderChunk.ts`
- 3.3 — `services/distributed/assemble.ts`
- 3.4 — `planDir` size cap (`PLAN_TOO_LARGE`)
- 3.5 — distributed format banlist (webm + HDR mp4)
- 3.6 — public exports + `@hyperframes/producer/distributed` subpath

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-13 20:46:51 -04:00
James Russo 73966cd53b feat(core): add 4k canvas resolution presets (#660) 
## What

Adds `landscape-4k` (3840×2160) and `portrait-4k` (2160×3840) presets to `CanvasResolution` and `CANVAS_DIMENSIONS` in `@hyperframes/core`. Foundation for end-to-end 4K rendering support.

## Why

There is no codified way today to mark a composition as 4K. The string union `CanvasResolution = "landscape" | "portrait"` is the only enum used by templates, generators, and stage-zoom math; without 4K members, scaffolds and helpers always emit 1080p dimensions even when the underlying engine + encoder pipeline can already handle larger viewports (Chrome `setViewport`, ffmpeg `libx264`/`libvpx-vp9`/`prores_ks` all scale fine).

This is PR 1 of a 3-PR stack making 4K a first-class option:

1. **PR #660 (this)** — core types/constants + parser detection.
2. **PR #661** — `hyperframes init --resolution 4k` flag to scaffold 4K projects.
3. **PR #662** — byte-budget the frame data-URI cache so 4K renders don't OOM.

## How

- `CanvasResolution` extended additively (no breaking change).
- `CANVAS_DIMENSIONS` gains `landscape-4k` / `portrait-4k` entries.
- `parseResolutionFromHtml` accepts `data-resolution="landscape-4k|portrait-4k"` and infers 4K from `data-composition-width`/`data-composition-height` (long side ≥ 2560 → UHD variant).
- `parseResolutionFromCss` reuses the same dimension-aware classifier so inline `#stage { width/height }` styles are detected too.
- Helper extracted: `resolveResolutionFromDimensions(w, h)`.
- `cli/info.ts` cleanup: replaces a nested `parsed.resolution === "portrait" ? 1080 : 1920` ternary with a `CANVAS_DIMENSIONS[parsed.resolution]` lookup so it stays correct for 4K.

Stage-CSS generators (`templates/base.ts`, `generators/hyperframes.ts`) already index `CANVAS_DIMENSIONS[resolution]`, so they pick up the new presets automatically.

## Test plan

- [x] Unit tests added/updated — 4 new parser tests, 1 expanded constants test
- [x] Manual testing performed — `bun run --cwd packages/core test` (679 pass), `bun run --cwd packages/cli test` (277 pass)
- [ ] Documentation updated (deferred to PR #661 where the user-facing flag lands)
 2026-05-06 23:09:12 -07:00
James Russo 0e7e33cf7d docs: add deployment guide for Vercel and Cloudflare templates (#653) 
## What

Adds a new guide at `docs/guides/deploy.mdx` covering the two official one-click deployment templates:

- [heygen-com/hyperframes-vercel-template](https://github.com/heygen-com/hyperframes-vercel-template) (Vercel Sandbox + Vercel Blob)
- [heygen-com/hyperframes-cloudflare-template](https://github.com/heygen-com/hyperframes-cloudflare-template) (Cloudflare Containers + R2)

Wires the new page into `docs.json` under Guides, slotted right after `guides/rendering` (logical follow-on: render locally → deploy to the cloud).

## Why

The two templates currently only exist as GitHub READMEs, so they're invisible to anyone reading the docs site. New users who want a hosted preview + render endpoint have no entry point in the docs to discover them.

## How

Single guide with:

- A comparison table (compute / storage / deploy button) so readers can pick at a glance.
- `<Tabs>` for per-platform details (deploy button, what you get, performance, pricing, "why this primitive").
- Shared architecture diagram and the common "pre-baked renderer" cost pattern.
- "Swapping the composition" steps that work for both templates.
- "When to use a template vs. roll your own" framing for queues, multi-tenant, self-hosted.

Single-page approach (vs. one page per template) chosen because content overlaps heavily — easier to discover and lower maintenance until more templates land.

## Test plan

- [x] No code changes — docs only
- [x] `bunx oxfmt --check` and `bunx oxlint` pass on changed files (no rules apply to `.mdx` / `.json` here)
- [ ] Render the docs site locally to verify nav placement and Mintlify components (`<Tabs>`, `<Steps>`, `<CardGroup>`, `<Note>`) render correctly
- [ ] Verify the Vercel and Cloudflare deploy buttons in the page open the correct template URLs
 2026-05-06 16:18:49 -07:00
James Russo f1d408eed3 test(producer): add variables-prod regression for the variables stack (PR 5/5) (#604) 
## What

End-to-end Docker regression test that exercises the **full variables chain** shipped in PRs #600-#603. Closes the loop between unit-tested seams and the actual rendered output.

This is **PR 5 of 5**, the regression cap. Stacked on `feat/get-variables-skills` (PR #603).

## Why

PRs 1-4 ship unit tests that cover the seams independently:
- Engine: \`evaluateOnNewDocument\` injection (mocked Puppeteer)
- Helper: \`getVariables()\` merge (jsdom)
- CLI: \`parseVariablesArg\` validation (pure function)
- Loader: \`__hfVariablesByComp\` population (jsdom)
- Validator: \`validateVariables\` type-checking (pure)

But none of those check that the chain actually works front-to-back inside the production Chrome+ffmpeg+harness combo. A type signature change on \`CaptureOptions.variables\`, a regression in \`evaluateOnNewDocument\` ordering, a bug in the runtime helper's attribute parsing — any of those could pass unit tests and silently render the wrong text. This regression catches it.

## How

**Fixture** (\`packages/producer/tests/variables-prod/\`):
- \`src/index.html\` — composition with three declared variables (\`title\`, \`subtitle\`, \`bgColor\`) read via \`window.__hyperframes.getVariables()\` and rendered as positioned text on a colored background. **No animation** — keeps the regression frame-stable so it isolates "did the variables flow through?" from motion concerns.
- \`meta.json\` — tags \`[\"variables\", \"composition\"]\` (runs in the existing fast shard's tag filter, no workflow YAML changes needed). \`renderConfig.variables\` provides override values that the baseline reflects (\"Override Title\", \"Override subtitle\", \`#0a3d62\`). **If variables don't propagate**, the rendered frame shows declared defaults (\"Default Title\", black) — visibly different from the baseline, so PSNR fails on dozens of frames.
- \`output/output.mp4\` — Docker-generated baseline per the project's CLAUDE.md golden-baseline rule. Host renders drift across Chrome/font versions and would fail PSNR even on green code.
- \`src/silence.wav\` — copied from \`missing-host-comp-id\`'s silence track to satisfy the audio-correlation check.

**Harness change** (\`packages/producer/src/regression-harness.ts\`):
- \`TestMetadata.renderConfig\` gains an optional \`variables: Record<string, unknown>\` field, validated as a JSON object (not array, not null) in the \`meta.json\` validator.
- The \`createRenderJob\` call site forwards \`renderConfig.variables\` to \`RenderConfig.variables\`, which the engine already consumes via \`evaluateOnNewDocument\` (PR #600).

## Test plan

- [x] **\`docker:test variables-prod\` PASSED** — 100/100 visual checkpoints, audio correlation 1.000.
- [x] **Defeated the bug it's meant to catch** — generated a baseline against an old image (without the harness change) and confirmed it shows defaults. After the harness change + image rebuild, the baseline correctly shows overrides.
- [x] **CI auto-includes** — fast shard's \`--exclude-tags slow,render-compat,hdr\` lets \`[variables, composition]\` through. No \`.github/workflows/regression.yml\` edits needed.

## Backwards compatibility

Additive. Existing fixtures don't set \`renderConfig.variables\` and behave identically. The harness validator only fires on the new field if it's present.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-04 13:54:08 -07:00
James Russo 62c2589839 docs(skills): document variables system in SKILL.md + docs (PR 4/4) (#603) 
## What

Distribution PR for the variables feature stack. Tells agents how to declare, read, and override variables across the four authoring surfaces — the two skill files agents load (`hyperframes`, `hyperframes-cli`), the public docs (`docs/packages/core.mdx`), and the in-CLI docs (`npx hyperframes docs compositions`).

This is **PR 4 of 4**, the final PR in the stack. Stacked on `feat/get-variables-validation` (PR #602).

## Why

PRs 1–3 added the runtime helper, sub-comp scoping, and schema validation, but the only places that mention them are the docs in those PRs. Agents loading `/hyperframes` or `/hyperframes-cli` skills won't know the new attributes/flags exist. This PR closes the loop.

## How

- **`skills/hyperframes/SKILL.md`** — new "Variables (Parametrized Compositions)" section right after "Composition Structure" with: declare/read/override pattern, full worked example (with enum), sub-comp per-instance pattern (two hosts sharing a source), rules of thumb (defaults always, read-once, `--strict-variables` in CI, type validation behavior). Also added `data-variable-values` + `data-composition-variables` rows to the existing data-attributes tables.
- **`skills/hyperframes-cli/SKILL.md`** — added `--variables`, `--variables-file`, `--strict-variables` to the render flag table; short paragraph forwarding to the hyperframes skill for the full pattern.
- **`docs/packages/core.mdx`** — added a code snippet showing `getVariables<T>()` and `validateVariables` / `formatVariableValidationIssue` for tooling that validates CLI / host overrides.
- **`packages/cli/src/docs/compositions.md`** — replaced the obsolete `JSON.parse(host.dataset.variableValues)` example with the modern `getVariables()` pattern.

The `openai/plugins` mirror is intentionally out of scope. Skills in this repo are the source of truth; the downstream mirror is updated after each release as a separate workflow.

## Test plan

- [x] Doc-only PR — no source code, no tests.
- [x] Format check passes via `bunx oxfmt --check` on the touched markdown files.
- [x] Manual review confirms each example compiles in head against the runtime/CLI surface that PRs 1–3 ship.

## Backwards compatibility

Doc-only — no behavior change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-04 13:33:34 -07:00
James Russo 1c9d726b5e feat(core,cli): variable schema validation + lint rules (PR 3/4) (#602) 
## What

Adds two lint rules + render-time validation for the variable system shipped in PR #600 + PR #601. Authors get fast feedback when JSON is malformed, declarations are missing required fields, or `--variables` values don't match the declared schema.

This is **PR 3 of a 4-PR stack** — based on `feat/get-variables-subcomp` (PR #601). Will retarget to `main` after PRs 1 + 2 merge.

## Why

The runtime today silently masks several classes of mistake:
- A typo in `data-variable-values` JSON makes the parser return `{}` and the script reads stale declared defaults, with no visible signal.
- A typo in a `data-composition-variables` declaration (missing `id`, wrong `type`) gets silently filtered out.
- `--variables '{"titel":"x"}'` instead of `"title"` renders fine and produces the wrong output.

These are exactly the cases lint + schema validation are good at catching.

## How

**Lint rules** (`packages/core/src/lint/rules/composition.ts`):
- `invalid_variable_values_json` — host's `data-variable-values` must parse as a JSON object.
- `invalid_composition_variables_declaration` — root `<html>`'s `data-composition-variables` must parse as an array of objects with `id`, `type` (one of string/number/color/boolean/enum), `label`, and `default`. Per-entry findings report which fields are missing or invalid.

Both rules read via a new `readJsonAttr` helper in `lint/utils.ts`. The existing `readAttr`'s regex `["']([^"']+)["']` truncates JSON-in-attribute values at the first internal quote — `data-variable-values='{"x":"y"}'` would capture only `{`. The new helper alternates double-vs-single-quoted branches with quote-specific char classes. A second helper `findHtmlTag` returns the `<html>` open tag (where `data-composition-variables` lives), distinct from `findRootTag` which returns the first in-body composition element.

**Render-time validation** (`packages/core/src/runtime/validateVariables.ts`):
- `validateVariables(values, declarations)` returns a structured array of `VariableValidationIssue`s — `undeclared`, `type-mismatch`, or `enum-out-of-range`. Pure / sync; works in any environment.
- `formatVariableValidationIssue` renders one-line user-facing strings for CLI output.
- Both exported from `@hyperframes/core`.

**CLI integration** (`packages/cli/src/commands/render.ts`):
- New `--strict-variables` flag. Default: print warnings, continue. Strict: print warnings, exit 1.
- New `validateVariablesAgainstProject(indexPath, values)` helper reads the project's `index.html`, pulls the declared schema via `extractCompositionMetadata`, validates the CLI payload. Uses the existing `ensureDOMParser` polyfill (same pattern as `compositions.ts`).

## Test plan

- [x] Unit tests added/updated
  - **11 `validateVariables` unit tests** — happy path, undeclared keys, type mismatches (every type), enum range, multi-issue aggregation, formatter output.
  - **11 `composition.test.ts` cases** — both lint rules: parse errors, shape errors, per-entry validation, unknown types, positive cases for valid declarations.
  - **5 `render.test.ts` cases** — `validateVariablesAgainstProject`: no-declarations, happy path, undeclared, type-mismatch, missing-file.
  - All existing tests green: core 646, cli 213.
- [x] Manual flow walkthrough
  - `--variables '{"title":"x"}'` against an index that declares `title` as string → no warnings.
  - `--variables '{"count":"three"}'` against `count: number` → warning printed, render continues.
  - Same with `--strict-variables` → exit 1 before render starts.
- [x] Documentation updated
  - `docs/packages/cli.mdx` — added `--strict-variables` flag row.

## Backwards compatibility

Fully additive. Existing compositions emit zero new lint findings (the rules only fire on malformed JSON or invalid declarations, which the runtime would have silently dropped anyway). Existing `hyperframes render` invocations behave identically without the new flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-04 13:25:33 -07:00
James Russo da92a17754 feat(core): scope getVariables() per sub-comp instance (PR 2/4) (#601) 
## What

Building on **PR #600** (`getVariables()` helper + `--variables` flag), this PR scopes the helper so embedded sub-compositions see their own per-instance values. The same composition source can now be embedded N times with different content via `data-variable-values` on each host.

This is **PR 2 of a 4-PR stack** — based on `feat/get-variables`. Will retarget to `main` after PR #600 merges.

## Why

`data-variable-values` was already documented as the per-instance attribute for nested compositions, but readers had to hand-roll `JSON.parse(host.dataset.variableValues)` and there was no scoping system — every sub-comp script had to re-implement the same pattern. This PR routes the values to a scoped `getVariables()` so authors use one API in both top-level and sub-comp contexts.

## How

```
[host element]                                    [sub-comp source HTML]
data-variable-values='{...}'        +    <html data-composition-variables='[...]'>
                                                  ^ declared defaults
        ↓                                                  ↓
        └──────────── compositionLoader ────────────────────┘
                            ↓
            window.__hfVariablesByComp[compId] = merged
                            ↓
                  scripts wrapped by compositionScoping
                            ↓
            __hyperframes.getVariables() → scoped lookup
```

Three small surgical changes:

- **`compositionLoader.ts`** — before injecting wrapped scripts, parse the host's `data-variable-values` JSON, merge it over `readDeclaredDefaults(doc.documentElement)` (reused from PR 1's runtime helper), and write the result to `window.__hfVariablesByComp[compositionId]`. Skipped when both sides are empty so the table only grows when there's actual data. Inline templates (no separate `<html>` root) get host overrides only.

- **`compositionScoping.ts`** — wrapper IIFE now takes a fourth parameter `__hyperframes` alongside the existing scoped `document` / `gsap` / `window`. Same shadowing pattern that already works for the other three. The scoped `__hyperframes` overrides `getVariables` to read from `__hfVariablesByComp[__hfCompId]` and returns a fresh object each call so script mutations don't leak into the shared table.

- **`getVariables.ts`** — `readDeclaredDefaults` becomes a named export (was a private helper) so the loader reuses the exact same defaults-extraction logic the top-level helper uses.

Top-level scripts that aren't wrapped by `compositionScoping` keep calling `window.__hyperframes.getVariables()` and get the unscoped path from PR 1 (declared defaults + CLI `--variables` override).

## Test plan

- [x] Unit tests added/updated
  - **3 new `compositionScoping.test.ts`** — scoped `getVariables` invocation routes to `__hfVariablesByComp[compId]`; missing-entry returns `{}`; mutation of the returned object doesn't leak into the shared table.
  - **5 new `compositionLoader.test.ts`** — merge order (host overrides win); declared-only path when host has no `data-variable-values`; empty-skip when neither side has data; invalid-host-JSON falls through to declared defaults; per-instance scoping across two hosts that share a source.
  - **3 new `getVariables.test.ts`** — covering the newly-public `readDeclaredDefaults` export (null root, valid attribute, invalid JSON / non-array).
  - All 622 existing core tests green.
- [x] Manual flow walkthrough — host with `data-variable-values='{"title":"Pro"}'` → sub-comp's `__hyperframes.getVariables()` returns `{title:"Pro", ...declaredDefaults}`. Two hosts pointing at the same source with different overrides → each script sees its own values.
- [x] Documentation updated
  - `docs/concepts/compositions.mdx` — switched the sub-comp example from `JSON.parse(host.dataset.variableValues)` to `__hyperframes.getVariables()`; added declared-defaults pattern and per-instance scoping note.
  - `docs/concepts/data-attributes.mdx` — clarified per-instance behavior on `data-variable-values`.

## Backwards compatibility

Fully backwards compatible. Compositions that read `host.dataset.variableValues` directly keep working — the host attribute is still set as before. The new path is a strict addition.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-04 13:05:25 -07:00
James Russo 03b82e6ff8 feat(core,cli,engine,producer): getVariables() helper + --variables render flag (PR 1/4) (#600) 
## What

Adds the parametrized-render primitive from [hf#592](https://github.com/heygen-com/hyperframes/issues/592) by introducing a `getVariables()` runtime helper plus a CLI `--variables` / `--variables-file` flag. Compositions declare variables once on the root `<html>` element (the existing `data-composition-variables` attribute, which already drives Studio editing UI), read them at runtime via `window.__hyperframes.getVariables()`, and CLI users override them at render time without touching the composition source.

This is **PR 1 of a 4-PR stack**:

1. **PR 1 (this one)** — runtime helper + CLI flag + engine injection (top-level renders).
2. PR 2 — sub-comp per-instance scoping (carry the host's `data-variable-values` into the inlined sub-comp's `getVariables()`).
3. PR 3 — schema validation + lint rules (warn on undeclared variable IDs, optional `--strict-variables`).
4. PR 4 — skill / scaffold distribution (SKILL.md, AGENTS.md scaffolds, openai/plugins mirror).

## Why

The existing `data-composition-variables` schema declares variable types and defaults but isn't readable from composition scripts and can't be overridden at render time. To produce N variations of a composition today, an agent has to fork the composition or edit the source HTML before each render. `--variables` collapses that into one render call per variation, matching Editframe's `--data` UX without copying their `getRenderData` framing — `getVariables()` is named for the codebase's existing "variables" terminology and works equally in dev preview and at render time.

## How

- **Runtime helper** (`packages/core/src/runtime/getVariables.ts`): reads `data-composition-variables` from `document.documentElement`, extracts `{id: default}` defaults, merges `window.__hfVariables` (override) on top, returns `Partial<T>`. Same code path in dev preview (no override) and at render (with override). Generic parameter for typed editor ergonomics. Exposed both as a named export from `@hyperframes/core` and on `window.__hyperframes.getVariables` for vanilla compositions.

- **CLI flag** (`packages/cli/src/commands/render.ts`): `--variables '<json>'` and `--variables-file <path>`. `parseVariablesArg` is split out as a pure function (returns a discriminated `{ ok: true } | { ok: false }` union) so all validation paths are unit-testable; the side-effecting `resolveVariablesArg` wraps it with `errorBox` + `process.exit`. Mutually exclusive with `--variables-file`; fail-fast on conflicts, missing file, unparseable JSON, or non-object payloads (string, number, array, null).

- **Engine injection** (`packages/engine/src/services/frameCapture.ts`): added an `evaluateOnNewDocument` step right after the `__name` polyfill that sets `window.__hfVariables` to the parsed JSON before any page script runs. Skipped when payload is empty so we don't add pointless init scripts. Plumbed through `CaptureOptions.variables` and `RenderConfig.variables`. Docker mode forwards the flag to the in-container CLI via `dockerRunArgs`.

- **Why a separate `__hfVariables` global** instead of writing into `__hyperframes.getVariables()` directly: the helper is an IIFE that has to be defined before composition scripts execute, but the *override* needs to land before *that*. `evaluateOnNewDocument` is the only reliable hook that runs before the runtime IIFE evaluates. Storing the raw value on `__hfVariables` and merging in the helper keeps both paths order-independent.

## Test plan

- [x] Unit tests added/updated
  - 9 jsdom tests for `getVariables()` covering empty state, declared defaults only, override merge, override-wins, declared-only, invalid JSON, non-array payloads, non-object overrides, typed generic.
  - 7 tests for `parseVariablesArg` covering all validation paths.
  - 2 integration tests for `renderLocal` confirming `variables` reach `createRenderJob`.
  - 3 new `dockerRunArgs` assertions for `--variables` passthrough (set / not-set / empty-object).
  - All existing tests green: core 611, cli 208, engine 519.
- [x] Manual testing performed
  - `npx tsx packages/cli/src/cli.ts render --help` shows both flags + the two new examples.
- [x] Documentation updated
  - `docs/packages/cli.mdx` — added flags to the table and a "Parametrized renders" section with a worked example.
  - `docs/concepts/data-attributes.mdx` — added `data-composition-variables` row.

## Backwards compatibility

Fully backwards compatible. Compositions without `data-composition-variables` work unchanged; `getVariables()` returns `{}` and the engine skips the injection step.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-04 12:41:18 -07:00
James Russo a68e840dda docs(remotion-skill): only trigger on explicit migration ask (#581) 
## Summary

Tightens the `remotion-to-hyperframes` SKILL.md trigger so the skill only fires when the user *explicitly* asks to migrate / port / convert a Remotion composition to HyperFrames — not when they merely have or mention Remotion code.

## Why

User feedback on X from [@jasonpurdy](https://x.com/jasonpurdy/status/2049985508701556855):

> I understand why you have the remotion migration skill, but I would encourage you to not have that on by default, it was basically the same video as remotion, I turned it off and like the native version more.

He was A/B-testing HyperFrames vs Remotion. The migration skill auto-triggered, produced a translated output (essentially the same video as his Remotion one), and only after disabling the skill did he get a *native* HyperFrames composition that he preferred.

The previous SKILL.md `description` listed four triggering conditions, three of which were context-detection patterns:

1. *the user provides Remotion source* (.tsx files using `useCurrentFrame`, `Sequence`, …) and asks to port
2. *the user pastes a Remotion entry point* and wants HTML
3. *the user links a Remotion repo* and asks for the HyperFrames equivalent
4. the user says "port my Remotion project", "translate this Remotion code", …

Conditions (1)-(3) gave agents enough latitude to fire the skill when the user shared Remotion code as reference material or A/B-test context, even if they hadn't actually asked for a migration. Condition (4) is the only explicit-ask gate.

## Fix

- Remove the context-detection conditions; gate strictly on *explicit migration verbs* (port, convert, migrate, translate, rewrite as HyperFrames) plus concrete trigger-phrase examples.
- Add explicit NOT clauses for the common false-positive cases:
  - authoring a NEW HyperFrames composition (even with similar Remotion code in the user's history)
  - mentioning Remotion in passing
  - sharing Remotion code as reference material
  - the *specific* @jasonpurdy case: "the same video as my Remotion one" — treat as a fresh build, not a migration
- Default recommendation when uncertain: route to the `hyperframes` skill instead.

The body of the SKILL.md is unchanged — translation guidance is correct once the gate is passed; this PR only tightens the gate itself.

## Distribution note

Per `project_hyperframes_plugin_distribution.md`, hyperframes-oss skills auto-propagate to Cursor + Claude Code (which consume this repo directly), but openai/plugins is a manual mirror that needs a sync PR after merge. Adding to my follow-up list.

## Test plan

- [x] SKILL.md frontmatter description updated; body unchanged
- [x] `bunx oxfmt --check skills/remotion-to-hyperframes/SKILL.md` passes
- [x] commitlint conventional-commit format passes
- [ ] After merge: sync to `openai/plugins/plugins/hyperframes/skills/remotion-to-hyperframes/SKILL.md`

— Rames Jusso
 2026-04-30 18:08:26 -07:00
James Russo d01685d695 Merge pull request #517 from heygen-com/skill/r2hf-skill-body 
feat(skills): remotion-to-hyperframes SKILL.md + orchestrator (7/7)
 2026-04-27 22:19:03 -07:00
James Russo f22d9bfabe Merge pull request #516 from heygen-com/skill/r2hf-references 
feat(skills): remotion-to-hyperframes references (6/7)
 2026-04-27 22:18:44 -07:00
James Russo 40339a65b9 Merge pull request #515 from heygen-com/skill/r2hf-corpus-t4 
feat(skills): remotion-to-hyperframes corpus T4 (5/7)
 2026-04-27 22:18:14 -07:00
James Russo 12945238cc Merge pull request #509 from heygen-com/skill/r2hf-corpus-t3 
feat(skills): remotion-to-hyperframes corpus T3 (4/7)
 2026-04-27 22:17:55 -07:00
James Russo 4ab4576adf Merge pull request #508 from heygen-com/skill/r2hf-corpus-t1-t2 
feat(skills): remotion-to-hyperframes corpus T1+T2 (3/7)
 2026-04-27 22:17:37 -07:00
James Russo b27385ba1a Merge pull request #507 from heygen-com/skill/r2hf-eval-harness 
feat(skills): remotion-to-hyperframes eval harness (2/7)
 2026-04-27 22:14:29 -07:00
James Russo 3fa094c5c5 Merge pull request #506 from heygen-com/skill/r2hf-scaffold 
feat(skills): scaffold remotion-to-hyperframes skill (1/7)
 2026-04-27 22:12:11 -07:00
James Russo 2915b68633 fix(cli): handle port collisions in dev server with auto-increment (#85) 
## Summary
- Fix silent failure when `hyperframes dev` port (default 3002) is already in use (e.g., by Cursor IDE)
- Replace TOCTOU-prone `isPortAvailable()` probe with `serveWithPortFallback()` that binds the real server directly
- Auto-increment to next available port with a visible yellow warning, or show a clear error when all ports (range of 10) are exhausted

## What changed
The old approach used a throwaway `net.createServer()` to test port availability, closed it, then opened the real Hono server — a classic TOCTOU race. The fallback also silently returned the original port when all 10 were taken.

Now we use `createAdaptorServer()` (creates the Hono HTTP server without binding) and manually call `.listen(port)`, catching `EADDRINUSE` to try the next port. This eliminates the race entirely.

**Before:** `hyperframes dev` says "Studio running at http://localhost:3002" even when port 3002 belongs to another process.

**After:**
- Port available: works as before
- Port taken: `Port 3002 is in use, using 3003 instead` (yellow warning)
- All ports taken: `Ports 3002–3011 are all in use. Use --port to specify a different port.` (error + exit)

## Testing
- Verified TypeScript compiles cleanly (`tsc --noEmit`)
- All pre-commit hooks (lint, format, commitlint) pass
- Manual test: run another server on 3002, then `hyperframes dev` → confirms auto-increment message appears

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-03-26 23:02:50 -07:00
James Russo d845068f16 feat(cli): add opt-out anonymous telemetry via PostHog (#52) 
## Summary

- Add anonymous usage telemetry to the CLI via PostHog's HTTP batch API (zero new dependencies)
- Track command invocations, render performance, template choices, and environment info
- Add `hyperframes telemetry [enable|disable|status]` command for user control
- Config stored at `~/.hyperframes/config.json` — future-proofed for more settings

## Design Decisions

| Decision | Rationale |
|---|---|
| **Raw `fetch` instead of `posthog-node`** | Zero new dependencies. Node 22 has built-in `fetch`. Can swap to SDK later if needed. |
| **Opt-out with first-run disclosure** | Industry standard (Next.js, Homebrew, .NET CLI). Opt-in gets <3% participation. |
| **Disabled in dev mode** | Uses `.ts` extension detection (shared `utils/env.ts`). Running via `tsx` = dev. |
| **`phc_` prefix check** | Safety net — if the API key is ever reverted to a placeholder, telemetry silently disables. |
| **5-second timeout, fail-silent** | Telemetry must never slow down or break the CLI. |
| **Detached spawn for exit flush** | `flushSync` spawns a detached child process so `process.exit()` paths don't block. |

## What's Collected

- Command names (init, render, dev, etc.)
- Render metrics (duration, fps, quality, workers, docker/gpu)
- Template choices during init
- OS, architecture, Node.js version, CLI version

## What's NOT Collected

- File paths, project names, or video content
- IP addresses — `$ip: null` on every event payload (client-side) + "Discard client IP data" enabled in PostHog project settings (server-side)
- Any personally identifiable information

## Opt-Out Mechanisms

- `hyperframes telemetry disable`
- `HYPERFRAMES_NO_TELEMETRY=1`
- `DO_NOT_TRACK=1`
- Automatically disabled in CI (`CI=true`)

## Files Changed

**New files:**
- `packages/cli/src/telemetry/config.ts` — Config read/write at `~/.hyperframes/config.json` (dir 0700, file 0600)
- `packages/cli/src/telemetry/client.ts` — PostHog HTTP client (queue, batch, flush, detached flushSync)
- `packages/cli/src/telemetry/events.ts` — Typed event helpers
- `packages/cli/src/telemetry/index.ts` — Barrel exports
- `packages/cli/src/commands/telemetry.ts` — `hyperframes telemetry` command
- `packages/cli/src/utils/env.ts` — Shared `isDevMode()` (extracted from dev.ts)

**Modified files:**
- `packages/cli/src/cli.ts` — Wire telemetry at entry point + add telemetry subcommand
- `packages/cli/src/commands/render.ts` — Track render success/failure metrics
- `packages/cli/src/commands/init.ts` — Track template selection
- `packages/cli/src/commands/browser.ts` — Track browser download events
- `packages/cli/src/commands/dev.ts` — Use shared `isDevMode()` from utils/env.ts

## Testing

- Verified typecheck passes (`tsc --noEmit`)
- Verified lint passes (`oxlint`)
- Verified format passes (`oxfmt --check`)
- Tested `hyperframes telemetry status/enable/disable` commands
- Verified first-run notice is suppressed in dev mode
- Verified `--help`/`--version` don't trigger telemetry
- Verified config file creation with correct permissions
- Verified telemetry is no-op when API key lacks `phc_` prefix

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-03-25 16:49:16 -07:00