ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 12:54:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
987 Commits 855 Branches 404 Tags
cd0e6b02a02a8b7094de79cfe4ad1f96b53c1489
Commit Graph
722 Commits
Author SHA1 Message Date
Miguel Ángel 5db554c7b6 feat(cli): silent auto-update on next run (#306) 
## Summary

Today users have to run `hyperframes upgrade` (or the right install command for their package manager) to get a new release — we ship fixes but they don't reach the install until the user remembers. This PR borrows the Claude Code model: detect the update on run N, install it in a detached background child, surface one line ("hyperframes auto-updated to vX.Y.Z") on run N+1. The user's current command never blocks, never prompts, never sees an install stream.

## Flow across two runs

```
Run N     → checkForUpdate() sees latest > current → spawn detached
            child running `npm install -g hyperframes@X` (or bun /
            pnpm / brew equivalent). Parent exits immediately.
(between) → detached child installs, writes completedUpdate into
            ~/.hyperframes/config.json, clears pendingUpdate.
Run N+1   → reportCompletedUpdate() prints one line and clears the
            marker. User is on the new version.
```

## Installer detection

Walks `realpathSync(process.argv[1])` against each package manager's well-known global prefix. Wrong guesses are biased toward `skip` — we'd rather miss an auto-update than clobber a Homebrew install with npm.

| Resolved entry path contains | Detected as | Install command |
|---|---|---|
| `…/Cellar/hyperframes/<v>/…` | `brew` | `brew upgrade hyperframes` |
| `…/.bun/…` | `bun` | `bun add -g hyperframes@<v>` |
| `…/pnpm/global/…` or `…/.pnpm/…` | `pnpm` | `pnpm add -g hyperframes@<v>` |
| `…/lib/node_modules/hyperframes/…` | `npm` | `npm install -g hyperframes@<v>` |
| `…/packages/cli/…` (workspace link) | `skip` | (no-op) |
| `…/_npx/…`, `…/bunx-…/…` | `skip` | (no-op) |
| Anything else | `skip` | (no-op) |

## Guardrails

- **Never auto-update across a major version.** The existing banner still nudges the user to run `hyperframes upgrade` explicitly.
- **Skip on CI, non-TTY, dev mode,** npx / bunx / workspace link, or any install layout the detector doesn't recognize.
- **`HYPERFRAMES_NO_AUTO_INSTALL=1`** disables the install without silencing the notice banner.
- **`HYPERFRAMES_NO_UPDATE_CHECK=1`** silences both (existing knob).
- **Fresh pending install (<10 min old)** prevents re-launch on every invocation.
- **Installer stdout + stderr go to `~/.hyperframes/auto-update.log`** for postmortem — the terminal stays clean.
- **Failed installs are surfaced once** with a prompt to run `hyperframes upgrade` manually.

## What changed

| File | Role |
|---|---|
| `packages/cli/src/utils/installerDetection.ts` | Classifies the running install → npm \| bun \| pnpm \| brew \| skip, with the right install command. |
| `packages/cli/src/utils/autoUpdate.ts` | `scheduleBackgroundInstall` + `reportCompletedUpdate`. Spawns a detached `node -e "..."` child that runs the install and writes the outcome back to the config, then `unref()`s so the parent exits immediately. |
| `packages/cli/src/telemetry/config.ts` | `pendingUpdate` + `completedUpdate` fields on the config schema. |
| `packages/cli/src/cli.ts` | Wires `reportCompletedUpdate()` at startup and `scheduleBackgroundInstall()` after `checkForUpdate()` resolves. |

## Verification

### Unit tests — 19 / 19 pass (full CLI suite 115 / 115)

- `installerDetection.test.ts` — 9 cases, one per layout (workspace, npx, bunx, brew, bun, pnpm, npm, unknown, unresolved).
- `autoUpdate.test.ts` — 10 scheduling-policy cases:
  - Minor/patch → schedules + writes pendingUpdate
  - Major bump → **does not** schedule
  - Dev mode → skipped
  - `CI=1` → skipped
  - `HYPERFRAMES_NO_AUTO_INSTALL=1` → skipped
  - Unknown installer → skipped
  - Already-on-latest → skipped
  - Fresh pending install → de-duplicated
  - Stale pending install (>10 min) → supersedes
  - Previous run already completed this version → skipped

Unit tests mock `spawn` and the installer — they verify the **policy**, not the real detached-child path.

### Live end-to-end smoke test (on this Mac, real processes)

To validate the parts the unit tests can't — actual detached spawn, real config writeback, banner surfacing in a fresh subsequent process — I wired a smoke script that exercises the exact same code path `autoUpdate.ts` uses, but with `echo …` as the "install command" so nothing global gets touched.

**Steps exercised:**

1. Backed up the user's real `~/.hyperframes/config.json`.
2. Wrote a `pendingUpdate` marker for version `0.4.99` (like `scheduleBackgroundInstall` does).
3. Spawned the **exact same detached `node -e "..."` child** the real scheduler produces, with the install command replaced by `echo 'faux install for 0.4.99'`.
4. The parent `unref()`d and continued; 800 ms later the parent re-read `config.json`.
5. Ran `reportCompletedUpdate()` in a **fresh subprocess** (via `bunx tsx -e ...`) to match the real "Run N+1" conditions, capturing its stderr.
6. Asserted the marker was cleared.
7. Restored the original config on exit.

**Observed output:**

```
  [setup] Backed up config to /Users/miguel/.hyperframes/config.json.smoke-backup
  [setup] Wrote pendingUpdate for v0.4.99
  [spawn] Detached child pid=49469
  [after] completedUpdate = {"version":"0.4.99","ok":true,"finishedAt":"2026-04-17T17:26:50.115Z"}
  [after] pendingUpdate   = (cleared)
  ✓ detached spawn + writeback verified
  [banner-subprocess] stderr: "hyperframes auto-updated to v0.4.99"
  ✓ banner fired in fresh process + marker cleared

  ALL CHECKS PASSED ✓
  [cleanup] Config restored
```

**What this proves:**

| Claim | Evidence |
|---|---|
| Detached spawn works (doesn't block the parent) | `[spawn] pid=49469` logged, parent continued immediately |
| Detached child is process-independent | Parent exited its own work while child ran `exec(CMD)` |
| Child writes correct config shape | `completedUpdate = { version: "0.4.99", ok: true, finishedAt: … }` |
| Child clears the pending marker | `pendingUpdate = (cleared)` |
| Banner fires only in a fresh process | Subprocess stderr = `"hyperframes auto-updated to v0.4.99"` |
| Banner message format | Matches the copy in `autoUpdate.ts:reportCompletedUpdate` exactly |
| Marker clears after banner | Second file read shows `completedUpdate` absent |

Both the original test-plan checkboxes (fresh install, `HYPERFRAMES_NO_AUTO_INSTALL=1`, `CI=1`) are covered by either the unit-test suite or this smoke test — the scheduling-policy gates are unit-tested under `CI=true`, and the real detached-spawn path is smoke-tested above.

### What's still worth doing

- **Physical installer test on a real `npm i -g` / `brew` / `bun add -g` environment** — the smoke test above replaces the install command with `echo`, so we've never actually seen npm/bun/brew run the real command. That's the one remaining unknown. Worth one manual run on the maintainer's machine before cutting v0.4.4.

## Test plan

- [x] `bunx vitest run` on `packages/cli` — 115 / 115 pass (incl. 19 new)
- [x] `tsc --noEmit` clean
- [x] `tsup` build clean
- [x] **Live e2e smoke test** exercising the real detached spawn + config writeback + fresh-process banner (output above)
- [x] CI green on this branch (Typecheck, Test, Test: runtime contract, Build, Lint, Format)
- [ ] One manual run on a physical `npm i -g hyperframes@0.4.2` install to confirm the real `npm install -g hyperframes@0.4.3` command actually runs when `autoUpdate.ts` delegates to it (the smoke test stopped short of executing `npm`)

## Notes

- Independent of any version bump — ship whenever.
- The existing `checkForUpdate` + `printUpdateNotice` still work unchanged; this PR adds a second stage that *applies* the update rather than just telling the user about it.
- `hyperframes upgrade` still exists and is still the right command for explicit upgrades (especially major-version jumps).
 2026-04-21 00:05:04 +02:00
Vance Ingalls acce9123b4 chore: release v0.4.11 2026-04-20 12:40:27 -07:00
Vance Ingalls 0d551e3614 chore: release v0.4.11-alpha.1 2026-04-20 12:38:05 -07:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00
Miguel Ángel a539266683 fix: harden CDN script inlining with linkedom (#352) 
* fix: harden CDN script inlining

* test: add spanish empire regression fixture
 2026-04-20 21:13:59 +02:00
James Russo 4a55bc8673 feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351) 
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix

`hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)`
with no language argument, so Kokoro's default phonemizer (en-us) was
applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha`
and feeding it Spanish or Japanese text produced English-phonemized
output.

Closes #349.

- `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and
  `isSupportedLang`. Attach a `defaultLang` field to every bundled voice
  and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`,
  `zf_xiaobei` so `--list` surfaces multilingual options.
- `synthesize.ts`: accept optional `lang: SupportedLang` in
  `SynthesizeOptions`, forward it to the Python worker as `argv[7]`.
  The worker introspects `Kokoro.create`'s signature and only passes
  `lang=` when the installed kokoro-onnx version supports it. Returned
  metadata now includes `lang` and `langApplied` so callers can detect
  silent no-ops. Bump the cached script filename to `synth-v2.py` so
  existing installs pick up the new script automatically.
- `commands/tts.ts`: add `--lang, -l` with validation against
  `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred
  from voice prefix > `en-us`. When explicit lang disagrees with the
  voice-implied lang (legitimate for stylized accents), emit a
  dim-level hint; suppress under `--json`. When kokoro-onnx silently
  ignores the kwarg, log that too. Update `--list` with a new
  "Lang code" column and add multilingual examples.
- Tests: new `manager.test.ts` covering every supported prefix, the
  unknown-prefix fallback, case-insensitivity, `isSupportedLang`
  validation, and a regression guard that every bundled voice has a
  valid `defaultLang` matching its ID.
- Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md`
  updated with the flag, examples, the espeak-ng dependency note for
  non-English phonemization, and the voice-prefix → lang table.

Backward compatibility:
- English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb
  — no change.
- Non-English voices now phonemize correctly by default (bug fix, not a
  regression).
- Older kokoro-onnx versions that don't know the `lang` kwarg keep
  working via signature introspection; the CLI logs a dim note if
  `--lang` was requested but ignored.

Verification:
- `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new).
- `bunx oxlint` and `bunx oxfmt --check` clean on changed files.
- `bun run build` succeeds.
- `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly;
  invalid `--lang` produces a clean error with the valid-codes list.

* refactor(cli): simplify tts --lang implementation

Post-review cleanup on #351. Net -21 lines.

- Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo —
  compute via `inferLangFromVoiceId(v.id)` at read time in listVoices.
  The only reader was the --list table; caching the derived value on
  every voice added a self-consistency invariant we had to test.
- Drop redundant `lang` field from SynthesizeResult — caller already
  knows the requested lang since it passed it in; only `langApplied`
  carries information the caller can't derive.
- Use `errorBox` for --lang validation to match the house style in
  render.ts (other validation errors already use errorBox).
- Reuse existing `langList` module constant in the validation error
  instead of re-joining SUPPORTED_LANGS.
- Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId.
- Trim WHAT-restating comments and the duplicate prefix-enumeration
  JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries
  per-row comments).
- Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts
  when writing the current versioned script, so repeated upgrades
  don't leak files.
- Drop the `EN-US` case-sensitive-rejection test assertion — the CLI
  lowercases input before validation, so accepting mixed case is a
  feature, not a bug.

Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass.
Lint + format + typecheck clean.
 2026-04-20 10:51:00 -07:00
Miguel Ángel 577e822fa6 chore: release v0.4.10 2026-04-20 11:06:52 -04:00
ukimsanov 0730f88dc7 fix(cli): address snapshot PR review feedback 
Incorporates four review points on #348:

1. Fix typecheck error — cast the dynamic `@hyperframes/engine` import to a
   typed shape and drop the `as typeof x` cast on a `null`-unioned variable
   that TS rejected. CI `Typecheck` was failing on main because the cast
   conflicted with the declared union.

2. Clear stale injected <img> overlays — always call
   `syncVideoFrameVisibility(page, activeIds)` on every seek (including
   `active.length === 0`), so injected frames from a previous timestamp
   don't leak into later snapshots. The runtime's visibility toggles act
   on the native <video> but not on its injected sibling, which Copilot
   correctly flagged as a leakage source.

3. URL-decode the served video path before resolving to a filesystem path.
   Files whose names contain spaces or other URL-encoded characters were
   skipped because `new URL().pathname` preserves `%20` while the file
   server decodes inbound requests and the file on disk lives at the
   decoded name.

4. Mirror `packages/core/src/runtime/media.ts` media math so clips with a
   non-1 `defaultPlaybackRate` get the correct active window and relTime.
   Specifically: clamp `defaultPlaybackRate` to [0.1, 5], apply
   `(t - start) * rate + mediaStart`, and adjust the duration-fallback
   branch by `/ rate` when only source duration is known.

5. Kill FFmpeg on a 30s timeout so a pathological clip cannot wedge
   snapshot indefinitely. Matches the default in
   `@hyperframes/engine`'s `runFfmpeg`.

Re-verified against the same 4-timestamp A/B on launch-video-2:
MD5s match the ffmpeg-from-render ground truth
  12.5s → ef9684e36fea53a0db7adf7cfcaacad3  (Stripe)
  16.0s → 487494ca16344d55d7181408dc439a56  (Framer)
  20.5s → 8835c34ad2a45755a1c98a7e079427a1  (HeyGen 3D)
  32.5s → 34dc9450f2bd661c12039d7aa82a30b0  (GitHub finale)

No-video projects (basecamp-tour, linear-brand, commissioned/github)
still produce identical output to the pre-fix path. Latency unchanged.

Made-with: Cursor
 2026-04-20 10:30:29 -04:00
ukimsanov a3e7c6176f fix(cli): inject real video frames in snapshot to match render 
The snapshot command previously just called `tl.seek(t)` + `page.screenshot`
and trusted Chrome to advance `<video>`-element decoders. Chrome headless
silently ignores `video.currentTime = X` writes — the setter is accepted
but the decoder never moves. Result: every snapshot of a composition that
uses body-level `<video data-start>` elements renders the same frame
regardless of the requested timestamp (the z-topmost video's first-frame
paints through, because all clips share `position: absolute; inset: 0`
and visibility:hidden doesn't always prevent the GPU surface from
contributing to the composite).

The render pipeline has already solved this: for each body-level video it
extracts the needed frame via FFmpeg and overlays it as an <img> sibling
via `injectVideoFramesBatch` (packages/engine/src/services/screenshot
Service.ts). This commit ports that same primitive into `snapshot`:

1. Added `extractVideoFrameToBuffer(videoPath, t)` — one FFmpeg spawn per
   active video, `-ss` keyframe seek (~100-200 ms), writes a temp PNG.
2. After the existing seek + settle, enumerate `<video data-start>`
   elements that are active at the target time, resolve each one's
   `currentSrc` URL back to a filesystem path under `projectDir`, extract
   the frame, and call `injectVideoFramesBatch`.
3. Then screenshot — as before.

Non-breaking: when no body-level `<video data-start>` elements exist (every
other project in the repo — basecamp, linear, stripe, github component), the
new block short-circuits on `active.length === 0` and behaves identically
to the pre-fix path. Verified against three no-video projects: bit-identical
snapshot output, no latency regression.

Measured on macOS M2 (4 frames, cold):
  launch-video-2 (11 timed videos): 6.48s → 6.16s   (-5%)
  basecamp-tour  (no timed videos): 5.67s → 4.87s   (-14%)

Proof: Pre-fix MD5 at t=12.5, 16.0, 20.5, 32.5 — all 4 identical (wrong frame)
  Post-fix MD5 at same timestamps  — all 4 distinct, match ffmpeg-from-render
Made-with: Cursor
 2026-04-20 09:36:33 -04:00
Miguel Ángel 62ce3679e6 chore: release v0.4.9 2026-04-19 23:03:07 -04:00
Miguel Ángel 479e6c3f81 chore: release v0.4.8 2026-04-19 22:40:57 -04:00
Miguel Ángel 03c2158e0f ci: verify on windows-latest + fix cross-platform build bugs it surfaced (#342) 
* fix(cli): make build copy cross-platform and deterministic

* fix(core): keep rewritten asset URLs POSIX on Windows

* ci(windows): add render verification workflow

* ci(windows): load canary gsap from cdn

* build: use dependency-aware workspace ordering

* Revert "build: use dependency-aware workspace ordering"

This reverts commit 99bc2ffbdf.
 2026-04-20 04:35:55 +02:00
Vance IngallsandClaude Opus 4.6 99a903be2f feat(hdr): layered HDR compositing, shader transitions, and HDR image support (#268) 
* feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes

- 15 GLSL→TypeScript shader transitions on rgb48le buffers
- Dual-scene compositing with scene detection via window.__hf.transitions
- --hdr flag gates ffprobe probing (zero overhead on SDR compositions)
- Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT
- Buffer.from() copy in writeFrame() fixes streaming encoder race condition
- SDR rendering fixes (three stacked bugs)
- Object.assign fix for window.__hf preservation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: tighten shader smoke thresholds + assert .scene contract

- Tighten the all-transitions smoke test thresholds: at progress=0 we now
  require the center pixel R-channel > 35000 (was > 25000) and at
  progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly
  halfway between the test from-pixel (40000) and to-pixel (10000), so a
  half-blended transition would silently pass.
- Add a runtime assertion in HyperShader.init() that every scene id
  resolves to a DOM element with the .scene class. Without this, missing
  ids silently no-op when textures + querySelectorAll(.scene) run later.

Addresses deferred review feedback from PR #268.

* fix(hdr): restore VIRTUAL_TIME_SHIM and applyRenderModeHints in renderOrchestrator

Commit c6b4619c ("feat(hdr): shader transitions, --hdr flag, and SDR
rendering fixes") accidentally removed two pieces of the deterministic
rendering pipeline:

1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`,
   which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven
   animations advance only when `window.__hf.seek(t)` is called.
2. The `applyRenderModeHints` function and its post-`compileForRender`
   call site, which auto-forces screenshot capture mode for compositions
   the compiler flagged as needing it (RAF, iframes, etc.).

Without (1), RAF animations advanced by wall-clock between the main-loop
seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the
sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer
seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat`
lost its automatic fallback to screenshot mode and the child-document
motion stopped being captured.

Both helpers are still produced by `htmlCompiler` and exercised by
`renderOrchestrator.test.ts` — the orchestrator just stopped calling
them. Restored:

- Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js`
- Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer`
  call sites (probe + main render)
- Re-add `applyRenderModeHints` (matching the test expectations) and
  call it immediately after `compileForRender`
- Persist `renderModeHints` in `summary.json` and the
  "Compiled composition metadata" log line

Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression
failures on `feat/hdr-layered-compositing`.

Made-with: Cursor

* test(engine): expand sampleRgb48le coverage + audit Uint16Array alignment

Adds:
- 8 new sampleRgb48le bilinear-interpolation tests covering boundary
  pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers.
- uint16-alignment-audit.test.ts documenting the alignment requirement
  for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE.

Background: ~105 hot-loop sites in shader transitions still use
readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut
overhead but requires guaranteed even byteOffsets — these tests document
the contract before any future refactor lands.

* fix(engine,producer): mask DOM layers during HDR layered compositing

The HDR layered compositor blits z-ordered layers over a shared canvas. DOM
layers used a full-page screenshot from `captureAlphaPng`, which captures
*every* painted pixel on the page — root background, sibling-scene content,
overlay UI elements that aren't part of the current layer. Those opaque
pixels were then blitted over the canvas, overwriting any HDR content
composited beneath in earlier layers.

The previous workaround toggled `display:none` on hide ids via
`hideVideoElements`/`showVideoElements`. That correctly hid native videos
but did nothing about the root composition's background or about overlay
elements that the layer grouping considered part of a different layer.

This commit replaces the workaround with a precise CSS mask installed
before each DOM screenshot:

1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and
   re-shows the layer's elements (and their descendants and their injected
   `__render_frame_*` siblings) with `visibility: visible !important`. CSS
   visibility is *not* multiplicative through descendants — a child with
   `visibility: visible` overrides an ancestor's `visibility: hidden`, so
   deeply nested layer content still paints even though every intermediate
   ancestor is hidden by the mass-hide rule.
2. Non-layer data-start ids are inline-hidden with
   `visibility: hidden !important`. Inline `!important` beats stylesheet
   `!important`, so this overrides the show rule for elements that fall
   under a show selector but should NOT paint — most importantly HDR
   videos and other-layer SDR videos that live as descendants of `#root`.
3. `removeDomLayerMask` tears the stylesheet down and clears the inline
   `visibility`/`opacity` properties so subsequent video frame injection
   gets a clean slate.

Crucially the mask only sets `visibility`, never `opacity`. CSS opacity
*is* multiplicative — `opacity: 0` on `#root` would zero out every
descendant including layer videos, even with `visibility: visible`. We
also extend `initTransparentBackground` to force the composition root
(`[data-composition-id]`) transparent in addition to `html`/`body`,
because compositions almost always set `#root { background: ... }` and
that background paints across the whole viewport otherwise.

Both compositing paths use the new helpers:
- The per-layer DOM branch (`compositeToBuffer`) for normal frames.
- The transition path (single DOM screenshot per scene) so transition
  frames also get a clean per-scene capture.

Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`:
per-layer pixel-add accounting, dumps of every captured DOM PNG, and a
periodic raw `rgb48le` snapshot of the composite buffer. These were
essential to diagnosing the root-overwrite bug and stay zero-cost in
normal renders. Also stops the workDir / per-video frame-dir cleanup
when `KEEP_TEMP=1` so the dumps survive past frame N.

Made-with: Cursor

* fix(engine): preserve GSAP-applied opacity across DOM-layer captures

SDR clips inside an HDR composition were rendering at full opacity even
when the user had animated their wrapper opacity (e.g. fade-in or
yoyo). Two bugs in the per-layer screenshot path conspired to drop the
GSAP-applied opacity on the floor:

1. removeDomLayerMask was unconditionally calling
   `el.style.removeProperty("opacity")` on every wrapper after each
   layer capture. applyDomLayerMask only ever sets `visibility`, so the
   only inline opacity present is the value GSAP wrote. Stripping it
   between layer captures means that on the next capture (at the same
   timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline
   is already at that time — the opacity is never restored, and the
   wrapper renders fully opaque.

2. injectVideoFramesBatch was reading the source <video>'s computed
   opacity via `parseFloat(computedStyle.opacity) || 1` and copying it
   onto the injected <img>. Because syncVideoFrameVisibility forces the
   <video> to `opacity: 0 !important` to hide it during capture, the
   computed value is always 0, which `|| 1` then silently flips to
   full opacity. The <img> is a sibling of the <video> inside the same
   wrapper, so it should inherit opacity from the wrapper directly
   instead of having a value hard-set on it.

Fix both: drop the opacity removal in removeDomLayerMask, skip opacity
when copying visual properties from <video> to <img>, and explicitly
clear any stale inline opacity on the <img> so it inherits from the
wrapper that GSAP is animating.

Made-with: Cursor

* fix(producer): correct hdrLayerStartTimes typo to hdrVideoStartTimes

The diagnostic logging block in executeRenderJob's HDR layer composite
path referenced an undeclared `hdrLayerStartTimes` map. The correct
variable, declared and populated earlier in the same function, is
`hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer
masking work and broke the producer build/typecheck on CI.

Made-with: Cursor

* fix(engine): restore video opacity copy to injected frame img

Commit 188ebcca removed the opacity copy from `injectVideoFramesBatch` on
the assumption that the <img> sibling would inherit GSAP's opacity from
a shared wrapper. That breaks any composition where GSAP animates opacity
directly on the <video> element itself: the <img> has no animated
ancestor and renders at full opacity throughout any fade, even when the
user's intent is partial or zero opacity.

The CI `style-7-prod` and `style-8-prod` regressions caught this:
the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut
because the <img> inherited opacity 1 regardless of GSAP's tween.

Restore the old explicit copy from `computedStyle.opacity` to the
<img>'s inline opacity, with the `|| 1` fallback intentionally
preserved. The fallback is load-bearing: GSAP's seek does not re-apply
tweens that have already completed, so post-fade frames read opacity 0
from the stale `opacity: 0 !important` we apply to hide the native
<video>. The `|| 1` recovers the tween's end-state opacity 1 for
those frames, matching the final on-screen intent and the existing
baseline renders.

Handles both DOM shapes:
- GSAP on wrapper: video's own computed opacity is 1, img set to 1,
  wrapper's opacity applies via stacking as before.
- GSAP on <video>: video's computed opacity is the tween value, copied
  to img directly since they are siblings.

Fixes:
- style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33)
- style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-19 19:00:58 -07:00
Vance Ingalls 8548a17771 feat(hdr): GSAP transforms and border-radius masks on HDR video (#290) 
## Summary

HDR video elements with GSAP animations (position, scale, rotation, opacity) and CSS border-radius rendered without any transforms applied — the video just sat at (0,0) full-size. This PR adds affine transform support and rounded-corner masking for natively-composited HDR video.

## What it does

**Affine blit with bilinear interpolation:**
- `blitRgb48leAffine()` — Takes a 4x4 DOMMatrix and maps each destination pixel back to source coordinates via the inverse transform. Bilinear interpolation between the 4 nearest source pixels produces smooth edges under rotation and non-integer scaling. Optional opacity and border-radius parameters.
- `parseTransformMatrix()` — Parses CSS `matrix(a,b,c,d,e,f)` strings into `[a,b,c,d,e,f]` tuples.

**Accumulated viewport matrix:**
- `getViewportMatrix()` — Walks the `offsetParent` chain from element to viewport, accumulating position offsets and CSS transforms at each level. Correctly handles `transform-origin` using the CSS sandwich: `translate(origin) × M × translate(-origin)`. This is critical because GSAP animates transforms on wrapper divs, not directly on the video element.

**Effective opacity:**
- `getEffectiveOpacity()` — Multiplies opacity values walking up the ancestor chain. Uses `Number.isNaN()` (not `|| 1`) so opacity:0 isn't incorrectly treated as 1.

**Border-radius masks:**
- `roundedRectAlpha()` — Per-pixel anti-aliased rounded-rectangle mask with support for independent corner radii.
- `getEffectiveBorderRadius()` — Walks ancestors for `overflow:hidden` + border-radius. Resolves percentage values (e.g., `50%` for circles) via `offsetWidth`/`offsetHeight`.

**Layout dimensions for extraction:**
- Uses `offsetWidth`/`offsetHeight` (unaffected by CSS transforms) instead of `getBoundingClientRect()` (which returns the transformed bounding box and wobbles under rotation).

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | `blitRgb48leAffine()`, `parseTransformMatrix()`, `roundedRectAlpha()`, `cornerAlpha()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `getViewportMatrix()`, `getEffectiveOpacity()`, `getEffectiveBorderRadius()`, `layoutWidth`/`layoutHeight` on `ElementStackingInfo` |
| `packages/producer/src/services/renderOrchestrator.ts` | Affine blit path, extraction at layout dimensions, border-radius parameter passing |

## How to test

Render a composition with an HDR video that has GSAP scale + rotation animation and a `border-radius: 50%` wrapper (circle mask). The video should rotate smoothly with round edges — no wobble, no sharp corners.

## Stack position

**5 of 6** — Stacked on #289 (z-ordered layers). Adds transform and masking support to the HDR blit that the layer compositor uses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-19 17:03:29 -07:00
James f622d39962 chore: release v0.4.7 2026-04-19 23:35:35 +00:00
James Russo 7b0c7e73b2 refactor: frame reorder buffer + port probe cleanup; add CREDITS.md and missing skill (#341) 
* refactor(engine): restructure frame reorder buffer with Map-keyed storage

Rewrites createFrameReorderBuffer to use a Map<number, Array<() => void>>
keyed by frame index instead of a flat Array<{frame, resolve}> scanned on
every advance. O(1) lookups in enqueue/flush, fast-paths for the matching-
cursor and overshoot cases, and a small fix: waitForAllDone now coexists
with the writer still waiting on the final frame instead of colliding on
the same waiter slot.

Also adds 5 unit tests (there were none before) covering the fast-path,
out-of-order gating, multi-waiter-per-frame semantics, waitForAllDone
normal path, and the overshoot case.

Comment tweaks on buildChromeArgs — the flag profile is the standard
headless-for-capture set (Puppeteer / Playwright / Chrome headless-shell
all converge on similar flags); rephrased for clarity.

* refactor(cli): simplify port availability probe with async/await

Rewrites isPortAvailableOnHost from a single new-Promise callback into an
async/await form with an intermediate `bindError: ErrnoException | null`
variable. Makes the bind-then-release flow explicit as two sequential
awaits, and broadens the non-EADDRINUSE errno commentary (EADDRNOTAVAIL
for disabled IPv6, EACCES for privileged ports, EAFNOSUPPORT for missing
address families — all treated as "this host doesn't apply", not "port
occupied").

No behavior change to existing callers; all four portUtils tests still
pass.

* docs: add CREDITS.md and surface website-to-hyperframes skill

- New CREDITS.md acknowledging prior art in the browser-based video
  rendering space (Remotion) and the ecosystem HyperFrames builds on
  (Puppeteer, FFmpeg, GSAP, Hono). Standard OSS practice.

- Adds the `website-to-hyperframes` skill to the skills tables in
  README.md, docs/guides/prompting.mdx, and the project template at
  packages/cli/src/templates/_shared/CLAUDE.md. The skill ships in
  skills/ but was missing from every table.

- Adds `/hyperframes-registry` to the prose mention in the repo
  CLAUDE.md.
 2026-04-19 16:33:42 -07:00
Vance IngallsandClaude Opus 4.6 0cc79a35b0 feat(hdr): z-ordered multi-layer compositing with PQ support (#289) 
* feat(hdr): add z-ordered multi-layer compositing with PQ support

Per-frame z-order analysis groups elements into DOM and HDR layers,
composited bottom-to-top. Adjacent DOM elements merge into single
screenshots. PQ (HDR10/smpte2084) support via sRGB-to-PQ LUT with
203-nit SDR reference white. queryElementStacking walks DOM for
effective z-index, groupIntoLayers splits on HDR/DOM boundaries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(hdr): address review feedback across stack

- Document groupIntoLayers tie-break (V8 stable sort → DOM order).
- Expand layerCompositor docstring: merge rationale, visibility inclusion.
- Add tests: empty input, negative z-index, stable tie-break at equal z.
- Document getEffectiveZIndex CSS stacking-context limitations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-19 16:29:24 -07:00
Vance Ingalls a21a62b574 feat(engine): add HDR two-pass compositing — DOM layer + native HLG video (#288) 
## Summary

Compositions with HDR video AND DOM overlays (text, graphics, SDR video) couldn't render both correctly — either HDR data was lost (Chrome captures sRGB only) or DOM overlays were missing (FFmpeg pass-through skips Chrome). This PR adds in-memory alpha compositing that combines both.

## What it does

**Per-frame two-pass capture:**
1. **DOM pass** — Chrome screenshots the page with a transparent background (CDP alpha). HDR videos are hidden, leaving transparent holes where they go.
2. **HDR pass** — Pre-extracted native HLG/PQ frames (16-bit PNG from FFmpeg) are read from disk.
3. **Composite** — DOM pixels (sRGB RGBA8) are alpha-composited over HDR pixels (rgb48le) in Node.js memory, with sRGB→HLG/PQ conversion via a 256-entry lookup table.

**Key components:**
- `decodePng()` / `decodePngToRgb48le()` — Pure Node.js PNG decoders (no native dependencies). Support all 5 PNG filter types.
- `blitRgba8OverRgb48le()` — Alpha composite with per-pixel sRGB→HDR LUT conversion. Fast paths for alpha=0 (skip) and alpha=255 (overwrite).
- `initTransparentBackground()` + `captureAlphaPng()` — Split CDP transparent background setup (once) from per-frame screenshot capture (eliminates 2 CDP round-trips per frame).
- Single-pass FFmpeg extraction — All HDR frames extracted in one sequential FFmpeg run (avoids duplicate frames from per-frame `-ss` fast seek).

## Key design decisions

| Decision | Why |
|----------|-----|
| In-memory compositing (not FFmpeg overlay) | Eliminates ~2400 process spawns + temp files per render. Pure pixel math is 10x faster. |
| 16-bit PNG intermediate | Raw `-f rawvideo` loses color metadata, causing moiré artifacts. PNG is self-describing. |
| sRGB→HLG LUT (256 entries) | DOM content is sRGB. Without conversion, it appears orange-shifted in HLG stream. |
| Native HDR detection before extraction | `extractAllVideoFrames` converts SDR→HDR. Pre-extraction probe identifies original HDR sources so only truly-HDR videos get native extraction. |

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/alphaBlit.ts` | **NEW** — PNG decode, sRGB→HDR LUT, alpha compositing (14 tests) |
| `packages/engine/src/services/screenshotService.ts` | Transparent background CDP, `captureAlphaPng()` |
| `packages/engine/src/services/videoFrameInjector.ts` | `hideVideoElements()` / `showVideoElements()` |
| `packages/engine/src/services/streamingEncoder.ts` | Input color space tags for rgb48le |
| `packages/producer/src/services/renderOrchestrator.ts` | Two-pass HDR capture loop, native HDR detection |

## How to test

Render a composition with an HDR video background and text overlays. Both should be visible — HDR video at full quality, text crisp with correct colors (not orange-shifted).

## Stack position

**3 of 6** — Stacked on #265 (HDR output pipeline). This is the foundation for all layered compositing that follows.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-19 16:26:54 -07:00
Ular Kimsanov 6a587606e4 Merge pull request #339 from heygen-com/feat/capture-improvements-v2 
feat(capture): improve capture quality, clean CLAUDE.md, skill refinements
 2026-04-19 18:31:50 -04:00
Vance Ingalls 5a3fde19d4 feat(engine): add HDR video output pipeline (#265) 
## Summary

Adds the ability to render HDR video output (H.265 10-bit, BT.2020) from HyperFrames compositions. When the renderer detects HDR source video, it automatically switches to the HDR output pipeline — no flags needed.

## What it does

- **Auto-detection** — Probes each video source with `ffprobe`. If any has bt2020/PQ/HLG color metadata, the output switches to H.265 10-bit with correct color tags. SDR-only compositions are unaffected (H.264, bt709).
- **HLG pass-through** — Native HLG pixels from FFmpeg extraction are piped directly to the encoder without conversion. This avoids brightness loss from HLG→linear→PQ conversion (which requires an OOTF system gamma we can't reliably apply).
- **Encoder HDR support** — Both chunk and streaming encoders accept HDR presets: `libx265`, `yuv420p10le`, BT.2020 color primaries, `hvc1` codec tag (required for Apple playback).
- **WebGPU HDR capture (gated)** — A complete WebGPU float16 readback pipeline is implemented and tested but gated behind headed Chrome (headless doesn't expose WebGPU). Ready for future use with WebGPU canvas content.
- **HDR utilities** — `detectTransfer()` (PQ vs HLG), `getHdrEncoderColorParams()`, `analyzeCompositionHdr()`. 15 unit tests.

## Key design decisions

| Decision | Why |
|----------|-----|
| No `--hdr` flag | SDR content encoded as HDR causes orange shift in browsers. Auto-detect eliminates this. |
| HLG pass-through (not HLG→PQ) | Conversion loses brightness without OOTF. Pass-through matches source exactly. |
| `hvc1` codec tag | Apple QuickTime requires `hvc1` (not `hev1`) for HEVC playback. |
| 1-hour streaming timeout | HDR capture at ~6fps needs more time than the default 10-minute FFmpeg timeout. |

## Files changed

| File | What changed |
|------|-------------|
| `packages/engine/src/utils/hdr.ts` | **NEW** — HDR detection, transfer types, encoder params (15 tests) |
| `packages/engine/src/services/hdrCapture.ts` | **NEW** — WebGPU readback, HLG conversion, PQ encode |
| `packages/engine/src/services/streamingEncoder.ts` | HDR presets, raw rgb48le input, color tags |
| `packages/engine/src/services/chunkEncoder.ts` | HDR presets, conditional color tags |
| `packages/producer/src/services/renderOrchestrator.ts` | Auto-detection loop, HDR pass-through capture path |

## How to test

Render a composition with an HDR video source. The output should be H.265 10-bit with HDR metadata visible in `ffprobe` (bt2020, arib-std-b67 or smpte2084). Plays correctly in QuickTime and on HDR displays.

## Stack position

**2 of 6** — Stacked on #258 (SDR/HDR normalization). Provides the encoder infrastructure that phases 1-5 build on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-19 15:10:59 -07:00
ukimsanovandClaude Opus 4.6 4e87f28cd9 fix: address PR #339 review — 7 issues 
1. Catalog failure safety: warn when catalog is empty or throws, so
   capture doesn't silently produce zero images
2. Dead path: extract-audio-data.py → skills/gsap/scripts/ (was
   skills/hyperframes/scripts/)
3. --json fonts compat: emit both `fonts` (string[]) and
   `fontsDetailed` (FontToken[]) to avoid breaking external consumers
4. Restore .cursorrules writing alongside AGENTS.md + CLAUDE.md
5. .gitignore: remove over-broad `projects/` and `videos/` entries,
   keep scoped `cursor-tests/` and `launch-video*/`
6. agentPromptGenerator: mark unused params as reserved with comments,
   remove _animations from buildPrompt
7. Cookie filter: threshold 20 → 8 chars to preserve footer copy like
   "© 2026 Stripe" (16 chars) and "Privacy & Terms" (15 chars)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-19 15:34:15 -04:00
WangandClaude Opus 4.7 d1f992570a fix(cli): use 'where' instead of 'which' on Windows for FFmpeg and br… (#336) 
* fix(cli): use 'where' instead of 'which' on Windows for FFmpeg and browser detection

- findFFmpeg() now uses 'where ffmpeg' on Windows, 'which ffmpeg' on Unix
- whichBinary() now uses 'where' on Windows, 'which' on Unix

Fixes FFmpeg detection failure on Windows where 'which' command doesn't exist.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(cli): handle multi-line output from Windows 'where' command

Windows 'where' can return multiple paths (one per line) when there
are multiple matches on PATH. Take only the first non-empty line.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(cli): extend Windows 'where' fix to whisper, tts, and clipboard modules

- whisper/manager.ts: whichBinary() now uses 'where' on Windows
- tts/synthesize.ts: findPython() now uses 'where' on Windows
- utils/clipboard.ts: detectProvider() now uses 'where' on Windows

All functions handle multi-line output from 'where' command.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-04-19 20:05:24 +02:00
ukimsanov d8f1af1ef1 feat(capture): write AGENTS.md alongside CLAUDE.md + skill refinements from regression tests 
Capture pipeline:
- agentPromptGenerator now writes AGENTS.md + CLAUDE.md (drop legacy
  .cursorrules), matching the dual-file convention already used by the
  _shared templates in hyperframes init. AGENTS.md is picked up natively by
  Cursor, Codex, Gemini CLI, Windsurf, Aider, and Jules; CLAUDE.md covers
  Claude Code. Both files share the same content — a capture data inventory
  that points agents to the website-to-hyperframes skill.

website-to-hyperframes skill refinements (derived from 8-site regression test):
- Drop slash-command phrasing throughout SKILL.md and step-6-build.md so the
  skill works identically across Claude Code (slash), Cursor (auto-discover
  by description), and other agents.
- Remove stale HANDOFF.md references from SKILL.md step-7 summary and
  reference table — matches the intent of the prior step-7 cleanup.
- step-5-vo: specify narration.txt filename convention (pronunciation-
  substituted spoken text; distinct from SCRIPT.md the creative doc).
- step-6 self-review adds three rules derived from actual lint warnings
  observed across the 8 regression runs:
    - Every <template> root needs data-start + data-duration (catches
      root_composition_missing_data_start/duration, seen in 4/8 runs).
    - Caption exits need a hard tl.set kill after tl.to(opacity:0), or
      per-word karaoke tweens can leave captions stuck on screen
      (caption_exit_missing_hard_kill).
    - No duplicate media nodes with identical src + start + duration, or
      the compiler discovers them twice (duplicate_media_discovery_risk).

Housekeeping:
- .gitignore: add cursor-tests/, basecamp-video/, projects/, videos/ —
  local regression-test scratch dirs that should never be committed.
- Remove two broken symlinks from .claude/skills/ that pointed to paths
  which never existed in the repo (.claude/skills/ is already gitignored).

Made-with: Cursor
 2026-04-18 23:17:29 -04:00
Dylan woo 3079e8c950 fix: address review feedback on doctor --json 
Follows up on jrusso1020's review in #320.

Exit code no longer gated on check health
---------------------------------------
`doctor --json` previously set exitCode=1 when any check failed. Two
problems:

- `checkVersion` returns ok:false whenever a newer npm version is
  available, so any pipeline using `hyperframes doctor --json || fail`
  would start failing the next time a new CLI version was published.
- Asymmetric with bare `doctor` which always exits 0.

Exit code now strictly reflects whether the command executed, not
whether the environment is healthy. Consumers who want to gate do:

    hyperframes doctor --json | jq -e '.ok' > /dev/null || handle_failure

Documented that pattern in docs/packages/cli.mdx.

Schema locked with a snapshot test
----------------------------------
Extracted `buildDoctorReport()` as a pure function and added
`doctor.test.ts` covering:

- top-level key set (any accidental rename/addition fails the test)
- shape of each CheckOutcome entry
- ok flag true/false semantics
- check-order preservation
- hint field: omitted when absent, preserved when present
- redact option both on and off

Any future refactor that silently breaks the documented JSON contract
will now fail CI.

$HOME redaction for JSON mode
-----------------------------
JSON output is explicitly designed to be pasted into bug reports and
agent contexts. Added `redactHome()` so the user's home directory is
replaced with the literal `$HOME` in `detail`/`hint` when --json is
set. Human mode is unchanged (shows real paths).

Import grouping
---------------
Moved `node:os` + `_examples` imports up with the rest so `export const
examples` no longer sits between imports.
 2026-04-19 11:01:04 +08:00
ukimsanov 92a5ef419b feat(capture): improve capture quality + clean up CLAUDE.md 
Capture improvements:
- Font weights via document.fonts API + DOM sampling (variable font detection)
- Section background-image extraction (no more false #FFFFFF on hero photos)
- Detected libraries surfaced in CLAUDE.md brand summary
- Structured visible-text.txt with [tag] prefixes, cookie/nav noise filtered
- tokens.json cleaned: removed images/paragraphs/icons (duplicated elsewhere),
  filtered sections to heading-only, trimmed cssVariables to design-relevant
- Removed redundant scroll pass in htmlExtractor (2-5s faster per capture)
- Font cap at 20 families, Placeholder/Fallback fonts filtered

CLAUDE.md rewrite:
- Removed prescriptive tone ("use exact strings" → "rephrase freely")
- Removed fluff sections (How to Create, DESIGN.md warning, Example Prompts,
  Source Patterns)
- asset-descriptions.md promoted to first data row
- Removed assets-catalog.json from inventory

Skill fixes:
- Dead shader refs → point to packages/shader-transitions/README.md
- Google Fonts import in techniques.md → local @font-face placeholder
- Added Stripe DESIGN.md as light-brand example
- Removed HANDOFF.md generation from step-7
- Updated step-1 for new font weight + visible-text formats
 2026-04-18 18:48:37 -04:00
James 38b7cb1c66 chore: release v0.4.6 2026-04-18 22:36:20 +00:00
James RussoClaude Opus 4.7Codex
 		ad11de698c feat(producer): auto-fallback screenshot capture for raf and iframes (#331) 
* fix(core): drive adapter seeks when composition has no GSAP timeline

renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.

Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.

GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(producer): auto-fallback screenshot capture for raf and iframes

Co-Authored-By: Codex <codex@openai.com>

* test(producer): add render compatibility regression fixtures

Co-Authored-By: Codex <codex@openai.com>

* fix(core): scrub CSS animations via WAAPI currentTime

Co-Authored-By: Codex <codex@openai.com>

* test(producer): cover css keyframe renders

Co-Authored-By: Codex <codex@openai.com>

* fix(producer): propagate virtual time into iframe documents

Co-Authored-By: Codex <codex@openai.com>

* test(producer): refresh iframe docker golden

Co-Authored-By: Codex <codex@openai.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
 2026-04-18 15:08:52 -07:00
Miguel Ángel 59aa2c9ec9 chore: release v0.4.5 2026-04-18 20:24:03 +01:00
Miguel Ángel 37370e1e7d fix(cli): set GIT_CLONE_PROTECTION_ACTIVE=0 for skills (GH #316) (#328) 
## Summary

Fixes #316 — `hyperframes skills` (and `npx skills add heygen-com/hyperframes`) fails with:

```
■  Failed to clone repository
fatal: active \`post-checkout\` hook found during \`git clone\`
└  Installation failed
```

## Root cause

Two layers stacked:

1. **Git 2.45+ refuses to execute hooks during `git clone` by default.** The opt-in is `GIT_CLONE_PROTECTION_ACTIVE=0` — the env-var name is intentionally explicit about the trade-off.
2. **Users who ran `git lfs install` globally have a post-checkout hook registered at `core.hooksPath`.** When the upstream `skills` CLI shells out to `git clone` to fetch a repo's `skills/` directory, git detects the user's LFS hook and aborts.

The check fires for **any repo**, regardless of whether the cloned repo uses LFS itself — it's protection against the user's own hooks, not the repo's content. Users who have git-lfs installed (very common) hit this for every clone the `skills` CLI does.

## The fix

`hyperframes skills` wraps `npx skills add`. The wrapper now sets `GIT_CLONE_PROTECTION_ACTIVE=0` on the spawned child's env via a single helper (`gitCloneFriendlyEnv`) with a docstring at the call site explaining exactly why. The rest of `process.env` is preserved — proxy settings, extra CA certs, locale, etc. stay untouched.

## What this fix doesn't do (deliberately)

This is the **code-path-we-own** fix. The deeper root cause is that the upstream `skills` CLI (vercel-labs/skills) should set this env var when it shells out to `git clone`. That would fix the bug for every user invoking `skills` directly — not just those who route through our wrapper. An upstream issue should be opened separately; not landing it as part of this PR.

## Users who call `npx skills add` directly

Documented in the new troubleshooting subsection: set the env var manually.

```bash
GIT_CLONE_PROTECTION_ACTIVE=0 npx skills add heygen-com/hyperframes
```

## Tests

`packages/cli/src/commands/skills.test.ts` — 2 cases:
- Every spawned child has `GIT_CLONE_PROTECTION_ACTIVE=0`
- The rest of `process.env` is preserved (not a wiped env)

Uses `vi.mock` on `node:child_process` because ESM doesn't allow live-module `vi.spyOn` on re-exported bindings.

## Docs

`docs/packages/cli.mdx` — new **Troubleshooting** subsection under the `skills` command. Explains both the automatic fix (`hyperframes skills` users are already covered) and the manual workaround (`npx skills add …` users).

## Closes

- #316
 2026-04-18 21:17:26 +02:00
Miguel Ángel e8a48a62d0 fix(producer): external assets work on Windows (GH #321) (#324) 
* fix(producer): external assets work on Windows (GH #321)

Two Unix-only assumptions in the external-asset pipeline caused every
absolute path on Windows to be rejected as "unsafe" at render time:

1. Containment checks used `child.startsWith(parent + "/")`. On Windows
   the separator is `\`, so the predicate is always false unless the
   paths are equal — every external asset tripped the safety guard in
   `renderOrchestrator.ts`. The reporter saw:

     [Render] Skipping external asset with unsafe path:
       hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav

   Fix: use `path.relative()` through a shared helper
   `isPathInside(child, parent)` that normalises separators per-platform
   and correctly rejects siblings whose names start with the parent
   (e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`).

2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`.
   A Windows absolute path (`D:\coder\...`) became
   `"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a
   drive-letter prefix as absolute, `join(compileDir, key)` silently
   escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive
   colon and normalises to forward slashes, producing
   `hf-ext/D/coder/...` — a pure relative path that `path.join` cannot
   promote to absolute on any OS.

Both helpers live in `packages/producer/src/utils/paths.ts` and are
exercised by 14 unit tests covering Unix paths, Windows drive-letter
paths, mixed separators, sibling-prefix confusion, and `..` traversal.

Docs: new "External assets" section in `docs/packages/producer.mdx`
describes detection, sanitised keys, and the cross-platform containment
invariant.

Closes #321.

* fix(producer): address review on #324 — UNC + integration test

Addresses the non-blocking observations from the PR #324 staff review
(https://github.com/heygen-com/hyperframes/pull/324#issuecomment):

1. UNC and extended-length Windows paths.
   `toExternalAssetKey` now handles:
   - `\\?\D:\very\long\path\clip.mp4` (extended-length)      → `hf-ext/D/very/long/path/clip.mp4`
   - `\\server\share\file.wav` (plain UNC)                    → `hf-ext/unc/server/share/file.wav`
   - `\\?\UNC\server\share\file.wav` (extended-length UNC)   → `hf-ext/unc/server/share/file.wav`
   The UNC-collapsed form keeps the server boundary so two different
   servers exposing the same share/file name cannot collide under one
   relative key. Previously both edge cases silently produced keys with
   stray `?` or `:` characters that downstream `isPathInside` rejected —
   not a security hole, but a silent drop of user assets.

2. Short-circuit on already-sanitised input.
   `toExternalAssetKey("hf-ext/…")` now returns its input unchanged
   instead of prepending `hf-ext/` a second time. Makes the helper
   genuinely idempotent, which is what the unit test claimed all along.
   Renamed the test accordingly.

3. JSDoc caller contract.
   `toExternalAssetKey` now documents that it expects canonicalised
   input (`path.resolve`'d upstream) and does not strip `..`
   components. `isPathInside` at copy time is still the defensive
   backstop — called out explicitly in the doc so future callers read
   the contract before the code.

4. End-to-end integration test.
   `renderOrchestrator.test.ts` gains two seam tests that run the full
   external-asset pipeline — build the sanitised key, populate an
   `externalAssets` map, invoke `writeCompiledArtifacts`, and assert
   both the success path (the file lands under `<compileDir>/hf-ext/…`)
   and the escape-rejection path (a malicious `hf-ext/../../etc/passwd`
   key does NOT materialise above `compileDir`). `writeCompiledArtifacts`
   is exported for the test seam with a clear JSDoc disclaimer that
   it's not part of the public API.

22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5).

Out of scope for this follow-up (tracked as follow-ups):
- Centralising every `startsWith("/")` absolute-path check into a
  shared helper across htmlCompiler / audioExtractor / audioMixer /
  videoFrameExtractor. Mentioned in the review; touches 5 files and
  deserves its own PR.
- Windows CI runner.
 2026-04-18 20:59:51 +02:00
Dylanwoo 64e3735100 fix(cli): doctor shows platform-correct install hints (#319) 
The "FFmpeg not found" hint was hardcoded to `sudo apt install ffmpeg`
for any non-macOS platform — Windows users would see an apt command that
doesn't exist on their system, and Red Hat / Arch users got the wrong
package manager too.

`getFFmpegInstallHint()` already exists in browser/ffmpeg.ts (and is
already used by render.ts) and handles darwin / linux / win32 correctly.
Use it here too.

Also rewrite checkFFprobe:
- it previously used `which ffprobe` which is not available on Windows
  (cmd uses `where`), so on Windows the check always reported "Not
  found" even when ffprobe was on PATH
- run `ffprobe -version` directly instead, which works cross-platform
  whenever ffprobe is resolvable on PATH, and surfaces the version
  string in the same style as the FFmpeg check
 2026-04-18 11:59:49 -07:00
Dylan woo 143af6aec3 feat(cli): add --json output to doctor 
`doctor` is one of the first commands users and agents run when something
is off — remote bug reports (e.g. #294, #316, #317) typically include a
doctor screenshot, which is painful to parse programmatically.

Every other CLI command that reports state already supports `--json`
(info, lint, compositions, catalog, benchmark, validate, capture). This
brings `doctor` in line with that convention so:

- CI pipelines can gate on `hyperframes doctor --json` (exit 1 on
  failure) without scraping terminal output
- AI agents consuming telemetry / diagnostic data get structured input
- Bug report tooling can attach machine-readable doctor output

Schema:

    {
      "ok": boolean,
      "platform": "darwin" | "linux" | "win32",
      "arch": "arm64" | "x64" | ...,
      "checks": [
        { "name": "FFmpeg", "ok": true, "detail": "ffmpeg version 8.1 …" },
        { "name": "Docker", "ok": false, "detail": "Not found",
          "hint": "https://docs.docker.com/get-docker/" }
      ],
      "_meta": { "version", "latestVersion", "updateAvailable" }
    }

Uses the existing `withMeta()` helper so the `_meta` envelope matches
other `--json` commands. Human output format is unchanged.
 2026-04-18 23:58:38 +08:00
James 0f4fcbeed8 chore: release v0.4.4 2026-04-18 03:37:09 +00:00
James RussoandClaude Opus 4.7 686e45dac0 fix(engine): suppress font-load 404s by checking console location URL (#313) 
Chrome's "Failed to load resource" message text does not include the failing
URL — it's only on msg.location().url. The previous filter in frameCapture.ts
only checked msg.text(), so every font 404 (e.g. Google Fonts <link> tags
in sandboxed render environments) fell through to the "[non-blocking]"
prefix instead of being suppressed.

Extract the classifier into isFontResourceError() and match against both
text and location.url, and extend the extension match to .ttf/.otf. Adds
a unit test covering the URL-in-location, URL-in-text, and non-font cases.

This is a targeted fix for the render-output noise that PR #311 attempted
to address by adding a ~120-entry SYSTEM_FONTS skip list. That approach
silently shadowed existing FONT_ALIASES (arial→inter, helvetica→inter,
courier new→jetbrains-mono, segoe ui→roboto, etc.) and changed render
output on Linux fleets that don't have those fonts installed. Fixing the
console-log filter here suppresses the noise without changing any font
resolution behavior.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-17 17:38:46 -07:00
Miguel Ángel 42d39866ff fix(studio): iPhone Safari layout + touch-drag scrubber (#308) 
Stacked on top of #307. Fixes three mobile UX bugs that made the studio unusable on iPhone Safari — discovered while testing the audio-ownership work from #307 on a physical device.

## Bugs fixed

### 1. Untappable Play button / bottom controls

`#root` was set to `100vh`. iOS Safari reports `100vh` as the **largest** viewport (toolbar hidden) and never shrinks it — so with the toolbar visible, the bottom of the layout sits under it. The Play button + timecode were fully occluded.

### 2. Scrubber not draggable by touch

The seek bar had only `onMouseDown`. Mouse events don't fire for touches on iOS Safari, so nothing responded. You could tap to jump but not drag.

### 3. Safari's horizontal swipe hijacked scrubber drags

Even when the seek bar caught `pointerdown`, `touch-action: manipulation` still let Safari consume horizontal edge-swipes for back-navigation — dragging the scrubber left was impossible.

## What changed

| File | Fix |
|---|---|
| `packages/studio/src/styles/studio.css` | `#root` → `height: 100dvh` with `100vh` fallback. Dynamic viewport height shrinks when the iOS toolbar is visible, so the bottom of `#root` lines up with the visible area. |
| `packages/studio/src/App.tsx` | Two `h-screen` containers → `h-full` so nested children fill the now-dynamic parent instead of asserting `100vh` and overflowing. |
| `packages/studio/index.html` | Added `viewport-fit=cover` so iOS exposes real `env(safe-area-inset-bottom)` values. |
| `packages/studio/src/player/components/PlayerControls.tsx` | Controls row gets `padding-bottom: calc(0.5rem + env(safe-area-inset-bottom))` so it clears the landscape home indicator. Scrubber replaced `onMouseDown` with `onPointerDown` + `setPointerCapture`, plus `touch-action: none` so Safari doesn't hijack horizontal swipes. Added `pointercancel` + window-level `pointerup` fallbacks. |

All desktop code paths are unchanged: `100dvh` falls back to `100vh`, `env(safe-area-inset-bottom)` is `0` off-iOS, Pointer Events subsume Mouse Events on desktop.

## Verified live

Via the `cloudflared` tunnel I ran during review on the factory-series-c-video project:

- iPhone Safari, portrait: Play button now fully visible and tappable. Bottom controls sit just above the URL bar.
- iPhone Safari, landscape: controls clear the home indicator.
- Finger-drag the scrubber left and right: tracks the touch smoothly, finger can leave the 6 px bar height without losing the drag.
- Desktop click-to-seek and click-drag: still work.
- Arrow-key seeking: still works.

## Stacked dependency

Base is `fix/player-audio-ownership-review` (PR #307). Once #307 merges, rebase this branch onto `main` — the changes are fully independent; the stacking is just to avoid waiting on the review for #307 before shipping pure UX wins.

## Test plan

- [x] `tsc --noEmit` on `packages/studio` — clean
- [x] `bun run --filter @hyperframes/studio build` — clean
- [x] Live repro on iPhone Safari via cloudflared tunnel: Play button tappable, scrubber drags with touch
- [ ] Android Chrome sanity pass before release (same Pointer Events code path, but worth eye-balling)
 2026-04-18 01:44:46 +02:00
Miguel Ángel c49181f1fa fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307) 
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment.

## Significant

### 1\. Drift threshold 150 ms → 50 ms

_mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity.

**Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction.

### 2\. Dynamic sub-composition media proxies

Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track.

Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect.

### 3\. `bridgeMuted` sticky in `syncRuntimeMedia`

The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message.

`syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs.

### 4\. Reset `_audioOwner` on iframe reload

The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently).

`_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`.

## Worth addressing

### 5\. Promotion → observable event + reason

Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state.

**Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once.

### 6\. Parent proxy play() rejection → `playbackerror` event

Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back.

### 7\. Mobile verification on real hardware

Tested with a tunnel in a real iOS device.

## Test gaps (from review)

- `userMuted` stickiness (mirror of the existing `outputMuted` test).
- **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`.
- **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic.
- **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once.
- **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote).
- **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately.
- **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag.

## Minor

- One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear).

## What's good (from the review)

Kept as-is — noted for posterity:

- `muted` vs `volume` framing (orthogonal channels).
- Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing.
- Two orthogonal mute channels.
- Backwards compat (new actions / messages safely ignored by either side).

## Test results

- `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe)
- `packages/core/src/runtime/bridge.test.ts` — **15 tests pass**
- `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface)
- Typecheck green on `core` + `player`
- `tsup` build green on `core` / `player` / `cli`
- Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold.

## Test plan

- [x] Unit tests (83 total across touched files)
- [x] Typecheck clean
- [x] Build clean
- [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened
- [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block
- [x] Physical iOS / Android device verification (unchanged since #298)
 2026-04-18 00:49:57 +02:00
Miguel Ángel e4cfcd3f61 fix(cli): serialize port-availability probes (#309) (#310) 
Closes #309. Full credit to @gigadeniga for the diagnosis — the root cause + proposed fix in that issue are exactly what landed here.

## The bug

\`npx hyperframes preview\` failed deterministically on Crostini (ChromeOS Linux) with \`Ports 3002–3101 are all in use\`, even when nothing was actually listening on any of them.

## Why

\`testPortOnAllHosts\` ran four probes in parallel:

\`\`\`ts
const hosts = ["127.0.0.1", "0.0.0.0", "::1", "::"];
const results = await Promise.all(hosts.map((h) => isPortAvailableOnHost(port, h)));
\`\`\`

Each probe binds a socket and then calls \`server.close()\`. Close is async — the socket stays open until its callback fires on the next event-loop tick. While it's open, the wildcard binds (\`0.0.0.0\`, \`::\`) that include the loopback address race the still-open loopback socket and return \`EADDRINUSE\` spuriously. On Crostini this happens 100% of the time; other Linux configs hit it intermittently; macOS is less predictable. Net effect: every port in the 100-port scan range appears busy and the preview refuses to start.

Reproduces on any Linux box with the standalone snippet from the issue:

\`\`\`
127.0.0.1: OK
0.0.0.0: EADDRINUSE   ← false positive
::1: OK
::: EADDRINUSE        ← false positive
\`\`\`

## Fix

Serialize the probes. Each socket is fully closed before the next opens, eliminating the race window entirely.

\`\`\`ts
for (const host of hosts) {
  const available = await isPortAvailableOnHost(port, host);
  if (!available) return false;
}
return true;
\`\`\`

Kept the four-host check rather than collapsing to just \`0.0.0.0\` + \`::\` — the multi-host coverage is load-bearing for the devbox / SSH-forwarding case where a port is free on loopback but held on the wildcard. Sequentializing is the smaller, less-behaviourally-affecting fix.

## Regression tests

\`packages/cli/src/server/portUtils.test.ts\` — three cases binding real sockets, no mocks:

- **Returns true for a genuinely free port** — directly reproduces the Crostini bug; would fail on Linux against the parallel implementation.
- **Returns false when the port is occupied on \`0.0.0.0\`** — confirms the multi-host check still catches the devbox scenario.
- **Releases each probe socket before the next run** — two back-to-back calls for the same free port both return true, pinning the sequential contract against future refactors that might try to reparallelize for perf.

## Test plan

- [x] \`bunx vitest run packages/cli/src/server/portUtils.test.ts\` — 3/3 pass
- [x] Full CLI suite — 109/109 pass
- [x] \`tsc --noEmit\` clean

## Notes

- Independent of any version bump; ship whenever.
- Probing 4 hosts serially adds at most ~tens of milliseconds per port on the scan (binds are very fast on loopback). The worst-case cost shows up when the first port in the range is free — previously 1 parallel round-trip, now 4 sequential — and it's imperceptible (\`preview\` bind is a one-time startup cost, not a hot path).
 2026-04-18 00:44:58 +02:00
Ular Kimsanov a95539a0fd Merge pull request #299 from heygen-com/feat/capture-improvements-v2 
fix: double-audio scaffold, lint rules, docs guide, Gemini 3.1
 2026-04-17 14:49:05 -04:00
ukimsanovandClaude Opus 4.6 274db7a5ef fix: address PR #299 review — lint correctness, docs, Gemini benchmark 
- lintMultipleRootCompositions: scan filesystem for HTML files with
  data-composition-id (was filtering results array — always 1 entry)
- lintDuplicateAudioTracks: order-independent attribute extraction,
  dedup by (src,start,duration,trackIndex), Infinity fallback for
  missing data-duration (matches runtime behavior)
- 10 new tests for both lint rules
- docs: explicit skill invocation, remove gsap-skills, fix indentation
- Gemini: env override (HYPERFRAMES_GEMINI_MODEL), benchmark data in
  code comment (49 imgs: 3.1-lite ~507ms/img, 2.5-lite ~230ms/img)
- cli.mdx: version-agnostic "Gemini vision" reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-17 10:15:26 -04:00
Miguel Ángel 46afe2ea5d chore: release v0.4.3 2026-04-17 12:15:37 +02:00
ularkim de5b53c08b feat(capture): switch Gemini 2.5 Flash → 3.1 Flash Lite 
Gemini 3.1 Flash Lite Preview: 2.5x faster TTFT, 45% faster output,
slightly cheaper ($0.25/M vs $0.30/M input), near-2.5-Flash quality.
Descriptions are actually more detailed in testing.
 2026-04-16 22:58:50 -04:00
ularkim a77a6cbbf7 fix: double-audio bug + lint rules + docs guide + capture improvements 
Double-audio bug fix:
- scaffolding.ts: stop writing index.html in captures/ (root cause —
  runtime discovered scaffold + real index.html as two compositions)
- New lint rule: multiple_root_compositions — errors if >1 root HTML
- New lint rule: duplicate_audio_track — warns on overlapping audio

Capture improvements (from testing 30+ websites):
- Catalog runs BEFORE extractHtml (which mutates DOM — converts img src
  to data URLs). HeyKuba: 2 images → 78.
- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load image wait, CSS background-image cataloging
- SVG naming from class/id/parent (not just aria-label)
- Gemini batch 5→20, pause 12s→2s, maxOutputTokens 300→500
- Asset descriptions sorted: captioned first

Docs:
- New guide: guides/website-to-video.mdx (full tutorial)
- CLI docs: added capture and snapshot commands
- docs.json: website-to-video in Guides nav

C
 2026-04-16 22:58:50 -04:00
Miguel Ángel 3256551a5e fix(player): single-owner audio to prevent double voice in preview (#298) 
## Summary

Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip).

## Root cause

Two audio pipelines were playing the same source in parallel:

1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path.
2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message.

The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks.

Confirmed via agent-browser instrumentation on `factory-series-c-video`:
- 6 `volumechange` events per play cycle (mute-fight signature)
- Both copies audible at `volume=1`, offset growing 23ms → 80ms
- Every seek widened the drift further

PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback.

## Fix

Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime.

- **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash.
- **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold.

Two orthogonal mute channels replace the volume fight:

| Channel | Purpose |
|---|---|
| `set-muted` | User's mute preference (existing, unchanged) |
| `set-media-output-muted` | Internal ownership handoff (new) |

`syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it.

## Why this shape

- **Single owner, explicit transition.** No races, no tug-of-war.
- **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing.
- **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact.
- **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift.
- **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior.
- **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files.

## Files changed

- `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types.
- `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields.
- `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`.
- `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`.
- `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session.
- `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership.

## Verified end-to-end with agent-browser on `factory-series-c-video`

**Runtime ownership (default — desktop studio):**

| | Before | After |
|---|---|---|
| `PARENT.play()` calls per play cycle | 1 | **0** |
| iframe `volumechange` events | 6 | **0** |
| Audible streams | 2 (drifting) | **1 (iframe)** |

**Parent ownership (simulated autoplay block — direct message):**

| | Value |
|---|---|
| iframe audio | `muted=true`, `volume=1` (untouched) |
| parent audio | `muted=false`, `volume=1`, audible |
| Parent ↔ iframe `currentTime` offset | ~6 ms steady state |
| Offset > 150 ms | corrected by mirror sync |

**Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:**

Event timeline captured via agent-browser instrumentation:

```
t=0.0 ms   IFRAME.play() called                       ← runtime attempts playback
t=0.4 ms   IFRAME.play() REJECTED: NotAllowedError    ← simulated mobile gate
t=0.4 ms   →IFRAME bridge set-media-output-muted=true ← player promotes
t=0.6 ms   PARENT.play() called                       ← parent proxy starts
t=0.8 ms   ←IFRAME msg media-autoplay-blocked         ← runtime signal
t=1.0 ms   PARENT.play() resolved                     ← audible
t=1.3 ms   IFRAME muted=true, volume=1                ← iframe silenced via native muted
```

Steady state at t=4 s under promoted parent ownership:

| Element | currentTime | paused | volume | muted |
|---|---|---|---|---|
| Parent audio | 4.060 s | false | 1.0 | **false** (audible) |
| Iframe audio | 4.068 s | false | 1.0 | **true** (silent) |

**Offset: 8 ms**, single audible stream, orthogonal mute channel respected.

## Test plan

- [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`)
- [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow)
- [x] `bun run build` — all packages green
- [x] Fresh preview + browser repro on `factory-series-c-video`:
  - [x] Runtime ownership: single audio stream, no drift
  - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible
  - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset
  - [x] No `volumechange` thrash in either ownership mode
- [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
 2026-04-17 04:46:22 +02:00
Miguel Ángel d291358cbc chore: release v0.4.2 2026-04-17 01:16:46 +02:00
Miguel Ángel 5e52e27872 fix(engine): auto-fall back to screenshot mode when chrome-headless-shell drops HeadlessExperimental.beginFrame (#296) 
Closes #294.

## Summary

Recent `chrome-headless-shell` builds (observed on 147) no longer expose `HeadlessExperimental.beginFrame`. The domain's `enable`/`disable` methods are deprecated upstream and appear to have been dropped alongside `beginFrame` in these builds, so on Linux with chrome-headless-shell the engine aborts with

\`\`\`
Protocol error (HeadlessExperimental.beginFrame):
'HeadlessExperimental.beginFrame' wasn't found
\`\`\`

and — because the browser was launched with `--enable-begin-frame-control` — the compositor waits for beginFrames the engine can no longer deliver, so every subsequent screenshot also comes back blank. Today users have to discover `PRODUCER_FORCE_SCREENSHOT=true` themselves (openclaw did exactly that — see the issue body).

## Fix

One-time probe, right after the browser launches in beginframe mode:

1. Create a disposable CDP session.
2. `await client.send("HeadlessExperimental.enable")`.
3. Send one no-op `HeadlessExperimental.beginFrame` raced against a 2s timeout.
4. If anything throws / times out — missing method, protocol error, stuck call — close the browser, strip beginframe-only chrome flags, relaunch in screenshot mode, and set \`captureMode = "screenshot"\` for the returned session.

Probing `beginFrame` directly rather than `enable` alone is important because some builds keep the domain registered (so `.enable()` succeeds) while dropping the method itself — that's exactly the failure shape in #294.

Cost on happy path: one extra CDP round-trip per browser acquisition (≈ a few ms, since in beginframe-control mode the command returns as soon as the compositor acks). Cost on broken path: one extra launch, which is what the env-var escape hatch already forces manually.

The beginframe-only flag set is enumerated in-module and matched by the stripper, so adding/removing flags stays in one place with `buildChromeArgs`.

## Test plan

- [x] `bun run --filter=@hyperframes/engine test` — all 42 tests pass
- [x] `bun run --filter=@hyperframes/engine build` — typechecks
- [x] `bunx oxlint` + `bunx oxfmt --check` clean
- [x] Manual: standalone test on Linux x86_64 with chrome-headless-shell 146 — probe returns `supported=true`, no fallback (happy path)
- [x] Manual: same test with `--force-fail` simulating openclaw's missing-method condition — fallback triggers, flags stripped, relaunch succeeds, 6.8 KB PNG captured (broken path)
- [ ] Verify on openclaw / real chrome-headless-shell 147 build that the fallback triggers automatically without `PRODUCER_FORCE_SCREENSHOT`

## Notes

- `probeBeginFrameSupport` catches any failure generically; we trust that a working browser answers the no-op beginFrame in well under 2s.
- Warning is logged once per browser acquisition, not per frame.
- Browser pool interaction: pooled browsers cache the resolved `captureMode`, so subsequent acquires in the same process reuse the post-fallback mode without re-probing.
 2026-04-17 01:10:08 +02:00
Miguel Ángel 96376c9be0 chore: release v0.4.1 2026-04-17 00:25:34 +02:00
Miguel Ángel e70687b66c fix(player): resolve iframe media src against iframe baseURI (#295) 
## Summary

`_setupParentMedia` scans the iframe for `audio[data-start]` / `video[data-start]` and creates parallel media elements in the host document (so the studio can scrub audio at sub-frame precision without iframe cross-origin restrictions). It was reading the raw `src` attribute string and assigning it directly to the host-document element, which then resolved relative URLs against the **studio root** instead of the **iframe**.

Result: a composition like

\`\`\`html
<audio id="narration" data-start="0" data-duration="53" src="assets/narration.wav"></audio>
\`\`\`

played fine in rendered MP4 output but 404'd silently in the studio preview (parent audio got `src = http://localhost:PORT/assets/narration.wav` instead of `http://localhost:PORT/api/projects/<name>/preview/assets/narration.wav`).

## Fix

Resolve the src against \`iframeEl.ownerDocument.baseURI\` before passing it to \`_createParentMedia\`. Also read the raw \`src\` attribute on \`<source>\` fallbacks so both paths go through the same resolution.

Diff is 2 lines of meaningful change (9 total once you include the comment).

## Reproduction

1. Create a project with a narration at \`assets/narration.wav\`
2. Reference it in \`index.html\` with \`<audio data-start="0" data-duration="53" src="assets/narration.wav">\`
3. \`npx hyperframes preview\` → open, click play
4. Before: silent (parent audio's \`error.code === 4\` / \`MEDIA_ERR_SRC_NOT_SUPPORTED\`)
5. After: narration plays, scrubbing syncs

## Test plan

- [x] Existing 21 player tests pass (`bun run --filter=@hyperframes/player test`)
- [x] oxlint + oxfmt clean on changed file
- [x] Manual: verified in-studio playback of a narration sourced via relative URL
- [x] Reviewer: confirm render pipeline unaffected (render doesn't go through `_setupParentMedia`)

## Notes

No tests added for this path because the existing harness covers only the `audio-src` attribute codepath — `_setupParentMedia` is triggered by an internal probe interval against a live iframe, which the current fixture doesn't build. Happy to add one in a follow-up if reviewers want that coverage before merge.
 2026-04-17 00:23:07 +02:00
James 60780774bb chore: release v0.4.0 2026-04-16 19:58:39 +00:00
Miguel Ángel f1a37400b4 fix(runtime): silent-first-play + loading overlay for preview (#293) 
## Summary

Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit.

### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`)

`syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window):

- `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click."
- The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged.

`HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`.

### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`)

The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s.

### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`)

Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration.

Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR.

### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`)

Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync.

## Verification

- 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset.
- Full monorepo build green (`bun run build`), typecheck clean, lint/format clean.
- End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass:

| Scenario | Metric | Result |
| --- | --- | --- |
| Normal first play | Audio plays from click, smooth progression | ✅ |
| Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ |
| Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ |
| Scrub mid-playback | Lands exactly at target frame | ✅ |

## Files changed

- `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve.
- `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset.
- `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes.
- `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check.

## Follow-ups (deferred)

- Tight-threshold short-window drift correction for lip-synced dialogue.
- A perf-regression test that fails on `waiting`\-event resurgence.

## Test plan

- [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words.
- [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events).
- [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers.
- [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame.
 2026-04-16 21:51:06 +02:00
Ular KimsanovandClaude Opus 4.6 87f4c77e2f feat: website capture pipeline + 7-step video production skill (#284) 
* feat(cli): add website capture with AI-powered DESIGN.md generation

Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:

- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
  components, do's/don'ts) with programmatic asset catalog (136+ assets
  with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering

* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update session handoff with slack research findings

- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps

* refactor(cli): simplify capture pipeline, remove replica generator

* feat(capture): add Lottie detection and WebGL shader extraction

Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(capture): clean pipeline + shader-first creative workflow

Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>

Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill architecture redesign spec

Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add implementation plan for skill architecture redesign

13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(capture): remove AI auto-generation and SDK dependencies

* fix(capture): convert extracted colors to HEX format

* refactor(capture): remove AI key path, add asset descriptions generator

* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions

* feat(capture): pre-wire shader transitions in index.html scaffold

* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)

* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator

* feat(skill): add Phase 1 understand reference

* feat(skill): add Phase 2 design reference with full DESIGN.md schema

* feat(skill): add Phase 3 creative direction reference

* feat(skill): add Phase 4 build reference with inline shader example

* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md

* docs: update CLAUDE.md skill references for phase-based workflow

* fix: address code review findings

- Remove orphaned `false` argument in generateAgentPrompt call (critical:
  was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions

- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
  so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
  to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
  nonexistent htmlContext field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: code review — 16 bugs, 7-step skill rewrite, cleanup

Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
  for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
  batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection

Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1

Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dev artifacts, research docs, wrong lockfiles

Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention

Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add capture command to help groups

The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format skill reference files (oxfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate bun.lock after rebase

The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments + improve capture quality

Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)

Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
  elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
  to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note

* fix: address PR review comments + improve capture quality

Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
  across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
  (avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
  is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
  like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)

Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.

Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.

* refactor(capture): split index.ts (1175 to 566 lines) into modules

Mechanical extraction, zero logic changes.

New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation

Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.

* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)

The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.

No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.

Backed up to ~/Desktop/capture-split-backup/ for reference.

* fix(security): add ssrf protection, lottie injection fix, oom guard

- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
  (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
  endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
  then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
  Lottie network interception to avoid OOM on multi-GB responses

* fix(capture): security fixes, timeout, sub-agent dispatch instructions

Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()

Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
  pass file PATHS not contents, use local fonts not Google Fonts,
  verify ../assets/ references after each beat

* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini

Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.

- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 12:03:16 -07:00