Commit Graph
350 Commits
Author SHA1 Message Date
JamesandClaude Opus 4.7 fd3fce9955 refactor(producer): tighten resolveChunkPlan assertion + trim comments
Address self-review findings:
- assertPositiveInteger now only runs on the caller-supplied path so the
  error message names `configChunkSize` only when the caller actually
  passed one. Previously, the assertion fired against `resolvedChunkSize`
  on both paths and would have lied about the offending input.
- Drop the call-site comment that narrated the diff/history; the
  function docstring already covers the contract.
- Drop the internal-track name and date from the MIN_CHUNK_SIZE rationale
  and the test block header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 20:37:25 +00:00
JamesandClaude Opus 4.7 e79a8faa42 feat(producer): auto-size chunkSize from maxParallelChunks when undefined
Previously, plan() defaulted chunkSize to 240 on a `?? DEFAULT_CHUNK_SIZE`
line, so a 660-frame composition with maxParallelChunks=16 ended up at 3
chunks (ceil(660/240)) regardless of the caller's fan-out intent.

When config.chunkSize is undefined, auto-size from maxParallelChunks:
  effectiveChunkSize = max(MIN_CHUNK_SIZE, ceil(totalFrames / maxParallelChunks))

MIN_CHUNK_SIZE=10 keeps per-chunk fixed overhead from swamping the
parallelism gain on tiny renders. Explicit numbers, including 240, take
precedence over the auto-sizer — no behavior change for callers that set
chunkSize explicitly.

Surfaced by the lever-1 chunk-scaling benchmark on 2026-05-17.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 20:29:34 +00:00
Miguel Ángel 27efcd0f80 chore: release v0.6.22 2026-05-18 14:23:43 -04:00
Miguel Ángel 8163f38077 chore: release v0.6.21 2026-05-18 11:29:01 -04:00
Miguel Ángel 4bf2fa5cd1 fix: revert flattenInnerRoot, use host-level authored-id + count-based rebind
Three changes to fix regression failures without breaking baselines:

1. Revert flattenInnerRoot in producer — use the original innerHTML
   inlining that preserves the existing DOM structure. Instead, set
   data-hf-authored-id on the HOST element so the scoped proxy can
   still rewrite #id selectors for sub-composition scripts.

2. Revert compiled.html baselines to main (no DOM structure changes).

3. Use timeline-count comparison instead of poll duration to decide
   whether to call __hfForceTimelineRebind. Compare timeline count
   before vs after the poll — rebind only when new timelines appeared
   during polling. This correctly identifies async compositions
   regardless of fetch speed, while leaving sync compositions
   untouched.
2026-05-18 10:20:17 -04:00
Miguel Ángel 04c35ce24c fix: regression fixes — conditional rebind + updated compilation baselines
1. Only call __hfForceTimelineRebind() when the timeline poll actually
   had to wait (pollDuration > 2 intervals). For compositions with
   synchronous timeline registration, the rebind was unnecessary and
   shifted render timing, causing PSNR regressions in chat and
   gsap-letters-render-compat.

2. Regenerate compiled.html baselines for missing-host-comp-id and
   overlay-montage-prod to match the new flattenInnerRoot behavior
   (data-composition-id stripped from inlined inner roots, replaced
   with data-hf-authored-id).

3. Add late-bind polling to runtime init.ts — after external
   compositions load, poll for 5s to detect async timelines that
   register after initial binding (e.g. from fetch callbacks).
2026-05-18 03:46:58 -04:00
Miguel Ángel 0f1c64dcae fix: address review feedback — observability, dedup, query-strip, catalog
Review items addressed:

1. Mirror video-failure warning in beginFrame path (was screenshot-only)
2. Fix resolveProjectRelativeSrc escape-fallback to use query-stripped
   cleanSrc instead of raw src for the normalize/strip arm
3. Export prepareFlattenedInnerRoot from @hyperframes/core/compiler and
   consume in the producer instead of duplicating the implementation
4. Use typed Window cast instead of (window as any) for __hfForceTimelineRebind
5. Regenerate docs/public/catalog-index.json with all 6 map blocks
6. Restore Maps nav group in docs.json (catalog generator had merged
   them into Data)
2026-05-18 01:41:15 -04:00
Miguel Ángel cf2addfb92 chore: remove accidentally committed producer node_modules 2026-05-18 01:28:31 -04:00
Miguel Ángel 4b9abaaf6f fix: deterministic stagger, lint importmap/module, cache-bust previews
- Replace from:"random" with from:"center" stagger in us-map,
  world-map, spain-map — random stagger is non-deterministic across
  parallel render workers, causing visual jumps at chunk boundaries.

- Exempt type="importmap" and type="module" inline scripts from the
  invalid_inline_script_syntax lint rule. The rule used new Function()
  to parse, which rejects import statements and JSON import maps.
  Closes #929.

- Cache-bust all map MDX preview video URLs after re-rendering with
  the deterministic stagger fix.
2026-05-18 01:18:24 -04:00
Miguel Ángel 5aff328a9b fix(producer): preserve inner root element when inlining sub-compositions
The producer's inlineSubCompositions was not passing flattenInnerRoot,
causing sub-composition inner root elements to be unwrapped during
compilation. Scripts using #id selectors (rewritten to
[data-hf-authored-id] by the scoping proxy) could not find their DOM
root because the authored-id attribute was never set.

The core bundler (bundleToSingleHtml) already passed flattenInnerRoot
correctly. This aligns the producer's render compilation with the
same behavior: clone the inner root, strip timing/composition attrs,
replace id with data-hf-authored-id, and mark with
data-hf-inner-root.
2026-05-18 00:27:20 -04:00
Miguel Ángel 5beeb4227e Merge pull request #927 from heygen-com/fix/caption-polish
fix(registry): polish caption timing and add highlight style
2026-05-18 03:06:57 +02:00
Miguel Ángel c8e8fdcf6d fix(producer): fetch missing font weights from Google Fonts at render time
The deterministic font system now supplements its embedded font bundle
with Google Fonts fetches for any weights not in the pre-bundled set.
This fixes compositions that request font weights (e.g. Montserrat 300)
not included in the CANONICAL_FONTS faces array — previously those
weights were silently dropped, causing invisible text in renders.

Also replaces caption-glitch-rgb and caption-weight-shift with improved
versions from avatar preview compositions, adapted to 1920x1080 with
standard demo transcript.
2026-05-17 20:41:30 -04:00
James Russo 4f274d3605 Merge pull request #920 from heygen-com/chore/aws-lambda-publish-ready
chore(lambda): publish-readiness for @hyperframes/aws-lambda
2026-05-17 19:41:04 -04:00
James 187dfd5b6c fix(producer): indirect lambda-local import path so tsc can't resolve it
The tsconfig `exclude` list isn't enough to keep producer's tsc emit pass
from pulling `regression-harness-lambda-local.ts` (and its
`@hyperframes/aws-lambda` static imports) into the program — tsc still
statically resolves the path in
`await import("./regression-harness-lambda-local.js")` from
`regression-harness.ts`, walks into the excluded file, and fails on
the missing aws-lambda type declarations.

Reproduction (clean workspace, no aws-lambda dist yet, mirrors CI):

  rm -rf packages/{aws-lambda,producer,core}/dist
  bun run build
  # @hyperframes/producer build: src/regression-harness-lambda-local.ts(36,70):
  # error TS2307: Cannot find module '@hyperframes/aws-lambda' or its
  # corresponding type declarations.

Fix: route the dynamic import path through a top-level string constant
so tsc can't statically resolve the target. tsc keeps the type-only
imports (`RunLambdaLocalRender` from the no-aws-lambda types file) and
treats the dynamic-import target as opaque. `tsx` resolves the path
normally at runtime, so `--mode=lambda-local` is unchanged.
2026-05-17 23:11:45 +00:00
Miguel Ángel 20a6ff6505 Merge pull request #925 from heygen-com/fix/regenerate-style-baselines
test: regenerate style-1-prod and style-12-prod baselines
2026-05-18 01:08:38 +02:00
Miguel Ángel 6c533c0b0f chore: release v0.6.20 2026-05-17 18:07:59 -04:00
Miguel ÁngelandClaude Sonnet 4.6 4d403b0d73 test: regenerate style-1-prod and style-12-prod baselines in Docker
Sub-comp visibility fix (PR #918) changed rendered output for these two
tests but the baselines on main were stale. Regenerated inside
Dockerfile.test to match CI's Chrome + ffmpeg build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 22:03:17 +00:00
Miguel Ángel 04aa6a644f chore: release v0.6.19 2026-05-17 17:55:32 -04:00
James a18fc04bf4 fix(producer): break aws-lambda↔producer build cycle for lambda-local harness
The aws-lambda publish-readiness changes earlier in this PR moved
aws-lambda's types from `./src/index.ts` → `./dist/index.d.ts`. That
flipped producer's emit pass from "always works" to "needs aws-lambda
built first," because producer's `regression-harness-lambda-local.ts`
imports `@hyperframes/aws-lambda{,/handler}`. aws-lambda's own emit
pass in turn imports `@hyperframes/producer{,/distributed}` → circular
build dependency, so neither side can emit declarations in a single
pass. CI's first run on this PR failed Build, Typecheck, CLI smoke,
windows tests, and every perf job for this reason.

Fix: extract the public surface of `regression-harness-lambda-local.ts`
into a new types-only file that has no aws-lambda imports, point
`regression-harness.ts` at that types file, and exclude
`regression-harness-lambda-local.ts` from producer's tsconfig
`include`. The implementation file is still loaded at runtime via
`tsx` (lambda-local mode runs the harness through tsx, not from
producer's dist/), so the runtime contract is unchanged — producer's
tsc just no longer has to type-check it.

  - `regression-harness-lambda-local-types.ts` (new): exports
    `RunLambdaLocalInput` + `RunLambdaLocalRender` with zero
    aws-lambda imports.
  - `regression-harness-lambda-local.ts`: re-exports
    `RunLambdaLocalInput` from the new types file (so the public
    name stays stable for any future direct imports).
  - `regression-harness.ts`: drops the `typeof import("...")` trick
    and uses the explicit `RunLambdaLocalRender` signature for the
    dynamically-loaded function.
  - `tsconfig.json`: excludes `src/regression-harness-lambda-local.ts`
    so producer's emit pass never resolves `@hyperframes/aws-lambda`.

Verified:

  bun run build                   # full root build green
  bun run verify:packed-manifests # all packages publish-safe
2026-05-17 21:43:46 +00:00
Miguel Ángel 1ac70e8968 Merge pull request #922 from heygen-com/fix/css-import-inlining
fix(core): resolve CSS @import and rebase url() paths during bundling
2026-05-17 23:28:36 +02:00
Miguel Ángel aa2f59c1d2 chore: untrack packages/producer/dist build artifacts
The root .gitignore already has `dist/` but these files were
force-added at some point. No other package has dist/ tracked.
2026-05-17 17:24:20 -04:00
Miguel Ángel 7a1e8769a5 Merge pull request #918 from heygen-com/refactor/unify-subcomp-inlining
fix: hold external sub-compositions in render mode + regenerate baseline
2026-05-17 20:22:21 +02:00
James Russo b05a22e69e feat(producer): add --mode=lambda-local to the regression harness (#913)
* feat(producer): add --mode=lambda-local to the regression harness

Third harness mode that drives the OSS @hyperframes/aws-lambda handler
through the exact event sequence Step Functions produces in
production:

  handler({Action: "plan"})             → planDir tarball on fake S3
  handler({Action: "renderChunk"}) × N  → chunk artifacts on fake S3
  handler({Action: "assemble"})         → final mp4/mov/png-sequence

The S3 client is a filesystem-backed fake (every s3://<bucket>/<key>
URI maps to <tempRoot>/s3/<key>), so the harness exercises the
handler's event-parsing + tar/S3 conventions + dispatch logic on top
of the underlying producer primitives. Regressions in event JSON
shape, S3 key layout, or plan-hash boundary checks now surface in
the same CI run as the in-process and distributed-simulated modes
without paying for a real AWS round-trip.

Deliberately NOT a Docker/RIE invocation — that would gate the
producer test suite on Docker-in-Docker support which most CI
runners lack. Real-ZIP-via-RIE tests live in
packages/aws-lambda/scripts/ (probe:beginframe) and the
maintainer-run smoke.sh.

Wired up via:
  - HarnessMode union extended to include "lambda-local"
  - parseHarnessModeFlag accepts --mode=lambda-local
  - regression-harness.ts dispatches to runLambdaLocalRender for
    the new mode, sharing the distributed-support gate +
    pathology-floor threshold with distributed-simulated mode
  - package.json scripts: test:lambda-local + docker:test:lambda-local
  - producer.devDependencies += @hyperframes/aws-lambda (workspace)
  - producer/tsconfig.json gains path mappings to self so the type
    cycle through aws-lambda's source resolves at typecheck time
    without needing producer to be pre-built

Tests: 3 new unit tests on parseHarnessModeFlag + resolveMinPsnrForMode
cover the new mode. End-to-end PSNR contract still runs through
Dockerfile.test (manual + CI).

* refactor(producer): /simplify pass on lambda-local harness imports

Three small cleanups on top of the lambda-local harness:

  - Drop the unused createReadStream import + its `void` workaround
    comment. The aws-lambda handler's tar / S3 transport pulls
    createReadStream from its own imports; this file never references
    it directly.

  - Hoist the dynamic `await import("node:fs")` calls for
    writeFileSync out of FilesystemBackedFakeS3.send into the static
    import block. Repeated PutObject calls don't need to repay the
    dynamic-import cost.

  - Hoist the dynamic `await import("@hyperframes/aws-lambda")` call
    for untarDirectory similarly. Drops the now-redundant duplicate
    aws-lambda import statement.

The PutObject body branch also collapses: `body instanceof Buffer`
and `typeof body === "string"` both call writeFileSync identically,
so they share one branch.

No behavior changes.

* fix(producer): lazy-import lambda-local harness module

The static import of regression-harness-lambda-local.ts pulled
@hyperframes/aws-lambda (and its @aws-sdk/* + @sparticuz/chromium
transitive deps) at module-load time. Dockerfile.test only copies
the producer's own files into the container, so aws-lambda's src
isn't present at runtime — and even `--mode=in-process` failed:

  Error [ERR_MODULE_NOT_FOUND]: Cannot find module
  '/app/packages/producer/node_modules/@hyperframes/aws-lambda/src/index.ts'
  imported from /app/packages/producer/src/regression-harness-lambda-local.ts

Load the module on demand instead. `--mode=lambda-local` callers
pay the import cost; the existing in-process and distributed-
simulated modes don't.

* fix(producer): address PR review on lambda-local harness

Three review items from Vai:

  - `Config.width`/`Config.height` are now plumbed through
    RunLambdaLocalInput rather than hardcoded inside
    runLambdaLocalRender. Lambda-local's whole point is to catch
    event-shape drift; if the handler ever starts honouring
    Config.width/height (e.g. for canvas sizing), having those
    values flow from the caller means the harness sees what the
    fixture authored. The interface change makes the eventual
    upgrade-to-real-fixture-resolution a one-line dispatch swap.

  - Drop the dead `export type { Fps }` and its unused import
    from @hyperframes/core. The module never re-exports it.

  - The dispatch site in regression-harness.ts now passes 1920×1080
    explicitly with a comment marking it as a placeholder until
    the harness compiles the composition HTML up-front to surface
    the authored data-width/data-height. distributed-simulated
    mode uses the same placeholder internally, kept for parity.

No behavior change in the existing modes; lambda-local now has a
clear extension point for honouring fixture dimensions.
2026-05-17 14:04:49 -04:00
Miguel ÁngelandClaude Sonnet 4.6 54afeccde7 test: regenerate all affected baselines in Docker
Regenerated baselines for all regression tests with sub-compositions
in the cancelled shards: style-3-prod, style-5-prod, style-9-prod,
style-15-prod, style-16-prod, style-17-prod, style-18-prod,
sub-composition-video, many-cuts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 17:47:25 +00:00
Miguel ÁngelandClaude Sonnet 4.6 819b00b9fe test: regenerate style-5-prod baseline
Sub-composition visibility fix changes output for compositions
with external sub-compositions. Baseline regenerated in Docker.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 17:37:37 +00:00
Miguel ÁngelandClaude Sonnet 4.6 d242c73948 test: regenerate style-1-prod baseline after linkedom fragment fix
The sub-composition inlining now correctly preserves composition IDs
when the host data-composition-id differs from the inner root's
(e.g., host "captions-comp" with inner root "captions"). The captions
layer renders with proper scoping, changing visual output.

Baseline regenerated inside Docker per CLAUDE.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 17:13:52 +00:00
Miguel ÁngelandClaude Sonnet 4.6 581e7a7eda refactor: extract shared inlineSubCompositions from bundler and producer
Both the core bundler (htmlBundler.ts) and the producer (htmlCompiler.ts)
had parallel ~200-line implementations of sub-composition inlining. This
divergence caused bug #911 (producer didn't set data-composition-file).

Extract the shared logic into core/compiler/inlineSubCompositions.ts:
- Single function handles: template/body extraction, CSS/script scoping,
  asset path rewriting, data-composition-file attribution, content injection
- Callers provide environment-specific callbacks (HTML resolution, parsing,
  variable handling, inner root flattening)
- Core bundler passes its advanced features (runtime IDs, variables,
  inline style rewriting, inner root flattening)
- Producer passes a simpler resolver (map + filesystem fallback) and
  adds pixel sizing post-hoc

Net: -215 lines, one source of truth for sub-comp inlining.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 16:28:20 +00:00
Miguel ÁngelandClaude Sonnet 4.6 b9bdc80db7 test: regenerate style-12-prod baseline after sub-comp duration hold fix
The sub-composition visibility fix (2b46565c) correctly holds external
compositions through their authored data-duration. This changes
style-12-prod output from t=8.26s onward: the mondrian-colors
sub-composition now stays visible instead of going black when its GSAP
timeline ends.

Baseline regenerated inside Docker per CLAUDE.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 16:14:43 +00:00
Miguel ÁngelandClaude Sonnet 4.6 78fce8bd8a chore: release v0.6.18
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 15:32:53 +00:00
Miguel ÁngelandClaude Sonnet 4.6 2b46565c65 fix(runtime): hold external sub-compositions in render mode
PR #917 fixed visibility clamping for external sub-compositions in
preview mode by checking data-composition-src. However, the producer's
htmlCompiler strips that attribute during inlining without setting the
data-composition-file marker that the core bundler sets. This caused
the runtime to still clamp duration to Math.min(authored, live) in
rendered output.

Two fixes:
- Runtime: also check data-composition-file (set by the core bundler
  after inlining)
- Producer: set data-composition-file before removing
  data-composition-src, matching the core bundler's behavior

Closes #911

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 15:30:23 +00:00
Miguel ÁngelandClaude Sonnet 4.6 3f976d454c chore: release v0.6.17
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 15:15:54 +00:00
James b30fd29695 chore: release v0.6.16 2026-05-17 08:20:08 +00:00
James Russo f01fccb0ea perf(distributed): skip eager probe session when chunkWorkerCount > 1 (#916) 2026-05-17 04:18:47 -04:00
James Russo 34d1f0e1d0 feat(lambda): add TypeScript SDK and CDK construct (#909)
* feat(lambda): add TypeScript SDK and CDK construct

Adds the client-side surface on top of the Phase 6a Lambda handler so
adopters can drive a deployed stack from Node without writing AWS-SDK
boilerplate:

- renderToLambda(opts) starts a Step Functions execution and returns a
  handle. Does NOT poll.
- getRenderProgress({ executionArn }) returns a snapshot of progress,
  frames rendered, cost (Lambda GB-seconds + SFN transitions), errors,
  and the final output object once Assemble completes.
- deploySite({ projectDir, bucketName }) content-addresses the project
  tree, tar.gzs it, and uploads to S3 with a HeadObject short-circuit so
  re-renders of the same tree skip the tar+PUT.
- validateDistributedRenderConfig throws a typed InvalidConfigError
  before StartExecution, so shape errors surface synchronously.
- computeRenderCost is exposed for callers who want to format cost out
  of band.

Also ships HyperframesRenderStack, an aws-cdk-lib L2 construct that
emits the same topology as examples/aws-lambda/template.yaml. Lives on
the ./cdk subpath export so SDK-only consumers don't pull aws-cdk-lib
into their runtime graph (declared as an optional peer dependency).

Tests: 24 new unit tests across the SDK plus 9 CDK synth / contract /
snapshot tests. All 83 tests in packages/aws-lambda/src pass.

* refactor(lambda): /simplify pass on the SDK + CDK PR

Pulls shared logic out so the SDK doesn't re-invent things the handler
and the producer already have:

- `formatExtension` extracted to packages/aws-lambda/src/formatExtension.ts.
  handler.ts and renderToLambda.ts both used identical 12-line copies of
  this switch.
- `PLAN_PROJECT_DIR_SKIP_SEGMENTS` is now exported from
  @hyperframes/producer/distributed. deploySite consumes it instead of
  its own duplicate SKIP_TOP_LEVEL set; the two lists were trivially
  identical and would have drifted silently.
- `FakeS3` + `drainBody` factored out of the two SDK test files into
  src/sdk/__fixtures__/fakeS3.ts. Drops ~110 lines of test-file
  duplication and gives future SDK tests a one-line FakeS3 import.
- S3 URI building in deploySite and renderToLambda routes through the
  existing `formatS3Uri` helper instead of inline `s3://...`
  concatenation; matches the convention already in handler.ts.

Net -133 lines across the touched files. All 83 aws-lambda tests still
pass; all 60 producer distributed tests still pass.

* fix(lambda): bump CDK test timeouts for CI cold-start synth

The bun:test default 5s timeout tripped the first CDK snapshot test
in CI when the cold-start `Template.fromStack(stack)` synth took ~5-8s
on the slowest GitHub Actions runner. Locally on a warm shell the
synth measures <1s, so the failure didn't reproduce until PR #909 hit
CI.

Two changes:

  - Both CDK test files cache one synth in `beforeAll(..., 30000)` and
    reuse the result across every test that uses the default props.
    Each individual test now runs in microseconds (pure assertions
    against the already-synthed template), so the 5s timeout no longer
    applies on the hot path.

  - The two contract tests that exercise non-default props
    (reservedConcurrency, projectName) still synth fresh per-test; they
    get a per-test `it(..., 30000)` timeout.

No behavior changes.

* fix(lambda): address PR review on SDK + CDK construct

Three correctness + ergonomics fixes raised in Vai's review:

  - getRenderProgress over-counted SFN transitions by 3-5×. Step
    Functions Standard Workflows bill per state-entry, not per
    history event. Each Task produces ~5-7 history events
    (Scheduled / Started / Succeeded / TaskStateExited / …);
    counting `events.length` reported the runaway. Switch to
    counting `*StateEntered` events explicitly.

  - assembleComplete + outputFile detection was coupled to the
    Lambda payload's `Action` field. Move both signals onto the
    enclosing state name (`StateExited.name === "Assemble"`), which
    is the state-machine identity rather than the Lambda event
    contract. framesRendered increment moves to the same boundary
    (RenderChunk state).

  - SiteHandle now carries `bucketName` directly so README + CLI
    callers don't have to re-parse `projectS3Uri.split("/")[2]`.

Test updates: getRenderProgress tests wrap renderChunk/assemble
events in matching StateEntered + StateExited pairs so the new
state-name-driven dispatch is exercised end-to-end. SiteHandle
fixture in renderToLambda.test.ts gets the new bucketName field.

All 83 aws-lambda tests still pass.
2026-05-17 03:03:51 -04:00
James 5f8391bd96 chore: release v0.6.15 2026-05-16 23:30:38 +00:00
James Russo 22363a11c9 perf(distributed): parallelize chunk capture across multiple workers (#906)
* perf(distributed): parallelize chunk capture across multiple workers

The distributed `renderChunk` primitive hardcoded `workerCount: 1` and
`captureStage` explicitly forbade `workerCount > 1` when `frameRange` was
set, with the comment:

  "Distributed chunk workers fan out at the activity layer; reduce
   workerCount to 1 when passing frameRange."

The assumption was that orchestration-layer fan-out (Temporal / Lambda /
K8s Jobs / SSH) saturates the available CPU on its own. In practice
adopters that deploy chunks onto multi-core hosts (8-24 vCPU is the
standard producer-worker pod sizing) end up pinning only ~3-4 cores per
chunk while the rest sit idle: chunk-level fan-out at the orchestration
layer gives each pod one chunk at a time, and the chunk render itself
was single-threaded.

Validated against a real 1080p / 30fps / 22-second shader-heavy
composition on a 22-vCPU Temporal pod: each chunk rendered at
165-273ms per frame (vs 94-98ms for the in-process streaming render
which runs `workerCount=2` by default). The slowest chunk gates total
wall-clock under parallel chunk fan-out, so the 2-3x per-frame gap
compounds and `distributed` was net-slower than `in-process` on every
composition smaller than ~5min of texture-class content. Lifting the
restriction is a measured ~2x per-chunk speedup with no contract
change at the framesDir or encoder layer.

Wire-up:

  * `WorkerTask.outputFrameOffset` — optional offset subtracted from the
    absolute frame index when computing the captured file's name.
    Default 0 (the in-process contract; file name == absolute index).
    Distributed chunks set this to the chunk's startFrame so file names
    land 0-indexed within the chunk's range, matching the sequential
    chunk-capture contract and the encoder's expectation that frames
    are read sequentially without an `-start_number` override.

  * `distributeFrames(totalFrames, workerCount, workDir, rangeStart=0)` —
    offsets both `startFrame`/`endFrame` (used for per-frame time math
    on the page's virtual clock) by `rangeStart`, and threads
    `outputFrameOffset = rangeStart` onto each task it emits. With the
    default `rangeStart=0` it is a no-op for in-process renders.

  * `executeWorkerTask` — uses `i - (task.outputFrameOffset ?? 0)` for
    the captured file name, leaving the per-frame TIME computation
    `(i * fps.den) / fps.num` untouched so the page's virtual clock is
    unchanged.

  * `executeDiskCaptureWithAdaptiveRetry({ frameRangeStart? })` — accepts
    the chunk's absolute startFrame and forwards it to `distributeFrames`
    and `buildMissingFrameRetryBatches`. Default `undefined` preserves
    the in-process contract.

  * `buildMissingFrameRetryBatches(ranges, ..., rangeStart=0)` —
    `findMissingFrameRanges` walks LOCAL 0-indexed file names; the retry
    batch translates the local missing-range pair back to ABSOLUTE
    composition indices for `WorkerTask.startFrame/endFrame` and sets
    `outputFrameOffset = rangeStart` so the retried capture writes back
    to the same local file name.

  * `captureStage` — drops the assert; passes
    `frameRangeStart: frameRange?.startFrame` to the parallel branch so
    workers land on absolute composition frame indices for time math
    while file names stay 0-indexed within the chunk range. Docstring
    updated to reflect that the parallel branch is now supported.

  * `renderChunk` — `workerCount: 1` → `workerCount: 2`. The pre-warmed
    `probeSession` is consumed only by the sequential branch; the
    parallel branch closes it during stage entry and creates its own
    worker sessions. Documented as a follow-up: skip probeSession
    creation when `workerCount > 1` to recover the ~3-5s warmup cost.

Backwards compatibility: every change is gated on a parameter that
defaults to the prior behavior. In-process callers (`executeRenderJob`)
pass no `frameRangeStart`, so `rangeStart === 0`, `outputFrameOffset`
defaults to 0, and the file-name math collapses to the prior `i` value.
The framesDir contract (`frame_0..frame_(totalFrames-1)`) and the
WorkerTask interface are extended, not replaced.

Tests: 24 pass / 0 fail across the distributed test suite (renderChunk,
plan, assemble, planFormatBanlist, planSizeCap, publicExports). 7 pass /
0 fail in `parallelCoordinator.test.ts`. The renderOrchestrator suite
has one pre-existing Windows-only failure
(`writeCompiledArtifacts — external assets on Windows drive-letter
paths`) unrelated to this change; the other 56 tests pass.

Refs: distributed-vs-inprocess benchmark thread at
heygen-com/experiment-framework#36950

* perf(distributed): auto-size chunk workerCount via calculateOptimalWorkers

Match the in-process renderer's worker selection instead of hardcoding 2.
`calculateOptimalWorkers(framesInChunk, undefined, cfg)` is the same call
`resolveRenderWorkerCount` makes under the hood, minus the capture-cost
calibration reduction (which would require plumbing the chunk's compiled
metadata through — left as a follow-up).

For a typical 22-vCPU producer-worker pod with `cfg.concurrency: "auto"`
this resolves to ~6 workers for a 240-frame chunk (capped by
`defaultSafeMaxWorkers() = max(6, min(16, floor(cpuCount/8)))`), matching
what `executeRenderJob` (the in-process path) already does. The prior
hardcoded `workerCount: 2` was a safe-minimum starting point that
undersized chunks vs prod's auto behavior.

Tests: 12/12 pass in `renderChunk.test.ts` (unchanged — the test suite
mocks the inner runCaptureStage call so workerCount selection is opaque
to it).

* refactor(distributed): /simplify pass on PR #906

Review pass on the parallel-capture frame-range change. Four targeted
cleanups identified by code-quality and efficiency review agents:

1. Add the missing `frameRange.endFrame - frameRange.startFrame === totalFrames`
   assert. The parallel branch forwards `totalFrames` separately from
   `frameRangeStart`; a caller passing mismatched values would have got a
   silently wrong distribution. The sequential branch already implicitly
   relied on this via its `rangeFrames = rangeEnd - rangeStart` arithmetic.

2. Collapse three near-duplicate docstrings (on `WorkerTask.outputFrameOffset`,
   `executeDiskCaptureWithAdaptiveRetry.frameRangeStart`, and `runCaptureStage`'s
   `frameRange`) so only the WorkerTask field carries the full contract. The
   other two cross-reference it.

3. Drop the WHAT-narrating comments inside `executeWorkerTask`'s per-frame
   loop. The variable names (`fileFrameIdx = i - outputOffset`) already say
   what the line does; the only remaining comment flags the non-obvious
   contract that the streaming callback gets the absolute index.

4. Trim the 30-line `chunkWorkerCount` block in `renderChunk` to one paragraph
   explaining the one non-obvious thing (why we use `calculateOptimalWorkers`
   directly instead of `resolveRenderWorkerCount`). The probeSession-wasted-on-
   parallel acknowledgement stays as a 3-line follow-up flag — investigated
   skipping it in this pass, but the SwiftShader probe is safety-critical and
   has no per-worker equivalent, so deferred to a separate change with proper
   per-worker assertion plumbing.

Tests + format + lint clean:
  * `bun test parallelCoordinator.test.ts` — 7/7
  * `bun test distributed/{renderChunk,plan}.test.ts` — 24/24
  * `bunx oxfmt` + `bunx oxlint` — clean
2026-05-16 16:29:34 -07:00
James Russo c50f59a53b feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe (#878)
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe

Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).

This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.

Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.

The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.

Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.

This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.

* fix(lambda): address PR 878 review feedback

- Verify event.PlanHash against the untarred plan.json at the handler
  boundary before invoking the producer primitive. Throws typed
  PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
  non-retryable; previously the field was schema bloat the handler
  ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
  the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
  but use binary mebibytes); previously mixed units made the 248 MiB
  budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
  comparison instead of lexicographic sort+reverse — the latter would
  silently pick "99.x" over "131.x" once Chrome cached three-digit
  majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
  Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
  imports — file is ESM and the same module is already imported.

* docs(lambda): drop internal plan-doc refs from package README

* ci(windows): fix bun filter UNION bug excluding producer from Windows tests

`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.

Enumerate the packages we DO want to test instead.
2026-05-16 18:08:47 -04:00
Miguel Ángel 1fca35b625 chore: release v0.6.14 2026-05-16 13:02:23 -07:00
James Russo 3569c4830c feat(producer): add Rio-style residual-RMS check to regression harness (#882)
* feat(producer): add Rio-style residual-RMS check to regression harness

The existing audio comparison in the regression harness measures the
Pearson correlation between RMS envelopes of the rendered and snapshot
streams. That catches shape-level drift but is insensitive to level
shifts, phase offsets, or codec-quantization noise — two streams can
correlate >0.9 while differing audibly.

Rio's approach (rio/tests/checksum.py:compare_audio_files_ffmpeg) is
sample-level: subtract the snapshot from the rendered stream, run
`astats`, read the residual Overall RMS in dBFS. Identical streams
cancel to silence (-inf, or sub -90 dBFS for AAC-vs-AAC); anything
>= -50 dBFS is considered drift.

This commit adds the same check as an optional secondary gate:

  - utils/audioRegression.ts: new `computeAudioResidualRmsDb()` that
    spawns ffmpeg with the same filter graph Rio uses (aresample +
    pan + volume=-1 + amix + astats) and returns the parsed Overall
    RMS plus a pass/fail flag.
  - utils/audioRegression.test.ts: 3 new tests covering identical
    streams (-inf result), drifted streams (440Hz vs 880Hz sine),
    and missing-audio-stream input.
  - regression-harness.ts: optional `maxAudioResidualRmsDb` field in
    meta.json. Default is undefined (skip the check) so legacy
    fixtures aren't retroactively gated; new fixtures opt in by
    setting a threshold (e.g. -50). Harness emits `residualRmsDb` in
    the audio_comparison_complete JSON event and the pretty log line.

The existing correlation check stays in place; the new residual check
is independent. They measure complementary properties (shape vs
sample-cancellation) and both should hold for a faithful render.

* fix(producer): harden residual-RMS check (parser, duration guard, error surfacing)

Addresses review feedback on PR #882:

- Stateful astats parse: modern ffmpeg emits `Overall` on its own line
  followed by per-stat lines, so the single-line `Overall RMS level dB:`
  regex never fires on 6.x/7.x/8.x. Find the `Overall` header, take the
  next `RMS level dB:` line. Single-line fallback preserved for 4.x.
- Pre-probe both inputs' audio durations and fail up-front if they differ
  by >5 ms — `amix=duration=shortest` was silently masking trailing
  audio differences.
- Surface ffmpeg/ffprobe spawn errors, signal kills, and non-zero exits
  with a stderr tail. Previously every failure mode collapsed into
  "NaN, fail" with no diagnostic.
- Extend `TestResult.audio` with `residualRmsDb` + `residualError`,
  propagate to `audio-failures.json`.
- Fix `residualSuffix` formatter: NaN (real failure) was being rendered
  as "-inf dBFS" (perfect match). Split the branch on `Number.isNaN`
  separately from `Number.isFinite` and add an explicit error label.
2026-05-16 15:55:03 -04:00
James Russo cad9160cd8 chore(producer): drop internal plan-doc refs from source + docs (#903)
The producer source + docs referenced an internal coordination doc
(DISTRIBUTED-RENDERING-PLAN.md) that doesn't ship in the OSS repo,
leaving broken cross-links for adopters. Drops the references and the
bare section-number shorthand that depended on them; behavioural
content (hash contract, retry semantics, threshold rationale) is
preserved inline where it was previously offloaded to a section number.
2026-05-16 15:54:50 -04:00
Miguel ÁngelandClaude Sonnet 4.6 883260aae3 chore: release v0.6.13
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-16 16:35:46 +00:00
Miguel Ángel 2355d505e1 chore: release v0.6.12 2026-05-16 00:48:57 -07:00
Miguel Ángel 4212a28312 chore: release v0.6.11 2026-05-15 23:43:21 -07:00
Miguel Ángel 1e05d78378 fix(engine): enable browser pool and deduplicate concurrent Chrome launches (#889)
## Summary

- **Enable browser pool by default** (`enableBrowserPool: true`) — parallel capture workers now share a single Chrome process via reference-counted pool instead of each spawning their own (~256MB each). A 6-worker render drops from 7+ browser parent processes to 1 shared pool.
- **Add launch-promise deduplication** in `acquireBrowser` — when multiple workers race into the pool simultaneously (via `Promise.all`), they await the same launch Promise instead of each triggering a separate Chrome spawn. Same pattern as the existing `_autoBrowserGpuModeCache` for GPU probes.
- **Add `connected` health check** on pool hit — if Chrome crashes mid-render, subsequent acquires detect the dead browser and launch fresh instead of returning a stale reference.
- **Add `drainBrowserPool()`** for explicit cleanup between independent render jobs.
- **CLI studio server** now uses the shared pool instead of its own redundant `enableBrowserPool: false` singleton, so thumbnail generation shares Chrome with render workers.

## Problem

The engine had a reference-counted browser pool (`browserManager.ts:73-75`) but it was **disabled by default** (`enableBrowserPool: false`). This meant:

1. **Every parallel worker spawned its own Chrome** — a `--workers 6` render launched 7+ independent Chrome processes (1 probe + 6 workers), each ~256MB.
2. **The pool had a race condition** — even if manually enabled, concurrent workers calling `acquireBrowser()` via `Promise.all` could all see `pooledBrowser === null` before the first launch completed, spawning N Chromes instead of 1.
3. **No crash recovery** — if Chrome died, the pool still held the dead reference. Subsequent acquires got a disconnected browser.
4. **CLI studio server ran its own singleton** — `studioServer.ts` explicitly set `enableBrowserPool: false` and managed a separate browser, so thumbnails and renders could never share.

Over time, orphaned Chrome processes accumulated across renders and previews. We observed **344 headless Chrome processes** consuming **569% CPU and 20% memory** on a dev machine.

## Before / After (6-worker parallel render)

| Metric | Before (pool off) | After (pool on) |
|--------|-------------------|-----------------|
| Browser parent processes | 7+ (1 probe + 6 workers) | **2** (1 GPU probe + 1 shared) |
| Total Chrome processes (with helpers) | 40-50+ | **14** |
| Memory during capture | ~20%+ | **4.6%** |
| Render time (1200 frames, 30fps) | ~64s | **53s** (~17% faster) |
| Post-render orphans | Accumulated over time | **0** |

## Changes

| File | Change |
|------|--------|
| `engine/src/config.ts` | `enableBrowserPool` default `false` → `true` |
| `engine/src/services/browserManager.ts` | Extract `launchBrowser()`, add `_pooledBrowserLaunchPromise` dedup, add `connected` check on pool hit, add `drainBrowserPool()` and `_resetBrowserPoolForTests()` |
| `engine/src/index.ts` | Export `drainBrowserPool` |
| `engine/src/services/browserManager.test.ts` | Pool dedup and drain tests |
| `cli/src/server/studioServer.ts` | Remove `enableBrowserPool: false` override — thumbnails now share the pool |
| `producer/src/services/browserManager.ts` | Re-export `drainBrowserPool` |

## Backward compatibility

- `PRODUCER_ENABLE_BROWSER_POOL=false` env var disables pooling (same as before).
- Callers passing `{ enableBrowserPool: false }` explicitly still get isolated browsers.
- Tests that set `enableBrowserPool: false` in their config fixtures continue to work.

## Test plan

- [x] Engine tests pass (597/597)
- [x] Producer tests pass (406/407, 1 pre-existing flaky test in `pngDecodeBlitWorkerPool`)
- [x] Build passes (lint, format, typecheck all green via lefthook pre-commit)
- [x] Manual render: `shortform-financial` with `--workers 6` → 1200 frames in 53s, 0 orphaned Chrome processes after completion
- [x] Process monitoring during render confirmed 2 browser parents (1 GPU probe + 1 shared pool) instead of 7+
2026-05-16 08:17:29 +02:00
Miguel Ángel d3c32a0b4d chore: release v0.6.10 2026-05-15 21:25:26 -07:00
Miguel Ángel 82c9b6b5ea chore: release v0.6.9 2026-05-15 19:45:01 -07:00
Miguel Ángel 4b501762e4 chore: release v0.6.8 2026-05-15 19:20:28 -07:00
f84cc492de perf(engine): faster shader transitions via page-side WebGL compositing (#832)
* fix(cli): prefer puppeteer cache + numeric version sort (staff review)

Two correctness fixes from PR #821 self-review:

1. Cache priority order. Previous order was hyperframes-managed cache →
   puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which
   lags 17+ releases behind upstream; if a user separately installed a
   newer chrome-headless-shell via @puppeteer/browsers install, the CLI
   would silently hand engine the older HF-cache binary while engine's
   own resolveHeadlessShellPath would have picked the newer one. Flip
   the priority so puppeteer cache wins, matching engine semantics.

2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()`
   over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would
   return `linux-99...` first because character '9' outranks '1'. Parse
   each name into integer segments and compare them numerically.

Tests: add both-caches-populated and linux-148-beats-linux-99 cases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(engine): page-side compositing for shader transitions (opt-in spike)

Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine
config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`.
When set, SDR shader-transition compositions skip the Node-side layered blend
(the hf#677 chain) and instead run the shader inside Chrome via a page-side
WebGL canvas; the engine then captures ONE opaque RGB frame per output frame
via the existing streaming capture path.

This is the strongest non-beginFrame perf lever for Mac users, who cannot
take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275).
Stacks on top of the hf#677 1.95× baseline.

Default OFF — existing fixture pins (byte-exact MP4 output) are preserved.
Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node
is f64). HDR content forces the existing layered path regardless.

Implementation:
- engine: new `EngineConfig.enablePageSideCompositing` (default false).
- producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script
  injected into the served HTML head when the flag is on.
- producer/renderOrchestrator: when the flag + no HDR + no png-sequence,
  route SDR transitions through the streaming path instead of the layered
  HDR stage.
- shader-transitions: new `engineModePageComposite.ts` installs a fullscreen
  WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside
  a transition window captures both scenes via the Chromium
  `drawElementImage` API to GL textures, runs the fragment shader, and
  displays the composited result on the overlay canvas. The engine takes
  one screenshot per frame and sees the composited overlay.
- cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true`
  before producer load.
- scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a
  representative fixture with and without the flag, validates the canary
  strings are in the shipped bundles, and writes a wall-time pair.

Determinism trade documented in the engine config doc-comment. The smoke
script enforces the bundled-CLI validation discipline from prior perf work
(see internal feedback note `validate_bundled_cli_not_dev_path`).

Runtime requirement: Chromium's `CanvasDrawElement` feature (already
enabled by the engine's `--enable-features=CanvasDrawElement` launch flag).
When the runtime feature is unavailable, the page-side installer logs a
warning and falls back to opacity-flip mode — the engine still takes the
streaming path; the transition window degrades to a hard scene swap. Vance
will validate on Mac Chrome where the feature is supported.

Co-Authored-By: Vai <vai@heygen.com>

* fix(shader-transitions): use html2canvas for page-side compositor capture

The original drawElementImage approach fails in engine render mode because
the virtual-time shim prevents Chromium from generating paint records for
cloned elements. drawElementImage requires a cached paint record from the
browser's compositor — clones created at capture time never receive one
because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original
rAFs don't produce real paints under virtual-time control, and
(c) layoutsubtree canvases don't apply CSS stylesheet rules to children.

Switch scene capture to html2canvas (foreignObjectRendering: false), the
same JS-based renderer already used by the preview-mode fallback path in
capture.ts. html2canvas reads computed styles and renders via its own
canvas drawing pipeline with no dependency on the browser paint cycle.

Also fixes:
- Engine seek must return the result so Puppeteer awaits async seek
  promises (frameCapture.ts).
- GSAP opacity cache: compositor must restore scene opacity before seek,
  not after — GSAP caches inline values and skips re-writes.
- Support check gates on WebGL availability, not drawElementImage.

Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps)
  Baseline (Node-side layered): 137s
  Page-side (html2canvas+WebGL): 33s → 4.1× speedup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shader-transitions): simplify review fixes for page-side compositor

- Use uploadTexture (zeroes canvas backing store after upload) to prevent
  ~2.2GB transient memory pressure across 280 html2canvas calls per render
- Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call,
  matching the preview-path capture.ts behavior
- Parallelize from/to scene captures with Promise.all
- Wrap post-capture render in try/finally so opacity is always restored
- Fix WebGL context leak in isPageSideCompositingSupported probe
- Remove dead ResolvedTransition.index field
- Export stabilizeTransformedBoxShadows from capture.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): unify page-side compositing gating and Docker forwarding

Addresses three issues from staff review:

1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3,
   p5.js) — narrowed to data-no-capture only since the compositor canvas
   is a body sibling never in the scene subtree.

2. Docker mode silently dropped --page-side-compositing — thread
   pageSideCompositing through DockerRenderOptions/buildDockerRunArgs
   with regression tests.

3. Fragmented gating across 4 independent sites could disagree:
   - Stub injection gated only on cfg flag (leaked into HDR/alpha)
   - Probe-created fileServer never got the stub
   - needsAlpha (WebM/MOV) not excluded from the gate
   - WebGL-unavailable fallback claimed layered path would run but
     orchestrator had already disabled it

   Fix: compute stub injection at the same site as the layered-bypass
   decision (after hasHdrContent is known), using addPreHeadScript on
   the already-running fileServer. Single predicate now gates both
   decisions, including !needsAlpha.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): two-phase drawElementImage capture for page-side compositing

Replace html2canvas with native drawElementImage for scene capture in
the page-side compositor. drawElementImage reads from the browser's own
paint cache, giving pixel-identical output to the preview path.

The blocker was that cloned elements inside layoutsubtree canvases have
no cached paint record under virtual time — the compositor only paints
when explicitly triggered. Fix: split the seek+composite into two phases
with an engine-forced paint between them.

Phase 1 (seek wrapper, page-side):
  - GSAP seek positions the timeline
  - Clone FROM/TO scenes into visible layoutsubtree staging canvases
  - Set window.__hf_page_composite_pending flag

Engine paint force (frameCapture.ts):
  - Detect pending flag after seek returns
  - Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the
    browser compositor to paint all visible elements including staging
    canvas children

Phase 2 (page.evaluate, page-side):
  - drawElementImage reads the now-valid paint records
  - Upload textures to WebGL, run shader, show GL overlay

Key insight: staging canvases must be visible (not opacity:0) for the
browser to paint their children. They sit at z-index:-9998, behind
the main DOM and covered by the GL overlay during transitions.

Perf: 15-scene fixture (28s, 14 transitions, 30fps):
  Baseline (Node-side layered): 137s
  html2canvas + WebGL:           33s (3.7×)
  drawElementImage + WebGL:      21s (6.6×)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(engine): optimize two-phase compositor hot path

- uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of
  canvas buffer alloc/dealloc churn (persistent staging canvases don't
  need the one-shot zeroing behavior)
- Fold hasPending check into seek page.evaluate: eliminates one CDP
  round-trip per frame (~700 unnecessary IPC calls on non-transition
  frames)
- Fix renderShader error handling: on failure, leave source scenes
  visible as fallback instead of hiding both scenes + GL overlay
  (which produced black frames)
- Move mutable state declarations above resolveComposite to prevent
  TDZ risk on refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): staff review — staging cleanup, pending flag, beginFrame guard

- Clear staging canvas children when leaving transition window (prevents
  visible clone bleed-through on transparent compositions)
- Clear __hf_page_composite_pending on all resolveComposite exit paths
- Guard micro-screenshot paint force against beginFrame mode (CDP
  Page.captureScreenshot conflicts with beginFrame compositor control)
- Update CLI flag description: document video/canvas limitation, remove
  stale PSNR claim

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): default-on page-side compositing for SDR shader transitions

Page-side compositing is now enabled by default for SDR shader-transition
renders without video content. The 6.6× speedup applies automatically —
no flag needed.

Auto-disables when:
- HDR content detected
- Alpha output (WebM/MOV/PNG-sequence)
- Composition contains <video> elements (cloneNode loses playback state)
- beginFrame capture mode (Linux headless)

Use --no-page-side-compositing to force the Node-side layered path.

Changes:
- Engine config: enablePageSideCompositing defaults to true
- CLI: flag default flipped to true; --no-page-side-compositing disables
- Orchestrator: added composition.videos.length === 0 gate
- Docker: forwards --no-page-side-compositing when explicitly disabled
- Config tests updated for new default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): support video elements on page-side compositing fast path

Three-phase capture protocol lets shader transitions render video scenes
without falling back to the slow Node-side layered pipeline:

1. Seek → compositor records transition metadata, sets pending flag
2. onBeforeCapture → video frame injector updates <img> replacements
3. prepare → cloneNode picks up current video frames, img.decode() awaits
4. micro-screenshot → forces browser to paint cloned elements
5. resolve → drawElementImage reads paint records, shader composites

Key changes:
- Remove `composition.videos.length === 0` gate from orchestrator
- Split compositor resolve into prepare (clone) + resolve (shader)
- Move onBeforeCapture before compositor prepare in frameCapture.ts
- Await img.decode() on cloned data-URI images to prevent stale frames
- Stop manipulating scene opacity in compositor (GL canvas overlay suffices)
- Add gsap.set declaration for shader-transitions ambient types
- Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end

Performance: compositions with video now render at 7.5s (6 workers) instead
of 2m38s on the layered path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs

The timing compiler now injects data-start="0" on <video> and <audio>
elements that lack it. This makes discoverMediaFromBrowser() find the
element (it queries video[data-start]), so the frame extraction pipeline
activates automatically. Videos "just work" without requiring authors to
add data-start, data-end, or id attributes.

Also removes the video_missing_timing_attrs lint rule — the compiler
handles the missing attributes automatically, so the lint rule would
only false-positive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(core): add data-hf-auto-start sentinel on auto-injected video timing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery

Seeks the GSAP timeline in Puppeteer to discover when each video's parent
scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed
by binary search refinement to frame-level precision (1/60s). Only processes
videos with the data-hf-auto-start sentinel so author-specified timing is
never overridden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(producer): integrate runtime video visibility discovery into probe stage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(producer): trigger browser probe for auto-start videos, remove debug logging

The probe stage was skipping browser launch when composition duration was
already known, which meant discoverVideoVisibilityFromTimeline never ran.
Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scripts): use mkdtempSync for smoke test work directory

Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory
via mkdtempSync to resolve CodeQL "insecure temporary file" alert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format smoke test script

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Vai <vai@heygen.com>
2026-05-15 16:23:12 -07:00
Miguel Ángel 9b23ccf665 feat(studio): html-backed motion panel (#873)
## Summary

Re-architects the studio motion panel to persist GSAP motion data directly in HTML element attributes instead of a `.hyperframes/studio-motion.json` JSON sidecar file. Same pattern as position/resize/rotation edits.

### Before
```
MotionPanel → commitStudioMotionManifestOptimistically()
  → writes .hyperframes/studio-motion.json
  → applyStudioMotionManifest(doc, manifest)
```

### After
```html
<div id="hero" data-hf-studio-motion='{"start":0.5,"duration":1,"ease":"power3.out","from":{"opacity":0,"y":40},"to":{"opacity":1,"y":0}}'>
```
```
MotionPanel → writeStudioMotionToElement(element, motion)
  → buildMotionPatches(element)
  → commitPositionPatchToHtml(selection, patches)
```

## What changed

- **studioMotionOps.ts** — Added `readStudioMotionFromElement()`, `writeStudioMotionToElement()`, `clearStudioMotionFromElement()` for attribute-based CRUD
- **studioMotion.ts** — Added `applyStudioMotionFromDom()` that reads motion from DOM attributes and builds GSAP timeline (kept `applyStudioMotionManifest` for render script compat)
- **manualEditsDom.ts** — Added `buildMotionPatches()` / `buildClearMotionPatches()`, integrated motion into `reapplyPositionEditsAfterSeek()`
- **useDomEditCommits.ts** — Rewrote `handleDomMotionCommit` / `handleDomMotionClear` to use HTML patching instead of manifest persistence
- **useManifestPersistence.ts** — Removed all motion manifest state (~200 lines): `studioMotionManifestRef`, `commitStudioMotionManifestOptimistically`, `applyStudioMotionToPreview`, motion SSE handler
- **App.tsx** — Reads motion from element attribute (`readStudioMotionFromElement`) instead of manifest ref
- **manualEditsRenderScript.ts** — Extended `studioPositionSeekReapplyRuntime` to rebuild GSAP motion timeline from `data-hf-studio-motion` attributes after each seek, including CustomEase support
- **htmlCompiler.ts** — Trigger seek-reapply script injection on `data-hf-studio-motion=` attributes

## Benefits

- No sidecar file — motion survives git, copy-paste, and manual HTML editing
- Undo/redo works via HTML source history (same as position edits)
- Renders correctly via CLI — seek-reapply script handles motion timeline rebuild
- Simpler architecture — one persistence path for all studio edits

## Test plan

- [x] `bun run build` passes
- [x] Pre-commit hooks pass (lint, format, typecheck)
- [ ] Set motion on element in Studio → `data-hf-studio-motion` attribute appears in HTML source
- [ ] Reload page → motion persists and plays correctly
- [ ] Clear motion → attribute removed, element returns to original state
- [ ] Undo/redo motion changes
- [ ] Render via CLI → motion visible in rendered video
- [ ] Seek animation → motion timeline re-syncs correctly
2026-05-15 21:45:57 +02:00
Miguel ÁngelandClaude Opus 4.6 8e0cfc33a7 fix(engine): preserve video frame replacement geometry (#838)
* fix(engine): preserve video frame replacement geometry

* test(producer): cover video overlay stretch regression

* fix(engine): always pass clip to Page.captureScreenshot

Without an explicit clip, Chrome can resolve replaced-element sizing
differently at dpr=1 when full-bleed absolute videos interact with
overlay layers — producing anisotropic frame stretching on some
compositor paths. Always passing clip with scale=dpr (including 1)
ensures geometry is locked to the measured viewport dimensions.

Credit: brian-t-allen (#837)

* test(producer): regenerate style-9-prod baseline for always-clip capture path

The always-clip change in screenshotService.ts routes Chrome through a
different compositor capture path at dpr=1, producing different video
frame compression artifacts. Regenerated inside Dockerfile.test to match
CI environment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-15 18:58:50 +02:00