mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 00:56:23 +00:00
v0.6.21
181
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
4bf2fa5cd1 |
fix: revert flattenInnerRoot, use host-level authored-id + count-based rebind
Three changes to fix regression failures without breaking baselines: 1. Revert flattenInnerRoot in producer — use the original innerHTML inlining that preserves the existing DOM structure. Instead, set data-hf-authored-id on the HOST element so the scoped proxy can still rewrite #id selectors for sub-composition scripts. 2. Revert compiled.html baselines to main (no DOM structure changes). 3. Use timeline-count comparison instead of poll duration to decide whether to call __hfForceTimelineRebind. Compare timeline count before vs after the poll — rebind only when new timelines appeared during polling. This correctly identifies async compositions regardless of fetch speed, while leaving sync compositions untouched. |
||
|
Miguel Ángel
|
0f1c64dcae |
fix: address review feedback — observability, dedup, query-strip, catalog
Review items addressed: 1. Mirror video-failure warning in beginFrame path (was screenshot-only) 2. Fix resolveProjectRelativeSrc escape-fallback to use query-stripped cleanSrc instead of raw src for the normalize/strip arm 3. Export prepareFlattenedInnerRoot from @hyperframes/core/compiler and consume in the producer instead of duplicating the implementation 4. Use typed Window cast instead of (window as any) for __hfForceTimelineRebind 5. Regenerate docs/public/catalog-index.json with all 6 map blocks 6. Restore Maps nav group in docs.json (catalog generator had merged them into Data) |
||
|
Miguel Ángel
|
5aff328a9b |
fix(producer): preserve inner root element when inlining sub-compositions
The producer's inlineSubCompositions was not passing flattenInnerRoot, causing sub-composition inner root elements to be unwrapped during compilation. Scripts using #id selectors (rewritten to [data-hf-authored-id] by the scoping proxy) could not find their DOM root because the authored-id attribute was never set. The core bundler (bundleToSingleHtml) already passed flattenInnerRoot correctly. This aligns the producer's render compilation with the same behavior: clone the inner root, strip timing/composition attrs, replace id with data-hf-authored-id, and mark with data-hf-inner-root. |
||
|
Miguel Ángel
|
5beeb4227e |
Merge pull request #927 from heygen-com/fix/caption-polish
fix(registry): polish caption timing and add highlight style |
||
|
Miguel Ángel
|
c8e8fdcf6d |
fix(producer): fetch missing font weights from Google Fonts at render time
The deterministic font system now supplements its embedded font bundle with Google Fonts fetches for any weights not in the pre-bundled set. This fixes compositions that request font weights (e.g. Montserrat 300) not included in the CANONICAL_FONTS faces array — previously those weights were silently dropped, causing invisible text in renders. Also replaces caption-glitch-rgb and caption-weight-shift with improved versions from avatar preview compositions, adapted to 1920x1080 with standard demo transcript. |
||
|
James
|
187dfd5b6c |
fix(producer): indirect lambda-local import path so tsc can't resolve it
The tsconfig `exclude` list isn't enough to keep producer's tsc emit pass
from pulling `regression-harness-lambda-local.ts` (and its
`@hyperframes/aws-lambda` static imports) into the program — tsc still
statically resolves the path in
`await import("./regression-harness-lambda-local.js")` from
`regression-harness.ts`, walks into the excluded file, and fails on
the missing aws-lambda type declarations.
Reproduction (clean workspace, no aws-lambda dist yet, mirrors CI):
rm -rf packages/{aws-lambda,producer,core}/dist
bun run build
# @hyperframes/producer build: src/regression-harness-lambda-local.ts(36,70):
# error TS2307: Cannot find module '@hyperframes/aws-lambda' or its
# corresponding type declarations.
Fix: route the dynamic import path through a top-level string constant
so tsc can't statically resolve the target. tsc keeps the type-only
imports (`RunLambdaLocalRender` from the no-aws-lambda types file) and
treats the dynamic-import target as opaque. `tsx` resolves the path
normally at runtime, so `--mode=lambda-local` is unchanged.
|
||
|
James
|
a18fc04bf4 |
fix(producer): break aws-lambda↔producer build cycle for lambda-local harness
The aws-lambda publish-readiness changes earlier in this PR moved
aws-lambda's types from `./src/index.ts` → `./dist/index.d.ts`. That
flipped producer's emit pass from "always works" to "needs aws-lambda
built first," because producer's `regression-harness-lambda-local.ts`
imports `@hyperframes/aws-lambda{,/handler}`. aws-lambda's own emit
pass in turn imports `@hyperframes/producer{,/distributed}` → circular
build dependency, so neither side can emit declarations in a single
pass. CI's first run on this PR failed Build, Typecheck, CLI smoke,
windows tests, and every perf job for this reason.
Fix: extract the public surface of `regression-harness-lambda-local.ts`
into a new types-only file that has no aws-lambda imports, point
`regression-harness.ts` at that types file, and exclude
`regression-harness-lambda-local.ts` from producer's tsconfig
`include`. The implementation file is still loaded at runtime via
`tsx` (lambda-local mode runs the harness through tsx, not from
producer's dist/), so the runtime contract is unchanged — producer's
tsc just no longer has to type-check it.
- `regression-harness-lambda-local-types.ts` (new): exports
`RunLambdaLocalInput` + `RunLambdaLocalRender` with zero
aws-lambda imports.
- `regression-harness-lambda-local.ts`: re-exports
`RunLambdaLocalInput` from the new types file (so the public
name stays stable for any future direct imports).
- `regression-harness.ts`: drops the `typeof import("...")` trick
and uses the explicit `RunLambdaLocalRender` signature for the
dynamically-loaded function.
- `tsconfig.json`: excludes `src/regression-harness-lambda-local.ts`
so producer's emit pass never resolves `@hyperframes/aws-lambda`.
Verified:
bun run build # full root build green
bun run verify:packed-manifests # all packages publish-safe
|
||
|
Miguel Ángel
|
7a1e8769a5 |
Merge pull request #918 from heygen-com/refactor/unify-subcomp-inlining
fix: hold external sub-compositions in render mode + regenerate baseline |
||
|
James Russo
|
b05a22e69e |
feat(producer): add --mode=lambda-local to the regression harness (#913)
* feat(producer): add --mode=lambda-local to the regression harness
Third harness mode that drives the OSS @hyperframes/aws-lambda handler
through the exact event sequence Step Functions produces in
production:
handler({Action: "plan"}) → planDir tarball on fake S3
handler({Action: "renderChunk"}) × N → chunk artifacts on fake S3
handler({Action: "assemble"}) → final mp4/mov/png-sequence
The S3 client is a filesystem-backed fake (every s3://<bucket>/<key>
URI maps to <tempRoot>/s3/<key>), so the harness exercises the
handler's event-parsing + tar/S3 conventions + dispatch logic on top
of the underlying producer primitives. Regressions in event JSON
shape, S3 key layout, or plan-hash boundary checks now surface in
the same CI run as the in-process and distributed-simulated modes
without paying for a real AWS round-trip.
Deliberately NOT a Docker/RIE invocation — that would gate the
producer test suite on Docker-in-Docker support which most CI
runners lack. Real-ZIP-via-RIE tests live in
packages/aws-lambda/scripts/ (probe:beginframe) and the
maintainer-run smoke.sh.
Wired up via:
- HarnessMode union extended to include "lambda-local"
- parseHarnessModeFlag accepts --mode=lambda-local
- regression-harness.ts dispatches to runLambdaLocalRender for
the new mode, sharing the distributed-support gate +
pathology-floor threshold with distributed-simulated mode
- package.json scripts: test:lambda-local + docker:test:lambda-local
- producer.devDependencies += @hyperframes/aws-lambda (workspace)
- producer/tsconfig.json gains path mappings to self so the type
cycle through aws-lambda's source resolves at typecheck time
without needing producer to be pre-built
Tests: 3 new unit tests on parseHarnessModeFlag + resolveMinPsnrForMode
cover the new mode. End-to-end PSNR contract still runs through
Dockerfile.test (manual + CI).
* refactor(producer): /simplify pass on lambda-local harness imports
Three small cleanups on top of the lambda-local harness:
- Drop the unused createReadStream import + its `void` workaround
comment. The aws-lambda handler's tar / S3 transport pulls
createReadStream from its own imports; this file never references
it directly.
- Hoist the dynamic `await import("node:fs")` calls for
writeFileSync out of FilesystemBackedFakeS3.send into the static
import block. Repeated PutObject calls don't need to repay the
dynamic-import cost.
- Hoist the dynamic `await import("@hyperframes/aws-lambda")` call
for untarDirectory similarly. Drops the now-redundant duplicate
aws-lambda import statement.
The PutObject body branch also collapses: `body instanceof Buffer`
and `typeof body === "string"` both call writeFileSync identically,
so they share one branch.
No behavior changes.
* fix(producer): lazy-import lambda-local harness module
The static import of regression-harness-lambda-local.ts pulled
@hyperframes/aws-lambda (and its @aws-sdk/* + @sparticuz/chromium
transitive deps) at module-load time. Dockerfile.test only copies
the producer's own files into the container, so aws-lambda's src
isn't present at runtime — and even `--mode=in-process` failed:
Error [ERR_MODULE_NOT_FOUND]: Cannot find module
'/app/packages/producer/node_modules/@hyperframes/aws-lambda/src/index.ts'
imported from /app/packages/producer/src/regression-harness-lambda-local.ts
Load the module on demand instead. `--mode=lambda-local` callers
pay the import cost; the existing in-process and distributed-
simulated modes don't.
* fix(producer): address PR review on lambda-local harness
Three review items from Vai:
- `Config.width`/`Config.height` are now plumbed through
RunLambdaLocalInput rather than hardcoded inside
runLambdaLocalRender. Lambda-local's whole point is to catch
event-shape drift; if the handler ever starts honouring
Config.width/height (e.g. for canvas sizing), having those
values flow from the caller means the harness sees what the
fixture authored. The interface change makes the eventual
upgrade-to-real-fixture-resolution a one-line dispatch swap.
- Drop the dead `export type { Fps }` and its unused import
from @hyperframes/core. The module never re-exports it.
- The dispatch site in regression-harness.ts now passes 1920×1080
explicitly with a comment marking it as a placeholder until
the harness compiles the composition HTML up-front to surface
the authored data-width/data-height. distributed-simulated
mode uses the same placeholder internally, kept for parity.
No behavior change in the existing modes; lambda-local now has a
clear extension point for honouring fixture dimensions.
|
||
|
Miguel ÁngelandClaude Sonnet 4.6
|
581e7a7eda |
refactor: extract shared inlineSubCompositions from bundler and producer
Both the core bundler (htmlBundler.ts) and the producer (htmlCompiler.ts) had parallel ~200-line implementations of sub-composition inlining. This divergence caused bug #911 (producer didn't set data-composition-file). Extract the shared logic into core/compiler/inlineSubCompositions.ts: - Single function handles: template/body extraction, CSS/script scoping, asset path rewriting, data-composition-file attribution, content injection - Callers provide environment-specific callbacks (HTML resolution, parsing, variable handling, inner root flattening) - Core bundler passes its advanced features (runtime IDs, variables, inline style rewriting, inner root flattening) - Producer passes a simpler resolver (map + filesystem fallback) and adds pixel sizing post-hoc Net: -215 lines, one source of truth for sub-comp inlining. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Miguel ÁngelandClaude Sonnet 4.6
|
2b46565c65 |
fix(runtime): hold external sub-compositions in render mode
PR #917 fixed visibility clamping for external sub-compositions in preview mode by checking data-composition-src. However, the producer's htmlCompiler strips that attribute during inlining without setting the data-composition-file marker that the core bundler sets. This caused the runtime to still clamp duration to Math.min(authored, live) in rendered output. Two fixes: - Runtime: also check data-composition-file (set by the core bundler after inlining) - Producer: set data-composition-file before removing data-composition-src, matching the core bundler's behavior Closes #911 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
James Russo
|
f01fccb0ea | perf(distributed): skip eager probe session when chunkWorkerCount > 1 (#916) | ||
|
James Russo
|
34d1f0e1d0 |
feat(lambda): add TypeScript SDK and CDK construct (#909)
* feat(lambda): add TypeScript SDK and CDK construct
Adds the client-side surface on top of the Phase 6a Lambda handler so
adopters can drive a deployed stack from Node without writing AWS-SDK
boilerplate:
- renderToLambda(opts) starts a Step Functions execution and returns a
handle. Does NOT poll.
- getRenderProgress({ executionArn }) returns a snapshot of progress,
frames rendered, cost (Lambda GB-seconds + SFN transitions), errors,
and the final output object once Assemble completes.
- deploySite({ projectDir, bucketName }) content-addresses the project
tree, tar.gzs it, and uploads to S3 with a HeadObject short-circuit so
re-renders of the same tree skip the tar+PUT.
- validateDistributedRenderConfig throws a typed InvalidConfigError
before StartExecution, so shape errors surface synchronously.
- computeRenderCost is exposed for callers who want to format cost out
of band.
Also ships HyperframesRenderStack, an aws-cdk-lib L2 construct that
emits the same topology as examples/aws-lambda/template.yaml. Lives on
the ./cdk subpath export so SDK-only consumers don't pull aws-cdk-lib
into their runtime graph (declared as an optional peer dependency).
Tests: 24 new unit tests across the SDK plus 9 CDK synth / contract /
snapshot tests. All 83 tests in packages/aws-lambda/src pass.
* refactor(lambda): /simplify pass on the SDK + CDK PR
Pulls shared logic out so the SDK doesn't re-invent things the handler
and the producer already have:
- `formatExtension` extracted to packages/aws-lambda/src/formatExtension.ts.
handler.ts and renderToLambda.ts both used identical 12-line copies of
this switch.
- `PLAN_PROJECT_DIR_SKIP_SEGMENTS` is now exported from
@hyperframes/producer/distributed. deploySite consumes it instead of
its own duplicate SKIP_TOP_LEVEL set; the two lists were trivially
identical and would have drifted silently.
- `FakeS3` + `drainBody` factored out of the two SDK test files into
src/sdk/__fixtures__/fakeS3.ts. Drops ~110 lines of test-file
duplication and gives future SDK tests a one-line FakeS3 import.
- S3 URI building in deploySite and renderToLambda routes through the
existing `formatS3Uri` helper instead of inline `s3://...`
concatenation; matches the convention already in handler.ts.
Net -133 lines across the touched files. All 83 aws-lambda tests still
pass; all 60 producer distributed tests still pass.
* fix(lambda): bump CDK test timeouts for CI cold-start synth
The bun:test default 5s timeout tripped the first CDK snapshot test
in CI when the cold-start `Template.fromStack(stack)` synth took ~5-8s
on the slowest GitHub Actions runner. Locally on a warm shell the
synth measures <1s, so the failure didn't reproduce until PR #909 hit
CI.
Two changes:
- Both CDK test files cache one synth in `beforeAll(..., 30000)` and
reuse the result across every test that uses the default props.
Each individual test now runs in microseconds (pure assertions
against the already-synthed template), so the 5s timeout no longer
applies on the hot path.
- The two contract tests that exercise non-default props
(reservedConcurrency, projectName) still synth fresh per-test; they
get a per-test `it(..., 30000)` timeout.
No behavior changes.
* fix(lambda): address PR review on SDK + CDK construct
Three correctness + ergonomics fixes raised in Vai's review:
- getRenderProgress over-counted SFN transitions by 3-5×. Step
Functions Standard Workflows bill per state-entry, not per
history event. Each Task produces ~5-7 history events
(Scheduled / Started / Succeeded / TaskStateExited / …);
counting `events.length` reported the runaway. Switch to
counting `*StateEntered` events explicitly.
- assembleComplete + outputFile detection was coupled to the
Lambda payload's `Action` field. Move both signals onto the
enclosing state name (`StateExited.name === "Assemble"`), which
is the state-machine identity rather than the Lambda event
contract. framesRendered increment moves to the same boundary
(RenderChunk state).
- SiteHandle now carries `bucketName` directly so README + CLI
callers don't have to re-parse `projectS3Uri.split("/")[2]`.
Test updates: getRenderProgress tests wrap renderChunk/assemble
events in matching StateEntered + StateExited pairs so the new
state-name-driven dispatch is exercised end-to-end. SiteHandle
fixture in renderToLambda.test.ts gets the new bucketName field.
All 83 aws-lambda tests still pass.
|
||
|
James Russo
|
22363a11c9 |
perf(distributed): parallelize chunk capture across multiple workers (#906)
* perf(distributed): parallelize chunk capture across multiple workers
The distributed `renderChunk` primitive hardcoded `workerCount: 1` and
`captureStage` explicitly forbade `workerCount > 1` when `frameRange` was
set, with the comment:
"Distributed chunk workers fan out at the activity layer; reduce
workerCount to 1 when passing frameRange."
The assumption was that orchestration-layer fan-out (Temporal / Lambda /
K8s Jobs / SSH) saturates the available CPU on its own. In practice
adopters that deploy chunks onto multi-core hosts (8-24 vCPU is the
standard producer-worker pod sizing) end up pinning only ~3-4 cores per
chunk while the rest sit idle: chunk-level fan-out at the orchestration
layer gives each pod one chunk at a time, and the chunk render itself
was single-threaded.
Validated against a real 1080p / 30fps / 22-second shader-heavy
composition on a 22-vCPU Temporal pod: each chunk rendered at
165-273ms per frame (vs 94-98ms for the in-process streaming render
which runs `workerCount=2` by default). The slowest chunk gates total
wall-clock under parallel chunk fan-out, so the 2-3x per-frame gap
compounds and `distributed` was net-slower than `in-process` on every
composition smaller than ~5min of texture-class content. Lifting the
restriction is a measured ~2x per-chunk speedup with no contract
change at the framesDir or encoder layer.
Wire-up:
* `WorkerTask.outputFrameOffset` — optional offset subtracted from the
absolute frame index when computing the captured file's name.
Default 0 (the in-process contract; file name == absolute index).
Distributed chunks set this to the chunk's startFrame so file names
land 0-indexed within the chunk's range, matching the sequential
chunk-capture contract and the encoder's expectation that frames
are read sequentially without an `-start_number` override.
* `distributeFrames(totalFrames, workerCount, workDir, rangeStart=0)` —
offsets both `startFrame`/`endFrame` (used for per-frame time math
on the page's virtual clock) by `rangeStart`, and threads
`outputFrameOffset = rangeStart` onto each task it emits. With the
default `rangeStart=0` it is a no-op for in-process renders.
* `executeWorkerTask` — uses `i - (task.outputFrameOffset ?? 0)` for
the captured file name, leaving the per-frame TIME computation
`(i * fps.den) / fps.num` untouched so the page's virtual clock is
unchanged.
* `executeDiskCaptureWithAdaptiveRetry({ frameRangeStart? })` — accepts
the chunk's absolute startFrame and forwards it to `distributeFrames`
and `buildMissingFrameRetryBatches`. Default `undefined` preserves
the in-process contract.
* `buildMissingFrameRetryBatches(ranges, ..., rangeStart=0)` —
`findMissingFrameRanges` walks LOCAL 0-indexed file names; the retry
batch translates the local missing-range pair back to ABSOLUTE
composition indices for `WorkerTask.startFrame/endFrame` and sets
`outputFrameOffset = rangeStart` so the retried capture writes back
to the same local file name.
* `captureStage` — drops the assert; passes
`frameRangeStart: frameRange?.startFrame` to the parallel branch so
workers land on absolute composition frame indices for time math
while file names stay 0-indexed within the chunk range. Docstring
updated to reflect that the parallel branch is now supported.
* `renderChunk` — `workerCount: 1` → `workerCount: 2`. The pre-warmed
`probeSession` is consumed only by the sequential branch; the
parallel branch closes it during stage entry and creates its own
worker sessions. Documented as a follow-up: skip probeSession
creation when `workerCount > 1` to recover the ~3-5s warmup cost.
Backwards compatibility: every change is gated on a parameter that
defaults to the prior behavior. In-process callers (`executeRenderJob`)
pass no `frameRangeStart`, so `rangeStart === 0`, `outputFrameOffset`
defaults to 0, and the file-name math collapses to the prior `i` value.
The framesDir contract (`frame_0..frame_(totalFrames-1)`) and the
WorkerTask interface are extended, not replaced.
Tests: 24 pass / 0 fail across the distributed test suite (renderChunk,
plan, assemble, planFormatBanlist, planSizeCap, publicExports). 7 pass /
0 fail in `parallelCoordinator.test.ts`. The renderOrchestrator suite
has one pre-existing Windows-only failure
(`writeCompiledArtifacts — external assets on Windows drive-letter
paths`) unrelated to this change; the other 56 tests pass.
Refs: distributed-vs-inprocess benchmark thread at
heygen-com/experiment-framework#36950
* perf(distributed): auto-size chunk workerCount via calculateOptimalWorkers
Match the in-process renderer's worker selection instead of hardcoding 2.
`calculateOptimalWorkers(framesInChunk, undefined, cfg)` is the same call
`resolveRenderWorkerCount` makes under the hood, minus the capture-cost
calibration reduction (which would require plumbing the chunk's compiled
metadata through — left as a follow-up).
For a typical 22-vCPU producer-worker pod with `cfg.concurrency: "auto"`
this resolves to ~6 workers for a 240-frame chunk (capped by
`defaultSafeMaxWorkers() = max(6, min(16, floor(cpuCount/8)))`), matching
what `executeRenderJob` (the in-process path) already does. The prior
hardcoded `workerCount: 2` was a safe-minimum starting point that
undersized chunks vs prod's auto behavior.
Tests: 12/12 pass in `renderChunk.test.ts` (unchanged — the test suite
mocks the inner runCaptureStage call so workerCount selection is opaque
to it).
* refactor(distributed): /simplify pass on PR #906
Review pass on the parallel-capture frame-range change. Four targeted
cleanups identified by code-quality and efficiency review agents:
1. Add the missing `frameRange.endFrame - frameRange.startFrame === totalFrames`
assert. The parallel branch forwards `totalFrames` separately from
`frameRangeStart`; a caller passing mismatched values would have got a
silently wrong distribution. The sequential branch already implicitly
relied on this via its `rangeFrames = rangeEnd - rangeStart` arithmetic.
2. Collapse three near-duplicate docstrings (on `WorkerTask.outputFrameOffset`,
`executeDiskCaptureWithAdaptiveRetry.frameRangeStart`, and `runCaptureStage`'s
`frameRange`) so only the WorkerTask field carries the full contract. The
other two cross-reference it.
3. Drop the WHAT-narrating comments inside `executeWorkerTask`'s per-frame
loop. The variable names (`fileFrameIdx = i - outputOffset`) already say
what the line does; the only remaining comment flags the non-obvious
contract that the streaming callback gets the absolute index.
4. Trim the 30-line `chunkWorkerCount` block in `renderChunk` to one paragraph
explaining the one non-obvious thing (why we use `calculateOptimalWorkers`
directly instead of `resolveRenderWorkerCount`). The probeSession-wasted-on-
parallel acknowledgement stays as a 3-line follow-up flag — investigated
skipping it in this pass, but the SwiftShader probe is safety-critical and
has no per-worker equivalent, so deferred to a separate change with proper
per-worker assertion plumbing.
Tests + format + lint clean:
* `bun test parallelCoordinator.test.ts` — 7/7
* `bun test distributed/{renderChunk,plan}.test.ts` — 24/24
* `bunx oxfmt` + `bunx oxlint` — clean
|
||
|
James Russo
|
c50f59a53b |
feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe (#878)
* feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption
(see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15).
This PR adds the new packages/aws-lambda/ workspace package that wraps
the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler,
plus a build pipeline that bundles the handler + Chromium runtime +
ffmpeg into a deployable ZIP.
Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium
with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan,
renderChunk, assemble}.
The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell
build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new
scripts/probe-beginframe.ts regression guard. Probe boots the runtime
inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and
asserts beginFrame returns a PNG buffer. Verified locally + inside the
Docker container; both pass with hasDamage=true.
Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under
the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling.
This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is
PR 6.1.
* fix(lambda): address PR 878 review feedback
- Verify event.PlanHash against the untarred plan.json at the handler
boundary before invoking the producer primitive. Throws typed
PLAN_HASH_MISMATCH on divergence so Step Functions routes it as
non-retryable; previously the field was schema bloat the handler
ignored, leaving enforcement entirely inside the producer.
- Standardize on MiB throughout build-zip.ts, verify-zip-size.ts, and
the README. Lambda's hard ceiling is 250 MiB (AWS docs label "250 MB"
but use binary mebibytes); previously mixed units made the 248 MiB
budget look like a ~5 MB margin instead of the 2 MiB it actually is.
- stageChromeHeadlessShell now picks Chrome versions via numeric semver
comparison instead of lexicographic sort+reverse — the latter would
silently pick "99.x" over "131.x" once Chrome cached three-digit
majors that aren't width-aligned.
- Drop _setSparticuzChromiumForTests from the public index barrel.
Test-only DI seam imported directly from ./chromium.js in tests.
- Replace require("node:fs") inside walkSize() with the top-level fs
imports — file is ESM and the same module is already imported.
* docs(lambda): drop internal plan-doc refs from package README
* ci(windows): fix bun filter UNION bug excluding producer from Windows tests
`bun run --filter "!a" --filter "!b" test` composes as a UNION (any
package matching either negation runs), not an intersection. Effect:
@hyperframes/producer was still being tested on Windows even though
it's explicitly excluded — its regression harness (Docker + LFS golden
mp4 baselines) is Linux-only and was driving the 32min timeout.
Enumerate the packages we DO want to test instead.
|
||
|
James Russo
|
3569c4830c |
feat(producer): add Rio-style residual-RMS check to regression harness (#882)
* feat(producer): add Rio-style residual-RMS check to regression harness
The existing audio comparison in the regression harness measures the
Pearson correlation between RMS envelopes of the rendered and snapshot
streams. That catches shape-level drift but is insensitive to level
shifts, phase offsets, or codec-quantization noise — two streams can
correlate >0.9 while differing audibly.
Rio's approach (rio/tests/checksum.py:compare_audio_files_ffmpeg) is
sample-level: subtract the snapshot from the rendered stream, run
`astats`, read the residual Overall RMS in dBFS. Identical streams
cancel to silence (-inf, or sub -90 dBFS for AAC-vs-AAC); anything
>= -50 dBFS is considered drift.
This commit adds the same check as an optional secondary gate:
- utils/audioRegression.ts: new `computeAudioResidualRmsDb()` that
spawns ffmpeg with the same filter graph Rio uses (aresample +
pan + volume=-1 + amix + astats) and returns the parsed Overall
RMS plus a pass/fail flag.
- utils/audioRegression.test.ts: 3 new tests covering identical
streams (-inf result), drifted streams (440Hz vs 880Hz sine),
and missing-audio-stream input.
- regression-harness.ts: optional `maxAudioResidualRmsDb` field in
meta.json. Default is undefined (skip the check) so legacy
fixtures aren't retroactively gated; new fixtures opt in by
setting a threshold (e.g. -50). Harness emits `residualRmsDb` in
the audio_comparison_complete JSON event and the pretty log line.
The existing correlation check stays in place; the new residual check
is independent. They measure complementary properties (shape vs
sample-cancellation) and both should hold for a faithful render.
* fix(producer): harden residual-RMS check (parser, duration guard, error surfacing)
Addresses review feedback on PR #882:
- Stateful astats parse: modern ffmpeg emits `Overall` on its own line
followed by per-stat lines, so the single-line `Overall RMS level dB:`
regex never fires on 6.x/7.x/8.x. Find the `Overall` header, take the
next `RMS level dB:` line. Single-line fallback preserved for 4.x.
- Pre-probe both inputs' audio durations and fail up-front if they differ
by >5 ms — `amix=duration=shortest` was silently masking trailing
audio differences.
- Surface ffmpeg/ffprobe spawn errors, signal kills, and non-zero exits
with a stderr tail. Previously every failure mode collapsed into
"NaN, fail" with no diagnostic.
- Extend `TestResult.audio` with `residualRmsDb` + `residualError`,
propagate to `audio-failures.json`.
- Fix `residualSuffix` formatter: NaN (real failure) was being rendered
as "-inf dBFS" (perfect match). Split the branch on `Number.isNaN`
separately from `Number.isFinite` and add an explicit error label.
|
||
|
James Russo
|
cad9160cd8 |
chore(producer): drop internal plan-doc refs from source + docs (#903)
The producer source + docs referenced an internal coordination doc (DISTRIBUTED-RENDERING-PLAN.md) that doesn't ship in the OSS repo, leaving broken cross-links for adopters. Drops the references and the bare section-number shorthand that depended on them; behavioural content (hash contract, retry semantics, threshold rationale) is preserved inline where it was previously offloaded to a section number. |
||
|
Miguel Ángel
|
1e05d78378 |
fix(engine): enable browser pool and deduplicate concurrent Chrome launches (#889)
## Summary
- **Enable browser pool by default** (`enableBrowserPool: true`) — parallel capture workers now share a single Chrome process via reference-counted pool instead of each spawning their own (~256MB each). A 6-worker render drops from 7+ browser parent processes to 1 shared pool.
- **Add launch-promise deduplication** in `acquireBrowser` — when multiple workers race into the pool simultaneously (via `Promise.all`), they await the same launch Promise instead of each triggering a separate Chrome spawn. Same pattern as the existing `_autoBrowserGpuModeCache` for GPU probes.
- **Add `connected` health check** on pool hit — if Chrome crashes mid-render, subsequent acquires detect the dead browser and launch fresh instead of returning a stale reference.
- **Add `drainBrowserPool()`** for explicit cleanup between independent render jobs.
- **CLI studio server** now uses the shared pool instead of its own redundant `enableBrowserPool: false` singleton, so thumbnail generation shares Chrome with render workers.
## Problem
The engine had a reference-counted browser pool (`browserManager.ts:73-75`) but it was **disabled by default** (`enableBrowserPool: false`). This meant:
1. **Every parallel worker spawned its own Chrome** — a `--workers 6` render launched 7+ independent Chrome processes (1 probe + 6 workers), each ~256MB.
2. **The pool had a race condition** — even if manually enabled, concurrent workers calling `acquireBrowser()` via `Promise.all` could all see `pooledBrowser === null` before the first launch completed, spawning N Chromes instead of 1.
3. **No crash recovery** — if Chrome died, the pool still held the dead reference. Subsequent acquires got a disconnected browser.
4. **CLI studio server ran its own singleton** — `studioServer.ts` explicitly set `enableBrowserPool: false` and managed a separate browser, so thumbnails and renders could never share.
Over time, orphaned Chrome processes accumulated across renders and previews. We observed **344 headless Chrome processes** consuming **569% CPU and 20% memory** on a dev machine.
## Before / After (6-worker parallel render)
| Metric | Before (pool off) | After (pool on) |
|--------|-------------------|-----------------|
| Browser parent processes | 7+ (1 probe + 6 workers) | **2** (1 GPU probe + 1 shared) |
| Total Chrome processes (with helpers) | 40-50+ | **14** |
| Memory during capture | ~20%+ | **4.6%** |
| Render time (1200 frames, 30fps) | ~64s | **53s** (~17% faster) |
| Post-render orphans | Accumulated over time | **0** |
## Changes
| File | Change |
|------|--------|
| `engine/src/config.ts` | `enableBrowserPool` default `false` → `true` |
| `engine/src/services/browserManager.ts` | Extract `launchBrowser()`, add `_pooledBrowserLaunchPromise` dedup, add `connected` check on pool hit, add `drainBrowserPool()` and `_resetBrowserPoolForTests()` |
| `engine/src/index.ts` | Export `drainBrowserPool` |
| `engine/src/services/browserManager.test.ts` | Pool dedup and drain tests |
| `cli/src/server/studioServer.ts` | Remove `enableBrowserPool: false` override — thumbnails now share the pool |
| `producer/src/services/browserManager.ts` | Re-export `drainBrowserPool` |
## Backward compatibility
- `PRODUCER_ENABLE_BROWSER_POOL=false` env var disables pooling (same as before).
- Callers passing `{ enableBrowserPool: false }` explicitly still get isolated browsers.
- Tests that set `enableBrowserPool: false` in their config fixtures continue to work.
## Test plan
- [x] Engine tests pass (597/597)
- [x] Producer tests pass (406/407, 1 pre-existing flaky test in `pngDecodeBlitWorkerPool`)
- [x] Build passes (lint, format, typecheck all green via lefthook pre-commit)
- [x] Manual render: `shortform-financial` with `--workers 6` → 1200 frames in 53s, 0 orphaned Chrome processes after completion
- [x] Process monitoring during render confirmed 2 browser parents (1 GPU probe + 1 shared pool) instead of 7+
|
||
|
|
f84cc492de |
perf(engine): faster shader transitions via page-side WebGL compositing (#832)
* fix(cli): prefer puppeteer cache + numeric version sort (staff review) Two correctness fixes from PR #821 self-review: 1. Cache priority order. Previous order was hyperframes-managed cache → puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which lags 17+ releases behind upstream; if a user separately installed a newer chrome-headless-shell via @puppeteer/browsers install, the CLI would silently hand engine the older HF-cache binary while engine's own resolveHeadlessShellPath would have picked the newer one. Flip the priority so puppeteer cache wins, matching engine semantics. 2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()` over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would return `linux-99...` first because character '9' outranks '1'. Parse each name into integer segments and compare them numerically. Tests: add both-caches-populated and linux-148-beats-linux-99 cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(engine): page-side compositing for shader transitions (opt-in spike) Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`. When set, SDR shader-transition compositions skip the Node-side layered blend (the hf#677 chain) and instead run the shader inside Chrome via a page-side WebGL canvas; the engine then captures ONE opaque RGB frame per output frame via the existing streaming capture path. This is the strongest non-beginFrame perf lever for Mac users, who cannot take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275). Stacks on top of the hf#677 1.95× baseline. Default OFF — existing fixture pins (byte-exact MP4 output) are preserved. Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node is f64). HDR content forces the existing layered path regardless. Implementation: - engine: new `EngineConfig.enablePageSideCompositing` (default false). - producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script injected into the served HTML head when the flag is on. - producer/renderOrchestrator: when the flag + no HDR + no png-sequence, route SDR transitions through the streaming path instead of the layered HDR stage. - shader-transitions: new `engineModePageComposite.ts` installs a fullscreen WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside a transition window captures both scenes via the Chromium `drawElementImage` API to GL textures, runs the fragment shader, and displays the composited result on the overlay canvas. The engine takes one screenshot per frame and sees the composited overlay. - cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true` before producer load. - scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a representative fixture with and without the flag, validates the canary strings are in the shipped bundles, and writes a wall-time pair. Determinism trade documented in the engine config doc-comment. The smoke script enforces the bundled-CLI validation discipline from prior perf work (see internal feedback note `validate_bundled_cli_not_dev_path`). Runtime requirement: Chromium's `CanvasDrawElement` feature (already enabled by the engine's `--enable-features=CanvasDrawElement` launch flag). When the runtime feature is unavailable, the page-side installer logs a warning and falls back to opacity-flip mode — the engine still takes the streaming path; the transition window degrades to a hard scene swap. Vance will validate on Mac Chrome where the feature is supported. Co-Authored-By: Vai <vai@heygen.com> * fix(shader-transitions): use html2canvas for page-side compositor capture The original drawElementImage approach fails in engine render mode because the virtual-time shim prevents Chromium from generating paint records for cloned elements. drawElementImage requires a cached paint record from the browser's compositor — clones created at capture time never receive one because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original rAFs don't produce real paints under virtual-time control, and (c) layoutsubtree canvases don't apply CSS stylesheet rules to children. Switch scene capture to html2canvas (foreignObjectRendering: false), the same JS-based renderer already used by the preview-mode fallback path in capture.ts. html2canvas reads computed styles and renders via its own canvas drawing pipeline with no dependency on the browser paint cycle. Also fixes: - Engine seek must return the result so Puppeteer awaits async seek promises (frameCapture.ts). - GSAP opacity cache: compositor must restore scene opacity before seek, not after — GSAP caches inline values and skips re-writes. - Support check gates on WebGL availability, not drawElementImage. Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps) Baseline (Node-side layered): 137s Page-side (html2canvas+WebGL): 33s → 4.1× speedup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(shader-transitions): simplify review fixes for page-side compositor - Use uploadTexture (zeroes canvas backing store after upload) to prevent ~2.2GB transient memory pressure across 280 html2canvas calls per render - Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call, matching the preview-path capture.ts behavior - Parallelize from/to scene captures with Promise.all - Wrap post-capture render in try/finally so opacity is always restored - Fix WebGL context leak in isPageSideCompositingSupported probe - Remove dead ResolvedTransition.index field - Export stabilizeTransformedBoxShadows from capture.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(producer): unify page-side compositing gating and Docker forwarding Addresses three issues from staff review: 1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3, p5.js) — narrowed to data-no-capture only since the compositor canvas is a body sibling never in the scene subtree. 2. Docker mode silently dropped --page-side-compositing — thread pageSideCompositing through DockerRenderOptions/buildDockerRunArgs with regression tests. 3. Fragmented gating across 4 independent sites could disagree: - Stub injection gated only on cfg flag (leaked into HDR/alpha) - Probe-created fileServer never got the stub - needsAlpha (WebM/MOV) not excluded from the gate - WebGL-unavailable fallback claimed layered path would run but orchestrator had already disabled it Fix: compute stub injection at the same site as the layered-bypass decision (after hasHdrContent is known), using addPreHeadScript on the already-running fileServer. Single predicate now gates both decisions, including !needsAlpha. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf(engine): two-phase drawElementImage capture for page-side compositing Replace html2canvas with native drawElementImage for scene capture in the page-side compositor. drawElementImage reads from the browser's own paint cache, giving pixel-identical output to the preview path. The blocker was that cloned elements inside layoutsubtree canvases have no cached paint record under virtual time — the compositor only paints when explicitly triggered. Fix: split the seek+composite into two phases with an engine-forced paint between them. Phase 1 (seek wrapper, page-side): - GSAP seek positions the timeline - Clone FROM/TO scenes into visible layoutsubtree staging canvases - Set window.__hf_page_composite_pending flag Engine paint force (frameCapture.ts): - Detect pending flag after seek returns - Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the browser compositor to paint all visible elements including staging canvas children Phase 2 (page.evaluate, page-side): - drawElementImage reads the now-valid paint records - Upload textures to WebGL, run shader, show GL overlay Key insight: staging canvases must be visible (not opacity:0) for the browser to paint their children. They sit at z-index:-9998, behind the main DOM and covered by the GL overlay during transitions. Perf: 15-scene fixture (28s, 14 transitions, 30fps): Baseline (Node-side layered): 137s html2canvas + WebGL: 33s (3.7×) drawElementImage + WebGL: 21s (6.6×) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf(engine): optimize two-phase compositor hot path - uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of canvas buffer alloc/dealloc churn (persistent staging canvases don't need the one-shot zeroing behavior) - Fold hasPending check into seek page.evaluate: eliminates one CDP round-trip per frame (~700 unnecessary IPC calls on non-transition frames) - Fix renderShader error handling: on failure, leave source scenes visible as fallback instead of hiding both scenes + GL overlay (which produced black frames) - Move mutable state declarations above resolveComposite to prevent TDZ risk on refactor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): staff review — staging cleanup, pending flag, beginFrame guard - Clear staging canvas children when leaving transition window (prevents visible clone bleed-through on transparent compositions) - Clear __hf_page_composite_pending on all resolveComposite exit paths - Guard micro-screenshot paint force against beginFrame mode (CDP Page.captureScreenshot conflicts with beginFrame compositor control) - Update CLI flag description: document video/canvas limitation, remove stale PSNR claim Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): default-on page-side compositing for SDR shader transitions Page-side compositing is now enabled by default for SDR shader-transition renders without video content. The 6.6× speedup applies automatically — no flag needed. Auto-disables when: - HDR content detected - Alpha output (WebM/MOV/PNG-sequence) - Composition contains <video> elements (cloneNode loses playback state) - beginFrame capture mode (Linux headless) Use --no-page-side-compositing to force the Node-side layered path. Changes: - Engine config: enablePageSideCompositing defaults to true - CLI: flag default flipped to true; --no-page-side-compositing disables - Orchestrator: added composition.videos.length === 0 gate - Docker: forwards --no-page-side-compositing when explicitly disabled - Config tests updated for new default Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): support video elements on page-side compositing fast path Three-phase capture protocol lets shader transitions render video scenes without falling back to the slow Node-side layered pipeline: 1. Seek → compositor records transition metadata, sets pending flag 2. onBeforeCapture → video frame injector updates <img> replacements 3. prepare → cloneNode picks up current video frames, img.decode() awaits 4. micro-screenshot → forces browser to paint cloned elements 5. resolve → drawElementImage reads paint records, shader composites Key changes: - Remove `composition.videos.length === 0` gate from orchestrator - Split compositor resolve into prepare (clone) + resolve (shader) - Move onBeforeCapture before compositor prepare in frameCapture.ts - Await img.decode() on cloned data-URI images to prevent stale frames - Stop manipulating scene opacity in compositor (GL canvas overlay suffices) - Add gsap.set declaration for shader-transitions ambient types - Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end Performance: compositions with video now render at 7.5s (6 workers) instead of 2m38s on the layered path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs The timing compiler now injects data-start="0" on <video> and <audio> elements that lack it. This makes discoverMediaFromBrowser() find the element (it queries video[data-start]), so the frame extraction pipeline activates automatically. Videos "just work" without requiring authors to add data-start, data-end, or id attributes. Also removes the video_missing_timing_attrs lint rule — the compiler handles the missing attributes automatically, so the lint rule would only false-positive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(core): add data-hf-auto-start sentinel on auto-injected video timing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery Seeks the GSAP timeline in Puppeteer to discover when each video's parent scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed by binary search refinement to frame-level precision (1/60s). Only processes videos with the data-hf-auto-start sentinel so author-specified timing is never overridden. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(producer): integrate runtime video visibility discovery into probe stage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(producer): trigger browser probe for auto-start videos, remove debug logging The probe stage was skipping browser launch when composition duration was already known, which meant discoverVideoVisibilityFromTimeline never ran. Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(scripts): use mkdtempSync for smoke test work directory Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory via mkdtempSync to resolve CodeQL "insecure temporary file" alert. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format smoke test script Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Vai <vai@heygen.com> |
||
|
Miguel Ángel
|
9b23ccf665 |
feat(studio): html-backed motion panel (#873)
## Summary
Re-architects the studio motion panel to persist GSAP motion data directly in HTML element attributes instead of a `.hyperframes/studio-motion.json` JSON sidecar file. Same pattern as position/resize/rotation edits.
### Before
```
MotionPanel → commitStudioMotionManifestOptimistically()
→ writes .hyperframes/studio-motion.json
→ applyStudioMotionManifest(doc, manifest)
```
### After
```html
<div id="hero" data-hf-studio-motion='{"start":0.5,"duration":1,"ease":"power3.out","from":{"opacity":0,"y":40},"to":{"opacity":1,"y":0}}'>
```
```
MotionPanel → writeStudioMotionToElement(element, motion)
→ buildMotionPatches(element)
→ commitPositionPatchToHtml(selection, patches)
```
## What changed
- **studioMotionOps.ts** — Added `readStudioMotionFromElement()`, `writeStudioMotionToElement()`, `clearStudioMotionFromElement()` for attribute-based CRUD
- **studioMotion.ts** — Added `applyStudioMotionFromDom()` that reads motion from DOM attributes and builds GSAP timeline (kept `applyStudioMotionManifest` for render script compat)
- **manualEditsDom.ts** — Added `buildMotionPatches()` / `buildClearMotionPatches()`, integrated motion into `reapplyPositionEditsAfterSeek()`
- **useDomEditCommits.ts** — Rewrote `handleDomMotionCommit` / `handleDomMotionClear` to use HTML patching instead of manifest persistence
- **useManifestPersistence.ts** — Removed all motion manifest state (~200 lines): `studioMotionManifestRef`, `commitStudioMotionManifestOptimistically`, `applyStudioMotionToPreview`, motion SSE handler
- **App.tsx** — Reads motion from element attribute (`readStudioMotionFromElement`) instead of manifest ref
- **manualEditsRenderScript.ts** — Extended `studioPositionSeekReapplyRuntime` to rebuild GSAP motion timeline from `data-hf-studio-motion` attributes after each seek, including CustomEase support
- **htmlCompiler.ts** — Trigger seek-reapply script injection on `data-hf-studio-motion=` attributes
## Benefits
- No sidecar file — motion survives git, copy-paste, and manual HTML editing
- Undo/redo works via HTML source history (same as position edits)
- Renders correctly via CLI — seek-reapply script handles motion timeline rebuild
- Simpler architecture — one persistence path for all studio edits
## Test plan
- [x] `bun run build` passes
- [x] Pre-commit hooks pass (lint, format, typecheck)
- [ ] Set motion on element in Studio → `data-hf-studio-motion` attribute appears in HTML source
- [ ] Reload page → motion persists and plays correctly
- [ ] Clear motion → attribute removed, element returns to original state
- [ ] Undo/redo motion changes
- [ ] Render via CLI → motion visible in rendered video
- [ ] Seek animation → motion timeline re-syncs correctly
|
||
|
James Russo
|
e8372d45d6 |
Merge pull request #857 from heygen-com/05-15-test_producer_extend_cross-worker_idempotency_to_non-zero_chunks_dedupe_soft-skip_regex
test(producer): extend cross-worker idempotency to non-zero chunks + dedupe soft-skip regex |
||
|
Miguel Ángel
|
225010800a |
feat(studio): persist element positions in HTML, fix resize overlay drift and GSAP double-translation (#829)
* feat(studio): add pasteboard background to preview viewport Adds bg-neutral-800 to the preview viewport so the area outside the canvas is visually distinct from the composition content — consistent with professional video editors (Premiere, DaVinci, Figma). * feat(studio): pasteboard background and canvas outline around preview - NLEPreview: viewport gets bg-neutral-700 (#404040) as the pasteboard color surrounding the canvas — distinct from the app chrome (#0a0a0a) - Player wrapper: drop bg-black so the pasteboard shows around the canvas (loading overlays still cover the area with bg-black during load) - Player: set host background to transparent via inline style (overrides :host { background: #000 } in shadow DOM), and inject a style rule into the open shadow root so .hfp-container has overflow:visible and the canvas iframe gets a thin white ring + soft drop-shadow — making the canvas boundary legible against the pasteboard * feat(studio): disable manual positioning JSON by default, add toggle Manual edits were always stored in `.hyperframes/studio-manual-edits.json`, making it hard to share source without the sidecar file and easy to accidentally reposition elements via drag. Changes: - `enabled` field added to `StudioManualEditManifest` (defaults to `false` when absent — existing projects are unaffected until they opt in) - Drag handles, resize, and rotation handles are hidden when disabled - Layout X/Y/W/H/R fields in the Design panel are read-only when disabled - "Manual positioning" toggle added at the bottom of the Design panel, visible whether or not an element is selected - Toggle state is persisted to `.hyperframes/studio-manual-edits.json` so each project can opt in independently - `STUDIO_PREVIEW_MANUAL_EDITING_ENABLED` env flag still acts as a hard cap (env off → feature off regardless of project setting) * feat(studio): enable manual positioning by default (opt-out) * feat(studio): allow absolute elements to drag without toggle; gate JSON-backed drag behind toggle * feat(studio): persist positions directly to HTML; remove JSON sidecar and manual positioning toggle Replace the `.hyperframes/studio-manual-edits.json` sidecar with inline-style persistence baked directly into the HTML source. Drag/resize/rotation values are written as CSS custom properties (`--hf-studio-offset-x/y`, `--hf-studio-width/height`, `--hf-studio-rotation`) plus `translate`/`width`/`height`/`rotate` inline styles via `persistDomEditOperations` — no re-apply step needed on load. Key changes: - `sourcePatcher`: add `value: string | null` to `PatchOperation` — null removes the property/attribute from the HTML tag instead of setting it - `manualEditsDom`: add `build*Patches` / `buildClear*Patches` helpers that capture live element state into `PatchOperation[]` for HTML source writes; add `reapplyPositionEditsAfterSeek` (DOM-query-based seek hook, queries data-attribute markers) - `manualEdits.ts`: remove `applyStudioManualEditManifest` and all manifest target resolution; export `reapplyPositionEditsAfterSeek`; keep seek/play wrap infrastructure - `useManifestPersistence`: remove all JSON I/O — no disk read on load, no manifest state, no toggle state; `applyCurrentStudioManualEditsToPreview` now only installs seek hooks via `reapplyPositionEditsAfterSeek` - `useDomEditCommits`: replace `commitStudioManualEditManifestOptimistically` calls with direct DOM apply + `commitPositionPatchToHtml` (queued HTML patch write, skipRefresh) - `DomEditOverlay`: remove `manualEditsEnabled` prop; revert all `canMove || manualEditsEnabled` gates to just `canApplyManualOffset` — every draggable element is always draggable - `PropertyPanel`: remove `ManualPositioningToggle` component and all toggle props - `manualEditsParsing/manualEditsTypes`: remove manifest types, upsert functions, and `STUDIO_MANUAL_EDITS_PATH`; keep `finiteNumber`, `readStudioFileChangePath`, `roundRotationAngle`, and snapshot/CSS-property types * fix(studio): sync keyboard shortcut handler with main; fix keepPlaying seek assertions in test * fix(studio): strip GSAP-cached translate from transform on path offset apply * fix(studio): remove Reset edits button from design panel * feat(studio): wire reloadPreview into manifest persistence; drop stale group-selection refresh - Pass `reloadPreview` into `useManifestPersistence` so undo/redo reloads via the refresh-key path instead of directly touching the iframe. - Remove `refreshDomEditGroupSelectionsFromPreview` from commit handlers; HTML is now the source of truth so no stale-ref refresh is needed. - Add `manualEditsRenderScript` helper; export via studio-api and apply it in `htmlCompiler` during HTML compilation. * fix(studio): prevent root composition from being selected; correct overlay drift on resize - Guard `getDomLayerPatchTarget` against elements with `data-composition-id` so the root composition div is never returned as a visual selection target. - Apply the same guard to the raw `elementFromPoint` fallback in `getPreviewTargetFromPointer`, which was the actual escape path. - Thread `iframeRef` into gesture handler opts; after applying draft dimensions during resize, re-read the element BCR via `toOverlayRect` and update the overlay box position to compensate for visual drift on elements with centered transform-origin (e.g. GSAP scale tweens). * fix(studio): correct resize overlay for scaled elements; block invisible element selection - Resize: use BCR from `toOverlayRect` for both position and size after applying draft dimensions — GSAP scale makes visual size diverge from raw CSS size, BCR is the only accurate source during a gesture. - Click selection: add `isElementComputedVisible` guard to the `elementFromPoint` fallback so opacity-0 / autoAlpha-hidden elements cannot be selected even though the browser hit-test returns them. * fix(studio): reload preview on external file changes via SSE/HMR Share the app-level domEditSaveTimestampRef with useManifestPersistence so the SSE/HMR handler can suppress echoes from all studio saves (code tab, timeline, DOM edits), then call reloadPreview() for non-motion external changes that aren't echoes of our own saves. * fix(studio): suppress post-resize click to keep selection on resized element * fix(studio): serve registry blocks without index.html in preview Blocks ship as {id}.html + assets/ with no index.html. The preview route hard-coded index.html so these projects returned 404 and their assets (e.g. korea-map.png, map-nyc-paris.png) were never served. Add resolveProjectMainHtml() that falls back to {id}.html, thread the resolved compositionPath through transformPreviewHtml and injectStudioPreviewAugmentations, and update listProjects() in the vite adapter to surface block directories in the project list. * fix(render): preserve studio drag/resize/rotation offsets in rendered video Three issues caused studio-edited positions to be lost during rendering: 1. The seek-reapply script used setInterval to wrap window.__hf.seek, but Puppeteer's page.evaluate() calls don't yield the event loop for macrotasks — the interval never fired, so reapplyAll() never ran after GSAP seeks. Fix: use Object.defineProperty to trap writes to the seek property, wrapping it synchronously the instant the bridge assigns it. 2. MEDIA_VISUAL_STYLE_PROPERTIES (copied from <video> to proxy <img> during render) included "transform" but not "translate", "rotate", or "scale" — the CSS Transforms Level 2 individual properties used by studio drag/resize/rotation. The proxy was positioned at offsetLeft/ offsetTop without the translate offset. 3. getViewportMatrix (HDR compositor) only read cs.transform, missing individual transform properties entirely. Added composeIndividualTransforms to build the translate × rotate × scale matrix and compose it before the legacy transform matrix. * fix(studio): select elements with pointer-events: none in preview Compositions often set pointer-events: none on scenes, avatar wrappers, and decorative layers. elementsFromPoint() skips these elements entirely, making them unselectable in the Studio. Fix: temporarily inject a * { pointer-events: auto !important } stylesheet during hit-testing, then remove it immediately after. Also adds a pointer_events_none lint rule (info severity, visible with --verbose) so authors know which selectors may affect Studio selection. |
||
|
James
|
47fe69aff0 | test(producer): extend cross-worker idempotency to non-zero chunks + dedupe soft-skip regex | ||
|
James Russo
|
808b10cb0e |
Merge pull request #844 from heygen-com/05-14-test_producer_add_cross-worker_idempotency_unit_test
test(producer): add cross-worker idempotency unit test |
||
|
James
|
6b3ad09436 |
fix(producer): tighten chunk-boundary test gates + narrow VIDEO_EXT indexing
Address @vanceingalls and @miguel-heygen review findings on #852: 1. Asymmetric soft-skip — only the N=1 plan+render+assemble call was wrapped in the host-Chrome-failure catch; an SwiftShader / cold-Chrome flake on the N=4 call would hard-fail instead of soft-skip. Factor a local runRender() helper and wrap both calls. 2. Vacuously-passing length assertion — 'expect(framesOne.length).toBe( framesFour.length)' passes when both runs produce 0 frames. Pin the absolute count (EXPECTED_FRAME_COUNT = 60) so a regression that identically truncates both renders shows red. 3. CDN version drift — anime-boundary loaded gsap@3.14.2 from jsdelivr while every other boundary fixture loaded 3.12.2 from cdnjs. Unify on cdnjs@3.12.2 so the next reader doesn't have to wonder why one fixture diverges. (gsap is an empty duration-driver in all six fixtures so the version was never load-bearing — but the divergence reads as intentional and isn't.) 4. VIDEO_EXT type narrowing — the lookup is Record<"mp4"|"mov"|"webm"> but outputFormat includes "png-sequence". The isPngSequence ternary short-circuits before png-sequence can reach the indexing site, but TS can't narrow through that. Add an explicit cast at the indexing site (not the lookup definition — over-widening to include "png-sequence": undefined would defeat the existence guarantee). 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
bd21d00b13 |
refactor(producer): /simplify Phase 4 distributed-rendering changes
Address findings from a three-agent code-review pass over the Phase 4 stack:
- regression-harness: hoist `readdirSync` out of the per-checkpoint
failure-extraction loop (was running 20 redundant syscalls on every
failing png-sequence test). Drop redundant `existsSync` guards before
`mkdirSync(recursive: true)` and `rmSync(force: true)`. Replace the
three-deep ternary that built the output filename suffix with a
single `Record<format, ext>` lookup.
- regression-harness-distributed: flatten the `format === "mp4" ? {...} : {...}`
branching in the `plan()` call into a single config object with a
conditional spread. `plan()` already accepts `codec: undefined` for
non-mp4 formats, so the duplicate object was unnecessary.
- chunkBoundary.test: rename the stale "byte-identical mp4" test title
to "byte-identical frames" (the test now uses png-sequence). Trim the
10-line comment justifying `rejectOnSystemFonts: false` to the
essential WHY.
- renderChunk / plan.test / regression-harness: drop trailing-edge
comment phrases that pinned the prose to the PR's calendar context
("today", "v1.5", "pre-codec-knob output", section-numbered cross-
references to the planning doc).
No behavior change. All 49 distributed unit tests pass. Smoke + four
distributed format fixtures pass in --mode=distributed-simulated.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
James
|
0b31465b2e | test(producer): add chunk-boundary fixtures per first-party adapter | ||
|
James
|
6dbdac8118 |
fix(producer): normalize default-format check + carry no-audio rationale to mp4-h265-sdr fixture
Address @vanceingalls review on #851: 1. validateMetadata's codec/format check read 'rc.codec !== undefined && rc.format !== undefined && rc.format !== "mp4"'. The behavior was correct (omitted format defaults to mp4 downstream so codec is legal) but relied on the reader knowing that default. Normalize 'effectiveFormat = rc.format ?? "mp4"' before the comparison so the intent reads directly. 2. The mp4-h264-sdr sibling carries inline rationale for the no-audio choice (AAC frame quantization extends container.duration past nb_frames/fps and trips the harness PSNR sampler) and the chunk-seam mapping (crossfade window 0.9-1.1s straddles frame 30). mp4-h265-sdr stripped both. Carry them back so the two fixtures stay parallel. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
5298b12c79 | test(producer): add mp4 H.265 SDR distributed fixture | ||
|
James
|
38e08e81c7 |
style(producer): apply oxfmt to resolvePresetForLockedEncoder signature
The generic parameter constraint exceeded oxfmt's line width, so the formatter wraps the type-param list onto its own line. Applies the same formatting locally that CI's 'Format' job would have produced via 'bun run format:check' — no behavior change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
db62e31d39 |
fix(producer): reject unknown codec strings + extract testable preset-override helper
Address @vanceingalls review on #850: 1. Unknown codec strings (typos like 'H265', future additions like 'av1') silently fell through to libx264 in resolveEncoderTriple. Add an explicit throw symmetric to the non-mp4-format branch already there. A JS caller building config from JSON who passes 'codec: "h266"' now gets a clear error at plan time instead of unflagged h264 output. 2. The preset.codec override in renderChunk had no fast unit coverage — only the heavyweight Docker fixture in #851 would catch a regression if someone refactored the spread (e.g. moved it into getEncoderPreset itself). Extract resolvePresetForLockedEncoder() and add 4 fast unit tests pinning the four encoder shapes (libx265/libx264/prores/png-seq). 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
eccea88daf | feat(producer): add codec knob to DistributedRenderConfig | ||
|
James
|
29436d997c | test(producer): add png-sequence distributed fixture | ||
|
James
|
6c98393ec9 |
fix(producer): detect duplicate fixture IDs across tests/<x>/ and tests/distributed/<x>/
Address @vanceingalls review on #845: the new discoverTestSuites dispatch was silently allowing a future tests/distributed/<x>/ fixture to collide with an existing tests/<x>/ fixture of the same name. Both would push under the same suite.id and stomp each other's failures/ output, baseline lookup, and CLI --filter match. Detect the collision at discovery time and throw with both source dirs named, so the conflict is fixable at author time. Easier to enforce now (one fixture in the new namespace) than after the rest of the Phase 4 fixtures land. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
James
|
4b9a17d520 | test(producer): add cross-worker idempotency unit test | ||
|
James
|
2974bcb1a9 | test(producer): add mp4 H.264 SDR distributed fixture | ||
|
JamesandClaude Opus 4.7
|
e50587496f |
fix(producer): address PR review feedback on harness mode + plan() copy filter
Miguel (approved) and Vai (commented) both flagged the same PSNR-threshold doc/code mismatch; Vai additionally flagged a path-anchoring bug in the projectDir-copy filter and a dishonest type cast. Addressed all five findings: PSNR threshold doc/code mismatch (important): - Module docstring, `resolveMinPsnrForMode` JSDoc, and tests/README.md all claimed distributed-simulated tightens to ≥50 dB. The actual code uses `max(fixture.minPsnr, 10)` — 10 dB is a pathology floor, the per-test gate is the fixture's authored `minPsnr`. Updated all three doc sites to describe what the code does. The 50 dB target in §5.1 is a per- render distributed-vs-in-process contract; against the frozen baseline it's unreachable for either mode (shared encoder/JPEG jitter), so it can't be a per-fixture gate. `PLAN_PROJECT_DIR_COPY_SKIP` regex matched absolute paths (important): - `cpSync` calls the filter with the absolute source path, so a `projectDir` whose absolute path happens to contain a blocklisted segment (`/home/user/work/output/comp/`, `~/projects/dist/foo/`, etc.) caused the filter to return false for every descendant — empty compiled directory, broken render. Now matches relative-to-projectDir segments via `path.relative()` + `split(sep)`. Switched from a regex to a Set for clarity. Harness fixtures don't hit this because they live under `tests/<name>/src/`, but adapters call `plan()` with caller-supplied paths. Dishonest type cast in regression-harness.ts (important): - `as "mp4" | "mov" | "png-sequence"` claimed reachability for formats that `validateMetadata` doesn't accept (the schema is `"mp4" | "webm"`, and webm is rejected by `checkDistributedSupport`). Narrowed to hardcoded `format: "mp4"` with a comment naming the metadata-schema invariant that lets us do that. Renamed `chunkVideoInjectorFactory` (nit): - The variable was invoked once and never used again — "factory" implied repeated calls. Inlined as a plain `videoInjector: BeforeCaptureHook | null` ternary. Replaced tautology test (nit): - `expect(DISTRIBUTED_SIMULATED_MIN_PSNR_DB).toBe(10)` was a value-pin over an exported constant. The invariant the JSDoc actually asserts is "10 dB is below any real fixture's authored minPsnr"; if someone lands a permissive fixture (minPsnr: 5), the value-pin doesn't catch it. Replaced with a test that walks `tests/*/meta.json` and asserts every authored `minPsnr` is ≥ the floor. Validated in `docker:test --mode=distributed-simulated`: font-variant-numeric, many-cuts, gsap-letters-render-compat, style-1-prod, sub-composition-video — all PASSED. Unit tests: 15/15 pass (new fixture-scan test included). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
b8e8617f80 |
refactor(producer): apply /simplify cleanups across distributed harness stack
Code reuse:
- Move `PlanVideosJson` interface + `meta/videos.json` path constant into
`services/distributed/shared.ts`; plan.ts and renderChunk.ts import from
there instead of redeclaring the same shape with the "duplicated here so
renderChunk doesn't import from plan.ts" comment.
- Replace hand-rolled `framePattern.slice(lastIndexOf("."))` with
`extname()` from `node:path` in `rebuildExtractedFramesFromPlanDir`.
Efficiency:
- Hoist `rebuildExtractedFramesFromPlanDir` + `createFrameLookupTable` +
`createVideoFrameInjector` out of the per-chunk closure in renderChunk.
Computed once per chunk now, not once per `createRenderVideoFrameInjector`
callsite (which `runCaptureStage` may invoke multiple times).
- Add a regex filter to `cpSync(projectDir → planDir/compiled/)` so
`node_modules`, `.git`, `output/`, `failures/`, `dist/`, etc. are not
copied. Real projects can have hundreds of MB in those directories;
shipping them to S3/Lambda /tmp on every render bloats cost and time.
- Drop redundant `if (!existsSync(metaDir)) mkdirSync(metaDir, {recursive:true})`
guards; `mkdirSync({recursive:true})` is already idempotent.
Quality:
- Strip narrative comments that told the story of debugging:
- renderChunk's 30-line "Two failure modes made the call actively
harmful" block → 4-line invariant on why `discardWarmupCapture` is
omitted.
- plan.ts's "DO NOT call cleanup()" block → 3 lines naming the
invariant.
- plan.ts's pre-seed-projectDir block → 7 lines on the file-server
invariant.
- renderChunk.ts top docstring's discardWarmupCapture paragraph.
- regression-harness-distributed.ts's PSNR-drift table (belongs in
DISTRIBUTED-RENDERING-PLAN.md, not the source).
- test file's docstring about which tests live where.
- Drop the unreachable IIFE-throw on `format === "webm"` in the harness
(the support check above rejected webm); replace with a plain
`as` cast.
- Replace dynamic `await import("node:fs")` with a top-level import in
`regression-harness-distributed.ts`.
All 54 distributed unit tests still pass in Docker. Full fixture sweep
in `docker:test --mode=distributed-simulated` (font-variant-numeric,
many-cuts, gsap-letters-render-compat, style-1-prod, sub-composition-video)
all PASSED.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JamesandClaude Opus 4.7
|
30f931b503 |
fix(producer): wire video frame injector into renderChunk
The chunk worker passed `createRenderVideoFrameInjector: () => null` to `runCaptureStage`, leaving the page's `<video>` elements to decode the source mp4 against the virtual clock. Chrome's native video pipeline seeks ±1 frame off what the in-process renderer captures (which uses pre-extracted frames injected as images via createVideoFrameInjector). That ±1 frame drift produced the PSNR gap on sub-composition-video and style-1-prod against the in-process baselines. Two pieces: 1. `plan()` now persists the engine's `VideoElement[]` (composition.videos) and a serialized form of `extractionResult.extracted` (videoId, srcPath, framePattern, fps, totalFrames, metadata — paths omitted) to `<planDir>/meta/videos.json`. This is the data renderChunk needs to reconstruct a `FrameLookupTable` without re-running the extract stage. 2. `plan()` no longer calls `frameLookup.cleanup()` after extraction. That cleanup was rm-rf-ing each video's outputDir, which for the in-process orchestrator is a scratch tree the renderer owns — but for plan() that "scratch" IS `compiledDir/__hyperframes_video_frames/<videoId>/`, the source material that the subsequent rename moves into `planDir/video-frames/`. Cleaning it up before the rename left planDir/video-frames/ with only the `_downloads/` subdirectory and no actual frame files. Both `style-1-prod` and `sub-composition-video` reproduced this on every distributed-simulated run; both pass after the cleanup is dropped. 3. `renderChunk` reads `meta/videos.json`, rebuilds `ExtractedFrames[]` by re-listing `planDir/video-frames/<videoId>/` for each video, calls `createFrameLookupTable(videos, extracted)`, and wraps the result in `createVideoFrameInjector` — the same hook the in-process renderer uses. The rebuilt entries set `ownedByLookup: false` so any later cleanup() call from the engine doesn't rm the planDir bytes another worker may still be reading. Validated in `docker:test --mode=distributed-simulated`: font-variant-numeric: PASSED many-cuts: PASSED gsap-letters-render-compat: PASSED style-1-prod: PASSED (was: 15 frames at 26-29 dB) sub-composition-video: PASSED (was: most frames at 21-25 dB) In-process unchanged; 54 distributed unit tests still pass in Docker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
24f64f02c0 |
fix(producer): remove discardWarmupCapture call entirely
Validating the harness against a multi-chunk render (chunkSize=50 on many-cuts, N=4 chunks) revealed that the previous "discard at startFrame-1 for chunk N>0" fix had a second deadlock mode: the discard's frameTimeTicks (base + 49*interval) ended up LARGER than the captureStage first-call's frameTimeTicks (base + 0). Chrome's compositor wedges when asked to go backward in time as predictably as it wedges on a same-time duplicate. Both attempted fixes were trying to work around a problem that doesn't exist: lastFrameCache is only consulted when Chrome returns hasDamage=false, and every chunk frame seeks fresh DOM via __hf.seek() before the screenshot, so hasDamage is always true and the cache is never read. The priming step is unnecessary. Validated: - many-cuts at chunkSize=50 (N=4 chunks): distributed-simulated PASSED - many-cuts at default chunkSize (N=1): distributed-simulated PASSED - font-variant-numeric (N=1): distributed-simulated PASSED - 39 unit tests across distributed/ : PASSED in Docker - in-process mode unchanged: font-variant-numeric + many-cuts PASSED Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
9273eb2229 |
fix(producer): correct discardWarmupCapture chunk-0 deadlock and walk back probe overcorrection
Empirical investigation of --mode=distributed-simulated against many-cuts revealed that the BeginFrame "hang" attributed earlier to a Chrome 148 SwiftShader compositor wedge was actually a renderChunk bug: discardWarmupCapture was called with frameIndex=slice.startFrame, then captureStage immediately captured frame 0 (relative) of the chunk's range. For chunk 0 (slice.startFrame=0) these two calls produced the same frameTimeTicks. Chrome's HeadlessExperimental.beginFrame deadlocks when called twice in a row with the same frameTimeTicks — the compositor has no new damage to advance for, and the second call hangs until the Puppeteer protocolTimeout fires. Tracing the chunk worker confirmed: warmup call 1 t=0 -> ok warmup call 60 t=1947 -> ok (loop exited) beginFrame call #1 t=2333.33 -> returned, hasData=true, hasDamage=true beginFrame call #2 t=2333.33 -> HANG Fix: discardWarmupCapture skips chunk 0 (no prior frame to prime, and the in-process renderer also has an empty cache at frame 0) and uses slice.startFrame - 1 for chunk N>0 (the actual previous absolute frame, which more accurately matches what the in-process renderer's cache holds at the start of frame N). The engine probe complications I added earlier — multi-step screenshot test, inline data:URL pre-navigation, rastered-bytes assertion — were chasing a phantom and are reverted to the original simple form. chrome-headless-shell @stable on Linux with --use-angle=swiftshader renders BeginFrame screenshots correctly after the warmup loop; what looked like "wedged compositor" was the same frameTimeTicks deadlock masquerading as a Chrome regression. Also lowers the harness's distributed-simulated PSNR floor from 45 dB to 10 dB and switches to using the fixture's own minPsnr for both modes. The 45 dB floor was set against font-variant-numeric's static-content baseline drift (~48 dB), but dynamic compositions like many-cuts produce 34-44 dB baseline drift even in-process — both renderers share the same encoder/JPEG jitter floor, so requiring distributed to clear a tighter threshold than in-process catches no real regression. 10 dB remains as an absolute-pathology guard for fixtures with a permissive authored threshold. Validated end-to-end in `docker:test --mode=distributed-simulated`: font-variant-numeric: PASSED (PSNR ~48 dB, audio correlation 1.000) many-cuts: PASSED (PSNR 37-44 dB across rapid transitions) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JamesandClaude Opus 4.7
|
e80bf61d61 |
fix(producer,engine): make distributed renderChunk actually work end-to-end
Three Phase 3 regressions surfaced when validating --mode=distributed-simulated:
engine: probeBeginFrameSupport approved chrome-headless-shell 148 even when
its SwiftShader compositor was wedged. The existing noDisplayUpdates:true
probe returns instantly on 148 and the screenshot variant returned empty
data without erroring. The real capture loop then hung on first frame with
"HeadlessExperimental.beginFrame timed out". Probe now navigates to a small
inline page (matching the real capture's compositor state, not about:blank)
and asserts that 3 back-to-back beginFrame calls each return non-empty
screenshotData. Catches the 148 soft-failure mode; falls back to
Page.captureScreenshot.
producer/plan: plan() didn't copy local assets (style.css, script.js, etc.
referenced by relative URL) into planDir/compiled/. The in-process file
server serves these from projectDir, but the distributed chunk worker's
file server only sees compiledDir. Result: every composition with external
local files rendered as unstyled HTML. Now plan() pre-seeds compiledDir
with cpSync(projectDir, ..., {dereference:true}) before compileStage
overwrites the entry HTML, so the planDir is the self-contained bundle
the docstring claims.
producer/renderChunk: force forceScreenshot:true in the chunk worker's
EngineConfig. Chrome 148's BeginFrame screenshot wedge is content-dependent
— the engine probe (now improved) catches it for some pages but not all,
and the real capture loop hangs on composition-shaped content the probe
can't simulate. Page.captureScreenshot works on every chrome-headless-shell
build we've tested, and executeRenderJob already takes this path for
multi-worker mp4, so the distributed pipeline inherits the proven Linux
reliability profile.
Also lowers the harness's distributed-simulated PSNR floor to 45 dB.
The plan's 50 dB target was written for per-render comparison; against
the frozen baseline file, the in-process renderer itself drifts ~2 dB
due to libx264/JPEG-capture jitter, so 50 dB is empirically unreachable
for either mode. 45 dB tracks the observed ~47-48 dB floor and stays
well above the 30 dB fixture threshold.
Validated:
- font-variant-numeric in distributed-simulated: PASSED (PSNR ~48 dB
across 100 checkpoints, audio correlation 1.000).
- many-cuts surfaces a fourth Phase 3 issue: timing drift on compositions
with external script src= files. First ~5 frames render the
pre-script-execution state and later variants come in ~200 ms late vs
baseline. Tracking separately — the harness mode is correctly detecting
it as a regression, which is the point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
James
|
415ad8b5a6 | feat(producer): add harness mode --mode=distributed-simulated | ||
|
James Russo
|
0df6985d34 |
Merge pull request #816 from heygen-com/05-13-feat_producer_export_distributed_render_primitives
feat(producer): public exports for distributed render primitives |
||
|
Vance Ingalls
|
acc83bd4bb |
perf(producer): pipeline capture and shader-blend per-frame (hf#732 PR 5/5) (#760)
## Summary PR 5 of 5 in the hf#732 decomposition stack. Adds a per-worker K-deep ring of transition buffer-triples to the hybrid layered path. Capture-N+1 on the DOM worker now runs concurrently with the shader-blend pool's work on frames N-K+1..N instead of being serialized behind each blend. ### Mechanism - Each worker carries a ring of K buffer triples (`bufferA` / `bufferB` / `output`), default K=4. - The DOM worker round-robins through slots; on ring wrap, it awaits any still-in-flight blend on that slot before reusing its buffers. - The shader-blend dispatch is no longer awaited inline. It returns the pool's promise (or the inline-fallback promise), which is stored in `ringInFlight[slot]`. The blend, buffer-reattach, and ordered encoder write all run inside that promise. - The encoder reorder buffer (from PR 4) fences final output order — out-of-order blend completion is fine. ### Why K=4 The optimal K is `blend_per_frame / capture_per_frame`. For 854×480 rgb48le with complex shaders this is ~910ms / ~175ms ≈ 5. K=4 balances perf vs. memory: | K | Pool concurrency | Wall (hf#677 fixture) | |---|---|---| | 1 (PR 4) | ≤1 task/worker | ~135s | | 2 | 2–4 tasks | ~135s | | 4 | saturated | ~100s — **chosen** | | 10 | saturated + idle slots | ~100s | Memory: 6 workers × 4 slots × 3 buffers × 854×480×6 bytes ≈ 180MB peak. Override at runtime via `HF_TRANSITION_RING_DEPTH`. ### Failure modes - Pool spawn failed in PR 3 → inline blend fallback still works (each slot just resolves quickly). - Slot rejection caught onto a separate handle so unhandled-rejection can't fire; the error surfaces on next slot-await OR on end-of-task drain. - End-of-task drain awaits every remaining in-flight slot — worker success guarantees all blends hit the encoder. ## Stack Top of the hf#732 decomposition stack. Stacked on top of #759 (PR 4: hybrid path). ## Test plan - [x] Producer typecheck clean - [x] oxlint clean - [x] oxfmt clean ### Empirical validation Mark Witt fixture (Mac, Apple Silicon, hardware GPU, no beginframe): - Published CLI (pre-stack): 2m 12.2s - Cascade CLI (full hf#732 stack): 1m 07.7s - **Measured speedup: ~2× on Mac (1.95× exact).** (Earlier "2.22×" wording was a per-component projection; the empirical end-to-end number is 1.95× on the validated fixture.) Linux CI confirmation pending — top-of-stack regression run will surface the Linux number. — Vai |
||
|
James
|
91a91f66ba | feat(producer): export distributed render primitives | ||
|
James Russo
|
b47a3e798b |
feat(producer): refuse distributed-unsupported formats (webm + HDR mp4) (#815)
## What Phase 3 of the distributed rendering plan: §11 PR 3.5 format banlist. Extends `plan()` to refuse two v1-unsupported formats up front with a typed non-retryable `FormatNotSupportedInDistributedError` (`code === "FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED"`). ## Why Both webm and HDR mp4 are documented as deferred to v1.5 (§7.2 + §12), but until this PR the only signal at the runtime layer is the in-process pipeline silently producing wrong output (chunk concat-copy doesn't round-trip VP9; HDR signaling gets stripped at the chunk boundary). Failing fast at `plan()` time keeps adopters from spending fan-out compute on a render that can't succeed and gives them a typed error code their workflow adapter can route on. ## How - New exports in `services/distributed/plan.ts`: - `FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED` — non-retryable error code matching §11's wording. - `FormatNotSupportedInDistributedError` — typed error class with `code`, `format`, and `reason` fields. Message names the rejected format and tells adopters to fall back to the in-process renderer (`executeRenderJob`) which has full format support. - `rejectUnsupportedDistributedFormat(config)` — pure helper exported separately so adapters can run the same gate at their input layer (Step Functions input validation, Temporal workflow start) before the activity even runs. - `plan()` calls `rejectUnsupportedDistributedFormat(config)` as the first line of the function — BEFORE `mkdirSync(planDir)` so a banned input never produces a partial planDir. - Replaced the previous ad-hoc `if (hdrMode === "force-hdr") throw new Error(...)` with the typed error class. ### What did NOT change `executeRenderJob`, the in-process orchestrator, the `hyperframes render` CLI, producer HTTP routes — all unchanged. The in-process renderer continues to accept webm + HDR (its existing functionality). ## Test plan - [x] Unit tests added — `packages/producer/src/services/distributed/planFormatBanlist.test.ts`. 5 cases: - `rejectUnsupportedDistributedFormat` accepts the v1-supported formats (mp4, mov, png-sequence) with both `auto` and `force-sdr` hdrMode. - Rejects webm — error has `code === FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED`, `format === "webm"`, message mentions in-process renderer. - Rejects HDR mp4 (`hdrMode === "force-hdr"`) — error has `format === "mp4-hdr"`, message mentions HDR. - End-to-end via `plan()`: webm throws with no planDir leaking to disk. - End-to-end via `plan()`: HDR mp4 throws with no planDir leaking to disk. - [x] `bun test packages/producer/src/services/distributed/` — 30 pass. - [x] `bun run --filter @hyperframes/producer typecheck` — clean. - [x] `bunx oxlint` + `bunx oxfmt --check` — clean on changed files. - [ ] Producer Docker regression harness — pending CI. `executeRenderJob` is unchanged; PSNR baselines should hold. This is PR 5 of a 6-PR Phase 3 stack: - 3.1 — `services/distributed/plan.ts` (#808) - 3.2 — `services/distributed/renderChunk.ts` (#809) - 3.3 — `services/distributed/assemble.ts` (#813) - 3.4 — `planDir` size cap (`PLAN_TOO_LARGE`) (#814) - **3.5 (this PR)** — distributed format banlist (webm + HDR mp4) - 3.6 — public exports + `@hyperframes/producer/distributed` subpath 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Vance Ingalls
|
1596fcbe70 |
perf(producer): hybrid layered/parallel path for SDR shader-transition renders (hf#732 PR 4/5) (#759)
## Summary PR 4 of 5 in the hf#732 decomposition stack. **This is where the bulk of the shader-transition speedup lives** (`~2×` verified — see Empirical validation below). Spreads per-frame DOM capture work across N DOM worker sessions and offloads the per-pixel shader-blend onto a `worker_threads` pool (the pool added in #758). ### Gating The hybrid path is gated by `shouldUseHybridLayeredPath`: - SDR content only — HDR raw-frame sources are fd-bound to one worker (per-worker `dup(fd)` is out of scope here). - `workerCount >= 2`. - Not every frame inside a transition window. When the gate trips, the hybrid loop spawns `workerCount - 1` extra DOM sessions, allocates per-worker scratch buffers, and partitions the frame range into contiguous slices via `distributeLayeredHybridFrameRanges`. Each worker walks its slice; transitions dispatch through the shader-blend pool (with inline fallback). A frame-reorder buffer fences the encoder. Pool teardown is guaranteed via try/finally on both the success and error paths. ### Structural change (heads-up to reviewers) `captureHdrStage.ts` on main was already 921 lines (over the project's 500-line ceiling). Adding the hybrid path on top would push it past 1100 and the local pre-commit hook refuses to stage files past 500. **PR 4 splits `captureHdrStage.ts` into 5 files**: - `captureHdrStage.ts` (orchestrator + cleanup invariants, 469 lines) - `captureHdrResources.ts` (HDR video extraction + image decode + dim probing) - `captureHdrFrameShared.ts` (gating predicates, partitioning, per-scene capture) - `captureHdrSequentialLoop.ts` (legacy single-session loop) - `captureHdrHybridLoop.ts` (new multi-worker path) No behavior change in any pre-existing code path: the sequential loop is byte-equivalent to the previous inline implementation (both consume `captureSceneIntoBuffer` from the shared module, so behavior parity is enforced structurally rather than by comment-keeping). `renderOrchestrator.ts` is intentionally unchanged — the stage computes its own worker budget via `calculateOptimalWorkers` rather than receiving it through the call signature. ## Stack Stacked on top of #758 (PR 3: shaderTransition pool). ## Test plan - [x] 14 new vitest tests in `captureHdrFrameShared.test.ts` pinning the hybrid gating predicate and the contiguous-chunking partitioner — all pass - [x] Producer typecheck clean - [x] oxlint clean ### Empirical validation Mark Witt fixture (Mac, Apple Silicon, hardware GPU, no beginframe): - Published CLI (pre-stack): 2m 12.2s - Cascade CLI (this stack): 1m 07.7s - **Measured speedup: 1.95× on Mac.** (Earlier "2.22×" wording was a projection from per-component micro-benchmarks; the empirical end-to-end number is 1.95× on the validated fixture.) Linux CI confirmation pending top-of-stack regression run. — Vai |
||
|
James
|
e5068487c1 | feat(producer): refuse distributed-unsupported formats early | ||
|
James
|
fbbd41a797 | feat(producer): enforce planDir size cap with PLAN_TOO_LARGE |