mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 10:06:21 +00:00
8224cb2060030fdc1dfb1eeb1e0b3f2142dbc47a
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
15ee63c6e7 |
fix: harden CLI edge-case repros (#591)
## Problem I reproduced the selected open issue batch one by one and confirmed the reports were valid. The fixes all touch the CLI/runtime capture boundary, then the follow-up regression run exposed one over-broad runtime change in sub-composition host visibility and one CI-only baseline trap. Closes #590, #589, #588, #587, #586, and #584. ## What this fixes ### CLI/runtime edge cases - Makes the GSAP infinite-repeat lint rule ignore JavaScript comments, so literal `repeat:-1` text in comments is not flagged. - Lets the compositions CLI inspect `<template>` content, count visual-only template descendants, estimate simple GSAP durations, and suppress root `data-start` warnings in sub-composition lint mode. - Preserves runtime bootstrap scripts when body scripts are coalesced, and injects the runtime into a real `<head>` when source HTML has no head. - Keeps #589 fixed by loading and rendering template-wrapped sub-composition content, while restoring host visibility to the shorter of the authored parent clip window and the child composition live timeline. - Resolves snapshot/validate viewport size from root `data-width` / `data-height` instead of falling back to 1920x1080. - Skips fully off-frame text boxes during contrast sampling and bounds-checks ring samples so contrast output no longer emits `null:1` / `NaN:1`. - Marks muted videos as `data-has-audio="false"` in the core timing compiler, which fixes the same-src muted `<video>` + separate `<audio>` StaticGuard case. - Keeps user-authored `hf-seek` listeners reachable during capture by preventing author scripts from being merged into the runtime bootstrap path. ### Shared helper cleanup - Removes the stale producer-local timing compiler duplicate; producer compilation now consumes the core timing compiler. - Centralizes HTML document helpers in core: fragment parsing, embedded runtime stripping, head/body script injection, and early-head injection. - Centralizes the CLI layout/snapshot static HTML server. - Adds browser-safe core subpath helpers for Lottie readiness and CLI screenshot clip calculation; Studio's Vite config keeps the screenshot clip helper self-contained so clean-checkout test startup does not value-import core `.ts` source. - Replaces the engine parity-contract copy with a core re-export. - De-duplicates render-job cleanup and Studio static file-serving callbacks. ### Regression hardening - Replaces the embedded-runtime script stripping regex with a script-tag scanner that handles closing tags like `</script >`. - Escapes inline script bodies before wrapping them in `<script>` tags, so authored `</script` and `<!--` text cannot break out of the injected wrapper script. - Shares media-duration clamping between core and producer, with a 50 ms tolerance for ffprobe precision drift between local and CI media stacks. - Pins the affected style fixture SFX durations in source so style-1 and style-9 compile deterministically. - Restores the `vfr-screen-recording` video golden to the CI-stable baseline; the current CI failure showed the Linux render matches the old golden, while the locally refreshed macOS golden was the mismatch. ## Root cause The CLI paths had accumulated assumptions that held for simple direct-root landscape compositions but not for current composition patterns: DOM queries did not enter template content, snapshot/validate used a fixed viewport, runtime and author scripts shared a coalescing bucket, and timing compilation treated every video as audio-bearing unless authors manually overrode it. The style shard failures were not product regressions. Local and CI media probing disagreed on the short SFX clip duration by about 45 ms, and the compiler was clamping authored durations to the locally probed value. The shared clamp tolerance preserves explicit author/source durations for small probe precision differences while still clamping real overflows. The vfr fast-shard failure was a bad baseline refresh: CI actual frames matched the old `vfr-screen-recording` baseline at 40+ dB PSNR, but mismatched the macOS-refreshed golden at ~18-22 dB. The fix is to keep the Docker/Linux-stable video golden and only retain the deterministic compiled snapshot change. The sub-composition regression came from treating a host's authored parent window as the only visibility boundary. That made settled child overlays stay visible after their own live GSAP timeline ended. The corrected runtime behavior respects both contracts: parent clips still bound where the host can appear, and the child live timeline can end the host earlier. ## Verification ### Local checks - `bunx oxfmt --check packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bunx oxlint packages/core/src/runtime/init.ts packages/core/src/runtime/init.test.ts` - `bun run --cwd packages/core test src/runtime/init.test.ts` - `bun run --cwd packages/cli test src/commands/compositions.test.ts src/utils/compositionViewport.test.ts` - `bun run build:hyperframes-runtime` - `bun run --cwd packages/producer test --keep-temp --sequential style-12-prod style-5-prod` - `bun run --cwd packages/producer test --sequential vfr-screen-recording hdr-hlg-regression style-7-prod` - `bun run --cwd packages/core test src/compiler/htmlCompiler.test.ts src/compiler/timingCompiler.test.ts src/index.test.ts` - `bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/htmlCompiler.ts packages/core/src/compiler/htmlCompiler.test.ts packages/core/src/compiler/index.ts packages/core/src/index.ts packages/core/src/index.test.ts packages/producer/src/services/htmlCompiler.ts` - `bun run --cwd packages/core typecheck` - `bun run --cwd packages/producer typecheck` - `bun run --cwd packages/producer test --sequential style-1-prod style-9-prod` - `bun run --filter @hyperframes/studio test` with `packages/core/dist` temporarily hidden to simulate clean-checkout config loading - `git diff --check` ### CI artifact checks - Inspected failed run `25225854394` job `73969147096`: style-1 failed only on `click-sfx` `1.044898` vs `1` duration/end. - Inspected failed run `25225854394` job `73969147061`: style-9 failed only on SFX `1.044898`-based duration/end mismatches. - Inspected failed run `25225854394` job `73969147048`: `vfr-screen-recording` compilation/audio passed, visual failed after comparing against the macOS-refreshed golden. - Compared the first 10 uploaded CI vfr failure frames against the restored old baseline; minimum PSNR was `40.444705`, above the fixture threshold of `28`. ### Repro checks - `bun packages/cli/src/cli.ts lint /tmp/hf-590-repro` now passes without `gsap_infinite_repeat`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-587-repro --at 0.5 --timeout 1000` now writes a 1080x1920 PNG. - `bun packages/cli/src/cli.ts validate /tmp/hf-588-repro --timeout 500` no longer emits `null:1` / `NaN:1` contrast output. - `bun packages/cli/src/cli.ts validate /tmp/hf-586-repro --timeout 500 --contrast false` no longer emits the muted-video StaticGuard contract error. - `bun packages/cli/src/cli.ts compositions /tmp/hf-589-gsap-repro` now reports `foo 0.5s 1920x1080 1 element`. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-589-gsap-repro --at 0.25 --timeout 2000` captures the expected template-backed red frame. - `bun packages/cli/src/cli.ts snapshot /tmp/hf-584-repro --at 0.5,1.5 --timeout 500` captures the expected post-seek green frame. ### Browser verification - Refreshed the local side-by-side comparison page at `qa-artifacts/pr-591-video-compare/index.html`. - Served the comparison page locally and used `agent-browser` to load `style-12-prod`, play both videos quickly to the failed window, pause, and inspect the side-by-side frame. - Browser proof screenshot: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12-labeled.png`. - Browser proof recording: `qa-artifacts/pr-591-video-compare/browser-proof/fixed-style12.webm`. - Earlier Studio proof artifacts remain local-only: `qa-artifacts/dedupe-refactor-preview.png`, `qa-artifacts/dedupe-refactor-preview-after-play.png`, `qa-artifacts/dedupe-refactor-preview.webm`. ## Notes - Browser proof and CI diagnostic artifacts are intentionally local-only and not committed. - Studio's Vite config intentionally keeps the thumbnail clip helper inline because Vite/Vitest config startup runs through Node's loader before package source `.ts` imports are transformed. - The committed PR diff changes `vfr-screen-recording/output/compiled.html` but no longer changes `vfr-screen-recording/output/output.mp4` relative to `main`. - I attempted a local `linux/amd64` Docker validation to mirror CI, but the local Docker build was blocked by Debian package download failures. The arm64 Docker image also cannot launch the x64 Puppeteer headless shell under OrbStack. The vfr baseline decision is therefore based on the uploaded CI artifact comparison above. - I kept this validated issue batch in one PR because the fixes overlap the same CLI/runtime capture surfaces. |
||
|
Miguel Ángel
|
267ffd3fca |
fix(engine,producer): preserve template-wrapped sub-composition media offsets (#476)
## Problem Template-wrapped sub-compositions could still lose correct parent timing during render in more than one place. In the validated repros, a host sub-composition starting after the intro (and in one follow-up repro, starting at `20s` after earlier compositions) contained scene-local media inside it. On the broken paths: - template-wrapped media could be missed during compile and scheduled at raw scene-local time - already-correct first-pass offsets could be clobbered during `recompileWithResolutions()` - even after those two fixes, the browser-metadata reconcile step in `executeRenderJob()` could still overwrite a compiled global `end` with a scene-local `data-end` from the inlined DOM, clipping the tail off late-start sub-composition media ## What this fixes ### Template-wrapped media discovery - `parseVideoElements`, `parseImageElements`, and `parseAudioElements` now unwrap a single top-level `<template>` wrapper before scraping media - the unwrap helper is DOM-based, not regex-based, so it avoids the CodeQL backtracking warning and only unwraps the exact single-wrapper shape we want - multiple sibling templates or other top-level content are left untouched instead of being rewritten heuristically ### Offset preservation after duration resolution - `recompileWithResolutions()` now preserves the first-pass sub-composition media arrays when the already-inlined HTML no longer contains `[data-composition-src]` hosts - that prevents correctly offset media metadata from being overwritten by scene-local media parsed from the merged DOM ### Browser metadata reconciliation in the compiled time origin - browser-discovered media can still report scene-local `data-start` / `data-end` from the merged DOM after inlining - the producer now reprojects browser `end` values into the compiled element's time origin before reconciling them back into `composition.videos` / `composition.audios` - this prevents late-start sub-composition media from getting truncated back to a scene-local end during the probe phase ### Regression coverage - adds focused engine tests for the template unwrap helper - adds producer regression coverage for both the initial compile path and the post-inline `recompileWithResolutions()` path - adds producer regression coverage for late-start host compositions (`t≈20`) with scene-local media inside them - adds producer unit coverage for the browser-end reprojection helper used by the reconcile path ## Root cause There were three distinct renderer failures behind the bug: ### 1. Template contents were invisible to the media scrapers `parseSubCompositions()` reads raw sub-composition HTML and applies the host offset to discovered media. But the engine media helpers were querying the parsed document directly, and linkedom follows browser semantics here: top-level `<template>` contents live in a `DocumentFragment`, so `querySelectorAll()` never saw those `<video>` / `<audio>` / `<img>` nodes. That meant template-wrapped sub-compositions could silently produce zero discovered media during the first pass. ### 2. The duration-resolution recompile could clobber already-correct offsets After the browser resolves composition durations, `recompileWithResolutions()` reparses the already-inlined HTML. By that point the original `[data-composition-src]` hosts are gone, so `parseSubCompositions()` legitimately returns no nested media. The old code still rebuilt the deduped media arrays from the merged DOM, which let scene-local media parsed from the inlined HTML overwrite the correctly offset first-pass metadata. ### 3. The browser probe reconcile path mixed two timing coordinate systems `discoverMediaFromBrowser()` reads `data-start` / `data-end` directly from the live DOM after sub-compositions are already inlined. For nested media, those attributes can still be scene-local even though the compiled metadata has already been offset into the parent host timeline. The old reconcile path compared those values directly and overwrote `existing.end` whenever the numbers differed. For a late-start sub-composition, that could replace a correct global end like `25.5` with a scene-local end like `5.5`, cutting the clip off during render. ## Verification ### Local checks - `bun test packages/engine/src/utils/htmlTemplate.test.ts` - `bun test packages/producer/src/services/htmlCompiler.test.ts` - `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts` - `bun run --filter @hyperframes/engine test` - `bun run --filter @hyperframes/engine typecheck` - `bun run --filter @hyperframes/producer typecheck` - `bunx oxlint packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts packages/producer/src/services/htmlCompiler.test.ts` - `bunx oxfmt --check packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts` - `bun run build:producer` ### Render / browser verification Verified against two local repros: 1. **Early offset repro** - host starts at `2s` - child media is scene-local `0-4s` - compiled render summary keeps the child video/audio at `start: 2` - browser verification via `agent-browser` confirmed the `2.2s` frame still shows the child clip active in the host timeline 2. **Late offset repro** - earlier compositions run first, then the target host starts at `20s` - child media starts scene-local at `1.5s` and should remain visible through `24.5s` - compiled render summary keeps the child video/audio at `start: 21.5`, `end: 25.5` - browser verification via `agent-browser` confirmed the `24.5s` frame still shows the late clip visible, which is the exact tail-clipping case the old reconcile path could break ## Notes - the `/tmp/hf-pr475-repro` and `/tmp/hf-pr476-late-offset-repro` projects plus their browser-proof artifacts are verification-only and are not part of this PR - this PR stays narrowly scoped to sub-composition media timing across compile, recompile, and browser probe reconciliation; it does not broaden into general sub-composition HTML normalization beyond the single-wrapper case |