Files
hyperframes/.github/workflows/windows-render.yml
T
Miguel Ángel 267ffd3fca fix(engine,producer): preserve template-wrapped sub-composition media offsets (#476)
## Problem

Template-wrapped sub-compositions could still lose correct parent timing during render in more than one place.

In the validated repros, a host sub-composition starting after the intro (and in one follow-up repro, starting at `20s` after earlier compositions) contained scene-local media inside it. On the broken paths:

- template-wrapped media could be missed during compile and scheduled at raw scene-local time
- already-correct first-pass offsets could be clobbered during `recompileWithResolutions()`
- even after those two fixes, the browser-metadata reconcile step in `executeRenderJob()` could still overwrite a compiled global `end` with a scene-local `data-end` from the inlined DOM, clipping the tail off late-start sub-composition media

## What this fixes

### Template-wrapped media discovery

- `parseVideoElements`, `parseImageElements`, and `parseAudioElements` now unwrap a single top-level `<template>` wrapper before scraping media
- the unwrap helper is DOM-based, not regex-based, so it avoids the CodeQL backtracking warning and only unwraps the exact single-wrapper shape we want
- multiple sibling templates or other top-level content are left untouched instead of being rewritten heuristically

### Offset preservation after duration resolution

- `recompileWithResolutions()` now preserves the first-pass sub-composition media arrays when the already-inlined HTML no longer contains `[data-composition-src]` hosts
- that prevents correctly offset media metadata from being overwritten by scene-local media parsed from the merged DOM

### Browser metadata reconciliation in the compiled time origin

- browser-discovered media can still report scene-local `data-start` / `data-end` from the merged DOM after inlining
- the producer now reprojects browser `end` values into the compiled element's time origin before reconciling them back into `composition.videos` / `composition.audios`
- this prevents late-start sub-composition media from getting truncated back to a scene-local end during the probe phase

### Regression coverage

- adds focused engine tests for the template unwrap helper
- adds producer regression coverage for both the initial compile path and the post-inline `recompileWithResolutions()` path
- adds producer regression coverage for late-start host compositions (`t≈20`) with scene-local media inside them
- adds producer unit coverage for the browser-end reprojection helper used by the reconcile path

## Root cause

There were three distinct renderer failures behind the bug:

### 1. Template contents were invisible to the media scrapers

`parseSubCompositions()` reads raw sub-composition HTML and applies the host offset to discovered media. But the engine media helpers were querying the parsed document directly, and linkedom follows browser semantics here: top-level `<template>` contents live in a `DocumentFragment`, so `querySelectorAll()` never saw those `<video>` / `<audio>` / `<img>` nodes.

That meant template-wrapped sub-compositions could silently produce zero discovered media during the first pass.

### 2. The duration-resolution recompile could clobber already-correct offsets

After the browser resolves composition durations, `recompileWithResolutions()` reparses the already-inlined HTML. By that point the original `[data-composition-src]` hosts are gone, so `parseSubCompositions()` legitimately returns no nested media.

The old code still rebuilt the deduped media arrays from the merged DOM, which let scene-local media parsed from the inlined HTML overwrite the correctly offset first-pass metadata.

### 3. The browser probe reconcile path mixed two timing coordinate systems

`discoverMediaFromBrowser()` reads `data-start` / `data-end` directly from the live DOM after sub-compositions are already inlined. For nested media, those attributes can still be scene-local even though the compiled metadata has already been offset into the parent host timeline.

The old reconcile path compared those values directly and overwrote `existing.end` whenever the numbers differed. For a late-start sub-composition, that could replace a correct global end like `25.5` with a scene-local end like `5.5`, cutting the clip off during render.

## Verification

### Local checks

- `bun test packages/engine/src/utils/htmlTemplate.test.ts`
- `bun test packages/producer/src/services/htmlCompiler.test.ts`
- `bunx vitest run packages/producer/src/services/renderOrchestrator.test.ts`
- `bun run --filter @hyperframes/engine test`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `bunx oxlint packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/renderOrchestrator.test.ts packages/producer/src/services/htmlCompiler.test.ts`
- `bunx oxfmt --check packages/engine/src/utils/htmlTemplate.ts packages/engine/src/utils/htmlTemplate.test.ts`
- `bun run build:producer`

### Render / browser verification

Verified against two local repros:

1. **Early offset repro**
   - host starts at `2s`
   - child media is scene-local `0-4s`
   - compiled render summary keeps the child video/audio at `start: 2`
   - browser verification via `agent-browser` confirmed the `2.2s` frame still shows the child clip active in the host timeline

2. **Late offset repro**
   - earlier compositions run first, then the target host starts at `20s`
   - child media starts scene-local at `1.5s` and should remain visible through `24.5s`
   - compiled render summary keeps the child video/audio at `start: 21.5`, `end: 25.5`
   - browser verification via `agent-browser` confirmed the `24.5s` frame still shows the late clip visible, which is the exact tail-clipping case the old reconcile path could break

## Notes

- the `/tmp/hf-pr475-repro` and `/tmp/hf-pr476-late-offset-repro` projects plus their browser-proof artifacts are verification-only and are not part of this PR
- this PR stays narrowly scoped to sub-composition media timing across compile, recompile, and browser probe reconciliation; it does not broaden into general sub-composition HTML normalization beyond the single-wrapper case
2026-04-24 19:00:42 +02:00

267 lines
10 KiB
YAML

name: Windows render verification
# Manually triggered smoke test that renders a HyperFrames composition on a
# real Windows runner. Proves the PR #336 `where ffmpeg` fix actually works
# end-to-end: FFmpeg is discovered natively on Windows, Chrome is installed
# and launched, frames are captured, and an MP4 is produced — without Docker
# or WSL.
on:
pull_request:
# `edited` is required so the workflow re-fires when a PR's base ref is
# set back to `main` after a Graphite stack restack momentarily flips
# the base off of `main`. Without it, `pull_request` triggers are not
# re-evaluated on `base_ref_changed`, leaving required checks skipped
# for that head SHA forever.
types: [opened, synchronize, reopened, edited]
branches: [main]
push:
branches: [main]
workflow_dispatch:
inputs:
ref:
description: "Git ref to render (branch / tag / SHA)."
required: false
default: "main"
concurrency:
group: windows-render-${{ github.ref }}
cancel-in-progress: true
jobs:
changes:
name: Detect changes
runs-on: ubuntu-latest
timeout-minutes: 2
outputs:
code: ${{ steps.filter.outputs.code }}
steps:
# Force git-based change detection instead of the pull_request REST API.
# The API path can fail the workflow on transient listFiles timeouts
# before the Windows render jobs even start.
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: dorny/paths-filter@v4
id: filter
with:
token: ""
filters: |
code:
- "packages/**"
- "scripts/**"
- "package.json"
- "bun.lock"
- ".github/workflows/windows-render.yml"
render-windows:
name: Render on windows-latest
needs: changes
if: needs.changes.outputs.code == 'true' || github.event_name == 'workflow_dispatch'
runs-on: windows-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.ref }}
lfs: true
- name: Show platform info
shell: pwsh
run: |
Write-Host "OS: $([System.Environment]::OSVersion.VersionString)"
Write-Host "PowerShell: $($PSVersionTable.PSVersion)"
Write-Host "Runner: windows-latest"
# -----------------------------------------------------------------
# Install FFmpeg via the shared composite action so the install logic
# stays identical between this job and `test-windows` below. See
# .github/actions/install-ffmpeg-windows for why we bypass Chocolatey.
# -----------------------------------------------------------------
- name: Install FFmpeg
uses: ./.github/actions/install-ffmpeg-windows
# -----------------------------------------------------------------
# Verify FFmpeg feature inventory.
#
# The engine shells out to a fixed set of encoders (libx264 for MP4,
# libx265 for HEVC, libvpx-vp9 for WebM, prores_ks for transparent
# MOV, aac for audio), muxers (mp4 / mov / webm), and demuxers
# (image2pipe for streaming RGBA frames, rawvideo for HDR PQ frames,
# mov,mp4 for video frame extraction). Some of these are GPL-only,
# so a future build swap could silently drop one and break a code
# path the canary render doesn't exercise. Fail fast here instead.
# -----------------------------------------------------------------
- name: Verify FFmpeg feature inventory
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
function Assert-FfmpegFeature {
param(
[Parameter(Mandatory)] [string] $Listing,
[Parameter(Mandatory)] [string] $Name,
[Parameter(Mandatory)] [string] $Kind
)
# `ffmpeg -encoders` etc. emit one feature per line as
# `<flags> <name> <description>`, so a whitespace boundary on
# each side is enough to disambiguate (e.g. `mov` vs `movflags`).
$pattern = "(^|\s)$([regex]::Escape($Name))(\s|$)"
if ($Listing -notmatch $pattern) {
throw "Required FFmpeg $Kind '$Name' not present in this build"
}
Write-Host " ok: $Kind $Name"
}
Write-Host "--- encoders ---"
$encoders = (& ffmpeg -hide_banner -encoders 2>&1) -join "`n"
foreach ($enc in @('libx264', 'libx265', 'libvpx-vp9', 'prores_ks', 'aac')) {
Assert-FfmpegFeature -Listing $encoders -Name $enc -Kind 'encoder'
}
Write-Host "--- muxers ---"
$muxers = (& ffmpeg -hide_banner -muxers 2>&1) -join "`n"
foreach ($mux in @('mp4', 'mov', 'webm')) {
Assert-FfmpegFeature -Listing $muxers -Name $mux -Kind 'muxer'
}
Write-Host "--- demuxers ---"
$demuxers = (& ffmpeg -hide_banner -demuxers 2>&1) -join "`n"
foreach ($dem in @('image2pipe', 'rawvideo', 'mov,mp4,m4a,3gp,3g2,mj2')) {
Assert-FfmpegFeature -Listing $demuxers -Name $dem -Kind 'demuxer'
}
- name: Install Bun
uses: oven-sh/setup-bun@v2
- name: Install Node
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install dependencies
shell: pwsh
run: bun install --frozen-lockfile
- name: Build all packages
shell: pwsh
run: bun run build
# -----------------------------------------------------------------
# Prove the PR #336 fix: hyperframes doctor exercises findFFmpeg()
# and whichBinary() — both must pass on Windows without workarounds.
# -----------------------------------------------------------------
- name: hyperframes doctor (verifies `where ffmpeg` fix)
shell: pwsh
run: node packages/cli/dist/cli.js doctor
- name: Scaffold canary composition
shell: pwsh
run: |
New-Item -ItemType Directory -Force -Path "$env:RUNNER_TEMP\windows-canary" | Out-Null
cd "$env:RUNNER_TEMP\windows-canary"
node "$env:GITHUB_WORKSPACE\packages\cli\dist\cli.js" init canary --example blank --non-interactive --skip-skills
$fixtures = "$env:GITHUB_WORKSPACE\.github\workflows\fixtures"
Copy-Item "$fixtures\windows-canary.html" "canary\index.html" -Force
- name: Render canary composition
shell: pwsh
run: |
cd "$env:RUNNER_TEMP\windows-canary\canary"
node "$env:GITHUB_WORKSPACE\packages\cli\dist\cli.js" render `
--fps 30 `
--quality draft `
--workers 2 `
--output renders\canary.mp4
- name: Verify rendered MP4
shell: pwsh
run: |
$mp4 = "$env:RUNNER_TEMP\windows-canary\canary\renders\canary.mp4"
if (-not (Test-Path $mp4)) { throw "canary.mp4 not produced" }
$probe = ffprobe -v error -select_streams v:0 `
-show_entries stream=width,height,r_frame_rate -show_entries format=duration `
-of default=noprint_wrappers=1 $mp4
Write-Host $probe
# Parse probe output
$width = ($probe | Select-String '^width=(.+)$').Matches.Groups[1].Value
$height = ($probe | Select-String '^height=(.+)$').Matches.Groups[1].Value
$fps = ($probe | Select-String '^r_frame_rate=(.+)$').Matches.Groups[1].Value
$duration = [double]($probe | Select-String '^duration=(.+)$').Matches.Groups[1].Value
if ([int]$width -ne 1920) { throw "expected 1920 width, got $width" }
if ([int]$height -ne 1080) { throw "expected 1080 height, got $height" }
if ($fps -ne "30/1") { throw "expected 30fps, got $fps" }
if ($duration -lt 7.5 -or $duration -gt 8.5) { throw "expected ~8s duration, got $duration" }
Write-Host "canary.mp4 ok: ${width}x${height} @ $fps, ${duration}s"
- name: Upload rendered MP4 artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: windows-render-${{ github.run_id }}
path: ${{ runner.temp }}/windows-canary/canary/renders/canary.mp4
if-no-files-found: error
retention-days: 7
# -------------------------------------------------------------------
# Unit-test suites on Windows. Mirrors the Linux `test` job in ci.yml
# so we catch Windows-specific regressions (path separators, shell
# invocations, CRLF, file URLs, etc.) in existing vitest suites.
# The producer package is skipped because its tests require Docker /
# Linux-only tooling (Dockerfile.test, LFS golden MP4 baselines).
# -------------------------------------------------------------------
test-windows:
name: Tests on windows-latest
needs: changes
if: needs.changes.outputs.code == 'true' || github.event_name == 'workflow_dispatch'
runs-on: windows-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.ref }}
lfs: true
# -----------------------------------------------------------------
# Install FFmpeg so vitest suites that gate on `HAS_FFMPEG`
# (e.g. packages/engine videoFrameExtractor.test.ts) actually run on
# Windows. Without it those suites `describe.skipIf(!HAS_FFMPEG)`
# themselves silently and any Windows-specific regression in the
# FFmpeg-driven code paths would not be caught here.
# -----------------------------------------------------------------
- name: Install FFmpeg
uses: ./.github/actions/install-ffmpeg-windows
- name: Install Bun
uses: oven-sh/setup-bun@v2
- name: Install Node
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install dependencies
shell: pwsh
run: bun install --frozen-lockfile
- name: Build
shell: pwsh
run: bun run build
- name: Run tests (all packages except producer)
shell: pwsh
run: bun run --filter "!@hyperframes/producer" test
- name: Run runtime contract test
shell: pwsh
run: bun run --filter "@hyperframes/core" test:hyperframe-runtime-ci