mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
Vance IngallsandClaude Fable 5
ec06f4bf89
* feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net Flip useDrawElement + worker-encode defaults on (HF_DE_BATCH default 4), clamped in resolveConfig to hosts where drawElement can engage (macOS + hardware-GPU browser) so page-side shader compositing is untouched everywhere else; explicit env opt-in keeps attempt-and-gate semantics. Safety net makes default-on safe: the compile/init gates catch predictable incompatibility; this catches the intermittent residue no static analysis can see (stale paints, dropped background images, transient blank frames). - engine: captureDeVerificationFrames — K=4 (HF_DE_VERIFY) ground-truth screenshots at init, after gates + armStaticDedup, BEFORE canvas injection (post-injection screenshots show the canvas bitmap, not the DOM). Runs the video-injection hook per sample; double-captures so rAF-driven text counters settle (a single immediate screenshot captures stale text and false-positives). Skips png, <10 frames, implausible __hf.duration (infinite-repeat GSAP sentinel). - producer: guardFrame on both worker-encode drains — rolling-median blank guard with retry-once at drain (byte-identical retry ⇒ deterministic dark frame, accepted; retry save/restores the static-dedup anchor) + ffmpeg PSNR self-verify vs ground truth (HF_DE_VERIFY_MIN_DB, default 32dB; natural agreement ≥45dB, damage ≤25dB). Breach dumps the frame pair to tmpdir and throws DrawElementVerificationError. - orchestrator: one-shot retry — on verification error the whole render re-runs with forceScreenshot (slower, never wrong); telemetry flag deSelfVerifyFallback. - tooling: de-canary-suite.sh (7-comp release gate with expected verdicts), de-gatecheck.sh (init-only corpus routing classifier), we-render.mjs. Validated: canary suite 7/7; 611-comp routing sample 54% drawelement / 37.5% gated / 8.3% comp-defect; 12/12 risk-band renders clean on bare defaults (48/48 verify samples); engine suite 888 passed; caught two real intermittent damage classes in the wild (background-image drop, root-props offset) that previously shipped silently. Kill switches: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false, HF_DE_WORKER_ENCODE=false, HF_DE_BATCH=0, HF_DE_VERIFY=0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(engine,producer): harden the drawElement self-verification net (max code-review findings) 15 confirmed findings from the adversarial review of the default-on flip; the load-bearing five: - Ground-truth capture no longer scrubs GSAP state: seek(0) + forced frame FIRST (lazy .from()/overlap tweens record start values on first seek — mid-timeline scrubs corrupted them for the whole render, and since DE frames and truth shared the corruption, PSNR passed on damaged output), then ascending even-spread fractions, page left at frame 0. - Default-on drawElement is confined to the verified path: resolveConfig requires worker-encode (the drain that runs the net), the orchestrator disengages the default when the render takes the disk path or parallel capture (no drain verification there), and closes a drawElement-initialized probe session rather than letting the unverified path reuse it. Explicit PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true keeps old attempt-and-gate behavior. - Blank-frame retry can no longer splice wrong-frame pixels: recapture goes through recaptureDrawElementFrameForVerify — no static-dedup shortcut (lastEncodeResult runs ahead of the drain) and no "No cached paint record" screenshot fallback (post-injection that captures the canvas = the LAST drawn frame); any recapture failure falls back the whole render. - Verify indices derive from the producer-resolved duration (CaptureOptions.compositionDurationSeconds) instead of raw __hf.duration, so samples always land inside the drained range. - The platform clamp accepts "auto" GPU mode — the stock CLI resolves auto, and the literal-"hardware" clamp made default-on a no-op for the primary audience (masked in validation by explicitly-set env). Also: NaN-safe env parses (HF_DE_VERIFY / HF_DE_VERIFY_MIN_DB / HF_DE_BATCH); video comps skip verification when the session has no frame injector (probe sessions — black-video truth false-positived); psnr infrastructure failures skip the sample instead of failing the render; boundary-saturated sample indices are skipped; shader-transition comps prefer page-side compositing over default drawElement and compile-gated comps get page-side compositing restored; observability.clearFailure un-brands the recovered first streaming attempt; canary suite exempts known-marginal "any" comps from the cross-path PSNR gate; dead we-render options removed; clamp tests pin their env. Validated: canary suite 7/7; auto-GPU bare render engages the full stack; disk-path and worker-encode-off renders disengage default drawElement; malformed HF_DE_VERIFY_MIN_DB still verifies at the default threshold; engine suite 890 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(engine,producer): review fixes — PSC intent, verify-threshold clamp, fail-closed canaries Addresses miguel-heygen's review on #1998: - Page-side compositing restore preserves explicit caller intent (blocker): resolveConfig now records pageSideCompositingAutoDisabled only when IT turned page-side compositing off because drawElement was on; the compile-time drawElement gates restore page-side compositing only when that flag is set. An explicit enablePageSideCompositing:false from the programmatic API or HF_PAGE_SIDE_COMPOSITING=false stays off. Pinned by two config tests. - HF_DE_VERIFY_MIN_DB clamped to [10, 60] with a warning on out-of-range values: below ~10dB the check passes severe damage; above ~60dB natural encoder differences force a screenshot fallback on every verified render. - de-canary-suite.sh + de-gatecheck.sh run under set -euo pipefail with explicit `|| true` on expected-nonzero commands (render exits handled by the suite's own checks, grep no-match, kill/pkill/wait races) and a hard FAIL when the PSNR compare produces no value — release canaries fail closed. Full suite re-run green (7/7) under the new flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
65 lines
3.8 KiB
Bash
65 lines
3.8 KiB
Bash
#!/usr/bin/env bash
|
|
# drawElement release canary suite — run before/after any DE-path change and on a
|
|
# schedule once DE is default-on. macOS + hardware GPU only (DE never engages
|
|
# elsewhere). Each canary renders with the full DE stack + safety net; the suite
|
|
# asserts the render COMPLETES and the safety net's verdict matches expectation:
|
|
# clean — DE end-to-end, no fallback; cross-checked vs a screenshot render (PSNR)
|
|
# fallback — self-verify/blank-guard fires and the auto-fallback completes (these
|
|
# comps have known intermittent DE damage or are defect comps; the net
|
|
# diverting them IS the pass condition)
|
|
# any — either outcome passes (known-flaky DE damage: fallback ~2/3 of runs)
|
|
# Exit nonzero on any FAIL. Usage: COMP_ROOT=/tmp/cc-all bash de-canary-suite.sh
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")"
|
|
ROOT="${COMP_ROOT:-/tmp/cc-all}"
|
|
OUT="${1:-/tmp/de-canary-suite}"
|
|
MIN_DB="${MIN_DB:-32}"
|
|
mkdir -p "$OUT"
|
|
|
|
# id expectation — see fast-capture-architecture.md § Runtime safety net for provenance
|
|
CANARIES="
|
|
d95f20b6 any # paint-race canary; intermittent background-image drop (16.4dB when it fires)
|
|
0531c45f clean # video comp + clip-boundary frames (Lim 6 regression guard)
|
|
3bea8c73 clean # dense motion graphics, deterministic small dark frames (blank-guard guard)
|
|
11c1c878 clean # media-heavy; false de-fail history (harness artifacts)
|
|
398c0655 any # marginal DE-vs-truth divergence hovering at the 32dB line (32-34dB post seek-fix, 23dB before); flips run to run
|
|
3e051c00 fallback # broken comp (runtime error banner) — must divert, not ship
|
|
42e7c33e fallback # error comp — must divert, not ship
|
|
"
|
|
|
|
DE_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true HF_DE_WORKER_ENCODE=true HF_DE_BATCH=4"
|
|
SS_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false"
|
|
|
|
fail=0
|
|
echo "$CANARIES" | grep -v '^\s*$' | while read -r pre expect _; do
|
|
dir=$(find "$ROOT/" -maxdepth 1 -type d -name "${pre}*" | head -1)
|
|
if [ -z "$dir" ]; then echo "FAIL $pre: comp not found under $ROOT"; exit 1; fi
|
|
log="$OUT/$pre.log"
|
|
env $DE_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-de.mp4" </dev/null >"$log" 2>&1 || true
|
|
if ! grep -q "RENDER_OK" "$log"; then echo "FAIL $pre: render did not complete (see $log)"; exit 1; fi
|
|
if grep -q "re-rendering via screenshot" "$log"; then verdict=fallback; else verdict=clean; fi
|
|
case "$expect" in
|
|
any) : ;;
|
|
"$verdict") : ;;
|
|
*) echo "FAIL $pre: expected $expect, got $verdict (see $log)"; exit 1 ;;
|
|
esac
|
|
# Clean DE renders get a cross-path quality check against a screenshot render.
|
|
# "any"-expectation comps are exempt: they are in the suite BECAUSE their
|
|
# DE-vs-screenshot agreement is marginal/intermittent — the check would fail
|
|
# on their intrinsic divergence, not on a regression.
|
|
if [ "$verdict" = clean ] && [ "$expect" != any ]; then
|
|
env $SS_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-ss.mp4" </dev/null >"$OUT/$pre-ss.log" 2>&1 || true
|
|
if ! grep -q "RENDER_OK" "$OUT/$pre-ss.log"; then echo "FAIL $pre: screenshot arm failed"; exit 1; fi
|
|
db=$(ffmpeg -nostdin -hide_banner -i "$OUT/$pre-de.mp4" -i "$OUT/$pre-ss.mp4" -lavfi psnr -f null - 2>&1 \
|
|
| grep -oE "average:(inf|[0-9.]+)" | head -1 | cut -d: -f2 || true)
|
|
if [ -z "$db" ]; then echo "FAIL $pre: PSNR compare produced no value"; exit 1; fi
|
|
if [ "$db" != "inf" ] && awk -v d="$db" -v m="$MIN_DB" 'BEGIN{exit !(d<m)}'; then
|
|
echo "FAIL $pre: DE-vs-screenshot ${db}dB < ${MIN_DB}dB"; exit 1
|
|
fi
|
|
echo "PASS $pre ($verdict, ${db:-?}dB)"
|
|
else
|
|
echo "PASS $pre ($verdict)"
|
|
fi
|
|
done || fail=1
|
|
[ "$fail" = 0 ] && echo "CANARY_SUITE_OK" || { echo "CANARY_SUITE_FAILED"; exit 1; }
|