snapshot: report loaded vs failed fonts via document.fonts API
step-3: pacing is FIRST decision at top of file
step-2 + SKILL.md: no more creative summary confirmation
step-5: sub-agent prompt includes explicit snapshot + lint commands
cli: auto-load .env from CWD so GEMINI_API_KEY/HEYGEN_API_KEY work without
source .env — agents no longer need to manually export keys
snapshot: rename frames to seconds (e.g. frame-00-at-4.2s) instead of
pct (frame-00-at-20pct) — more useful when comparing against storyboard
screenshotCapture: dismiss marketing banners, cookie consents, and
modals before scrolling — prevents popups contaminating screenshots
step-6: replace verbose manual review instructions with:
- required reads: descriptions.md AND contact-sheet.jpg (both)
- critic sub-agent prompt for honest constructive quality assessment
before showing anything to the user
Catches ../capture/ in sub-composition files — the most persistent
sub-agent mistake across v1-v4 (100% failure rate on framer, 4+ rounds).
Documentation rules in sub-agent prompts were not sufficient.
The rule fires as a hard error so lint fails until paths are corrected.
../capture/ works on disk but breaks in Studio and renders (Studio rewrites
base URLs to the project root). Must use capture/ not ../capture/.
Verified: fires correctly on compositions with ../capture/ paths.
snapshot: add --describe flag (default on) for Gemini vision analysis
- describes every frame with the default prompt or a custom question
- auto-resizes 1920x1080 PNGs via sharp to stay under Gemini 4MB limit
- writes snapshots/descriptions.md for agents to compare against storyboard
- errors visible in descriptions.md instead of silently swallowed
- silent skip when GEMINI_API_KEY not set
step-5: block-name vs shader-name, css centering, last-scene workaround
step-4: kokoro ssml tags read as literal text
skill: autonomous mode when user signals skip collaborative gates
step-6: document --describe usage and descriptions.md workflow
Bug 1: snapshot fired before all sub-compositions loaded
- Old: resolved when keys.length >= 2 (first sub-comp registered)
- Fix: wait until all data-composition-src hosts have timelines
- Verified: beats 2-4 now visible in huly-promo v4 snapshots
Bug 2: sub-comp timelines seeked to global time, not local time
- Old: tls[key].seek(globalTime) — with exit animations, global t=29
clamps to GSAP end (t=7), elements at opacity:0 from exit fade
- Fix: seek to global_t - host.data_start (correct local time)
- Verified: beat-5 (starts at t=26.7) seeked to local t=2.3 at
global t=29 — 'Join the Movement' now visible in snapshots
Both bugs confirmed via render vs snapshot comparison before fixing.
Also fix HeyGen v3 API docs: response is data.audio_url not audio_url
directly; word_timestamps uses 'word' key not 'text' — normalize before
saving as transcript.json
snapshot: auto-clear png/jpg files before each run so old frames don't
mix into contact sheets (confirmed issue from loom v3 agent report)
step-4-vo: update HeyGen TTS to v3 API (current; v1/v2 deprecated Oct 2026)
- list voices: GET /v3/voices?engine=starfish, response is data[] direct list
- generate: POST /v3/voices/speech, returns audio_url + word_timestamps
- HeyGen v3 no longer needs separate transcription step
- v2 data.voices shape documented as legacy
step-5-build RULES: add HyperShader+beat timing rules from v3 findings
- timeline:tl pattern breaks scrubber — let HyperShader create timeline
- proxy+onUpdate doesn't fire in seek mode — use discrete tl.set() calls
- shader name validation: check ls registry/blocks/ before writing
step-3-storyboard: add Beat Timing section to each beat template
- agents specify transition-in time + GSAP duration so step-5 can set
data-start and data-duration without guessing
snapshot: fix two shader pre-render wait failure modes
- Cold cache: overlay stays in DOM as display:none after hiding,
element-absence check never resolved, always timed out at 60s
- Warm cache: IndexedDB hydration runs without overlay, absence check
resolved instantly while prewarming was still in progress
- Fix: primary signal is window.__hf.shaderTransitions[].ready;
overlay display:none is fallback for older builds
- Verified: huly-promo snapshots show composition content, not loader
shader-transitions: optional shader field for CSS crossfade mixing
- TransitionConfig.shader is now optional (ShaderName | undefined)
- Omitting shader = smooth CSS opacity crossfade via fallback path
- CSS-only transitions skip program compilation and prewarming
- HfTransitionMeta.shader made optional in engine/src/types.ts
- Verified: mixed CSS + WebGL composition renders all three beats correctly
step-3-storyboard: SFX assigned here with exact files, not in Step 5
- Agent reads sfx/manifest.json and assigns exact filenames per moment
- Less-is-more: most beats need zero SFX, one max per beat
- Removed duplicate HTML-in-Canvas API detail (build concern, not storyboard)
step-5-build: implement storyboard SFX exactly, zero creative decisions
step-4-vo: required timing reconciliation gate before Step 5
- Compare real audio vs planned total; if delta >15% rescale or trim
- Hard cap on CTA hold: 2-3s after last word, not extended dead silence
- contactSheet.ts (new): paginated grids for all asset types
- fit:contain replaces fit:cover — no more aspect-ratio cropping
- SVG scanner covers both assets/svgs/ and assets/ root directories
- agentPromptGenerator: semantic color role labels + dynamic page listing
- snapshot.ts: updated for string[] return from contact sheet functions
Snapshot: HyperShader shows a loading overlay ([data-hyper-shader-loading])
while pre-rendering transition frames. Snapshots taken during loading show
"Preparing scene transitions" instead of content. Now waits up to 60s for
the overlay to disappear before capturing.
SVG captioning: Gemini captions for svgs/ subdirectory were generated but
never used in asset-descriptions.md. The generateAssetDescriptions function
had a separate code path for svgs/ that used bare "icon:" labels instead
of checking geminiCaptions. Now checks geminiCaptions first.
The producer's deterministicFonts.ts pre-bundles 18 font families as
data URIs (Inter, Roboto, JetBrains Mono, etc.). The lint rule was
falsely flagging these as missing @font-face. Now only warns for fonts
not in the pre-bundled FONT_ALIASES list — brand-specific fonts like
GT Walsheim or Aeonik that the producer genuinely can't resolve.
Fonts available via Google Fonts (Path 2) still trigger warnings since
they require network access at render time and may fail in sandboxed
environments.
The vision API rejects image/svg+xml, so SVGs were silently skipped during
Gemini captioning — leaving 97% of icons/logos undescribed. This caused
asset utilization as low as 3.8% on SVG-heavy sites (Huly, Raycast).
Now sends SVG source code as text to Gemini, which reads the XML structure
and describes what it renders. Truncates SVGs >10KB to keep token costs low.
Tested on Huly: 2/52 → 37/52 assets described (3.8% → 71%).
Two new lint rules that catch the #1 composition quality failure:
- google_fonts_import: warns on fonts.googleapis.com in <link> or @import
- font_family_without_font_face: warns when font-family CSS has no matching @font-face
Verified against 3 baseline sites (framer, raycast, workos) — correctly detects
all font loading failures found in manual A/B testing.
The property delegation on window.__player used Object.defineProperty
with only a getter, causing "Cannot set property renderSeek which has
only a getter" when Studio's motion-wrapping code tried to reassign
__player.renderSeek with a wrapped version. This cascaded into an
infinite error loop making the timeline unusable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The postMessage state path set `_currentTime` without clamping, while the
direct timeline path already used `Math.min(currentTime, _duration)`. A
final-frame state message with a frame count slightly past the end would
set `_currentTime > _duration`, causing the progress bar (position:
absolute, no overflow guard) to bleed out of the scrubber track and
visually cover the volume button, and the time display to show values
like "0:05 / 0:04".
- Clamp `_currentTime` in `_onMessage` to match the direct timeline path
- Clamp defensively in `updateTime` so the display layer never overflows
- Add `overflow: hidden` + `min-width: 0` to `.hfp-scrubber` as a CSS
safety net; remove now-redundant `border-radius` from `.hfp-progress`
(parent `overflow: hidden` handles clipping to the rounded shape)
- Apply the same `overflow: hidden` fix to `.hfp-volume-slider` for
consistency; remove redundant `border-radius` from `.hfp-volume-fill`
- Add regression test covering the postMessage over-duration case
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Clips whose compositionStart is ahead of the current timeline position
were starting immediately because sourceNode.start() always received
when=0. Use the AudioContext scheduling API to defer future clips:
sourceNode.start(ctx.currentTime + delay, mediaStart).
Closes#674
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the two-clock architecture (GSAP rAF ticker + HTMLMediaElement
pipeline reconciled by a 50ms polling loop) with a single TransportClock.
GSAP is always paused and seeked to clock.now() on each rAF tick.
Drift between visual timeline and audio is structurally impossible.
Architecture:
TransportClock.now() ──rAF──▶ timeline.seek(t) + el.currentTime
▲
AudioContext.currentTime (~21µs) ← WebAudio active
OR
audio.currentTime (~33ms) ← HTMLMediaElement fallback
OR
performance.now() (~1ms) ← no audio
Key changes:
- TransportClock class with monotonic + audio-master clock sources
- WebAudioTransport: routes audio through AudioBufferSourceNode for
sample-accurate scheduling, falls back gracefully to HTMLMediaElement
- rAF tick loop replaces 50ms setInterval poll; GSAP always paused
- Strict sync (40ms threshold, consecutive-sample gated) + forceSync
on play/pause/seek transitions for sub-frame media accuracy
- Buffer-stall: visuals freeze when audio is buffering instead of
running ahead
- Frame quantization preserved in seek/renderSeek (parity contract)
Browser-verified: 0.0ms drift after 40 pause/play cycles (was 400ms+).
Also fixes: CDN script HTML error responses in validate (pre-existing).
54 tests across clock, clock-drift, webAudioTransport, and media.
Closes#668
The first dynamic `await import("./render.js")` cold-load takes >5 s on
Windows runners — long enough to blow vitest's default 5 s timeout in
whichever test ran it first. Subsequent imports are <10 ms because the
module is now cached, so only test #1 ever times out.
The downstream failure is more subtle: when test #1 times out, vitest
moves on, but its leaked async function eventually hits the synchronous
`producer.createRenderJob(...)` line and pushes a stale config to
`producerState.createdJobs`. That push lands AFTER test #2's `beforeEach`
clears the array, so test #2's `createdJobs[0]` is the leaked test #1
entry instead of its own. That's why test #2 saw `browserGpuMode: 'software'`
when it expected `'auto'`.
Hoist the import into `beforeAll` (matching the pattern the existing
`parseVariablesArg` and `validateVariablesAgainstProject` describe blocks
in this file already use). Cold-load happens once outside any test's
timeout window, every test stays fast, no leaked promise can corrupt
state.
Failing run: https://github.com/heygen-com/hyperframes/actions/runs/25470257972/job/74732502915
Started failing on main with the merge of #642 (auto-detect-browser-gpu),
which added the "forwards browserGpuMode='auto'" test as test #2.