Commit Graph
557 Commits
Author SHA1 Message Date
ukimsanov ec5edfa1df feat(snapshot): font verification + pacing-first + creative summary removal
snapshot: report loaded vs failed fonts via document.fonts API
step-3: pacing is FIRST decision at top of file
step-2 + SKILL.md: no more creative summary confirmation
step-5: sub-agent prompt includes explicit snapshot + lint commands
2026-05-18 01:45:06 -07:00
ukimsanov f606cdfc0f refactor(pipeline): strip dead capture outputs, add easing vocabulary, streamline skill
Capture CLI:
- Remove detected-libraries.json, assets-catalog.json, full-page.png generation
- Remove detectedLibraries from AGENTS.md generator and buildPrompt

Skill:
- step-5: pro easing vocabulary + stacked-beats code pattern for fast pacing
- step-3: contact sheet asset discovery, marketing assets vs screenshots distinction
- SKILL.md: lighter DESIGN.md for fast pacing, no arbitrary GSAP call count
2026-05-17 21:26:56 -07:00
ukimsanov 5e33c32565 feat(pipeline): pacing system, launch-video skill, verification gates, skill cleanup
Pipeline pacing:
- step-3: add pacing → beat structure table mapping Fast/Moderate/Slow/Arc to
  concrete beat count, duration, and architecture choices
- step-5: add architecture selection based on pacing (stacked beats for fast,
  HyperShader sub-comps for moderate/slow) with cinematic beat patterns

Launch-video skill (from upstream PR #699):
- Billboard-per-beat pattern for fast-paced videos
- Stacked divs with GSAP opacity sequencing, no sub-compositions

Verification improvements:
- step-6: Definition of Done checklist, per-beat verdict protocol, critic scoring

Skill cleanup (~1,600 lines saved per session):
- capabilities.md (704 lines) → capabilities-overview.md (34 lines)
- beat-direction.md inlined into step-3 (motion verbs + transition matrix)
- narration.md inlined into step-3 (pacing formula + hook patterns)

Other fixes:
- agentPromptGenerator.ts: fix outputDir scope crash
- step-1: font availability check
- step-3: block name ≠ shader name warning
- step-4: TTS test clip, background music prompt
- step-5: HyperShader.init() pattern, sequential track indices, GSAP FROM TRAP
- step-6: GEMINI_API_KEY export for subshell inheritance
2026-05-17 16:21:23 -07:00
ukimsanov a997f79c37 fix(pipeline): auto-load .env, second-based snapshot names, banner dismissal, critic sub-agent
cli: auto-load .env from CWD so GEMINI_API_KEY/HEYGEN_API_KEY work without
source .env — agents no longer need to manually export keys

snapshot: rename frames to seconds (e.g. frame-00-at-4.2s) instead of
pct (frame-00-at-20pct) — more useful when comparing against storyboard

screenshotCapture: dismiss marketing banners, cookie consents, and
modals before scrolling — prevents popups contaminating screenshots

step-6: replace verbose manual review instructions with:
- required reads: descriptions.md AND contact-sheet.jpg (both)
- critic sub-agent prompt for honest constructive quality assessment
  before showing anything to the user
2026-05-16 14:34:07 -07:00
ukimsanov 215b41dc46 fix(linter): add invalid_capture_path error rule
Catches ../capture/ in sub-composition files — the most persistent
sub-agent mistake across v1-v4 (100% failure rate on framer, 4+ rounds).
Documentation rules in sub-agent prompts were not sufficient.

The rule fires as a hard error so lint fails until paths are corrected.
../capture/ works on disk but breaks in Studio and renders (Studio rewrites
base URLs to the project root). Must use capture/ not ../capture/.

Verified: fires correctly on compositions with ../capture/ paths.
2026-05-16 12:22:05 -07:00
ukimsanov 841f877061 feat(snapshot): gemini vision descriptions + v4 skill fixes
snapshot: add --describe flag (default on) for Gemini vision analysis
- describes every frame with the default prompt or a custom question
- auto-resizes 1920x1080 PNGs via sharp to stay under Gemini 4MB limit
- writes snapshots/descriptions.md for agents to compare against storyboard
- errors visible in descriptions.md instead of silently swallowed
- silent skip when GEMINI_API_KEY not set

step-5: block-name vs shader-name, css centering, last-scene workaround
step-4: kokoro ssml tags read as literal text
skill: autonomous mode when user signals skip collaborative gates
step-6: document --describe usage and descriptions.md workflow
2026-05-16 12:18:02 -07:00
ukimsanov 4ae59e5a7d fix(snapshot): two sub-composition seek bugs causing black beats
Bug 1: snapshot fired before all sub-compositions loaded
- Old: resolved when keys.length >= 2 (first sub-comp registered)
- Fix: wait until all data-composition-src hosts have timelines
- Verified: beats 2-4 now visible in huly-promo v4 snapshots

Bug 2: sub-comp timelines seeked to global time, not local time
- Old: tls[key].seek(globalTime) — with exit animations, global t=29
  clamps to GSAP end (t=7), elements at opacity:0 from exit fade
- Fix: seek to global_t - host.data_start (correct local time)
- Verified: beat-5 (starts at t=26.7) seeked to local t=2.3 at
  global t=29 — 'Join the Movement' now visible in snapshots

Both bugs confirmed via render vs snapshot comparison before fixing.

Also fix HeyGen v3 API docs: response is data.audio_url not audio_url
directly; word_timestamps uses 'word' key not 'text' — normalize before
saving as transcript.json
2026-05-16 11:48:09 -07:00
ukimsanov ebdc8d3725 fix(pipeline): loom v3 fixes, HeyGen v3 API, snapshot auto-clear, beat timing docs
snapshot: auto-clear png/jpg files before each run so old frames don't
mix into contact sheets (confirmed issue from loom v3 agent report)

step-4-vo: update HeyGen TTS to v3 API (current; v1/v2 deprecated Oct 2026)
- list voices: GET /v3/voices?engine=starfish, response is data[] direct list
- generate: POST /v3/voices/speech, returns audio_url + word_timestamps
- HeyGen v3 no longer needs separate transcription step
- v2 data.voices shape documented as legacy

step-5-build RULES: add HyperShader+beat timing rules from v3 findings
- timeline:tl pattern breaks scrubber — let HyperShader create timeline
- proxy+onUpdate doesn't fire in seek mode — use discrete tl.set() calls
- shader name validation: check ls registry/blocks/ before writing

step-3-storyboard: add Beat Timing section to each beat template
- agents specify transition-in time + GSAP duration so step-5 can set
  data-start and data-duration without guessing
2026-05-16 06:26:05 -07:00
ukimsanov a2ccc1e6fa fix(pipeline): transcript CWD footgun and template script/style rules
cli/transcribe: default output dir to dirname(inputPath) instead of CWD
- transcript.json now lands next to narration.wav regardless of CWD
- fixes silent stale-transcript poisoning when running from repo root

step-5 sub-agent RULES: script+style inside template (verified root causes)
- mercury v3: scripts outside template caused ALL GSAP to silently fail
- workos v3: CSS style blocks inside template not applied after injection
- expanded querySelector null guard: getTotalLength crash documented
2026-05-16 04:20:13 -07:00
ukimsanov f8e733b976 fix(pipeline): snapshot wait, SFX ownership, VO timing, optional shader
snapshot: fix two shader pre-render wait failure modes
- Cold cache: overlay stays in DOM as display:none after hiding,
  element-absence check never resolved, always timed out at 60s
- Warm cache: IndexedDB hydration runs without overlay, absence check
  resolved instantly while prewarming was still in progress
- Fix: primary signal is window.__hf.shaderTransitions[].ready;
  overlay display:none is fallback for older builds
- Verified: huly-promo snapshots show composition content, not loader

shader-transitions: optional shader field for CSS crossfade mixing
- TransitionConfig.shader is now optional (ShaderName | undefined)
- Omitting shader = smooth CSS opacity crossfade via fallback path
- CSS-only transitions skip program compilation and prewarming
- HfTransitionMeta.shader made optional in engine/src/types.ts
- Verified: mixed CSS + WebGL composition renders all three beats correctly

step-3-storyboard: SFX assigned here with exact files, not in Step 5
- Agent reads sfx/manifest.json and assigns exact filenames per moment
- Less-is-more: most beats need zero SFX, one max per beat
- Removed duplicate HTML-in-Canvas API detail (build concern, not storyboard)

step-5-build: implement storyboard SFX exactly, zero creative decisions

step-4-vo: required timing reconciliation gate before Step 5
- Compare real audio vs planned total; if delta >15% rescale or trim
- Hard cap on CTA hold: 2-3s after last word, not extended dead silence
2026-05-16 01:55:39 -07:00
ukimsanov 3cffd37386 fix(cli): paginated contact sheets, fit-contain, SVG root scan, semantic colors
- contactSheet.ts (new): paginated grids for all asset types
- fit:contain replaces fit:cover — no more aspect-ratio cropping
- SVG scanner covers both assets/svgs/ and assets/ root directories
- agentPromptGenerator: semantic color role labels + dynamic page listing
- snapshot.ts: updated for string[] return from contact sheet functions
2026-05-15 21:36:50 -07:00
ukimsanov a8e470bb9c fix(cli): wait for shader loading overlay before snapshots + fix svgs/ captioning
Snapshot: HyperShader shows a loading overlay ([data-hyper-shader-loading])
while pre-rendering transition frames. Snapshots taken during loading show
"Preparing scene transitions" instead of content. Now waits up to 60s for
the overlay to disappear before capturing.

SVG captioning: Gemini captions for svgs/ subdirectory were generated but
never used in asset-descriptions.md. The generateAssetDescriptions function
had a separate code path for svgs/ that used bare "icon:" labels instead
of checking geminiCaptions. Now checks geminiCaptions first.
2026-05-11 01:09:49 -07:00
ukimsanov 0cdbe0311e fix(core): font lint rule skips producer-bundled fonts
The producer's deterministicFonts.ts pre-bundles 18 font families as
data URIs (Inter, Roboto, JetBrains Mono, etc.). The lint rule was
falsely flagging these as missing @font-face. Now only warns for fonts
not in the pre-bundled FONT_ALIASES list — brand-specific fonts like
GT Walsheim or Aeonik that the producer genuinely can't resolve.

Fonts available via Google Fonts (Path 2) still trigger warnings since
they require network access at render time and may fail in sandboxed
environments.
2026-05-10 20:13:01 -07:00
ukimsanov acf11413a1 feat(cli): caption SVGs via Gemini text analysis in capture pipeline
The vision API rejects image/svg+xml, so SVGs were silently skipped during
Gemini captioning — leaving 97% of icons/logos undescribed. This caused
asset utilization as low as 3.8% on SVG-heavy sites (Huly, Raycast).

Now sends SVG source code as text to Gemini, which reads the XML structure
and describes what it renders. Truncates SVGs >10KB to keep token costs low.

Tested on Huly: 2/52 → 37/52 assets described (3.8% → 71%).
2026-05-10 20:13:01 -07:00
ukimsanov c9e114cbad feat(core): add font lint rules (google_fonts_import, font_family_without_font_face)
Two new lint rules that catch the #1 composition quality failure:
- google_fonts_import: warns on fonts.googleapis.com in <link> or @import
- font_family_without_font_face: warns when font-family CSS has no matching @font-face

Verified against 3 baseline sites (framer, raycast, workos) — correctly detects
all font loading failures found in manual A/B testing.
2026-05-10 20:13:01 -07:00
Miguel Ángel d4ba9080f3 Merge pull request #704 from WadydX/fix/init-video-short-flag-conflict
fix(cli): resolve init --video short-flag collision with global -V
2026-05-10 22:43:43 +02:00
WadydX 2778845bd3 fix(cli): handle init -V with explicit migration error 2026-05-10 21:31:17 +01:00
Miguel Ángel 57ea5641fe chore: release v0.5.7 2026-05-10 18:34:43 +00:00
Miguel ÁngelandClaude Opus 4.6 711ac22fd8 fix(runtime): add setter to delegated __player properties
The property delegation on window.__player used Object.defineProperty
with only a getter, causing "Cannot set property renderSeek which has
only a getter" when Studio's motion-wrapping code tried to reassign
__player.renderSeek with a wrapped version. This cascaded into an
infinite error loop making the timeline unusable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 18:27:58 +00:00
WadydX e61d1fe002 fix(cli): avoid -V collision for init video flag 2026-05-10 19:01:31 +01:00
Miguel Ángel bd7bbae42d chore: release v0.5.6 2026-05-10 05:40:30 +00:00
terencechoandClaude Sonnet 4.6 dd375e2784 fix(player): clamp scrubber progress when postMessage frame exceeds duration (#700)
The postMessage state path set `_currentTime` without clamping, while the
direct timeline path already used `Math.min(currentTime, _duration)`. A
final-frame state message with a frame count slightly past the end would
set `_currentTime > _duration`, causing the progress bar (position:
absolute, no overflow guard) to bleed out of the scrubber track and
visually cover the volume button, and the time display to show values
like "0:05 / 0:04".

- Clamp `_currentTime` in `_onMessage` to match the direct timeline path
- Clamp defensively in `updateTime` so the display layer never overflows
- Add `overflow: hidden` + `min-width: 0` to `.hfp-scrubber` as a CSS
  safety net; remove now-redundant `border-radius` from `.hfp-progress`
  (parent `overflow: hidden` handles clipping to the rounded shape)
- Apply the same `overflow: hidden` fix to `.hfp-volume-slider` for
  consistency; remove redundant `border-radius` from `.hfp-volume-fill`
- Add regression test covering the postMessage over-duration case

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:23:19 -07:00
Miguel ÁngelandClaude Opus 4.6 ae343bfdc8 chore: release v0.5.5
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-08 15:00:29 +00:00
Miguel ÁngelandClaude Opus 4.6 38522f996e fix(runtime): schedule future WebAudio clips with delay instead of playing immediately
Clips whose compositionStart is ahead of the current timeline position
were starting immediately because sourceNode.start() always received
when=0. Use the AudioContext scheduling API to defer future clips:
sourceNode.start(ctx.currentTime + delay, mediaStart).

Closes #674

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-08 14:56:33 +00:00
Miguel Ángel 8de7ad7f61 chore: release v0.5.4 2026-05-07 23:22:15 -07:00
Miguel Ángel f761ee2624 fix(player): drive src URL timelines without runtime (#673)
* fix(player): drive src URL timelines without runtime

* fix(player): pause direct timeline after seek
2026-05-08 08:19:29 +02:00
Miguel Ángel b009288df7 feat(runtime): single-clock transport — eliminate pause/play audio drift (#671)
Replace the two-clock architecture (GSAP rAF ticker + HTMLMediaElement
pipeline reconciled by a 50ms polling loop) with a single TransportClock.
GSAP is always paused and seeked to clock.now() on each rAF tick.
Drift between visual timeline and audio is structurally impossible.

Architecture:

  TransportClock.now() ──rAF──▶ timeline.seek(t) + el.currentTime
       ▲
  AudioContext.currentTime (~21µs)  ← WebAudio active
       OR
  audio.currentTime (~33ms)         ← HTMLMediaElement fallback
       OR
  performance.now() (~1ms)          ← no audio

Key changes:
- TransportClock class with monotonic + audio-master clock sources
- WebAudioTransport: routes audio through AudioBufferSourceNode for
  sample-accurate scheduling, falls back gracefully to HTMLMediaElement
- rAF tick loop replaces 50ms setInterval poll; GSAP always paused
- Strict sync (40ms threshold, consecutive-sample gated) + forceSync
  on play/pause/seek transitions for sub-frame media accuracy
- Buffer-stall: visuals freeze when audio is buffering instead of
  running ahead
- Frame quantization preserved in seek/renderSeek (parity contract)

Browser-verified: 0.0ms drift after 40 pause/play cycles (was 400ms+).

Also fixes: CDN script HTML error responses in validate (pre-existing).

54 tests across clock, clock-drift, webAudioTransport, and media.

Closes #668
2026-05-08 07:53:46 +02:00
James 453bd6e313 fix(engine,producer): reject alpha+supersample, unit-test clip plumbing 2026-05-07 16:58:26 +00:00
James 385917ea59 fix(engine): supersample screenshot capture to honor deviceScaleFactor 2026-05-07 16:58:26 +00:00
James 0eaa54e000 fix(core,engine): derive CanvasResolution from CANVAS_DIMENSIONS, snapshot eviction value 2026-05-07 16:58:26 +00:00
James 2b2281d490 refactor: dedupe resolution presets and clean up 4k stack 2026-05-07 16:58:26 +00:00
James bc02b8810c fix(studio): exhaustively type RESOLUTION_OPTIONS and document layout coupling 2026-05-07 16:58:26 +00:00
James 0c026bb0f6 fix(studio): forward outputResolution from vite dev adapter to producer 2026-05-07 16:58:26 +00:00
James 25edba803a feat(studio): add resolution selector to render export bar 2026-05-07 16:58:26 +00:00
James 1545763ea3 fix(cli,producer): cross-multiply aspect check, CLI HDR guard, honest banner 2026-05-07 16:58:25 +00:00
James e07aeba213 feat(cli): add --resolution flag to hyperframes render for one-line 4k 2026-05-07 16:58:25 +00:00
James Russo c1c7ba999a Merge pull request #661 from heygen-com/05-07-feat_cli_add_--resolution_flag_to_hyperframes_init_for_4k_scaffolding
feat(cli): add --resolution flag to hyperframes init for 4k scaffolding
2026-05-07 07:55:13 -07:00
Vance Ingalls edac92b431 docs: add texture mask text catalog entry (#650)
* feat(registry): add texture mask PNGs for texture-mask-text component

* feat(registry): add texture-mask-text CSS snippet

* feat(registry): add registry-item.json for texture-mask-text

* feat(registry): add texture-mask-text demo composition

* feat(registry): register texture-mask-text component in manifest

* style: format texture-mask-text files with oxfmt

* fix: set mask-image directly on texture classes instead of via CSS custom property

url() inside CSS custom properties doesn't resolve correctly with mask-image
in some browsers. Move mask-image declarations to each texture class directly.

* docs: add texture mask text catalog entry

* test: lint texture mask text usage

* fix: harden texture mask text docs and lint

* fix: stabilize texture mask asset paths

* fix: address texture catalog review feedback

* fix: harden texture mask text instructions

* docs: remove texture catalog intro copy

* docs: use canonical texture preview URL

* docs: use cdn texture mask assets

* fix: escape catalog frontmatter safely

* test: stabilize windows render cli test

* test: pin texture catalog instructions
2026-05-07 00:23:36 -07:00
James 8355b39ffc fix(engine): skip caching oversized frames + add eviction telemetry 2026-05-07 06:10:26 +00:00
James 8203005488 fix(engine): byte-budget the frame data uri cache to bound memory at 4k 2026-05-07 06:10:26 +00:00
James 9f0074e44a fix(cli): handle reverse-order CSS in applyResolutionPreset and document scope 2026-05-07 06:10:22 +00:00
James a4eea984d9 feat(cli): add --resolution flag to hyperframes init for 4k scaffolding 2026-05-07 06:09:44 +00:00
James 20f8318ee2 fix(core): tighten 4K threshold to 3840 and pin square=portrait convention 2026-05-07 05:08:12 +00:00
James 555c51fcf6 feat(core): add 4k canvas resolution presets 2026-05-07 02:36:16 +00:00
James a739be58b1 fix(cli): preload render.js once in renderLocal test suite to fix Windows CI
The first dynamic `await import("./render.js")` cold-load takes >5 s on
Windows runners — long enough to blow vitest's default 5 s timeout in
whichever test ran it first. Subsequent imports are <10 ms because the
module is now cached, so only test #1 ever times out.

The downstream failure is more subtle: when test #1 times out, vitest
moves on, but its leaked async function eventually hits the synchronous
`producer.createRenderJob(...)` line and pushes a stale config to
`producerState.createdJobs`. That push lands AFTER test #2's `beforeEach`
clears the array, so test #2's `createdJobs[0]` is the leaked test #1
entry instead of its own. That's why test #2 saw `browserGpuMode: 'software'`
when it expected `'auto'`.

Hoist the import into `beforeAll` (matching the pattern the existing
`parseVariablesArg` and `validateVariablesAgainstProject` describe blocks
in this file already use). Cold-load happens once outside any test's
timeout window, every test stays fast, no leaked promise can corrupt
state.

Failing run: https://github.com/heygen-com/hyperframes/actions/runs/25470257972/job/74732502915
Started failing on main with the merge of #642 (auto-detect-browser-gpu),
which added the "forwards browserGpuMode='auto'" test as test #2.
2026-05-07 02:06:03 +00:00
James Russo 31acf7fdec Merge pull request #654 from TheodorKleynhans/feat/cli-png-sequence-format
feat(cli): expose png-sequence format
2026-05-06 18:12:18 -07:00
James Russo 588639fc1a Merge pull request #657 from heygen-com/fix/runtime-swallow-helper
fix(runtime): swallow() helper replaces empty catches in inlined runtime
2026-05-06 18:11:38 -07:00
Miguel Ángel ea3f7273e7 fix(core): scope duplicate getElementById lookups (#655) 2026-05-07 02:49:30 +02:00
James Russo a327e13a1e Merge pull request #632 from kiyeonjeon21/fix/cli-bun-command-hints
fix(cli): use bun in command hints
2026-05-06 17:22:02 -07:00
James Russo 5212ed49c9 Merge pull request #320 from Dylanwooo/feat/doctor-json-output
feat(cli): add --json output to doctor
2026-05-06 17:21:33 -07:00