Commit Graph
811 Commits
Author SHA1 Message Date
ukimsanov 215b41dc46 fix(linter): add invalid_capture_path error rule
Catches ../capture/ in sub-composition files — the most persistent
sub-agent mistake across v1-v4 (100% failure rate on framer, 4+ rounds).
Documentation rules in sub-agent prompts were not sufficient.

The rule fires as a hard error so lint fails until paths are corrected.
../capture/ works on disk but breaks in Studio and renders (Studio rewrites
base URLs to the project root). Must use capture/ not ../capture/.

Verified: fires correctly on compositions with ../capture/ paths.
2026-05-16 12:22:05 -07:00
ukimsanov 841f877061 feat(snapshot): gemini vision descriptions + v4 skill fixes
snapshot: add --describe flag (default on) for Gemini vision analysis
- describes every frame with the default prompt or a custom question
- auto-resizes 1920x1080 PNGs via sharp to stay under Gemini 4MB limit
- writes snapshots/descriptions.md for agents to compare against storyboard
- errors visible in descriptions.md instead of silently swallowed
- silent skip when GEMINI_API_KEY not set

step-5: block-name vs shader-name, css centering, last-scene workaround
step-4: kokoro ssml tags read as literal text
skill: autonomous mode when user signals skip collaborative gates
step-6: document --describe usage and descriptions.md workflow
2026-05-16 12:18:02 -07:00
ukimsanov 07c5679019 docs: add v4 agent feedback with root cause analysis 2026-05-16 11:52:50 -07:00
ukimsanov 4ae59e5a7d fix(snapshot): two sub-composition seek bugs causing black beats
Bug 1: snapshot fired before all sub-compositions loaded
- Old: resolved when keys.length >= 2 (first sub-comp registered)
- Fix: wait until all data-composition-src hosts have timelines
- Verified: beats 2-4 now visible in huly-promo v4 snapshots

Bug 2: sub-comp timelines seeked to global time, not local time
- Old: tls[key].seek(globalTime) — with exit animations, global t=29
  clamps to GSAP end (t=7), elements at opacity:0 from exit fade
- Fix: seek to global_t - host.data_start (correct local time)
- Verified: beat-5 (starts at t=26.7) seeked to local t=2.3 at
  global t=29 — 'Join the Movement' now visible in snapshots

Both bugs confirmed via render vs snapshot comparison before fixing.

Also fix HeyGen v3 API docs: response is data.audio_url not audio_url
directly; word_timestamps uses 'word' key not 'text' — normalize before
saving as transcript.json
2026-05-16 11:48:09 -07:00
ukimsanov a5a405efd1 refactor(pipeline): reduce overengineering — cut reading burden
skill/SKILL.md: remove Step 5.5 Self-Critique (agents couldn't verify
visuals via snapshots anyway; step was theoretical without visual feedback)

step-1-design.md: 9 sections → 8
- remove Responsive Behavior section (video is fixed 1920x1080)
- remove Inputs/Forms, Navigation, Links from Component Stylings
  (these are never used in video compositions)
- rename and keep Agent Prompt Guide (most-consulted section per docs)
- remove 200-400 line target (was driving padding)

step-3-storyboard.md: remove Implementation References per-beat block
- was the most verbose part of the storyboard template
- sub-agent template in step-5 already handles targeted file reads
- agents were filling it mechanically without adding value

evidence: v2+v3 agents reporting 40% of session time spent reading docs;
storyboard writing cited as second biggest time sink
2026-05-16 06:48:36 -07:00
ukimsanov ebdc8d3725 fix(pipeline): loom v3 fixes, HeyGen v3 API, snapshot auto-clear, beat timing docs
snapshot: auto-clear png/jpg files before each run so old frames don't
mix into contact sheets (confirmed issue from loom v3 agent report)

step-4-vo: update HeyGen TTS to v3 API (current; v1/v2 deprecated Oct 2026)
- list voices: GET /v3/voices?engine=starfish, response is data[] direct list
- generate: POST /v3/voices/speech, returns audio_url + word_timestamps
- HeyGen v3 no longer needs separate transcription step
- v2 data.voices shape documented as legacy

step-5-build RULES: add HyperShader+beat timing rules from v3 findings
- timeline:tl pattern breaks scrubber — let HyperShader create timeline
- proxy+onUpdate doesn't fire in seek mode — use discrete tl.set() calls
- shader name validation: check ls registry/blocks/ before writing

step-3-storyboard: add Beat Timing section to each beat template
- agents specify transition-in time + GSAP duration so step-5 can set
  data-start and data-duration without guessing
2026-05-16 06:26:05 -07:00
ukimsanov d7d0ee6e6a docs: update HANDOFF with v3 run results and new P0 findings 2026-05-16 05:11:50 -07:00
ukimsanov ba9aa4c791 docs: data-start rule in sub-agent RULES, v3 feedback with arc root cause 2026-05-16 05:05:21 -07:00
ukimsanov f74beb8362 fix(pipeline): document data-start/data-duration rule for HyperShader beats
root cause verified from arc-launch v3 render analysis:
- all beats had data-start=0 and data-duration=26 (total video length)
- engine seeks each beat's sub-composition to global_time - data_start
- with data-start=0, beat-2 (BEAT=5.5s GSAP) is seeked to global_t=7
- gsap timeline exhausted at t=5.5, engine marks sub-comp invisible
- content disappears exactly at each beat's GSAP timeline end

fix verified: sequential data-start per transition point + data-duration
matching each beat's GSAP duration → all beats render correctly

added concrete code example to step-5-build.md with explanation
2026-05-16 04:40:25 -07:00
ukimsanov a2ccc1e6fa fix(pipeline): transcript CWD footgun and template script/style rules
cli/transcribe: default output dir to dirname(inputPath) instead of CWD
- transcript.json now lands next to narration.wav regardless of CWD
- fixes silent stale-transcript poisoning when running from repo root

step-5 sub-agent RULES: script+style inside template (verified root causes)
- mercury v3: scripts outside template caused ALL GSAP to silently fail
- workos v3: CSS style blocks inside template not applied after injection
- expanded querySelector null guard: getTotalLength crash documented
2026-05-16 04:20:13 -07:00
ukimsanov ca4c5cc18e docs: add v3 agent feedback with root cause analysis and cross-iteration comparison 2026-05-16 04:06:53 -07:00
ukimsanov 8db7300bda docs: mark v3-prep fixes in agent feedback status table 2026-05-16 02:19:31 -07:00
ukimsanov 36953172bf fix(pipeline): sfx timing rules, font wiring, asset paths, captions, kokoro
step-3: sfx placement rules grounded in audio production standards
- impact sounds: trigger at visual moment, let decay tail bleed (J-Cut)
- riser/build-up: trigger at climax_time - sfx_duration so peak lands on moment
- data-duration must equal manifest duration, never trimmed to fit beat time
- volume under narration: 0.2-0.3 max (no auto-ducking in HyperFrames)
- vo start timing: explicit storyboard decision, document in Global Direction

step-5: asset path rule as first sub-agent rule, impossible to miss
- @font-face: paste declaration into sub-agent prompt explicitly
- Inter Variable -> Inter (compiler only maps base name)
- querySelector null guard: getElementById for elements inside beat hosts
- wcag gradient false positive: data-layout-ignore for decorative elements
- captions stacking: initial opacity:0, one group visible, verify with snapshot

step-4: kokoro pronunciation guidance
- known failure patterns: Vercel, WorkOS, One API
- phonetic substitution workflow: test first sentence, fix in narration.txt
2026-05-16 02:18:30 -07:00
ukimsanov 65533efa8e docs: update HANDOFF and feedback doc with v2 run fixes and status 2026-05-16 01:59:25 -07:00
ukimsanov f8e733b976 fix(pipeline): snapshot wait, SFX ownership, VO timing, optional shader
snapshot: fix two shader pre-render wait failure modes
- Cold cache: overlay stays in DOM as display:none after hiding,
  element-absence check never resolved, always timed out at 60s
- Warm cache: IndexedDB hydration runs without overlay, absence check
  resolved instantly while prewarming was still in progress
- Fix: primary signal is window.__hf.shaderTransitions[].ready;
  overlay display:none is fallback for older builds
- Verified: huly-promo snapshots show composition content, not loader

shader-transitions: optional shader field for CSS crossfade mixing
- TransitionConfig.shader is now optional (ShaderName | undefined)
- Omitting shader = smooth CSS opacity crossfade via fallback path
- CSS-only transitions skip program compilation and prewarming
- HfTransitionMeta.shader made optional in engine/src/types.ts
- Verified: mixed CSS + WebGL composition renders all three beats correctly

step-3-storyboard: SFX assigned here with exact files, not in Step 5
- Agent reads sfx/manifest.json and assigns exact filenames per moment
- Less-is-more: most beats need zero SFX, one max per beat
- Removed duplicate HTML-in-Canvas API detail (build concern, not storyboard)

step-5-build: implement storyboard SFX exactly, zero creative decisions

step-4-vo: required timing reconciliation gate before Step 5
- Compare real audio vs planned total; if delta >15% rescale or trim
- Hard cap on CTA hold: 2-3s after last word, not extended dead silence
2026-05-16 01:55:39 -07:00
ukimsanov f4f6739510 docs: add HANDOFF.md, agent feedback, and CLAUDE.md session header
- HANDOFF.md: 747-line context doc for the May 15-16 pipeline quality session
  (git state, all skill changes, eval arena, video locations, key discoveries)
- AGENT-FEEDBACK-V2.md: 7-agent post-run friction report with priority action
  items (P0: HyperShader CSS crossfade bug, snapshot loading-screen blindspot)
- CLAUDE.md: add READ HANDOFF.md FIRST notice at top for new sessions
2026-05-16 00:55:09 -07:00
ukimsanov 309a641c38 chore: gitignore review-package and scratch files; fix SKILL.md step-1 gate 2026-05-15 23:11:58 -07:00
ukimsanov a80f761c48 chore(skills): remove old pipeline files left over from prior commits
Old step files from before the pipeline v2 restructure were still tracked
alongside the new versions, causing agents to see a conflicting dual pipeline.
Removed: step-1-capture, step-4-storyboard, step-5-vo, step-6-build,
step-7-validate, composition-guide, generate-skeleton.mjs
2026-05-15 22:51:20 -07:00
ukimsanov 91c16cd00b chore(skills): remove deprecated step-2-design and step-3-script files
These were superseded by the new 7-step pipeline structure in the
website-to-hyperframes skill restructure.
2026-05-15 22:28:54 -07:00
ukimsanov d895bdb94f feat(skills): website-to-hyperframes pipeline v2
- SKILL.md: creative tension principle, Step 5.5 self-critique, vision note
- visual-vocabulary.md: inverted to brand-cues→dimensions (replaces user-words→recipes table)
- step-0-capture.md: paginated contact sheet filenames, tighten individual asset reads
- step-1-design.md: user gate after DESIGN.md, generic-vs-specific iteration guide examples
- step-2-brief.md: personalized capabilities pitch + Q2/Q3 anti-prescriptive rewrites
- step-3-storyboard.md: creative tension check, Text Animations + Implementation References per beat
- step-4-vo.md: HeyGen TTS first, music gap for non-narrated videos
- step-5-build.md: agent-agnostic sub-agents, captions stub ban, orphaned comp check
- step-6-validate.md: snapshot formula max(beats×3, ceil(duration/2)), density vs storyboard spec
- capabilities.md: fix VFX count, add text-effects reference
- html-in-canvas-patterns.md: Three.js 0.147→0.181.2 ESM, Math.random→mulberry32
2026-05-15 22:07:18 -07:00
ukimsanov e643d8827d feat(skills): hyperframes core — remove prescriptive tables, fix tone, bundle text-effects
- visual-styles.md: replace YAML recipes with design tradition descriptions + pitfalls
- visual-vocabulary.md moved to w2h skill; house-style.md light/dark mapping removed
- transitions.md: Energy table converted to motion qualities; Mood table to characteristics
- beat-direction.md: rhythm table removed; verb table restructured by physical character
- dynamic-techniques.md: energy table restructured with explanatory principles
- motion-principles.md: replace shouty tone with common-defaults framing
- typography.md: replace guardrails with defaults-to-watch-for framing
- video-composition.md: fix density contradiction (3-layer rule vs 8-10 target)
- techniques.md: remove When-to-Use-What table; fix Lottie package name
- text-effects.md (new): 24 bundled text animation effects with GSAP specs
- assets/text-effects/ (new): full JSON specs for all 24 effects
2026-05-15 22:04:53 -07:00
ukimsanov 3cffd37386 fix(cli): paginated contact sheets, fit-contain, SVG root scan, semantic colors
- contactSheet.ts (new): paginated grids for all asset types
- fit:contain replaces fit:cover — no more aspect-ratio cropping
- SVG scanner covers both assets/svgs/ and assets/ root directories
- agentPromptGenerator: semantic color role labels + dynamic page listing
- snapshot.ts: updated for string[] return from contact sheet functions
2026-05-15 21:36:50 -07:00
ukimsanov d4f18acb0a feat(skills): add SFX manifest, device models, and local CLI documentation
- Add 20 SFX files to website-to-hyperframes/assets/sfx/ with manifest.json
- Add device GLTF models (iphone.glb, macbook.glb) and reference images
- CLAUDE.md: document local CLI path for capture, snapshot, and shader-transitions
- .gitignore: update patterns
2026-05-15 21:34:50 -07:00
ukimsanov 9af33071d3 feat(skills): add skeleton generator for website-to-hyperframes pipeline
Generates pre-valid index.html + composition templates from STORYBOARD.md.
Bakes in: position:absolute, HyperShader wiring, autoAlpha toggles,
SFX placement from manifest, narration audio, grain overlays, scene
visibility management.

Sub-agents only fill creative content — no pipeline mechanics to learn.
Follows the claude-design-hyperframes.md pattern: code handles structure,
AI handles creativity.

Tested: passes lint with 0 errors on first generation.
2026-05-11 01:51:00 -07:00
ukimsanov a8e470bb9c fix(cli): wait for shader loading overlay before snapshots + fix svgs/ captioning
Snapshot: HyperShader shows a loading overlay ([data-hyper-shader-loading])
while pre-rendering transition frames. Snapshots taken during loading show
"Preparing scene transitions" instead of content. Now waits up to 60s for
the overlay to disappear before capturing.

SVG captioning: Gemini captions for svgs/ subdirectory were generated but
never used in asset-descriptions.md. The generateAssetDescriptions function
had a separate code path for svgs/ that used bare "icon:" labels instead
of checking geminiCaptions. Now checks geminiCaptions first.
2026-05-11 01:09:49 -07:00
ukimsanov b1f773178a fix(skills): improve CTA beat guidance and remove implicit beat count
CTA beats are consistently under-animated (13 GSAP calls for 8-second
duration in v3 tests). Added explicit guidance: treat CTA with same
animation density as feature beats, minimum 15 GSAP calls, logo entrance
as an event not a fade.

Also clarified that beat count is agent's decision based on content —
no fixed number.
2026-05-10 21:21:44 -07:00
ukimsanov f14c781075 feat(skills): add SFX manifest with durations and AI descriptions
Created manifest.json for all 19 SFX files with exact duration from
ffprobe and Gemini audio analysis description. Updated Step 6 to
reference manifest instead of generic moment-type table.
2026-05-10 21:07:10 -07:00
ukimsanov aec70f8b57 fix(skills): add critical position:absolute CSS for beat host divs
Root cause of snapshot seeking bug: without position:absolute, beat divs
stack vertically in normal flow. The runtime sets visibility:hidden/visible
to toggle beats, but this only works when beats overlap in the same frame.

Verified: with position:absolute + local CLI, snapshots at t=15 correctly
show beat-4 content instead of beat-1 stuck at every timestamp.
2026-05-10 21:00:17 -07:00
ukimsanov 921d3d24b4 feat(skills): add verified shader transition pattern for sub-compositions
Tested and confirmed: shader transitions work with sub-composition
pipelines. The pattern:
1. Add class="scene" to beat host divs in index.html
2. Load shader-transitions CDN script
3. Call HyperShader.init() with host div IDs and transition configs
4. Engine detects window.__hf.transitions and composites with shaders

Verified with minimal test: cross-warp-morph between two sub-compositions
rendered successfully (layered composite, 180 frames, SDR).
2026-05-10 20:37:30 -07:00
ukimsanov a8ece0dacd fix(skills): address 4 root causes from transcript analysis
1. Capabilities Discovery: replace descriptive guidance with executable
   commands (npx hyperframes list, ls registry/blocks/). Agent was told
   to "check" but never ran the commands — now it has exact bash to run
   and paste results.

2. SFX decoupled from storyboard: removed SFX listing from Step 4
   Capabilities Audit. SFX are now wired in Step 6 Audio Wiring only,
   matched to the storyboard's creative moments. Prevents creative
   direction from revolving around the SFX inventory.

3. Font @font-face: rewritten to explain the auto-resolve system.
   Common fonts (Inter, Roboto, etc.) skip @font-face. Brand fonts
   use tokens.json for family mapping. Hashed filenames = Google Fonts
   subsets that auto-resolve. Prevents agents from giving up on hashed
   filenames.

4. SFX wiring table: added moment-type → SFX mapping so agents match
   sound to emotion instead of listing files arbitrarily.
2026-05-10 20:13:01 -07:00
ukimsanov 0cdbe0311e fix(core): font lint rule skips producer-bundled fonts
The producer's deterministicFonts.ts pre-bundles 18 font families as
data URIs (Inter, Roboto, JetBrains Mono, etc.). The lint rule was
falsely flagging these as missing @font-face. Now only warns for fonts
not in the pre-bundled FONT_ALIASES list — brand-specific fonts like
GT Walsheim or Aeonik that the producer genuinely can't resolve.

Fonts available via Google Fonts (Path 2) still trigger warnings since
they require network access at render time and may fail in sandboxed
environments.
2026-05-10 20:13:01 -07:00
ukimsanov acf11413a1 feat(cli): caption SVGs via Gemini text analysis in capture pipeline
The vision API rejects image/svg+xml, so SVGs were silently skipped during
Gemini captioning — leaving 97% of icons/logos undescribed. This caused
asset utilization as low as 3.8% on SVG-heavy sites (Huly, Raycast).

Now sends SVG source code as text to Gemini, which reads the XML structure
and describes what it renders. Truncates SVGs >10KB to keep token costs low.

Tested on Huly: 2/52 → 37/52 assets described (3.8% → 71%).
2026-05-10 20:13:01 -07:00
ukimsanov 3eca797b2a fix(skills): add SFX copy instruction so agents find bundled audio files 2026-05-10 20:13:01 -07:00
ukimsanov b0e42b8fcd fix(skills): resolve 8 audit findings across website-to-hyperframes
- Fix technique count: "10" → "11" in techniques.md and step-6 (was out of sync)
- Fix VFX registry names: npx hyperframes add --tag html-in-canvas (not bare name)
- Fix step-5 sequencing: remove "update index.html" before it exists in step-6
- Fix shader transitions: inline available shader names instead of repo-only path
- Fix launch-video-2 reference: replace inaccessible file path with inline metrics
- Upgrade Gemini API key from "optional" to recommended with user prompt
2026-05-10 20:13:01 -07:00
ukimsanov 9458eb3f11 feat(skills): pipeline quality improvements for website-to-hyperframes
Step 4 (storyboard):
- Add Capabilities Audit section (HTML-in-Canvas, texture mask text, SFX, rembg)

Step 6 (build):
- Pre-generated @font-face block instruction for sub-agent dispatch
- Per-composition lint gate after each beat
- Asset cross-reference as blocking check
- Depth Layers vocabulary (background/midground/foreground)
- SFX wiring convention with track index allocation
- Animation density minimum (15+ GSAP calls) with production reference
- Easing variety check (3+ distinct easings)

techniques.md:
- Add Easing Vocabulary section with full GSAP palette and mood mapping
2026-05-10 20:13:01 -07:00
ukimsanov c9e114cbad feat(core): add font lint rules (google_fonts_import, font_family_without_font_face)
Two new lint rules that catch the #1 composition quality failure:
- google_fonts_import: warns on fonts.googleapis.com in <link> or @import
- font_family_without_font_face: warns when font-family CSS has no matching @font-face

Verified against 3 baseline sites (framer, raycast, workos) — correctly detects
all font loading failures found in manual A/B testing.
2026-05-10 20:13:01 -07:00
Miguel Ángel d4ba9080f3 Merge pull request #704 from WadydX/fix/init-video-short-flag-conflict
fix(cli): resolve init --video short-flag collision with global -V
2026-05-10 22:43:43 +02:00
WadydX 2778845bd3 fix(cli): handle init -V with explicit migration error 2026-05-10 21:31:17 +01:00
Miguel Ángel d27c4a12be Merge pull request #705 from WadydX/docs/clarify-cli-interactivity-scope
docs(cli): align interactivity and --human-friendly guidance with actual behavior
2026-05-10 22:18:15 +02:00
Miguel Ángel 57ea5641fe chore: release v0.5.7 v0.5.7 2026-05-10 18:34:43 +00:00
Miguel Ángel 5595e38bfb Merge pull request #706 from heygen-com/fix/player-renderseek-setter 2026-05-10 20:33:27 +02:00
Miguel ÁngelandClaude Opus 4.6 711ac22fd8 fix(runtime): add setter to delegated __player properties
The property delegation on window.__player used Object.defineProperty
with only a getter, causing "Cannot set property renderSeek which has
only a getter" when Studio's motion-wrapping code tried to reassign
__player.renderSeek with a wrapped version. This cascaded into an
infinite error loop making the timeline unusable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 18:27:58 +00:00
WadydX e0573c1b94 docs(cli): remove remaining invalid init --human-friendly references 2026-05-10 19:13:14 +01:00
WadydX f083888030 docs(cli): clarify interactivity defaults and human-friendly scope 2026-05-10 19:02:36 +01:00
WadydX e61d1fe002 fix(cli): avoid -V collision for init video flag 2026-05-10 19:01:31 +01:00
Miguel Ángel bd7bbae42d chore: release v0.5.6 v0.5.6 2026-05-10 05:40:30 +00:00
terencechoandClaude Sonnet 4.6 dd375e2784 fix(player): clamp scrubber progress when postMessage frame exceeds duration (#700)
The postMessage state path set `_currentTime` without clamping, while the
direct timeline path already used `Math.min(currentTime, _duration)`. A
final-frame state message with a frame count slightly past the end would
set `_currentTime > _duration`, causing the progress bar (position:
absolute, no overflow guard) to bleed out of the scrubber track and
visually cover the volume button, and the time display to show values
like "0:05 / 0:04".

- Clamp `_currentTime` in `_onMessage` to match the direct timeline path
- Clamp defensively in `updateTime` so the display layer never overflows
- Add `overflow: hidden` + `min-width: 0` to `.hfp-scrubber` as a CSS
  safety net; remove now-redundant `border-radius` from `.hfp-progress`
  (parent `overflow: hidden` handles clipping to the rounded shape)
- Apply the same `overflow: hidden` fix to `.hfp-volume-slider` for
  consistency; remove redundant `border-radius` from `.hfp-volume-fill`
- Add regression test covering the postMessage over-duration case

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:23:19 -07:00
Miguel Ángel 811f309ea5 Merge pull request #685 from heygen-com/feat/contribute-skill
feat(skills): contribute skill for registry block authoring
2026-05-10 05:13:40 +02:00
Miguel ÁngelandClaude Opus 4.6 9fdedaf3e7 feat(docs): add catalog contributing guide and rename skill
Add mintlify docs page for contributing blocks/components to the
registry catalog. Rename skill from contribute to contribute-catalog
for clearer intent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-10 01:38:05 +00:00
Miguel ÁngelandClaude Opus 4.6 082e19df22 fix(skills): tighten trigger surface and add dimension/duration comments
- Narrow description to disambiguate from hyperframes-registry (install)
  and hyperframes (in-project authoring) skills
- Add "adjust" comments on dimensions/duration defaults so agents don't
  blindly copy 1920x1080/10s for all composition shapes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-09 23:45:35 +00:00