ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 23:03:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
317 Commits 856 Branches 404 Tags
237847e5c68f2a35ee39b617bcc335eb9c39913c
Commit Graph
5 Commits
Author SHA1 Message Date
Miguel Ángel 5655dabff6 feat: allow clip animation + ship <hyperframes-player> web component (#209) 
## Summary

Two independent initiatives that improve agent DX and expand HyperFrames' reach.

### Initiative 1: Fix the Clip Animation Footgun

- `gsap_animates_clip_element` lint rule now uses smart detection — only errors when GSAP animates `visibility` or `display` on a clip element
- All other properties (opacity, transform, x, y, scale, etc.) are allowed silently
- This was the #1 agent failure in QA (10/10 agents hit it on v0.2.1)

### Initiative 2: `<hyperframes-player>` Web Component

- New `@hyperframes/player` package — zero dependencies, 3.3KB gzipped
- Iframe-based web component with Shadow DOM for perfect isolation
- Video-like API: `play()`, `pause()`, `seek()`, `currentTime`, `duration`, events
- Controls overlay with play/pause, scrubber (mouse + touch), time display, auto-hide
- Full docs page at `docs/packages/player.mdx`

## Before / After

### Clip animation lint

**Before (10/10 agents hit this):**

```
✗ gsap_animates_clip_element: GSAP animation targets a clip element.
  Selector "#title" resolves to element <div id="title" class="clip">.
  The framework manages clip visibility — animate an inner wrapper instead.
  Fix: Wrap content in a child <div> and target that with GSAP.
```

**After (only errors on actual conflicts):**

```
# This passes lint — no error:
tl.from("#title", { opacity: 0, y: -50, scale: 0.8 }, 0);

# This still errors — actual conflict with runtime:
tl.to("#title", { visibility: "hidden" }, 3);
✗ gsap_animates_clip_element: GSAP animation sets visibility on a clip element.
  Fix: Remove the visibility/display tween. Use opacity for fade effects.
```

### Embeddable player

**Before:** No way to embed a composition in a web page.
**After:**

```html
<script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script>
<hyperframes-player src="./composition/index.html" controls></hyperframes-player>
```

```js
const player = document.querySelector('hyperframes-player');
player.play();
player.pause();
player.seek(2.5);
player.addEventListener('ready', (e) => console.log('Duration:', e.detail.duration));
```

## Test plan

- [x] 427 core tests pass (20 GSAP lint tests with smart detection)
- [x] 7 player tests pass (formatTime + element registration)
- [x] TypeScript compiles cleanly (core + player)
- [x] Lint: GSAP animating clip with safe props → 0 errors
- [x] Lint: GSAP animating clip with `visibility` → 1 error (correct)
- [x] Player builds to 3.3KB gzipped ESM
- [x] Lockfile updated for CI
- [x] Docs page added at `docs/packages/player.mdx`
 2026-04-06 19:59:39 +02:00
Miguel Ángel baa3d813be fix: address QA report P0-P2 issues for 10/10 agent experience (#208) 
## Summary

Addresses all 8 issues from the QA report to improve agent and user experience.

### P0 — Must Fix

- **Blank template broken captions**: Removed `compositions/captions.html` and its reference from the blank template. Every agent (10/10) hit 404 errors during render.
- **Inner-wrapper example**: Added a clear structural comment in the blank template showing the correct `class="clip"` + inner wrapper pattern.
- **Sub-composition introspection**: `hyperframes compositions` now reads external HTML files referenced via `data-composition-src` and shows their real duration/element count instead of `0.0s / 0 elements`.

### P1 — Fix Soon

- **Font mapping warnings**: Now lists all mapped fonts, suggests alternatives (use a mapped font, add @font-face, install locally), and links to docs.
- **Browser 404s**: Non-font "Failed to load resource" 404s now prefixed with `[non-blocking]` instead of `[Browser:ERROR]`.
- **Render concurrency**: Default workers increased from `cores/2` (max 4) to `cores*3/4` (max 6). Added `--concurrency` alias.

### P2 — Nice to Have

- **Transform conflict fix suggestion**: `gsap_css_transform_conflict` now suggests exact GSAP property replacements (e.g., `xPercent: -50, yPercent: -50`).
- **Upgrade --yes**: Now actually runs the install instead of just printing the command.

## Before / After

### Font mapping warning

**Before:**

```
[Compiler] No deterministic font mapping for: DM Sans
```

**After:**

```
[Compiler] No deterministic font mapping for: DM Sans
  Mapped fonts: arial → inter, courier → jetbrains-mono, ...
  To fix, pick one:
    1. Use a mapped font name instead (see list above)
    2. Add a @font-face block in your HTML with a local or hosted font file
    3. Install the font locally on the render machine (Docker: add to Dockerfile)
    4. Add an alias to FONT_ALIASES in deterministicFonts.ts (for contributors)
```

### Browser 404s during render

**Before:** `[Browser:ERROR] Failed to load resource: the server responded with a status of 404`
**After:** `[non-blocking] Failed to load resource: the server responded with a status of 404`

### Transform conflict lint

**Before:** `Fix: Remove the transform from CSS and use tl.fromTo...`
**After:** `Fix: Remove transform: translate(-50%, -50%) from CSS and replace with GSAP properties: xPercent: -50, yPercent: -50`

### Compositions command

**Before:**

```
overlay   0.0s   1920×1080   0 elements
```

**After:**

```
overlay   8.0s   1920×1080   2 elements ← compositions/overlay.html
```

## Test plan

- [x] All 422 core tests pass
- [x] TypeScript compiles cleanly (all 4 packages)
- [x] Full monorepo build succeeds
- [x] `hyperframes init --template blank` ships without broken captions reference
 2026-04-06 18:56:39 +02:00
James RussoandClaude Opus 4.6 2f99e33bbe feat(cli,core): standalone transcribe command, transcript normalization, caption lint rules (#151) 
* feat(cli,core): add standalone transcribe command, transcript normalization, and caption lint rules

- Add `hyperframes transcribe` command for transcribing audio/video and importing
  existing transcripts (SRT, VTT, OpenAI Whisper API JSON, whisper.cpp JSON)
- Add transcript format normalizer (normalize.ts) with auto-detection and
  conversion to standard [{text, start, end}] word arrays
- Upgrade default whisper model from base.en to small.en for better accuracy
- Add --model and --language flags to both `transcribe` and `init` commands
- Extract shared patchCaptionHtml() to eliminate duplication between init.ts
  and transcribe.ts (init.ts reduced by ~55 lines)
- Add 3 caption lint rules: caption_exit_missing_hard_kill,
  caption_text_overflow_risk, caption_container_relative_position
- Update captions skill with model guide, format docs, music guidance,
  text overflow prevention, caption exit guarantee pattern
- Expand captions skill trigger to cover lyrics, karaoke, lyric videos

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(cli): add transcribe command and --model/--language flags to CLI docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): fix blank template lint issues

- blank/index.html: remove data-start from video (was nested in timed parent),
  add class="clip" for initial hidden state
- blank/captions.html: add max-width + overflow:hidden to prevent text clipping,
  add tl.set hard kill after exit tween to prevent stuck captions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add lint-after-edit rule to repo and project CLAUDE.md

Agents must run `npx hyperframes lint` after editing compositions.
Also expand captions skill description in project template.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format _shared/CLAUDE.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 18:58:06 -07:00
Miguel ÁngelandClaude Opus 4.6 d52ea88df5 fix(cli): make blank template a full HTML document 
The blank template was a bare `<div>` fragment without `<!DOCTYPE>`,
`<head>`, or `<body>`. This caused:
- Blank preview (bundler/runtime can't initialize from a fragment)
- "Failed to run lint" (parsing errors on the malformed document)

All other templates (swiss-grid, vignelli, warm-grain, play-mode) are
proper HTML documents — blank was the only outlier.

Also improves lint error reporting to show the actual error message
instead of the generic "Failed to run lint."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 02:50:53 -04:00
Vance IngallsandClaude Opus 4.6 f4367d5726 feat(cli): add whisper transcription and template improvements (#53) 
* feat(cli): add whisper transcription to init flow

New modules:
- whisper/manager.ts: download/cache whisper.cpp binary + model
  (~/.cache/hyperframes/whisper/)
- whisper/transcribe.ts: extract audio, run whisper, save transcript.json

Init flow changes:
- "Got a video or audio file?" now accepts audio-only files (mp3, wav, m4a)
- "Generate captions from audio?" prompt after file selection
- Transcription produces transcript.json in project root
- Graceful fallback if whisper/ffmpeg unavailable

Supports: macOS ARM64/x86, Linux x86_64. Downloads whisper.cpp v1.7.3
from GitHub releases and ggml-base.en model from Hugging Face.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): use brew/system whisper instead of downloading binaries

whisper.cpp doesn't ship pre-built macOS/Linux CLI binaries.
Use brew install whisper-cpp on macOS (auto-installs if brew available),
system PATH lookup otherwise. Model still downloaded from Hugging Face.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): simplify whisper install — detect or instruct, don't build

Remove build-from-source complexity. If whisper-cpp is found on PATH,
use it. If not, show install instructions instead of blocking:

  "To generate captions, install whisper-cpp: brew install whisper-cpp"

The transcription prompt only appears when whisper is available.
When it's not, the user sees the install command and can re-run init.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): auto-install whisper via brew or build from source

ensureWhisper() now tries 4 strategies in order:
1. System PATH (whisper-cli or whisper already installed)
2. Homebrew (macOS: brew install whisper-cpp)
3. Build from source (git clone + cmake, ~30-60s)
4. Show install instructions as last resort

Init flow always asks "Generate captions?" — whisper is installed
automatically in the background if needed. No user intervention
required on macOS with Xcode CLI tools or any system with git+cmake.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add window.__timelines guard to all templates

The studio bundler doesn't always initialize window.__timelines
before template scripts run, causing "Cannot set properties of
undefined" errors. Add defensive guard to every template.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): patch template captions with actual transcript data

After scaffolding, if transcript.json exists, replace the hardcoded
word array in the template's captions composition with the real
transcript data. The template's caption animation and styling are
preserved — only the word data changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): show install notice when whisper needs to be installed

When whisper-cpp isn't found, show an info message before the spinner:
"whisper-cpp not found — installing automatically..."
Then the spinner shows "Installing whisper-cpp (this may take a moment)..."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add muted and playsinline to all template video elements

The framework requires video elements to have muted and playsinline
attributes. All four templates were missing these, causing video
to not play in the studio preview.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): flat asset structure + separate audio tracks in templates

Assets: video, images, fonts all go at project root (not assets/ or
fonts/ subdirectories). The studio preview can't resolve relative
paths from subdirectories due to the /preview URL suffix.

Audio: added <audio> elements alongside muted <video> in all 4
templates so the video's audio plays back. The framework requires
muted video + separate audio element.

Removed assets/ and fonts/ directory creation from scaffoldProject.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): inject base tag for asset resolution in preview

The preview iframe serves bundled HTML from /api/projects/:id/preview
but relative asset paths (video.mp4, font.woff2) resolve to the wrong
URL without a <base> tag. Now injects <base href="/api/projects/:id/preview/">
so relative paths route through the static asset handler.

Also adds proper MIME types for video, audio, image, and font files
served from the preview asset route.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): serve HyperFrames runtime in dev mode

The preview runtime script had an empty src — the framework never
loaded, so video playback and clip lifecycle didn't work.

Now auto-detects packages/cli/dist/hyperframe-runtime.js and serves
it at /api/runtime.js. No env var needed in dev mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): filter whisper special tokens from transcript

Use --output-json instead of --output-json-full to avoid special
tokens like [_TT_485] and [BLANK_AUDIO]. Also filter remaining
bracket tokens when building the word array for captions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): use --output-json-full for word-level timestamps

--output-json only produces segment-level timing (no tokens).
--output-json-full is required for word-level timestamps that
the captions template needs. Special tokens are filtered out
by the patchTranscript function.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): patch template durations to match uploaded video

Templates now use __VIDEO_DURATION__ placeholder that gets replaced
with the actual probed video duration. All data-duration values on
the root composition, video, audio, and caption clips are updated.

Without a video, defaults to 10 seconds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): merge punctuation tokens with preceding word

Whisper outputs punctuation (. , ! ?) as separate tokens. These
appeared as standalone words in captions, sometimes in the wrong
group. Now merged with the preceding word during transcript
normalization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): match both TRANSCRIPT and script variable names in templates

Three templates use `const TRANSCRIPT = [...]` while warm-grain uses
`const script = [...]`. The patchTranscript function now matches both.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): security and template fixes

- Replace shell injection risk (execSync rm) with unlinkSync in transcribe.ts
- Add GIT_TERMINAL_PROMPT=0 to whisper buildFromSource git clone
- Fix hardcoded data-duration="18" in warm-grain captions template
- Add data-start="0" to root compositions in swiss-grid, vignelli, warm-grain
- Add data-start="0" to warm-grain grain-overlay composition
- Deduplicate hasFFmpeg: remove from init.ts, import from whisper/manager.ts
- Add my-video/ and packages/studio/data/ to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): format warm-grain captions and fix TS nullability errors

- Format warm-grain/compositions/captions.html
- Add optional chaining on token.offsets (may be undefined)
- Use intermediate variable for lastWord to satisfy TS strict checks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add blank template option, smart defaults for video vs audio

- Blank template: minimal scaffolding (root composition, video, audio,
  GSAP timeline) with __VIDEO_SRC__ and __VIDEO_DURATION__ placeholders
- Template defaults: video uploads default to "blank" (user brings
  their own content), audio-only defaults to "warm-grain" (motion
  graphics template since there's no video to show)
- Audio-only projects now tracked with isAudioOnly flag

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): address whisper review feedback

- Clean stale builds: if BUILD_DIR exists but no binary, nuke and retry
- Build failures clean up BUILD_DIR so next attempt starts fresh
- patchTranscript regex scoped within <script> blocks to prevent
  matching across block boundaries
- Removed hardcoded model size hint (~148MB)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add missing rmSync import to whisper manager

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove test project and lock file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): address review items 7-12 — execFileSync, build diagnostics, WAV verification

- manager.ts: replace all execSync with execFileSync to prevent command injection
- manager.ts: capture cmake stderr and include in build failure error message
- transcribe.ts: verify WAV is 16kHz mono via ffprobe before passing to whisper
- init.ts: replace fragile JSON formatting with JSON.stringify(words, null, 2)
- init.ts: fix default duration from "10" to "5" matching DEFAULT_META
- init.ts: add probeAudioDuration() and --audio/--skip-transcribe flags
- init.ts: extract finalizeProject() to reduce code path duplication
- init.ts: wire transcription into non-interactive path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-26 11:05:15 -07:00