ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-05 10:14:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,173 Commits 885 Branches 406 Tags
b34e623bf8c5fb2ae9608edf1908c466584d627e
Commit Graph
9 Commits
Author SHA1 Message Date
Miguel Ángel b34e623bf8 fix(hyperframes-media): sfx offline path skips (loudly) a missing bundled file 2026-07-05 11:46:30 -07:00
Miguel Ángel 942080fa8e fix(hyperframes-media): install MusicGen deps via python3 -m pip, not bare pip (#1894) 
pipInstall() spawned a bare "pip" binary. Many Homebrew/system Python
installs expose only python3/pip3 on PATH, so the spawn silently ENOENTs
and the documented "auto-installed on demand" local MusicGen path never
actually installs anything - the failure is invisible since spawnSync's
status just comes back non-zero like any other install failure.

Switched to `python3 -m pip install`, matching this same file's own
pyOk() convention of always invoking python3 explicitly. This also
closes a second latent bug: a bare pip/pip3 could resolve to a different
Python installation than the python3 binary pyOk() probes against if
more than one is on PATH, so `-m pip` guarantees the install lands in the
exact interpreter being checked.

Manually verified `python3 -m pip --version` succeeds in this
environment. No automated test added - this is a literal command-array
swap with no new branching logic, and spawnSync is a named import from
node:child_process with no clean mocking path available without a
larger refactor disproportionate to the fix's size.
 2026-07-04 14:18:56 -07:00
Miguel Ángel 3388635f1d fix(hyperframes-media): resolve python3 -> python/py on Windows (#1922) 
The audio engine shells out to `python3` for ElevenLabs TTS
(tts.mjs) and the local Lyria/MusicGen BGM paths (bgm.mjs). `python3`
is correct on macOS/Linux, but a standard python.org install on
Windows only creates `python.exe` plus the `py` launcher -- there is
no `python3.exe` (only the Microsoft Store build adds one). So every
`spawn("python3", ...)`/`spawnSync("python3", ...)` ENOENTs on a normal
Windows Python setup, silently disabling all Python-backed audio
features until the user hand-creates a `python3.exe` shim (reported: a
user copied python.exe to python3.exe to work around it, and separately
another had to target a python3 stub specifically).

Fix: a shared lib/python.mjs resolver probes the platform's candidates
in order and returns the argv prefix that actually launches Python 3 --
`["python3"]` / `["python"]` / `["py", "-3"]` on win32, `["python3"]`
then `["python"]` elsewhere -- resolved once per process. All direct
`python3` spawn sites in tts.mjs (elevenlabs probe + synth) and bgm.mjs
(pyOk probe, Lyria recipe, MusicGen script) now route through it. On
macOS/Linux `python3` still wins first, so behavior there is unchanged;
if nothing probes OK the resolver falls back to `python3` so the spawn
fails loudly exactly as before, never worse.

Scope: only the direct python3 invocations. bgm.mjs's pipInstall()
still shells `pip` -- switching that to `<python> -m pip` is the
separate concern of the open PR #1894 (draft); noting the overlap so
the two don't collide. Windows-specific whisper.cpp-vs-openai-whisper
detection and the npm_execpath/npx spawn issue from the same report are
distinct root causes, not addressed here.

Test: python.test.mjs (node:test) covers every platform/probe branch
with an injected probe -- no real interpreter spawned: non-win32 picks
python3; win32 prefers python3, falls back to python, then to `py -3`;
the py launcher is probed as `py -3 --version`; nothing-runs falls back
to the canonical python3; and pythonInvocation keeps the launcher's -3
ahead of caller args. Existing tts.spawn.test.mjs still passes (6/6).
 2026-07-04 14:08:29 -07:00
Miguel Ángel 3b67918242 fix(hyperframes-media): npx spawn without shell:true fails silently on Windows (#1845) 
* fix(hyperframes-media): npx spawn without shell:true fails silently on Windows

Two independent user reports of Kokoro TTS silently failing on Windows,
both naming the same site: lib/tts.mjs synthesizeOne() spawns "npx" via
plain spawn(cmd, args). On Windows npx resolves to npx.cmd, which Node's
spawn() cannot exec without shell:true — it fails ENOENT, and spawnP's
"error" listener turns that into a plain ok:false ("TTS failed") with no
indication of the real cause.

Scope the fix to the npx call specifically (python3/ffmpeg are real
binaries and don't need it), with the platform/spawn function injectable
so the win32 branch is testable without mocking node:child_process (its
ESM exports are non-configurable, so mock.method can't patch it) or the
real process.platform.

* fix(hyperframes-media): avoid shell true for windows npx
 2026-07-02 18:04:14 -07:00
Miguel Ángel e2b43583fc fix(hyperframes-media): fall back to ffmpeg when ffprobe is missing (#1877) 
* fix(hyperframes-media): fall back to ffmpeg when ffprobe is missing

ffprobeDuration() returned NaN whenever the ffprobe spawn failed for any
reason, conflating "ffprobe binary not installed" with "file is corrupt".
Some ffmpeg-only distributions (common in curated Windows installs) ship
ffmpeg.exe without ffprobe.exe, so every TTS line hit the missing-binary
case and audio.mjs read the NaN as a bad WAV, silently dropping an already
successfully synthesized line. Now falls back to parsing ffmpeg's own
`Duration:` stderr banner when ffprobe specifically ENOENTs, and only
returns NaN when the file itself can't be probed by either tool.

* fix(hyperframes-media): update skills manifest for ffprobe fallback
 2026-07-02 17:57:24 -07:00
Miguel Ángel 2e9a33ca71 fix(hyperframes-media): cap TTS synthesis concurrency instead of firing every line at once (#1862) 
Two independent post-release feedback reports of the same mechanism from
two different skills (both delegate to this one shared engine): audio.mjs
fired every line's Kokoro TTS + whisper-transcribe subprocess concurrently
via a bare Promise.all with no cap.

- One OOM'd 12/13 lines on a resource-constrained laptop (32GB total, ~7GB
  free), requiring a manual patch to a sequential for-loop.
- The other saw 7/8 lines fail on first run, then pass on retry once the
  model was cached — concurrent cold-start model loads overwhelming the
  machine, not a real synthesis failure.

Kokoro/Whisper each load their own local model per subprocess, so firing
every line at once multiplies that cost by the line count. Extracted the
concurrency cap into lib/concurrency.mjs (audio.mjs is a script — it runs
CLI/exit side effects on import, so it can't be unit-tested directly;
the cap is small enough to pull out and test in isolation). Default 4,
overridable via HYPERFRAMES_TTS_CONCURRENCY, floored at 1 (matching one
report's own manual workaround).

hyperframes-media/scripts/audio.mjs is the single canonical engine per its
own header comment; product-launch-video, faceless-explainer, and
pr-to-video each carry a thin wrapper that spawns this file as a
subprocess (confirmed via their DEFAULT_ENGINE path), so this one fix
covers all four skills without touching the other three.

Tests: 4 new cases for mapWithConcurrency (order preserved regardless of
completion order, cap actually enforced, limit > item count doesn't hang,
empty input). Full skills test suite (514 tests) shows no new failures —
the 444 pre-existing failures are environment-dependent and reproduce
identically on unmodified main.
 2026-07-02 17:46:18 -07:00
WaterrrForeverandClaude Opus 4.8 05af482f22 feat(skills): product-launch-video skill + consolidate motion knowledge into hyperframes-animation (#1745) 
* feat(skills): product-launch-video + consolidate motion knowledge into hyperframes-animation

- Add the product-launch-video skill: shot-sequence architecture where each
  visual frame is a time-coded shot sequence picked from a blueprint menu and
  paced to the voiceover (anti-PowerPoint). Includes the frame-worker sub-agent,
  story/visual/motion-design references, and audio/captions/transitions/
  stage-assets/assemble-index scripts.
- Consolidate motion knowledge in hyperframes-animation as the single source of
  truth: promote the updated atomic rules (31 -> 36) and rename product-launch-
  video's archetypes into hyperframes-animation blueprints (13 -> 15, replacing
  the old set). product-launch-video, faceless-explainer, and pr-to-video now
  reference them via ../hyperframes-animation/{rules-index,blueprints-index}.md
  and the rules/blueprints dirs. Fixes the discrete-text-sequence broken links;
  blueprints no longer ship per-id runnable examples, so example references in
  the consumers were dropped.
- Default HeyGen TTS voice to Marcia (deterministic; was the API's first English
  voice, which drifts on catalog re-sort). Override with --voice.
- assemble-index pre-assembly frame guards: auto-repair a sub-comp root missing
  canvas dims; hard-fail on <video>/<audio> inside a sub-comp; hard-fail on a
  timed non-root element missing class="clip" or overlapping same-track clips.
- Lint/CLI: lint media inside sub-compositions as an error; stop false-positive
  caption layout/lint findings; contrast/layout-audit skip elements hidden by an
  invisible ancestor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): clear CodeQL alerts in assemble-index.mjs

- script/style blanking regex now matches closing tags with trailing
  whitespace (</script >, </style >) — js/bad-tag-filter (high).
- drop the existsSync precheck before reading/repairing a frame file; read
  directly and handle ENOENT, removing the check->write TOCTOU window —
  js/file-system-race (high).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-27 02:51:35 +08:00
WaterrrForeverandClaude Opus 4.8 54cab331d0 feat(cli): shared TTS/BGM auth preflight + caption and skill-workflow fixes (#1697) 
* fix: handle caption skin workflow

* docs(skills): simplify the finalize step across video workflows

- Drop --strict-layout; all skills use plain `hyperframes inspect`
- Add the caption text_box_overflow false-positive note to faceless-explainer
- On a failed check, the orchestrator makes the cheapest safe edit itself
  (no worker re-dispatch / Step 3 backtrack language)
- Snapshot: glance at the stitched contact-sheet.jpg and move on

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): onboarding-first `auth status` + shared TTS/BGM preflight

When no HeyGen credential is configured, `hyperframes auth status` now
prints registration-first guidance instead of a terse error:

- Interactive / agent-driven sessions get sign-in guidance led by
  `hyperframes auth login` (the OAuth step that also creates an account
  and is shared with heygen-cli), and never steer users to a per-repo
  `.env`. CI / non-interactive runs get a terse note. Exit 1 is kept so
  the "am I logged in?" `$?` contract still holds.
- It probes which local engine voice/music will fall back to (Kokoro /
  MusicGen, mirroring the skill resolution order) and whether their
  Python deps are installed, with a pip hint when missing. `--json`
  exposes `recommended_action` + `offline_engines` for skills to branch.
- `doctor` gains matching "TTS (Kokoro)" / "BGM (MusicGen)" checks via
  the same shared probe (findPython/hasPythonModules extracted to
  tts/python.ts; provider resolution in audio/providers.ts).

Every TTS/BGM workflow now relays this at Step 0 (setup) instead of
improvising its own "missing key" prompt: pr-to-video, product-launch-
video, faceless-explainer, website-to-video, music-to-video. The
canonical behavior + key-priority table live once in hyperframes-media.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(pr-to-video): scale recommended video length to PR change size

Step 0 led with a fixed ~60-90s length default. Now the recommended
length is derived from the PR's diff stat (lines added+deleted, nudged
by file count) on a tier scale (trivial ~20-40s → large ~110-180s, hard
cap ~3 min), reusing the same PR peek already done to infer the angle.
The agent states the basis when proposing it, and a huge PR with one
headline change still stays tight. User can always override.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(captions): embed brand fonts whose files use separators

brandFontFaces() matched font files by stripping only whitespace, so an
underscore/hyphen-named file (TT_Norms_Pro_Bold.woff2) never matched the
family key "ttnormspro" — captions shipped with no @font-face, the
font_family_without_font_face bug. Now both family and filename normalize
away all non-alphanumerics; families match longest-key-first so a parent
family can't swallow a more specific one's files (TT Norms Pro vs Mono);
each file is claimed once; "demibold" ranks before "bold"; and when
nothing matches it warns loudly at build time instead of returning "".

Also: parseFonts() falls back to h1/h2/title/hero display roles, and the
frame-worker + caption authoring docs spell out that only shipped font
files render — no system CJK/Devanagari families on the headless renderer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hyperframes-media): enforce sign-in preflight on standalone BGM/TTS

A one-off "generate me a BGM" request went straight to local MusicGen
without recommending sign-in: bgm.md/tts.md framed the no-credential path
as an automatic fallback, so the generation path bypassed the Preflight
stop, and the preflight used a bare `hyperframes auth status` that isn't
on PATH in a fresh `npx skills` project.

- Preflight now applies to one-off generation as well as workflows, uses
  `npx hyperframes auth status`, and says: if the CLI can't run, still
  recommend signing in and STOP — never treat "no credential" as a silent
  green light for local generation.
- bgm.md and tts.md point at the Preflight before generating, reframing
  local generation as the fallback the user opts into, not a default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(auth): add Authentication & API keys guide

Document signing in, the keys each capability (voice, music, capture)
uses, their resolution priority, and the fully local fallback. Add the
guide to the nav and cross-link it from the cloud deploy note and the
CLI env-var reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): strip HTML comments in a fixpoint loop (CodeQL)

Single-pass <!-- --> removal can re-form a complete comment from
adjacent markers (e.g. `<<!-- -->!-- ... -->`), letting a decoy
<template> survive and hijack the template-boundary match. Loop to a
fixpoint, mirroring the captions.mjs precedent; add a regression test
that fails on single-pass (2 root findings) and passes on the loop.

Also wrap the build-frame.mjs node:fs imports to satisfy oxfmt — the
new copyFileSync import pushed the line past the width limit, which
was the sole cause of the Format / Preflight CI failures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): strip HTML comments with a linear scan (CodeQL ReDoS)

The fixpoint loop still ran a /<!--[\s\S]*?-->/ regex per pass, which
backtracks O(n^2) on inputs with many unterminated "<!--" — CodeQL
js/polynomial-redos (high). Looping the same regex (the prescribed
fix) never addressed this; only the regex itself does.

Replace it with an indexOf-based linear strip in utils.ts
(stripHtmlComments), kept in a fixpoint loop so markers that re-form
when a comment is removed are still stripped. 200k unterminated
"<!--" now strips in ~3ms instead of quadratic time; behavior is
otherwise unchanged — unterminated comments are kept verbatim, as the
old regex left them. The re-forming regression test still guards it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): make TTS/BGM sign-in guidance accurate and runnable

From team review of the not-signed-in onboarding:

- OAuth is a `hyperframes auth login` feature only. The separate `heygen`
  CLI is API-key-only — `heygen auth login` stores a pasted key, it is not
  OAuth and does not create an account. Stop presenting the two CLIs as the
  same OAuth/sign-up step.
- Use `npx hyperframes` in every imperative and runtime hint. Bare
  `hyperframes` is not on PATH on a fresh machine (command not found); only
  `npx hyperframes` is guaranteed. Also updates the JSON recommended_action.
- Drop `heygen auth login` from the terminal/skill onboarding: it needs its
  own install and there is no `npx heygen`, so it was a command-not-found
  trap. The shared-credential fact stays in the reference docs.

Covers the `auth status` guidance + tests, the Authentication docs, the
shared hyperframes-media preflight (SKILL, requirements, tts, error hints),
and the `npx hyperframes auth status` preflight in every TTS/BGM workflow
(pr-to-video, product-launch-video, faceless-explainer, website-to-video,
music-to-video).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-24 23:20:39 +08:00
WaterrrForeverandClaude Opus 4.8 d0f0ec29e7 feat(skills): frame-preset library + shared audio engine (foundation) (#1632) 
* feat(hyperframes-creative): add frame-preset library

Add a library of ready-made visual frame presets (claude, biennale-yellow,
blockframe, blue-professional, bold-poster, broadside, capsule, cartesian,
cobalt-grid, coral, creative-mode, daisy-days, editorial-forest, …), each with
a FRAME.md spec, a frame-showcase.html, and a per-preset caption-skin.html.
Registered in the creative design-spec so workflows can remix a preset onto
brand tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(hyperframes-media): shared TTS/BGM/SFX audio engine

Add a shared audio engine under hyperframes-media (scripts/audio.mjs + lib/
tts.mjs, bgm.mjs, sfx.mjs, heygen.mjs) plus a bundled SFX pack and manifest.
Workflows resolve this engine by path (../../hyperframes-media/scripts/
audio.mjs) for text-to-speech, background music, and sound effects, so audio
is authored once and reused across skills instead of duplicated per workflow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(skills): gate render on user review; refresh router, core, general-video

- hyperframes-cli: render is now user-gated — preview opens Studio (the timeline
  editor where the user can hand-edit anything, not just watch); never
  auto-render once checks pass, pause at preview and render only after approval.
- hyperframes (router): tighten the entry SKILL.md description + routing.
- hyperframes-core: rewrite SKILL.md and add script-format.md + storyboard-format.md
  references for the script-driven authoring architecture.
- general-video: tidy the fallback-workflow description and routing table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(hyperframes-creative): reformat frame-preset showcase HTML

Run the HTML formatter over the frame-showcase.html files (indentation,
self-closing void tags, one CSS declaration per line). Formatting only — no
content or markup changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hyperframes-media): correct wait-bgm field mapping and guard credential parse

Two correctness fixes from review (#1632):

- wait-bgm.mjs read audioMeta.bgm_path / audioMeta.bgm_enabled, but audio.mjs
  writes the path nested as bgm.path and the flag as bgm_pending. The detached
  generate path (Lyria/MusicGen) therefore always saw an empty path and exited
  status: disabled, silently dropping the music track even while generation was
  running. Read audioMeta.bgm?.path and gate on bgm_pending.
- heygenCredential() had an unguarded JSON.parse despite documenting that it
  never throws — a malformed ~/.heygen credentials file crashed the engine at
  startup instead of degrading to no-credential. Wrap the parse and return null.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(hyperframes): add router tag to entry skill metadata

Fold the router metadata tag into the foundation rewrite of the entry SKILL.md.
This file is owned by this PR (the full router rewrite); keeping the tag tweak
here — instead of a separate edit on the pre-rewrite version in another PR —
avoids a guaranteed merge conflict between the two.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-22 22:49:10 +08:00