ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-03 12:54:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,127 Commits 855 Branches 404 Tags
2e9a33ca719a2b28bd8bb51083df32b8f74ca437
Commit Graph
4 Commits
Author SHA1 Message Date
Miguel Ángel 2e9a33ca71 fix(hyperframes-media): cap TTS synthesis concurrency instead of firing every line at once (#1862) 
Two independent post-release feedback reports of the same mechanism from
two different skills (both delegate to this one shared engine): audio.mjs
fired every line's Kokoro TTS + whisper-transcribe subprocess concurrently
via a bare Promise.all with no cap.

- One OOM'd 12/13 lines on a resource-constrained laptop (32GB total, ~7GB
  free), requiring a manual patch to a sequential for-loop.
- The other saw 7/8 lines fail on first run, then pass on retry once the
  model was cached — concurrent cold-start model loads overwhelming the
  machine, not a real synthesis failure.

Kokoro/Whisper each load their own local model per subprocess, so firing
every line at once multiplies that cost by the line count. Extracted the
concurrency cap into lib/concurrency.mjs (audio.mjs is a script — it runs
CLI/exit side effects on import, so it can't be unit-tested directly;
the cap is small enough to pull out and test in isolation). Default 4,
overridable via HYPERFRAMES_TTS_CONCURRENCY, floored at 1 (matching one
report's own manual workaround).

hyperframes-media/scripts/audio.mjs is the single canonical engine per its
own header comment; product-launch-video, faceless-explainer, and
pr-to-video each carry a thin wrapper that spawns this file as a
subprocess (confirmed via their DEFAULT_ENGINE path), so this one fix
covers all four skills without touching the other three.

Tests: 4 new cases for mapWithConcurrency (order preserved regardless of
completion order, cap actually enforced, limit > item count doesn't hang,
empty input). Full skills test suite (514 tests) shows no new failures —
the 444 pre-existing failures are environment-dependent and reproduce
identically on unmodified main.
 2026-07-02 17:46:18 -07:00
WaterrrForeverandClaude Opus 4.8 05af482f22 feat(skills): product-launch-video skill + consolidate motion knowledge into hyperframes-animation (#1745) 
* feat(skills): product-launch-video + consolidate motion knowledge into hyperframes-animation

- Add the product-launch-video skill: shot-sequence architecture where each
  visual frame is a time-coded shot sequence picked from a blueprint menu and
  paced to the voiceover (anti-PowerPoint). Includes the frame-worker sub-agent,
  story/visual/motion-design references, and audio/captions/transitions/
  stage-assets/assemble-index scripts.
- Consolidate motion knowledge in hyperframes-animation as the single source of
  truth: promote the updated atomic rules (31 -> 36) and rename product-launch-
  video's archetypes into hyperframes-animation blueprints (13 -> 15, replacing
  the old set). product-launch-video, faceless-explainer, and pr-to-video now
  reference them via ../hyperframes-animation/{rules-index,blueprints-index}.md
  and the rules/blueprints dirs. Fixes the discrete-text-sequence broken links;
  blueprints no longer ship per-id runnable examples, so example references in
  the consumers were dropped.
- Default HeyGen TTS voice to Marcia (deterministic; was the API's first English
  voice, which drifts on catalog re-sort). Override with --voice.
- assemble-index pre-assembly frame guards: auto-repair a sub-comp root missing
  canvas dims; hard-fail on <video>/<audio> inside a sub-comp; hard-fail on a
  timed non-root element missing class="clip" or overlapping same-track clips.
- Lint/CLI: lint media inside sub-compositions as an error; stop false-positive
  caption layout/lint findings; contrast/layout-audit skip elements hidden by an
  invisible ancestor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): clear CodeQL alerts in assemble-index.mjs

- script/style blanking regex now matches closing tags with trailing
  whitespace (</script >, </style >) — js/bad-tag-filter (high).
- drop the existsSync precheck before reading/repairing a frame file; read
  directly and handle ENOENT, removing the check->write TOCTOU window —
  js/file-system-race (high).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-27 02:51:35 +08:00
WaterrrForeverandClaude Opus 4.8 54cab331d0 feat(cli): shared TTS/BGM auth preflight + caption and skill-workflow fixes (#1697) 
* fix: handle caption skin workflow

* docs(skills): simplify the finalize step across video workflows

- Drop --strict-layout; all skills use plain `hyperframes inspect`
- Add the caption text_box_overflow false-positive note to faceless-explainer
- On a failed check, the orchestrator makes the cheapest safe edit itself
  (no worker re-dispatch / Step 3 backtrack language)
- Snapshot: glance at the stitched contact-sheet.jpg and move on

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): onboarding-first `auth status` + shared TTS/BGM preflight

When no HeyGen credential is configured, `hyperframes auth status` now
prints registration-first guidance instead of a terse error:

- Interactive / agent-driven sessions get sign-in guidance led by
  `hyperframes auth login` (the OAuth step that also creates an account
  and is shared with heygen-cli), and never steer users to a per-repo
  `.env`. CI / non-interactive runs get a terse note. Exit 1 is kept so
  the "am I logged in?" `$?` contract still holds.
- It probes which local engine voice/music will fall back to (Kokoro /
  MusicGen, mirroring the skill resolution order) and whether their
  Python deps are installed, with a pip hint when missing. `--json`
  exposes `recommended_action` + `offline_engines` for skills to branch.
- `doctor` gains matching "TTS (Kokoro)" / "BGM (MusicGen)" checks via
  the same shared probe (findPython/hasPythonModules extracted to
  tts/python.ts; provider resolution in audio/providers.ts).

Every TTS/BGM workflow now relays this at Step 0 (setup) instead of
improvising its own "missing key" prompt: pr-to-video, product-launch-
video, faceless-explainer, website-to-video, music-to-video. The
canonical behavior + key-priority table live once in hyperframes-media.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(pr-to-video): scale recommended video length to PR change size

Step 0 led with a fixed ~60-90s length default. Now the recommended
length is derived from the PR's diff stat (lines added+deleted, nudged
by file count) on a tier scale (trivial ~20-40s → large ~110-180s, hard
cap ~3 min), reusing the same PR peek already done to infer the angle.
The agent states the basis when proposing it, and a huge PR with one
headline change still stays tight. User can always override.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(captions): embed brand fonts whose files use separators

brandFontFaces() matched font files by stripping only whitespace, so an
underscore/hyphen-named file (TT_Norms_Pro_Bold.woff2) never matched the
family key "ttnormspro" — captions shipped with no @font-face, the
font_family_without_font_face bug. Now both family and filename normalize
away all non-alphanumerics; families match longest-key-first so a parent
family can't swallow a more specific one's files (TT Norms Pro vs Mono);
each file is claimed once; "demibold" ranks before "bold"; and when
nothing matches it warns loudly at build time instead of returning "".

Also: parseFonts() falls back to h1/h2/title/hero display roles, and the
frame-worker + caption authoring docs spell out that only shipped font
files render — no system CJK/Devanagari families on the headless renderer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hyperframes-media): enforce sign-in preflight on standalone BGM/TTS

A one-off "generate me a BGM" request went straight to local MusicGen
without recommending sign-in: bgm.md/tts.md framed the no-credential path
as an automatic fallback, so the generation path bypassed the Preflight
stop, and the preflight used a bare `hyperframes auth status` that isn't
on PATH in a fresh `npx skills` project.

- Preflight now applies to one-off generation as well as workflows, uses
  `npx hyperframes auth status`, and says: if the CLI can't run, still
  recommend signing in and STOP — never treat "no credential" as a silent
  green light for local generation.
- bgm.md and tts.md point at the Preflight before generating, reframing
  local generation as the fallback the user opts into, not a default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(auth): add Authentication & API keys guide

Document signing in, the keys each capability (voice, music, capture)
uses, their resolution priority, and the fully local fallback. Add the
guide to the nav and cross-link it from the cloud deploy note and the
CLI env-var reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): strip HTML comments in a fixpoint loop (CodeQL)

Single-pass <!-- --> removal can re-form a complete comment from
adjacent markers (e.g. `<<!-- -->!-- ... -->`), letting a decoy
<template> survive and hijack the template-boundary match. Loop to a
fixpoint, mirroring the captions.mjs precedent; add a regression test
that fails on single-pass (2 root findings) and passes on the loop.

Also wrap the build-frame.mjs node:fs imports to satisfy oxfmt — the
new copyFileSync import pushed the line past the width limit, which
was the sole cause of the Format / Preflight CI failures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): strip HTML comments with a linear scan (CodeQL ReDoS)

The fixpoint loop still ran a /<!--[\s\S]*?-->/ regex per pass, which
backtracks O(n^2) on inputs with many unterminated "<!--" — CodeQL
js/polynomial-redos (high). Looping the same regex (the prescribed
fix) never addressed this; only the regex itself does.

Replace it with an indexOf-based linear strip in utils.ts
(stripHtmlComments), kept in a fixpoint loop so markers that re-form
when a comment is removed are still stripped. 200k unterminated
"<!--" now strips in ~3ms instead of quadratic time; behavior is
otherwise unchanged — unterminated comments are kept verbatim, as the
old regex left them. The re-forming regression test still guards it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): make TTS/BGM sign-in guidance accurate and runnable

From team review of the not-signed-in onboarding:

- OAuth is a `hyperframes auth login` feature only. The separate `heygen`
  CLI is API-key-only — `heygen auth login` stores a pasted key, it is not
  OAuth and does not create an account. Stop presenting the two CLIs as the
  same OAuth/sign-up step.
- Use `npx hyperframes` in every imperative and runtime hint. Bare
  `hyperframes` is not on PATH on a fresh machine (command not found); only
  `npx hyperframes` is guaranteed. Also updates the JSON recommended_action.
- Drop `heygen auth login` from the terminal/skill onboarding: it needs its
  own install and there is no `npx heygen`, so it was a command-not-found
  trap. The shared-credential fact stays in the reference docs.

Covers the `auth status` guidance + tests, the Authentication docs, the
shared hyperframes-media preflight (SKILL, requirements, tts, error hints),
and the `npx hyperframes auth status` preflight in every TTS/BGM workflow
(pr-to-video, product-launch-video, faceless-explainer, website-to-video,
music-to-video).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-24 23:20:39 +08:00
WaterrrForeverandClaude Opus 4.8 d0f0ec29e7 feat(skills): frame-preset library + shared audio engine (foundation) (#1632) 
* feat(hyperframes-creative): add frame-preset library

Add a library of ready-made visual frame presets (claude, biennale-yellow,
blockframe, blue-professional, bold-poster, broadside, capsule, cartesian,
cobalt-grid, coral, creative-mode, daisy-days, editorial-forest, …), each with
a FRAME.md spec, a frame-showcase.html, and a per-preset caption-skin.html.
Registered in the creative design-spec so workflows can remix a preset onto
brand tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(hyperframes-media): shared TTS/BGM/SFX audio engine

Add a shared audio engine under hyperframes-media (scripts/audio.mjs + lib/
tts.mjs, bgm.mjs, sfx.mjs, heygen.mjs) plus a bundled SFX pack and manifest.
Workflows resolve this engine by path (../../hyperframes-media/scripts/
audio.mjs) for text-to-speech, background music, and sound effects, so audio
is authored once and reused across skills instead of duplicated per workflow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(skills): gate render on user review; refresh router, core, general-video

- hyperframes-cli: render is now user-gated — preview opens Studio (the timeline
  editor where the user can hand-edit anything, not just watch); never
  auto-render once checks pass, pause at preview and render only after approval.
- hyperframes (router): tighten the entry SKILL.md description + routing.
- hyperframes-core: rewrite SKILL.md and add script-format.md + storyboard-format.md
  references for the script-driven authoring architecture.
- general-video: tidy the fallback-workflow description and routing table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(hyperframes-creative): reformat frame-preset showcase HTML

Run the HTML formatter over the frame-showcase.html files (indentation,
self-closing void tags, one CSS declaration per line). Formatting only — no
content or markup changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(hyperframes-media): correct wait-bgm field mapping and guard credential parse

Two correctness fixes from review (#1632):

- wait-bgm.mjs read audioMeta.bgm_path / audioMeta.bgm_enabled, but audio.mjs
  writes the path nested as bgm.path and the flag as bgm_pending. The detached
  generate path (Lyria/MusicGen) therefore always saw an empty path and exited
  status: disabled, silently dropping the music track even while generation was
  running. Read audioMeta.bgm?.path and gate on bgm_pending.
- heygenCredential() had an unguarded JSON.parse despite documenting that it
  never throws — a malformed ~/.heygen credentials file crashed the engine at
  startup instead of degrading to no-credential. Wrap the parse and return null.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(hyperframes): add router tag to entry skill metadata

Fold the router metadata tag into the foundation rewrite of the entry SKILL.md.
This file is owned by this PR (the full router rewrite); keeping the tag tweak
here — instead of a separate edit on the pre-rewrite version in another PR —
avoids a guaranteed merge conflict between the two.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-22 22:49:10 +08:00