mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-05 00:56:23 +00:00
feat(media-use): v2 media OS core (resolve cascade, providers, local generation, telemetry) + retire hyperframes-media
This commit is contained in:
Miguel Angel Simon Sierra
@@ -96,24 +96,24 @@ Never use `position: absolute; top: Npx` on a content container — it overflows
|
||||
|
||||
## Build — delegate to the domain skills
|
||||
|
||||
This maps the skill's full surface (see the `description`) to its references — non-exhaustive; when an intent isn't listed, route through `hyperframes-creative` (look/concept), `hyperframes-animation` (motion), `hyperframes-core` (contract), `hyperframes-media` (audio/captions). **The first row is ADDITIVE — read it AND your intent row, not one or the other.**
|
||||
This maps the skill's full surface (see the `description`) to its references — non-exhaustive; when an intent isn't listed, route through `hyperframes-creative` (look/concept), `hyperframes-animation` (motion), `hyperframes-core` (contract), `media-use` (audio/captions). **The first row is ADDITIVE — read it AND your intent row, not one or the other.**
|
||||
|
||||
| Building… | Read first (in order) |
|
||||
| --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **ALWAYS — every non-trivial piece, on top of your intent row below** | `hyperframes-creative/references/house-style.md` + `references/video-composition.md` (also gated in Step 1 / HARD-GATE; the "produced, not generated" foreground detailing) |
|
||||
| **Kinetic typography / text-forward** | `hyperframes-animation/techniques.md` (kinetic type) + `adapters/gsap-easing-and-stagger.md` + `rules/kinetic-beat-slam.md` |
|
||||
| **Title card / lower-third / overlay / PiP / text-behind-subject** | `hyperframes-creative/references/composition-patterns.md` + (for the centered/sized frame) `hyperframes-core` → "Root must be sized" |
|
||||
| **Logo / brand-mark reveal** | `hyperframes-animation/rules/svg-path-draw.md` (draw-on) + `rules/3d-text-depth-layers.md` + `rules/scale-swap-transition.md` |
|
||||
| **Data / stats / numbers** | `hyperframes-animation/rules/counting-dynamic-scale.md` + `rules/stat-bars-and-fills.md` + `hyperframes-creative/references/data-in-motion.md` |
|
||||
| **Product / app / UI demo** | `hyperframes-animation/rules/3d-page-scroll.md` + `rules/cursor-click-ripple.md` + `rules/press-release-spring.md` |
|
||||
| **Audio-reactive / music-driven** | `hyperframes-creative/references/audio-reactive.md` (pre-extract bands; map to motion) |
|
||||
| **Narrated / voiceover / music / SFX / captions** | `hyperframes-media` → the shared audio engine `scripts/audio.mjs` (one call = TTS + BGM + SFX → `audio_meta.json`); caption authoring + asset placement via `hyperframes-core`. See **Audio** below. |
|
||||
| **Multi-scene / transitions** | `hyperframes-animation/transitions/overview.md` **then** `transitions/catalog.md` (you are not done after the overview — the GSAP recipe is in the catalog) |
|
||||
| **Modular / sub-compositions** | `hyperframes-core/references/composition-patterns.md` + `references/sub-compositions.md` |
|
||||
| Building… | Read first (in order) |
|
||||
| --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **ALWAYS — every non-trivial piece, on top of your intent row below** | `hyperframes-creative/references/house-style.md` + `references/video-composition.md` (also gated in Step 1 / HARD-GATE; the "produced, not generated" foreground detailing) |
|
||||
| **Kinetic typography / text-forward** | `hyperframes-animation/techniques.md` (kinetic type) + `adapters/gsap-easing-and-stagger.md` + `rules/kinetic-beat-slam.md` |
|
||||
| **Title card / lower-third / overlay / PiP / text-behind-subject** | `hyperframes-creative/references/composition-patterns.md` + (for the centered/sized frame) `hyperframes-core` → "Root must be sized" |
|
||||
| **Logo / brand-mark reveal** | `hyperframes-animation/rules/svg-path-draw.md` (draw-on) + `rules/3d-text-depth-layers.md` + `rules/scale-swap-transition.md` |
|
||||
| **Data / stats / numbers** | `hyperframes-animation/rules/counting-dynamic-scale.md` + `rules/stat-bars-and-fills.md` + `hyperframes-creative/references/data-in-motion.md` |
|
||||
| **Product / app / UI demo** | `hyperframes-animation/rules/3d-page-scroll.md` + `rules/cursor-click-ripple.md` + `rules/press-release-spring.md` |
|
||||
| **Audio-reactive / music-driven** | `hyperframes-creative/references/audio-reactive.md` (pre-extract bands; map to motion) |
|
||||
| **Narrated / voiceover / music / SFX / captions** | `media-use` → the shared audio engine `scripts/audio.mjs` (one call = TTS + BGM + SFX → `audio_meta.json`); caption authoring + asset placement via `hyperframes-core`. See **Audio** below. |
|
||||
| **Multi-scene / transitions** | `hyperframes-animation/transitions/overview.md` **then** `transitions/catalog.md` (you are not done after the overview — the GSAP recipe is in the catalog) |
|
||||
| **Modular / sub-compositions** | `hyperframes-core/references/composition-patterns.md` + `references/sub-compositions.md` |
|
||||
|
||||
### Audio: one engine (TTS · BGM · SFX)
|
||||
|
||||
Only when the piece calls for it (per "build exactly what was asked" — no ambient music on a title card). Don't hand-roll TTS or vendor a copy: write a neutral `audio_request.json` and call the shared engine in `hyperframes-media`. It auto-degrades on one switch — HeyGen credential present → HeyGen TTS + music/SFX **retrieval**; absent → ElevenLabs/Kokoro TTS, Lyria/MusicGen BGM **generation**, and the bundled SFX library. Full flag list + request/meta schema: the header comment of `hyperframes-media/scripts/audio.mjs`.
|
||||
Only when the piece calls for it (per "build exactly what was asked" — no ambient music on a title card). Don't hand-roll TTS or vendor a copy: write a neutral `audio_request.json` and call the shared engine in `media-use`. It auto-degrades on one switch — HeyGen credential present → HeyGen TTS + music/SFX **retrieval**; absent → ElevenLabs/Kokoro TTS, Lyria/MusicGen BGM **generation**, and the bundled SFX library. Full flag list + request/meta schema: the header comment of `media-use/audio/scripts/audio.mjs`.
|
||||
|
||||
```jsonc
|
||||
// audio_request.json — one line per narrated segment; `id` is yours (joins audio_meta back)
|
||||
@@ -127,11 +127,11 @@ Only when the piece calls for it (per "build exactly what was asked" — no ambi
|
||||
```
|
||||
|
||||
```bash
|
||||
# <MEDIA_DIR> = the installed hyperframes-media skill dir (sibling of this skill)
|
||||
# <MEDIA_DIR> = the installed media-use skill dir (sibling of this skill)
|
||||
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json
|
||||
```
|
||||
|
||||
Then read `audio_meta.json`: mount each `voices[].path` + (`bgm.path`, `sfx[]`) as `<audio>` tracks and use `voices[].words` for captions, all per `hyperframes-core` (audio tracks + caption authoring). If BGM took the generate path (`bgm_pending: true`), run `hyperframes-media/scripts/wait-bgm.mjs` before final render.
|
||||
Then read `audio_meta.json`: mount each `voices[].path` + (`bgm.path`, `sfx[]`) as `<audio>` tracks and use `voices[].words` for captions, all per `hyperframes-core` (audio tracks + caption authoring). If BGM took the generate path (`bgm_pending: true`), run `media-use/audio/scripts/wait-bgm.mjs` before final render.
|
||||
|
||||
## Output checklist → `hyperframes-cli`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user