mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 17:27:09 +00:00
feat(skills): sign-in and credits gate for media-use, draft reverse skill
- media-use: relay a HyperFrames-plan refusal (no plan, out of credits, sign-in not covered) as one line with the plans or credits link. Links are overridable via HYPERFRAMES_PLANS_URL / HYPERFRAMES_CREDITS_URL. - media-use SKILL.md: free / credits / local-only-when-asked columns, sign in with hyperframes auth, no silent local fallback, one budget up front, plan for the best result. - reverse-any-video: internal draft workflow skill. Work-in-progress branch, not for merge.
This commit is contained in:
Wenbo Zhu
@@ -18,13 +18,17 @@
|
||||
"files": 13
|
||||
},
|
||||
"media-use": {
|
||||
"hash": "938c00af915a46b5",
|
||||
"hash": "c049f659917e85ac",
|
||||
"files": 116
|
||||
},
|
||||
"remotion-to-hyperframes": {
|
||||
"hash": "37ed4536305b3c64",
|
||||
"files": 78
|
||||
},
|
||||
"reverse-any-video": {
|
||||
"hash": "f22cdd6b6eb671fc",
|
||||
"files": 1
|
||||
},
|
||||
"slideshow": {
|
||||
"hash": "c8a591e2aa7a6d86",
|
||||
"files": 2
|
||||
|
||||
+16
-16
@@ -15,27 +15,27 @@ npx hyperframes media-use resolve --type <type> --intent "<what you need>" --pro
|
||||
|
||||
It prints one line, `resolved <id> → <path> (<type>, <metadata>)`. List reusable assets first with `--candidates` and judge the fit yourself; reuse, ingest (`--from`) and adopt are in `references/resolve.md`.
|
||||
|
||||
## Two ways to get it
|
||||
## Where it comes from
|
||||
|
||||
| Need | HeyGen (best quality) | Free, no account |
|
||||
| --- | --- | --- |
|
||||
| Music (`bgm`) | Catalog of 10k+ tracks | Local MusicGen (`audio/references/bgm.md`) |
|
||||
| Sound effects (`sfx`) | | Openverse; 21 bundled offline |
|
||||
| Images, icons | Asset search, 75k+ | Local FLUX (mflux), if installed |
|
||||
| Logos (`logo`) | | theSVG, GitHub avatar, favicon; never redrawn |
|
||||
| Voiceover (`voice`) | HeyGen voices, voice cloning | `npx hyperframes tts` (Kokoro on device; Python 3.10+ with `pip install kokoro-onnx soundfile`) |
|
||||
| A presenter speaking | Avatar video (`--type video`) | Local LTX for generic clips, if installed |
|
||||
| Transcripts | | `npx hyperframes transcribe` (Parakeet or whisper) |
|
||||
| Cutouts | `heygen background-removal` | `npx hyperframes remove-background` |
|
||||
| Grades, LUTs, looks | | `npx hyperframes media-treatment` |
|
||||
| Need | Free | HyperFrames credits | Local or the person's own key (only when they ask) |
|
||||
| --- | --- | --- | --- |
|
||||
| Music (`bgm`) | HeyGen catalog of 10k+ tracks (signed in) | Generated track | Local MusicGen (`audio/references/bgm.md`) |
|
||||
| Sound effects (`sfx`) | Openverse; 21 bundled offline | | |
|
||||
| Images, icons | HeyGen asset search, 75k+ (signed in) | Generated images | Local FLUX (mflux), if installed |
|
||||
| Logos (`logo`) | theSVG, GitHub avatar, favicon; never redrawn | | |
|
||||
| Voiceover (`voice`) | HeyGen Voice Turbo (signed in) | Other HeyGen voices, voice cloning | `npx hyperframes tts` (Kokoro on device; Python 3.10+ with `pip install kokoro-onnx soundfile`) |
|
||||
| A presenter speaking | | Avatar video (`--type video`) | Local LTX for generic clips, if installed |
|
||||
| Transcripts | `npx hyperframes transcribe` (Parakeet or whisper) | | |
|
||||
| Cutouts | | `heygen background-removal` | `npx hyperframes remove-background` |
|
||||
| Grades, LUTs, looks | `npx hyperframes media-treatment` | | |
|
||||
|
||||
**Host apps.** Inside an app that gives you its own media tools, use those first. When `HEYGEN_API_BASE` is set, the host pays for HeyGen: don't ask for the `heygen` CLI or a sign-in, and when a call is refused, relay the host's message as written and stop.
|
||||
**Sign in.** When a request needs a HeyGen model or the catalog and there is no credential, tell the person in one line that sign-in is free and gives Voice Turbo, the music catalog and asset search, then run `npx hyperframes auth login` and show the link it prints. Check setup with `npx hyperframes media-use resolve --doctor`. Don't fall back to local models on your own; use local tools or the person's own keys only when they ask for that.
|
||||
|
||||
**HeyGen.** Install the [`heygen` CLI](https://developers.heygen.com/cli) and sign in with `heygen auth login --oauth`: the OAuth session carries a free allowance on subscription credits; an API key (`--api-key`) bills API credits. Check setup with `npx hyperframes media-use resolve --doctor`. When a request needs HeyGen (a presenter or avatar, a HeyGen voice, catalog music) and `heygen` is missing or signed out, tell the person how to get it: the install link, `heygen auth login --oauth` for the free allowance, or an API key for credits. Then offer the free path or wait. Beyond `resolve`, the same CLI makes text-to-video (`heygen video-agent create`), avatar videos with a recorded track (`heygen video create`), cloned voices (`heygen voice clone create`), dubbing (`heygen lipsync`), translated videos (`heygen video-translate`) and short clips from a long video (`heygen ai-clipping`): recipes in `references/operations.md`.
|
||||
**Host apps.** Inside an app that gives you its own media tools, use those first. When `HEYGEN_API_BASE` is set, the host pays for HeyGen: don't ask for a sign-in, and when a call is refused, relay the host's message as written and stop.
|
||||
|
||||
**Free.** Every tool in the free column needs no account; local models are opt-in installs (`references/setup-providers.md`). Without `heygen`, `resolve` for music, images and icons fails and says so; credit CC BY sounds (`audio/references/sfx.md`). `--local-only` skips every network provider.
|
||||
**Credits.** Plan for the best result; don't swap in cheaper or free options, or drop items, to save credits unless the person asks to save cost. Before the first paid item, tell the person the total for everything you intend to make and get one yes for that budget; inside it, don't ask again. When a call answers with a `Subscribe:` or `Add credits:` line, show it to the person exactly as printed and stop paid steps until they are done. `--provider <name>` pins one generator when the person names it. `--local-only` skips every network provider.
|
||||
|
||||
**Cost.** Confirm before a paid call you chose to make (an avatar video counts); one the person asked for just runs. `--provider <name>` pins one generator when the person names it.
|
||||
**More HeyGen recipes.** The [`heygen` CLI](https://developers.heygen.com/cli) makes text-to-video (`heygen video-agent create`), avatar videos with a recorded track (`heygen video create`), cloned voices (`heygen voice clone create`), dubbing (`heygen lipsync`), translated videos (`heygen video-translate`) and short clips from a long video (`heygen ai-clipping`): recipes in `references/operations.md`. Credit CC BY sounds (`audio/references/sfx.md`).
|
||||
|
||||
## Looks on real footage
|
||||
|
||||
|
||||
@@ -198,8 +198,38 @@ export async function heygenJSON(path, { method = "GET", headers = {}, body } =
|
||||
return text ? JSON.parse(text) : {};
|
||||
}
|
||||
|
||||
// HeyGen's own words for a failed call: its {"error":{"message"}}, else the raw body, else the error's message.
|
||||
// Where a refused HyperFrames-plan call sends the person. The plans and credits pages are not live yet;
|
||||
// override with $HYPERFRAMES_PLANS_URL / $HYPERFRAMES_CREDITS_URL until they are.
|
||||
const PLANS_URL = () => process.env.HYPERFRAMES_PLANS_URL || "https://hyperframes.studio/";
|
||||
const CREDITS_URL = () => process.env.HYPERFRAMES_CREDITS_URL || PLANS_URL();
|
||||
|
||||
function errorCode(body) {
|
||||
try {
|
||||
const parsed = JSON.parse(body);
|
||||
return parsed?.error?.code ?? parsed?.code ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// One line the agent relays as written when the HyperFrames plan can't pay: no plan, out of credits, or a
|
||||
// sign-in the plan doesn't cover yet. Null for every other failure.
|
||||
export function billingRefusal(e) {
|
||||
const code = e?.body ? errorCode(e.body) : null;
|
||||
if (code === "hf_subscription_required")
|
||||
return `This needs a HyperFrames plan. Subscribe: ${PLANS_URL()}`;
|
||||
if (code === "hf_usage_exhausted")
|
||||
return `Out of HyperFrames credits. Add credits: ${CREDITS_URL()}`;
|
||||
if (code === "api_key_required")
|
||||
return "This HeyGen feature isn't available with this sign-in yet. Nothing was charged. Use a HeyGen API key, or wait for HyperFrames billing in the CLI.";
|
||||
return null;
|
||||
}
|
||||
|
||||
// HeyGen's own words for a failed call: a billing refusal line first, then its {"error":{"message"}}, else the
|
||||
// raw body, else the error's message.
|
||||
export function heygenMessage(e) {
|
||||
const refusal = billingRefusal(e);
|
||||
if (refusal) return refusal;
|
||||
if (!e?.body) return e?.message ? String(e.message) : String(e);
|
||||
try {
|
||||
return JSON.parse(e.body).error?.message ?? e.body;
|
||||
|
||||
@@ -5,11 +5,13 @@ import { createServer } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
billingRefusal,
|
||||
heygenAuthHeaders,
|
||||
heygenAuthMethod,
|
||||
heygenBase,
|
||||
heygenCredential,
|
||||
heygenJSON,
|
||||
heygenMessage,
|
||||
loadEnvFromDir,
|
||||
} from "./heygen.mjs";
|
||||
|
||||
@@ -290,3 +292,50 @@ test("a base that isn't HeyGen's gets no stored or host OAuth credential, only a
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test("a refused HyperFrames-plan call becomes one line with the plans or credits link", () => {
|
||||
const refused = (code) => ({
|
||||
status: 402,
|
||||
body: JSON.stringify({ error: { code, message: "raw" } }),
|
||||
});
|
||||
const prev = {
|
||||
plans: process.env.HYPERFRAMES_PLANS_URL,
|
||||
credits: process.env.HYPERFRAMES_CREDITS_URL,
|
||||
};
|
||||
try {
|
||||
process.env.HYPERFRAMES_PLANS_URL = "https://example.test/plans";
|
||||
process.env.HYPERFRAMES_CREDITS_URL = "https://example.test/credits";
|
||||
assert.equal(
|
||||
heygenMessage(refused("hf_subscription_required")),
|
||||
"This needs a HyperFrames plan. Subscribe: https://example.test/plans",
|
||||
);
|
||||
assert.equal(
|
||||
heygenMessage(refused("hf_usage_exhausted")),
|
||||
"Out of HyperFrames credits. Add credits: https://example.test/credits",
|
||||
);
|
||||
assert.match(
|
||||
heygenMessage({ status: 403, body: JSON.stringify({ error: { code: "api_key_required" } }) }),
|
||||
/Nothing was charged/,
|
||||
);
|
||||
// Every other failure keeps HeyGen's own words.
|
||||
assert.equal(
|
||||
billingRefusal({
|
||||
status: 400,
|
||||
body: JSON.stringify({ error: { code: "invalid_parameter", message: "bad voice" } }),
|
||||
}),
|
||||
null,
|
||||
);
|
||||
assert.equal(
|
||||
heygenMessage({ status: 400, body: JSON.stringify({ error: { message: "bad voice" } }) }),
|
||||
"bad voice",
|
||||
);
|
||||
} finally {
|
||||
for (const [k, v] of [
|
||||
["HYPERFRAMES_PLANS_URL", prev.plans],
|
||||
["HYPERFRAMES_CREDITS_URL", prev.credits],
|
||||
]) {
|
||||
if (v === undefined) delete process.env[k];
|
||||
else process.env[k] = v;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: reverse-any-video
|
||||
description: Reverse any video. Remake a reference video's structure, pacing and look with the person's own product, copy and brand as an editable HyperFrames project. Use when the person shares a video and wants one like it.
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
**Plugin installs:** Before setup or freshness commands, follow [plugin execution rules](../hyperframes/references/plugin-installation.md) when this skill is inside a HyperFrames plugin.
|
||||
|
||||
# Reverse any video
|
||||
|
||||
The skill is free. Video understanding and generated media use HyperFrames credits; the person approves one budget up front. The `/hyperframes` contract applies to the remake; media comes through `/media-use`.
|
||||
|
||||
## Sign in
|
||||
|
||||
Understanding the reference uses a HeyGen model, so it needs sign-in. If there is no credential, tell the person in one line that sign-in is free and gives Voice Turbo, the music catalog and asset search, run `npx hyperframes auth login` and show the link it prints. If the person says to use their own API keys or only local tools, do that and don't bring sign-in up again.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Estimate once.** Tell the person what understanding the reference costs and the typical total for a high-quality remake, and get one budget, unless they already gave one. Don't ask about cost again while you stay inside it.
|
||||
2. **Understand the reference.** Shots with their timing, on-screen text, typography, palette, camera moves, speech and music. HeyGen video understanding for the CLI is not wired yet; until it is, use frames and the transcript (`npx hyperframes transcribe`) only when the person agreed to local tools, and say what that leaves out.
|
||||
3. **Plan for the best result.** Shot by shot: keep the reference's timing and rhythm, swap in the person's product, copy and brand. Choose the media that gives the highest-quality remake; don't trim the plan or swap in cheaper or free options unless the person asked to save cost. If the plan costs more than the budget, stop: show the total and ask the person to raise it or top up. Never cut the plan to fit.
|
||||
4. **Make the media** with `/media-use`. Relay any `Subscribe:` or `Add credits:` line exactly as printed.
|
||||
5. **Build** the composition, then `npx hyperframes lint` and `npx hyperframes check`; render locally with `npx hyperframes render` (free).
|
||||
6. **Wrap up.** Tell the person what the project used and what is left, and list what you could not match from the reference.
|
||||
Reference in New Issue
Block a user