mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 17:27:09 +00:00
* feat(cli): download Parakeet with models install parakeet and transcribe with it `hyperframes models install parakeet` installs the sherpa-onnx runtime from npm and the Parakeet TDT 0.6B v3 int8 model from Hugging Face, pinned to a revision. Every file is checked against its pinned size and sha256 under a temp name before it is renamed into place. Re-running it is a no-op. On Linux with glibc older than 2.32 it refuses up front. `transcribe --engine auto|parakeet` now prefers that engine when it is installed, then an installed parakeet-mlx runner, then whisper. Audio is cut into about 60 s windows at the quietest 100 ms near each cut, and decoding runs in a child process. If the child cannot load the runtime or crashes, transcribe falls back with one stderr line saying why. The JSON output and transcript.json shape are unchanged. * fix(skills): report the real word count and engine from transcribe fallbacks The media-use transcribe script read `words` from the CLI's transcript, which is a flat array, so its whisper path always reported an undefined word count. The embedded-captions script labelled every CLI transcript whisper.cpp even when `--engine auto` ran Parakeet; it now uses the engine the CLI reports. * fix(cli): pin the sherpa-onnx native package to the runtime's exact version * test(cli): type the sherpa install test doubles * fix(cli): transcribe audio with no speech as 0 words instead of failing Parakeet writes an empty word list for a music bed, a silent intro or a very short clip, and loading it failed with "Unrecognized JSON transcript format". An empty list is now read as a transcript with no words, as whisper's empty output already is. A non-empty list still has to look like words. * fix(cli): re-decode a Parakeet window that skipped the start of its speech At some window lengths the model silently drops the first clause: on a looped speech clip, 4 of 167 lengths between 55 and 60 s, and a 61.91 s window, lost their first 9 words. A window with a token gap of at least 1 s that is as loud as speech is decoded again with 0.5 s of leading silence, and the result with more tokens wins. Clean windows are unchanged. Decoding with 1.5 s of overlap before each cut was measured too and was worse on 573 s of audio (2.62% WER against 0.98%), so it is not used. Empty audio no longer makes a zero-length window, which aborted the decoder. * fix(cli): stop Parakeet on Ctrl-C and name the repair when its install breaks - Ctrl-C during a Parakeet transcribe now stops with exit 130 instead of starting whisper, and a SIGTERM to the CLI also stops its decode child. - Ctrl-C during models install parakeet aborts the download and the npm install, keeps no partial file, and exits 130. A staging dir left by a killed install is swept by the next run. - When Parakeet fails under --engine auto, the warning leads with the Parakeet error and names hyperframes models install parakeet. If the fallback fails too, the Parakeet error is reported rather than whisper_unavailable, so --optional no longer hides a broken install. - An explicit --engine parakeet no longer falls back: it fails with the Parakeet error and the repair command. - The install refuses up front on systems with no native package (naming the six supported ones) and on musl, instead of a raw npm error. - 24-bit and float WAV go through ffmpeg, since sherpa-onnx reads only 16-bit PCM. The decode timeout scales with the audio length. * docs(cli): point Parakeet setup at models install parakeet The transcribe docs and the media-use references named parakeet-mlx as the only way to get Parakeet. They now name hyperframes models install parakeet, and the CLI reference gets a models section. * test(cli): cover the npm cancel signal and decode timeout, and fix a doubled period The Parakeet failure line printed "failed.. To repair" when the decoder's own message ended in a period. Tests now also pin that Ctrl-C reaches the npm install and that the decode timeout scales with the audio length. * fix(cli): report unreadable audio as an input error, not a Parakeet failure Prepare the WAV once before the Parakeet fallback and hand the same file to whisper, so ffmpeg runs once. Ctrl-C while ffmpeg runs now ends as a cancel (exit 130) instead of starting whisper. * fix(cli): recover a clause Parakeet drops after a pause by decoding the gap alone Each loud gap without tokens is decoded on its own with 0.5 s either side, and only its new tokens are spliced in. The padded whole-window retry stays as the fallback. A window with no tokens at all is no longer decoded twice. * perf(cli): size the decode timeout from the WAV header instead of reading the file * docs(cli): say a broken Parakeet install still fails transcribe --optional * fix(cli): treat Ctrl-C during audio preparation as a cancel and drop the partial WAV ffmpeg traps SIGINT and exits 255 rather than dying by the signal, so that status now counts as a cancel unless the timeout fired. A failed or stopped ffmpeg removes the temp WAV it was writing. The decode timeout now sizes a prepared WAV from its file size, so a large header can no longer hide its length. * fix(cli): splice a re-decoded gap by whole words so its neighbours are never doubled The gap decode splits edge words into other tokens than the window decode did. Match neighbours as whole words within 0.3 s of each gap edge, add only whole words that start inside the gap, and decode the gap behind 0.5 s of silence like the padded retry. * fix(cli): count a cancel only when ffmpeg says it received a signal ffmpeg 7 and later also exit 255 when a file is not permitted (EPERM), so exit 255 alone read a permission error as a cancel. ffmpeg's stderr is now captured, a cancel needs its received-signal line, and any other failure reports ffmpeg's last line. The output path is passed to the shared runner explicitly. * fix(cli): keep a spliced gap word that has no leading space The filter dropped such a word; it is now kept and given its word boundary. * refactor(cli): trim three long comments in the Parakeet decoder and installer * fix(cli): repair a Parakeet runtime missing its native binary, and pick Parakeet by language The runtime dir is per platform and arch. Installed now needs the runtime manifest and the native sherpa-onnx.node; a half-installed dir is removed before npm runs, since install() keeps any dir holding the runtime manifest. The decode worker flattens a multi-line loader error onto its prefixed line and stops between windows once orphaned. transcribe opens its cancellation scope before audio prep, so Ctrl-C under --json cleans up. auto uses Parakeet only for its 25 languages and only where the host can run it; --engine parakeet with another language fails. * fix(cli): sweep reused-pid staging, name failed model downloads, keep ffmpeg's cause sweepStaleStaging also removes a staging dir older than 6 hours whose pid is alive, since containers reuse pids. A failed model download names the file, the host and a retry. runFfmpeg keeps ffmpeg's stderr on its error, so transcribe shows "Output file does not contain any stream" for a video without audio, and reads a cancel only from ffmpeg's own signal line. * fix(skills): make media-use --engine whisper run whisper and report the engine that ran The whisper path now passes --engine whisper when asked, or --engine auto, and reads the engine from the CLI's --json line, so a CLI Parakeet run is no longer labelled whisper. Docs state the language rule, the real repair behaviour and the platforms Parakeet runs on. * test(cli): pin that a long Parakeet decoder error is kept whole up to a bound * fix(cli): report a broken Parakeet runtime with the repair instead of skipping it * refactor(cli): name the staging age limit instead of commenting it * fix(skills): let media-use --engine parakeet use the CLI Parakeet when parakeet-mlx is absent * fix(cli): repair a Parakeet runtime that fails to load, and read Ctrl-C after whisper models install parakeet now loads the runtime in a child process, the way the decode worker does, and reinstalls it when that fails, so a missing shared library, a corrupt binary or missing runtime JS is repaired. It reports changed: false when the runtime loads. transcribe keeps its cheap manifest check. After a failure, transcribe lets the event loop turn once before reading the Ctrl-C scope, since a signal that landed during the synchronous whisper fallback is only handled then. * fix(cli): read Ctrl-C from the whisper child, and re-check the Parakeet runtime after a repair A whisper fallback killed by SIGINT or SIGTERM is a cancel (exit 130), the same rule decode and runFfmpeg use; the loop-turn wait is gone. The runtime probe treats a child stopped by Ctrl-C as a cancel and keeps the runtime; a reinstall is probed again and fails with the loader's error if it still does not load. media-use keeps the CLI's last lines, stdout included, when hyperframes transcribe fails. * fix(cli): give the user-stop rule one owner, SIGHUP included, and end a cancelled install at 130 stoppedByCancelSignal in renderCancellation.ts uses the scope's own signal list (SIGINT, SIGTERM, SIGHUP) and is false whenever Node stopped the child (a code, killed, or a spawn error). decode, runFfmpeg, the runtime probe and the whisper fallback all route through it, so a probe killed by SIGHUP keeps the runtime. isWhisperTimeoutError now keys on ETIMEDOUT only, which Node sets on execFileSync's own timeout, so a SIGTERM reaches the cancel rule. A cancelled models install sets exit 130 and returns instead of failing, and the probe says what went wrong (timed out, crashed, no output). * refactor(cli): let the stop-rule helper name carry its meaning
108 lines
4.2 KiB
TypeScript
108 lines
4.2 KiB
TypeScript
import { defineCommand } from "citty";
|
|
import * as clack from "@clack/prompts";
|
|
import type { Example } from "./_examples.js";
|
|
import { c } from "../ui/colors.js";
|
|
import { formatBytes } from "../ui/format.js";
|
|
import { failCommand, setCommandExitCode } from "../utils/commandResult.js";
|
|
import { createRenderCancellationScope } from "../utils/renderCancellation.js";
|
|
import { PARAKEET_MODEL_LABEL } from "../whisper/parakeet.js";
|
|
|
|
export const examples: Example[] = [
|
|
[
|
|
"Download the Parakeet speech model that transcribe uses",
|
|
"hyperframes models install parakeet",
|
|
],
|
|
];
|
|
|
|
function fail(message: string, json: boolean): never {
|
|
if (json) console.log(JSON.stringify({ ok: false, error: message }));
|
|
else console.error(c.error(message));
|
|
failCommand();
|
|
}
|
|
|
|
/** A cancel is the user's choice, not a command failure: exit 130 without a cli_error. */
|
|
function reportCancel(json: boolean): void {
|
|
const message = "Parakeet install cancelled; nothing partial was kept.";
|
|
if (json) console.log(JSON.stringify({ ok: false, error: message }));
|
|
else console.error(c.warn(message));
|
|
setCommandExitCode(130);
|
|
}
|
|
|
|
function downloadProgress(spin: Spinner) {
|
|
let lastPct = -1;
|
|
return (done: number, total: number) => {
|
|
const pct = Math.floor((done / total) * 100);
|
|
if (pct <= lastPct) return;
|
|
lastPct = pct;
|
|
spin?.message(
|
|
`Downloading Parakeet TDT 0.6B v3 — ${c.progress(pct + "%")} ${c.dim("(" + formatBytes(done) + " / " + formatBytes(total) + ")")}`,
|
|
);
|
|
};
|
|
}
|
|
|
|
type Sherpa = typeof import("../whisper/sherpa.js");
|
|
type Spinner = ReturnType<typeof clack.spinner> | null;
|
|
|
|
/** Installs what is missing; true when anything changed. */
|
|
async function installMissing(sherpa: Sherpa, spin: Spinner, signal: AbortSignal) {
|
|
const runtimeInstalled = await sherpa.installSherpaRuntime({ signal });
|
|
spin?.message("Verifying the Parakeet model...");
|
|
const modelFetched = await sherpa.ensureParakeetModel({
|
|
signal,
|
|
onBytes: downloadProgress(spin),
|
|
});
|
|
return runtimeInstalled || modelFetched;
|
|
}
|
|
|
|
/** A synchronous child's Ctrl-C shows as its error before the scope's listener runs. */
|
|
const wasCancelled = (err: unknown, signal: AbortSignal, sherpa: Sherpa) =>
|
|
signal.aborted || err instanceof sherpa.DecodeCancelled;
|
|
|
|
async function installParakeet(json: boolean): Promise<void> {
|
|
const sherpa = await import("../whisper/sherpa.js");
|
|
const unsupported = sherpa.sherpaUnsupportedReason();
|
|
if (unsupported) fail(unsupported, json);
|
|
|
|
const spin = json ? null : clack.spinner({ output: process.stderr });
|
|
// Ctrl-C must stop a 650 MB download, not just print "Canceled" over it.
|
|
const cancellation = createRenderCancellationScope();
|
|
spin?.start("Checking the sherpa-onnx runtime (installing it from npm if it does not load)...");
|
|
try {
|
|
const changed = await installMissing(sherpa, spin, cancellation.signal);
|
|
spin?.stop(c.success(changed ? "Parakeet installed" : "Parakeet is already installed"));
|
|
if (json) {
|
|
const { SHERPA_RUNTIME_DIR: runtimeDir, PARAKEET_MODEL_DIR: modelDir } = sherpa;
|
|
console.log(
|
|
JSON.stringify({ ok: true, model: PARAKEET_MODEL_LABEL, changed, runtimeDir, modelDir }),
|
|
);
|
|
}
|
|
} catch (err) {
|
|
const cancelled = wasCancelled(err, cancellation.signal, sherpa);
|
|
spin?.stop(
|
|
cancelled ? c.warn("Parakeet install cancelled") : c.error("Parakeet install failed"),
|
|
);
|
|
if (cancelled) return reportCancel(json);
|
|
fail(err instanceof Error ? err.message : String(err), json);
|
|
} finally {
|
|
cancellation.dispose();
|
|
}
|
|
}
|
|
|
|
export default defineCommand({
|
|
meta: { name: "models", description: "Download on-device models (models install parakeet)" },
|
|
args: {
|
|
action: { type: "positional", description: "install", required: true },
|
|
name: { type: "positional", description: "Model to install: parakeet", required: true },
|
|
json: { type: "boolean", description: "Print one JSON result, no progress", default: false },
|
|
},
|
|
async run({ args }) {
|
|
if (args.action !== "install" || args.name !== "parakeet") {
|
|
fail(
|
|
`Unknown: models ${args.action} ${args.name}. Try: hyperframes models install parakeet`,
|
|
args.json,
|
|
);
|
|
}
|
|
return installParakeet(args.json);
|
|
},
|
|
});
|