mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 07:19:52 +00:00
feat(cli): add whisper transcription and template improvements (#53)
* feat(cli): add whisper transcription to init flow New modules: - whisper/manager.ts: download/cache whisper.cpp binary + model (~/.cache/hyperframes/whisper/) - whisper/transcribe.ts: extract audio, run whisper, save transcript.json Init flow changes: - "Got a video or audio file?" now accepts audio-only files (mp3, wav, m4a) - "Generate captions from audio?" prompt after file selection - Transcription produces transcript.json in project root - Graceful fallback if whisper/ffmpeg unavailable Supports: macOS ARM64/x86, Linux x86_64. Downloads whisper.cpp v1.7.3 from GitHub releases and ggml-base.en model from Hugging Face. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): use brew/system whisper instead of downloading binaries whisper.cpp doesn't ship pre-built macOS/Linux CLI binaries. Use brew install whisper-cpp on macOS (auto-installs if brew available), system PATH lookup otherwise. Model still downloaded from Hugging Face. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): simplify whisper install — detect or instruct, don't build Remove build-from-source complexity. If whisper-cpp is found on PATH, use it. If not, show install instructions instead of blocking: "To generate captions, install whisper-cpp: brew install whisper-cpp" The transcription prompt only appears when whisper is available. When it's not, the user sees the install command and can re-run init. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): auto-install whisper via brew or build from source ensureWhisper() now tries 4 strategies in order: 1. System PATH (whisper-cli or whisper already installed) 2. Homebrew (macOS: brew install whisper-cpp) 3. Build from source (git clone + cmake, ~30-60s) 4. Show install instructions as last resort Init flow always asks "Generate captions?" — whisper is installed automatically in the background if needed. No user intervention required on macOS with Xcode CLI tools or any system with git+cmake. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add window.__timelines guard to all templates The studio bundler doesn't always initialize window.__timelines before template scripts run, causing "Cannot set properties of undefined" errors. Add defensive guard to every template. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): patch template captions with actual transcript data After scaffolding, if transcript.json exists, replace the hardcoded word array in the template's captions composition with the real transcript data. The template's caption animation and styling are preserved — only the word data changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): show install notice when whisper needs to be installed When whisper-cpp isn't found, show an info message before the spinner: "whisper-cpp not found — installing automatically..." Then the spinner shows "Installing whisper-cpp (this may take a moment)..." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add muted and playsinline to all template video elements The framework requires video elements to have muted and playsinline attributes. All four templates were missing these, causing video to not play in the studio preview. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): flat asset structure + separate audio tracks in templates Assets: video, images, fonts all go at project root (not assets/ or fonts/ subdirectories). The studio preview can't resolve relative paths from subdirectories due to the /preview URL suffix. Audio: added <audio> elements alongside muted <video> in all 4 templates so the video's audio plays back. The framework requires muted video + separate audio element. Removed assets/ and fonts/ directory creation from scaffoldProject. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): inject base tag for asset resolution in preview The preview iframe serves bundled HTML from /api/projects/:id/preview but relative asset paths (video.mp4, font.woff2) resolve to the wrong URL without a <base> tag. Now injects <base href="/api/projects/:id/preview/"> so relative paths route through the static asset handler. Also adds proper MIME types for video, audio, image, and font files served from the preview asset route. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(studio): serve HyperFrames runtime in dev mode The preview runtime script had an empty src — the framework never loaded, so video playback and clip lifecycle didn't work. Now auto-detects packages/cli/dist/hyperframe-runtime.js and serves it at /api/runtime.js. No env var needed in dev mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): filter whisper special tokens from transcript Use --output-json instead of --output-json-full to avoid special tokens like [_TT_485] and [BLANK_AUDIO]. Also filter remaining bracket tokens when building the word array for captions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): use --output-json-full for word-level timestamps --output-json only produces segment-level timing (no tokens). --output-json-full is required for word-level timestamps that the captions template needs. Special tokens are filtered out by the patchTranscript function. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): patch template durations to match uploaded video Templates now use __VIDEO_DURATION__ placeholder that gets replaced with the actual probed video duration. All data-duration values on the root composition, video, audio, and caption clips are updated. Without a video, defaults to 10 seconds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): merge punctuation tokens with preceding word Whisper outputs punctuation (. , ! ?) as separate tokens. These appeared as standalone words in captions, sometimes in the wrong group. Now merged with the preceding word during transcript normalization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): match both TRANSCRIPT and script variable names in templates Three templates use `const TRANSCRIPT = [...]` while warm-grain uses `const script = [...]`. The patchTranscript function now matches both. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): security and template fixes - Replace shell injection risk (execSync rm) with unlinkSync in transcribe.ts - Add GIT_TERMINAL_PROMPT=0 to whisper buildFromSource git clone - Fix hardcoded data-duration="18" in warm-grain captions template - Add data-start="0" to root compositions in swiss-grid, vignelli, warm-grain - Add data-start="0" to warm-grain grain-overlay composition - Deduplicate hasFFmpeg: remove from init.ts, import from whisper/manager.ts - Add my-video/ and packages/studio/data/ to .gitignore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): format warm-grain captions and fix TS nullability errors - Format warm-grain/compositions/captions.html - Add optional chaining on token.offsets (may be undefined) - Use intermediate variable for lastWord to satisfy TS strict checks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): add blank template option, smart defaults for video vs audio - Blank template: minimal scaffolding (root composition, video, audio, GSAP timeline) with __VIDEO_SRC__ and __VIDEO_DURATION__ placeholders - Template defaults: video uploads default to "blank" (user brings their own content), audio-only defaults to "warm-grain" (motion graphics template since there's no video to show) - Audio-only projects now tracked with isAudioOnly flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address whisper review feedback - Clean stale builds: if BUILD_DIR exists but no binary, nuke and retry - Build failures clean up BUILD_DIR so next attempt starts fresh - patchTranscript regex scoped within <script> blocks to prevent matching across block boundaries - Removed hardcoded model size hint (~148MB) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add missing rmSync import to whisper manager Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove test project and lock file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address review items 7-12 — execFileSync, build diagnostics, WAV verification - manager.ts: replace all execSync with execFileSync to prevent command injection - manager.ts: capture cmake stderr and include in build failure error message - transcribe.ts: verify WAV is 16kHz mono via ffprobe before passing to whisper - init.ts: replace fragile JSON formatting with JSON.stringify(words, null, 2) - init.ts: fix default duration from "10" to "5" matching DEFAULT_META - init.ts: add probeAudioDuration() and --audio/--skip-transcribe flags - init.ts: extract finalizeProject() to reduce code path duplication - init.ts: wire transcription into non-interactive path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
5fceab9279
commit
f4367d5726
@@ -10,11 +10,12 @@ import {
|
||||
} from "node:fs";
|
||||
import { resolve, basename, join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { execSync, execFileSync, spawn } from "node:child_process";
|
||||
import { execFileSync, spawn } from "node:child_process";
|
||||
import * as clack from "@clack/prompts";
|
||||
import { c } from "../ui/colors.js";
|
||||
import { TEMPLATES, type TemplateId } from "../templates/generators.js";
|
||||
import { trackInitTemplate } from "../telemetry/events.js";
|
||||
import { hasFFmpeg } from "../whisper/manager.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Install skills silently after scaffolding
|
||||
@@ -161,15 +162,23 @@ function isWebCompatible(codec: string): boolean {
|
||||
return WEB_CODECS.has(codec.toLowerCase());
|
||||
}
|
||||
|
||||
function hasFFmpeg(): boolean {
|
||||
function probeAudioDuration(filePath: string): number | undefined {
|
||||
try {
|
||||
execSync("ffmpeg -version", { stdio: "ignore", timeout: 5000 });
|
||||
return true;
|
||||
const raw = execFileSync(
|
||||
"ffprobe",
|
||||
["-v", "quiet", "-print_format", "json", "-show_format", filePath],
|
||||
{ encoding: "utf-8", timeout: 15_000 },
|
||||
);
|
||||
const parsed: { format?: { duration?: string } } = JSON.parse(raw);
|
||||
const duration = parseFloat(parsed.format?.duration ?? "");
|
||||
return Number.isNaN(duration) ? undefined : duration;
|
||||
} catch {
|
||||
return false;
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
// hasFFmpeg is imported from whisper/manager.ts to avoid duplication
|
||||
|
||||
function transcodeToMp4(inputPath: string, outputPath: string): Promise<boolean> {
|
||||
return new Promise((resolvePromise) => {
|
||||
const child = spawn(
|
||||
@@ -211,7 +220,11 @@ function getStaticTemplateDir(templateId: string): string {
|
||||
return existsSync(devPath) ? devPath : builtPath;
|
||||
}
|
||||
|
||||
function patchVideoSrc(dir: string, videoFilename: string | undefined): void {
|
||||
function patchVideoSrc(
|
||||
dir: string,
|
||||
videoFilename: string | undefined,
|
||||
durationSeconds?: number,
|
||||
): void {
|
||||
const htmlFiles = readdirSync(dir, { withFileTypes: true, recursive: true })
|
||||
.filter((e) => e.isFile() && e.name.endsWith(".html"))
|
||||
.map((e) => join(e.parentPath ?? e.path, e.name));
|
||||
@@ -224,11 +237,71 @@ function patchVideoSrc(dir: string, videoFilename: string | undefined): void {
|
||||
// Remove video elements with placeholder src
|
||||
content = content.replace(/<video[^>]*src="__VIDEO_SRC__"[^>]*>[\s\S]*?<\/video>/g, "");
|
||||
content = content.replace(/<video[^>]*src="__VIDEO_SRC__"[^>]*>/g, "");
|
||||
// Remove audio elements with placeholder src
|
||||
content = content.replace(/<audio[^>]*src="__VIDEO_SRC__"[^>]*>[\s\S]*?<\/audio>/g, "");
|
||||
content = content.replace(/<audio[^>]*src="__VIDEO_SRC__"[^>]*>/g, "");
|
||||
}
|
||||
// Patch duration — use probed duration or default (matches DEFAULT_META)
|
||||
const dur = durationSeconds ? String(Math.round(durationSeconds * 100) / 100) : "5";
|
||||
content = content.replaceAll("__VIDEO_DURATION__", dur);
|
||||
writeFileSync(file, content, "utf-8");
|
||||
}
|
||||
}
|
||||
|
||||
function patchTranscript(dir: string, transcriptPath: string): void {
|
||||
// Read the whisper transcript and normalize to [{text, start, end}]
|
||||
const raw = JSON.parse(readFileSync(transcriptPath, "utf-8"));
|
||||
const words: { text: string; start: number; end: number }[] = [];
|
||||
for (const seg of raw.transcription ?? []) {
|
||||
for (const token of seg.tokens ?? []) {
|
||||
const text = (token.text ?? "").trim();
|
||||
if (!text || text.startsWith("[_") || text.startsWith("[BLANK")) continue;
|
||||
|
||||
// Merge punctuation with the previous word
|
||||
const isPunctuation = /^[.,!?;:'")\]}>…–—-]+$/.test(text);
|
||||
const lastWord = words[words.length - 1];
|
||||
if (isPunctuation && lastWord) {
|
||||
lastWord.text += text;
|
||||
lastWord.end = Math.round(((token.offsets?.to ?? 0) / 1000) * 1000) / 1000;
|
||||
continue;
|
||||
}
|
||||
|
||||
words.push({
|
||||
text,
|
||||
start: Math.round(((token.offsets?.from ?? 0) / 1000) * 1000) / 1000,
|
||||
end: Math.round(((token.offsets?.to ?? 0) / 1000) * 1000) / 1000,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (words.length === 0) return;
|
||||
|
||||
const wordsJson = JSON.stringify(words, null, 2);
|
||||
|
||||
// Find captions HTML files and replace the hardcoded script array
|
||||
const htmlFiles = readdirSync(dir, { withFileTypes: true, recursive: true })
|
||||
.filter((e) => e.isFile() && e.name.endsWith(".html"))
|
||||
.map((e) => join(e.parentPath ?? e.path, e.name));
|
||||
|
||||
for (const file of htmlFiles) {
|
||||
let content = readFileSync(file, "utf-8");
|
||||
// Match within <script> blocks only to avoid crossing block boundaries
|
||||
const scriptBlocks = content.match(/<script>[\s\S]*?<\/script>/g) ?? [];
|
||||
let scriptMatch: RegExpMatchArray | null = null;
|
||||
let transcriptMatch: RegExpMatchArray | null = null;
|
||||
for (const block of scriptBlocks) {
|
||||
scriptMatch = scriptMatch ?? block.match(/const script = \[[\s\S]*?\];/);
|
||||
transcriptMatch = transcriptMatch ?? block.match(/const TRANSCRIPT = \[[\s\S]*?\];/);
|
||||
}
|
||||
const match = scriptMatch ?? transcriptMatch;
|
||||
if (match) {
|
||||
const varName = scriptMatch ? "script" : "TRANSCRIPT";
|
||||
content = content.replace(match[0], `const ${varName} = ${wordsJson};`);
|
||||
writeFileSync(file, content, "utf-8");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// handleVideoFile — probe, check codec, optionally transcode, copy to destDir
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -276,8 +349,16 @@ async function handleVideoFile(
|
||||
const transcode = await clack.select({
|
||||
message: "Transcode to H.264 MP4 for browser playback?",
|
||||
options: [
|
||||
{ value: "yes", label: "Yes, transcode", hint: "converts to H.264 MP4" },
|
||||
{ value: "no", label: "No, keep original", hint: "video won't play in browser" },
|
||||
{
|
||||
value: "yes",
|
||||
label: "Yes, transcode",
|
||||
hint: "converts to H.264 MP4",
|
||||
},
|
||||
{
|
||||
value: "no",
|
||||
label: "No, keep original",
|
||||
hint: "video won't play in browser",
|
||||
},
|
||||
],
|
||||
});
|
||||
if (clack.isCancel(transcode)) {
|
||||
@@ -329,12 +410,13 @@ function scaffoldProject(
|
||||
name: string,
|
||||
templateId: TemplateId,
|
||||
localVideoName: string | undefined,
|
||||
durationSeconds?: number,
|
||||
): void {
|
||||
mkdirSync(destDir, { recursive: true });
|
||||
|
||||
const templateDir = getStaticTemplateDir(templateId);
|
||||
cpSync(templateDir, destDir, { recursive: true });
|
||||
patchVideoSrc(destDir, localVideoName);
|
||||
patchVideoSrc(destDir, localVideoName, durationSeconds);
|
||||
|
||||
writeFileSync(
|
||||
resolve(destDir, "meta.json"),
|
||||
@@ -351,6 +433,37 @@ function scaffoldProject(
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// finalizeProject — shared scaffold + patch + skills logic
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function finalizeProject(opts: {
|
||||
destDir: string;
|
||||
name: string;
|
||||
templateId: TemplateId;
|
||||
localVideoName?: string;
|
||||
durationSeconds?: number;
|
||||
skipSkills: boolean;
|
||||
interactive: boolean;
|
||||
}): Promise<void> {
|
||||
scaffoldProject(
|
||||
opts.destDir,
|
||||
opts.name,
|
||||
opts.templateId,
|
||||
opts.localVideoName,
|
||||
opts.durationSeconds,
|
||||
);
|
||||
|
||||
const transcriptFile = resolve(opts.destDir, "transcript.json");
|
||||
if (existsSync(transcriptFile)) {
|
||||
patchTranscript(opts.destDir, transcriptFile);
|
||||
}
|
||||
|
||||
if (!opts.skipSkills) {
|
||||
await installSkills(opts.interactive);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// nextStepLoop — "What do you want to do?" loop after scaffolding
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -360,7 +473,11 @@ async function nextStepLoop(destDir: string): Promise<void> {
|
||||
const next = await clack.select({
|
||||
message: "What do you want to do?",
|
||||
options: [
|
||||
{ value: "dev", label: "Open in studio", hint: "full editor with timeline" },
|
||||
{
|
||||
value: "dev",
|
||||
label: "Open in studio",
|
||||
hint: "full editor with timeline",
|
||||
},
|
||||
{ value: "render", label: "Render to MP4", hint: "export video now" },
|
||||
{ value: "done", label: "Done for now" },
|
||||
],
|
||||
@@ -398,19 +515,41 @@ async function nextStepLoop(destDir: string): Promise<void> {
|
||||
export default defineCommand({
|
||||
meta: { name: "init", description: "Scaffold a new composition project" },
|
||||
args: {
|
||||
name: { type: "positional", description: "Project name", required: false },
|
||||
name: {
|
||||
type: "positional",
|
||||
description: "Project name (default: my-video)",
|
||||
required: false,
|
||||
},
|
||||
template: {
|
||||
type: "string",
|
||||
description: `Template: ${ALL_TEMPLATE_IDS.join(", ")}`,
|
||||
description: `Template: ${ALL_TEMPLATE_IDS.join(", ")}. Required for non-interactive mode.`,
|
||||
alias: "t",
|
||||
},
|
||||
video: { type: "string", description: "Path to a source video file", alias: "V" },
|
||||
"skip-skills": { type: "boolean", description: "Skip AI skills installation" },
|
||||
video: {
|
||||
type: "string",
|
||||
description: "Path to a source video file (auto-transcodes if needed)",
|
||||
alias: "V",
|
||||
},
|
||||
audio: {
|
||||
type: "string",
|
||||
description: "Path to a source audio file (cannot combine with --video)",
|
||||
alias: "A",
|
||||
},
|
||||
"skip-skills": {
|
||||
type: "boolean",
|
||||
description: "Skip AI skills installation",
|
||||
},
|
||||
"skip-transcribe": {
|
||||
type: "boolean",
|
||||
description: "Skip whisper transcription (default: transcribes when video/audio provided)",
|
||||
},
|
||||
},
|
||||
async run({ args }) {
|
||||
const templateFlag = args.template;
|
||||
const videoFlag = args.video;
|
||||
const audioFlag = args.audio;
|
||||
const skipSkills = args["skip-skills"] === true;
|
||||
const skipTranscribe = args["skip-transcribe"] === true;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Non-interactive mode: flags provided
|
||||
@@ -433,6 +572,13 @@ export default defineCommand({
|
||||
mkdirSync(destDir, { recursive: true });
|
||||
|
||||
let localVideoName: string | undefined;
|
||||
let videoDuration: number | undefined;
|
||||
let sourceFilePath: string | undefined;
|
||||
|
||||
if (videoFlag && audioFlag) {
|
||||
console.error(c.error("Cannot specify both --video and --audio"));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (videoFlag) {
|
||||
const videoPath = resolve(videoFlag);
|
||||
@@ -440,16 +586,55 @@ export default defineCommand({
|
||||
console.error(c.error(`Video file not found: ${videoFlag}`));
|
||||
process.exit(1);
|
||||
}
|
||||
sourceFilePath = videoPath;
|
||||
const result = await handleVideoFile(videoPath, destDir, false);
|
||||
localVideoName = result.localVideoName;
|
||||
videoDuration = result.meta.durationSeconds;
|
||||
} else if (audioFlag) {
|
||||
const audioPath = resolve(audioFlag);
|
||||
if (!existsSync(audioPath)) {
|
||||
console.error(c.error(`Audio file not found: ${audioFlag}`));
|
||||
process.exit(1);
|
||||
}
|
||||
sourceFilePath = audioPath;
|
||||
copyFileSync(audioPath, resolve(destDir, basename(audioPath)));
|
||||
videoDuration = probeAudioDuration(audioPath);
|
||||
}
|
||||
|
||||
scaffoldProject(destDir, basename(destDir), templateId, localVideoName);
|
||||
trackInitTemplate(templateId);
|
||||
if (!skipSkills) {
|
||||
await installSkills(false);
|
||||
// Transcribe if we have a source file and transcription isn't skipped
|
||||
if (sourceFilePath && !skipTranscribe) {
|
||||
try {
|
||||
const { ensureWhisper, ensureModel } = await import("../whisper/manager.js");
|
||||
await ensureWhisper({
|
||||
onProgress: (msg) => console.log(c.dim(` ${msg}`)),
|
||||
});
|
||||
await ensureModel(undefined, {
|
||||
onProgress: (msg) => console.log(c.dim(` ${msg}`)),
|
||||
});
|
||||
const { transcribe: runTranscribe } = await import("../whisper/transcribe.js");
|
||||
const result = await runTranscribe(sourceFilePath, destDir, {
|
||||
onProgress: (msg) => console.log(c.dim(` ${msg}`)),
|
||||
});
|
||||
console.log(
|
||||
c.success(
|
||||
`Transcribed ${result.wordCount} words (${result.durationSeconds.toFixed(1)}s)`,
|
||||
),
|
||||
);
|
||||
} catch (err) {
|
||||
console.log(c.dim(`Transcription skipped: ${err instanceof Error ? err.message : err}`));
|
||||
}
|
||||
}
|
||||
|
||||
await finalizeProject({
|
||||
destDir,
|
||||
name: basename(destDir),
|
||||
templateId,
|
||||
localVideoName,
|
||||
durationSeconds: videoDuration,
|
||||
skipSkills,
|
||||
interactive: false,
|
||||
});
|
||||
|
||||
console.log(c.success(`\nCreated ${c.accent(name + "/")}`));
|
||||
for (const f of readdirSync(destDir)) {
|
||||
console.log(` ${c.accent(f)}`);
|
||||
@@ -493,42 +678,47 @@ export default defineCommand({
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Got a video?
|
||||
// 2. Got a video or audio file?
|
||||
let localVideoName: string | undefined;
|
||||
let sourceFilePath: string | undefined;
|
||||
let videoDuration: number | undefined;
|
||||
let isAudioOnly = false;
|
||||
|
||||
if (videoFlag) {
|
||||
// Video supplied via --video flag even in interactive mode
|
||||
const videoPath = resolve(videoFlag);
|
||||
if (!existsSync(videoPath)) {
|
||||
clack.log.error(`Video file not found: ${videoFlag}`);
|
||||
clack.log.error(`File not found: ${videoFlag}`);
|
||||
clack.cancel("Setup cancelled.");
|
||||
process.exit(1);
|
||||
}
|
||||
mkdirSync(destDir, { recursive: true });
|
||||
sourceFilePath = videoPath;
|
||||
const result = await handleVideoFile(videoPath, destDir, true);
|
||||
localVideoName = result.localVideoName;
|
||||
videoDuration = result.meta.durationSeconds;
|
||||
} else {
|
||||
const videoChoice = await clack.select({
|
||||
message: "Got a video file?",
|
||||
const mediaChoice = await clack.select({
|
||||
message: "Got a video or audio file?",
|
||||
options: [
|
||||
{ value: "yes", label: "Yes", hint: "MP4 or WebM recommended" },
|
||||
{ value: "video", label: "Video", hint: "MP4, WebM, MOV" },
|
||||
{ value: "audio", label: "Audio only", hint: "MP3, WAV, M4A" },
|
||||
{
|
||||
value: "no",
|
||||
label: "No",
|
||||
hint: "Start with motion graphics or text",
|
||||
},
|
||||
],
|
||||
initialValue: "no" as "yes" | "no",
|
||||
initialValue: "no" as "video" | "audio" | "no",
|
||||
});
|
||||
if (clack.isCancel(videoChoice)) {
|
||||
if (clack.isCancel(mediaChoice)) {
|
||||
clack.cancel("Setup cancelled.");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (videoChoice === "yes") {
|
||||
if (mediaChoice === "video" || mediaChoice === "audio") {
|
||||
const pathResult = await clack.text({
|
||||
message: "Path to your video file (drag and drop or paste)",
|
||||
placeholder: "/path/to/video.mp4",
|
||||
message: `Path to your ${mediaChoice} file (drag and drop or paste)`,
|
||||
placeholder: mediaChoice === "video" ? "/path/to/video.mp4" : "/path/to/audio.mp3",
|
||||
validate(val) {
|
||||
const trimmed = val?.trim();
|
||||
if (!trimmed) return "Please enter a file path";
|
||||
@@ -541,15 +731,70 @@ export default defineCommand({
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const videoPath = resolve(String(pathResult).trim());
|
||||
|
||||
const filePath = resolve(String(pathResult).trim());
|
||||
sourceFilePath = filePath;
|
||||
mkdirSync(destDir, { recursive: true });
|
||||
const result = await handleVideoFile(videoPath, destDir, true);
|
||||
localVideoName = result.localVideoName;
|
||||
|
||||
if (mediaChoice === "video") {
|
||||
const result = await handleVideoFile(filePath, destDir, true);
|
||||
localVideoName = result.localVideoName;
|
||||
videoDuration = result.meta.durationSeconds;
|
||||
} else {
|
||||
// Audio file — copy to project root and probe duration
|
||||
isAudioOnly = true;
|
||||
copyFileSync(filePath, resolve(destDir, basename(filePath)));
|
||||
videoDuration = probeAudioDuration(filePath);
|
||||
clack.log.info(`Audio copied to ${c.accent(basename(filePath))}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Pick template — single list for all templates
|
||||
// 2b. Transcribe if we have a source file with audio
|
||||
if (sourceFilePath && !skipTranscribe) {
|
||||
const transcribeChoice = await clack.confirm({
|
||||
message: "Generate captions from audio?",
|
||||
initialValue: true,
|
||||
});
|
||||
if (!clack.isCancel(transcribeChoice) && transcribeChoice) {
|
||||
const { findWhisper } = await import("../whisper/manager.js");
|
||||
const needsInstall = findWhisper() === undefined;
|
||||
if (needsInstall) {
|
||||
clack.log.info(c.dim("whisper-cpp not found — installing automatically..."));
|
||||
}
|
||||
|
||||
const spin = clack.spinner();
|
||||
spin.start(
|
||||
needsInstall
|
||||
? "Installing whisper-cpp (this may take a moment)..."
|
||||
: "Preparing transcription...",
|
||||
);
|
||||
try {
|
||||
const { ensureWhisper, ensureModel } = await import("../whisper/manager.js");
|
||||
await ensureWhisper({
|
||||
onProgress: (msg) => spin.message(msg),
|
||||
});
|
||||
await ensureModel(undefined, {
|
||||
onProgress: (msg) => spin.message(msg),
|
||||
});
|
||||
|
||||
spin.message("Transcribing audio...");
|
||||
const { transcribe: runTranscribe } = await import("../whisper/transcribe.js");
|
||||
const transcribeResult = await runTranscribe(sourceFilePath, destDir, {
|
||||
onProgress: (msg) => spin.message(msg),
|
||||
});
|
||||
spin.stop(
|
||||
c.success(
|
||||
`Transcribed ${transcribeResult.wordCount} words (${transcribeResult.durationSeconds.toFixed(1)}s)`,
|
||||
),
|
||||
);
|
||||
} catch (err) {
|
||||
spin.stop(c.dim(`Transcription skipped: ${err instanceof Error ? err.message : err}`));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Pick template — default depends on media type
|
||||
const defaultTemplate = isAudioOnly ? "warm-grain" : "blank";
|
||||
const templateResult = await clack.select({
|
||||
message: "Pick a template",
|
||||
options: TEMPLATES.map((t) => ({
|
||||
@@ -557,7 +802,7 @@ export default defineCommand({
|
||||
label: t.label,
|
||||
hint: t.hint,
|
||||
})),
|
||||
initialValue: TEMPLATES[0]?.id,
|
||||
initialValue: defaultTemplate as TemplateId,
|
||||
});
|
||||
if (clack.isCancel(templateResult)) {
|
||||
clack.cancel("Setup cancelled.");
|
||||
@@ -566,15 +811,18 @@ export default defineCommand({
|
||||
|
||||
const templateId: TemplateId = templateResult;
|
||||
|
||||
// 4. Copy template and patch
|
||||
scaffoldProject(destDir, name, templateId, localVideoName);
|
||||
// 4. Copy template, patch, and install skills
|
||||
await finalizeProject({
|
||||
destDir,
|
||||
name,
|
||||
templateId,
|
||||
localVideoName,
|
||||
durationSeconds: videoDuration,
|
||||
skipSkills,
|
||||
interactive: true,
|
||||
});
|
||||
trackInitTemplate(templateId);
|
||||
|
||||
// 5. Install AI coding skills
|
||||
if (!skipSkills) {
|
||||
await installSkills(true);
|
||||
}
|
||||
|
||||
const files = readdirSync(destDir);
|
||||
clack.note(files.map((f) => c.accent(f)).join("\n"), c.success(`Created ${name}/`));
|
||||
|
||||
|
||||
Reference in New Issue
Block a user