Files
hyperframes/skills/media-use/scripts/lib/specs.mjs
James RussoandClaude Opus 5 8392e84a18 fix(media-use): repoint the dead videogen tier, demote past unusable local models (#3509)
* fix(media-use): repoint the dead videogen tier, demote past unusable models

`LOCAL_MODELS.videogen`'s `large` tier named `dgrauet/ltx-2.3-mlx-bf16`, which
returns HTTP 401 and cannot be downloaded at all. It was not a dormant entry:
`rankedByPreference` sorts by descending `needs.ramMB` when no `rank` is set,
so the largest fitting tier is tried FIRST by design. Any machine clearing
32 GB *available* RAM selected the dead entry, `ltxVideoGenerate` caught the
failure and returned a bare `null`, and since `ltx.local` is last in
`["heygen.video", "ltx.local"]` and network providers are skipped under
`--local-only` (`registry.mjs:206`), local video generation failed outright
instead of falling back to the tier that works.

It survived review because the table landed with "live verification on a 24GB
M-series Mac" - and a 24 GB machine cannot select a 32 GB tier, so that entry
was unreachable on the only machine that validated it. The unit fixtures
inherit the same ceiling (`fittingSpecs` is 20000MB), so every existing test
exercised the medium tier alone.

Two changes:

1. Repoint to `dgrauet/ltx-2.3-mlx-q8` (reachable) and correct `sizeMB` from
   45000 to 28800. Measured against the HF API: the q8 repo totals 87.5 GB,
   and the registry's own targeted `--include` subset is 28.76 GB. That
   matches the sibling q4 entry's convention (`sizeMB: 20000` vs a measured
   19.48 GB subset), so 45000 was wrong under either reading. `--low-ram` is
   added because the entry's own note calls it required at this tier's 32 GB
   floor, and the invoke omitted it.

2. A repoint alone is one bad URL from a repeat, so add the missing recovery.
   `selectModelLadder` returns every fitting model best-first;
   `selectModel`'s pick is now defined as that list's head. All three sites
   that previously selected exactly one model and failed terminally walk the
   ladder instead, demoting past a tier that cannot run here - gated weights,
   runner off PATH, an OOM at a tier that nominally fits:

   - `ltx-video-provider.mjs` (videogen, the reported failure)
   - `mflux-provider.mjs` (imagegen - same shape, and its 32 GB/64 GB tiers
     are equally unverifiable on a 24 GB machine)
   - `local-run.mjs` (tts/asr/upscale - `fish-speech` missing should still
     get you Kokoro)

   Every demotion is logged rather than silent, so a quietly smaller model is
   never mistaken for the tier the machine nominally qualified for.

Also fixes the `install` string both videogen entries share: it ended at
`uv sync --all-extras`, which leaves the entry point in `.venv/bin`, so the
"`ltx-2-mlx` not on PATH" hint named a command that following the instruction
would not put on PATH.

The q8 tier is NOT live-verified - no 32 GB+ Apple Silicon machine was
available - and its notes say so. Shipping it unverified is safe precisely
because of change 2: a wrong tier now costs one failed attempt, not the whole
local path.

- Rames Jusso

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(media-use): report the real videogen download size, disclose it, discard failed partials

Addresses review feedback on #3509 (CHANGES_REQUESTED at 90df164a), plus the
follow-on ask to tell the user what a download costs before they accept it.

1. `sizeMB` described a targeted `--include` subset that no run ever gets.
   Both videogen invokes pass a repo id to `--model`, and upstream
   `resolve_model_dir()` (`ltx_pipelines_mlx/utils/_orchestration.py:35-40`)
   calls `snapshot_download(repo)` with no `allow_patterns`, so the full repo
   lands regardless of what was pre-fetched. Corrected to measured repo
   totals: q8 87500 (87,511,991,375 B) and q4 59700 (59,686,429,583 B). q4 was
   wrong the same way at 20000, so both are fixed together rather than leaving
   one convention on each side.

   My earlier claim that 28800 "matches the sibling q4 entry's convention" was
   wrong in the way that matters: the convention itself described a subset the
   runner does not honor. The file's own comment already said "blind
   snapshot-downloads the lot (60 GB q4, 88 GB q8)" three lines above the
   fields that contradicted it, and the original report measured it too ("the
   q4 cache ended at 56 GB and q8 at 82 GB"), which reconciles exactly once
   read as GiB: 59.69 GB = 55.6 GiB, 87.51 GB = 81.5 GiB. So the download is
   the complete repo both times, not a partial fetch.

   Removed the `--include` recipe rather than repairing it: it is ineffective
   (the runner refetches at generate time) and insufficient (`--two-stage` is
   "dev model + CFG at half-res, upscale, distilled LoRA refine" per upstream's
   own help text, so it needs transformer-dev AND transformer-distilled AND
   spatial_upscaler_x2; `--distilled` needs an upscaler too). The q4 tier
   verified on a 24 GB Mac only worked BECAUSE the download is unfiltered.

2. Nothing told the user what they were agreeing to before a tool started
   pulling tens of GB. `describeDownload()` in `specs.mjs` names the size and
   the directory the weights land in, and checks free space with `statfs`
   against that directory rather than cwd, since the weights do not land in
   cwd. A tier that will not fit is still offered, with a plain statement that
   it will not fit: hiding it would make a machine that could free up space
   look like it has no large tier. Unknown free space reports as unknown, not
   as zero. Wired into both providers' install hints, the `runLocalModel`
   install payload (now carrying `sizeMB`), and `describeModelLadder`.

3. Each retry attempt mints its own timestamped temp path, so a partial
   artifact from a failed tier was orphaned rather than overwritten, and a
   lower tier then succeeding hid it. Both providers discard the partial before
   demoting, guarded so a file that cannot be removed never masks the generate
   failure it came from. Video is the material case: a partial mp4 is large.

   `local-run.mjs` is deliberately unchanged here. Its `out` is caller-provided
   and identical across attempts, so a partial is overwritten rather than
   orphaned, and unlinking a path the caller named would be a footgun. The rule
   the two providers follow is: clean up what you allocate.

Tests: 553/553 across `skills/**/*.test.mjs` (+15). New coverage pins the
cleanup (failed tier's partial removed, returned artifact survives, one discard
per attempt on the all-fail path, an unremovable partial still surfaces the
real failure) and the disclosure (cache-dir precedence, statfs walk-up to the
deepest existing ancestor, unknown-vs-zero, and the will-not-fit wording).
Every new guard mutation-tested: removing any one of them turns tests red.

- Rames Jusso

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 16:15:51 -07:00

133 lines
5.4 KiB
JavaScript

// Machine-capability probe for the spec-gated local-model fallback.
//
// Local models are USER-INSTALLED and local-use-only — media-use recommends,
// spec-checks, and assists install, but never bundles or runs them as a service.
// This probe answers "what tier can this machine actually run?" so selection can
// offer a medium/large local model, or fall back to recommending the CLI path.
//
// `osMod` and `exec` are injectable for tests. `exec(cmd)` returns the command's
// stdout as a string, or throws / returns null on failure.
import os from "node:os";
import { statfsSync } from "node:fs";
import { dirname, join } from "node:path";
import { execSync } from "node:child_process";
function defaultExec(cmd) {
return execSync(cmd, { encoding: "utf8", stdio: ["ignore", "pipe", "ignore"], timeout: 4000 });
}
// Available (not just total) RAM — the real budget for "will this model fit
// alongside the OS + open apps". On unified-memory Macs the model shares system
// RAM, so a big model on a busy machine OOMs/thrashes even if total RAM looks
// ample. macOS: vm_stat free + inactive + speculative + purgeable pages (all
// reclaimable). Linux: /proc/meminfo MemAvailable. Fallback: total (optimistic
// but stable when the probe is unavailable, e.g. in tests).
function availableRamMB(platform, exec, totalMB) {
try {
if (platform === "darwin") {
const out = String(exec("vm_stat"));
const pageSize = parseInt((out.match(/page size of (\d+)/) || [])[1] || "16384", 10);
const pages = (name) => {
const m = out.match(new RegExp(`${name}:\\s+(\\d+)`));
return m ? parseInt(m[1], 10) : 0;
};
const reclaimable =
pages("Pages free") +
pages("Pages inactive") +
pages("Pages speculative") +
pages("Pages purgeable");
const mb = Math.round((reclaimable * pageSize) / (1024 * 1024));
if (mb > 0) return mb;
} else if (platform === "linux") {
const out = String(exec("cat /proc/meminfo"));
const m = out.match(/MemAvailable:\s+(\d+)\s+kB/);
if (m) return Math.round(parseInt(m[1], 10) / 1024);
}
} catch {
// probe unavailable — fall through to the total-RAM estimate
}
return totalMB;
}
function detectGpu(platform, arch, ramMB, exec) {
// Apple Silicon: Metal GPU with unified memory — VRAM tracks system RAM.
if (platform === "darwin" && arch === "arm64") {
return { present: true, kind: "apple", vramMB: ramMB };
}
// NVIDIA: query total VRAM. Any failure (no driver, no GPU) -> no GPU.
try {
const out = exec("nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits");
const mb = parseInt(String(out).trim().split(/\r?\n/)[0], 10);
if (Number.isFinite(mb) && mb > 0) return { present: true, kind: "nvidia", vramMB: mb };
} catch {
// fall through — no usable GPU
}
return { present: false, kind: null, vramMB: 0 };
}
export function probeSpecs({ osMod = os, exec = defaultExec } = {}) {
const platform = osMod.platform();
const arch = osMod.arch();
const cpuCores = osMod.cpus().length;
const ramMB = Math.round(osMod.totalmem() / (1024 * 1024));
return {
platform,
arch,
cpuCores,
ramMB,
availableRamMB: availableRamMB(platform, exec, ramMB),
appleSilicon: platform === "darwin" && arch === "arm64",
gpu: detectGpu(platform, arch, ramMB, exec),
};
}
// Where Hugging Face actually puts downloaded weights. A free-space check
// against cwd measures the wrong filesystem, so the disk question has to be
// asked about this directory. Precedence follows huggingface_hub's own order.
export function weightsCacheDir({ env = process.env, osMod = os } = {}) {
if (env.HF_HUB_CACHE) return env.HF_HUB_CACHE;
if (env.HUGGINGFACE_HUB_CACHE) return env.HUGGINGFACE_HUB_CACHE;
if (env.HF_HOME) return join(env.HF_HOME, "hub");
return join(osMod.homedir(), ".cache", "huggingface", "hub");
}
// Free space on the filesystem that will hold the weights. The cache dir
// usually does not exist until the first download and statfs throws on a
// missing path, so walk up to the deepest ancestor that does exist. Returns
// null when even the root cannot be read, so callers can say "unknown" instead
// of implying zero and scaring someone off a download that would have worked.
export function freeSpaceMB(dir, statfsFn = statfsSync) {
let path = dir;
for (;;) {
try {
const { bavail, bsize } = statfsFn(path);
return (bavail * bsize) / 1e6;
} catch {
const parent = dirname(path);
if (parent === path) return null;
path = parent;
}
}
}
// One line the user reads BEFORE agreeing to a pull that can be tens of GB.
// Always names the size and where it lands. When it will not fit we say so
// plainly rather than withholding the tier: a machine that could free up space
// should still be told the tier exists. `statfsFn` / `env` / `osMod` are
// injectable for tests.
export function describeDownload(
sizeMB,
{ statfsFn = statfsSync, env = process.env, osMod = os } = {},
) {
const dir = weightsCacheDir({ env, osMod });
const gb = (mb) => (mb / 1000).toFixed(1);
const head = `downloads ~${gb(sizeMB)}GB of weights to ${dir}`;
const free = freeSpaceMB(dir, statfsFn);
if (free == null) return `${head} (free space unknown)`;
if (free < sizeMB) {
return `${head}, but only ${gb(free)}GB is free there, so it will NOT fit as-is`;
}
return `${head} (${gb(free)}GB free there)`;
}