mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 18:26:17 +00:00
* feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare CLI Add color grading to media-use as first-class resolve types plus a faithful comparison command. All local, offline, deterministic — no model, no GPU. - resolve -t grade / -t lut: produce a data-color-grading block (or a frozen .cube). Look cascade: core preset (no file) -> bundled .cube library -> parametric buildCube. Emitted .cube is Rec.709 and validated against core's colorLuts constraints (LUT_3D_SIZE <= 64) before it is frozen. - smart grade (grade --for <media>): ffmpeg signalstats -> adjust suggestion (exposure / contrast / white balance), surfaced with the measured evidence on stderr as a starting point; never auto-applied. - hyperframes grade-compare: renders N candidate grades onto a reference frame through the real runtime shader into one labeled comparison PNG, so an agent picks a look without opening Studio. Prepends an "original" baseline cell by default (--no-baseline to omit). Shares the headless-capture pipeline with snapshot via capture/captureCompositionFrame. - media-use SKILL: proactive "media opportunity pass" guidance (grounded signal -> offer, ask once, surface don't mutate). Verified: media-use 116/116, grade-compare 7/7, snapshot 9/9, lint + format clean, full build green, comparison renders end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * test(cli): narrow grade-compare baseline assertion off unknown-typed grading Assert the whole cell via toEqual instead of reaching into .grading.preset / .grading.lut on the unknown-typed field, keeping the test typecheck-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): agent-authored LUTs via --params + validate --from cube; never-read-.cube guardrail - resolve -t lut / -t grade --params '<json>': build a parametric .cube from explicit params (bypassing the intent cascade), validate, and freeze in one step. --intent becomes the optional description. Lets an agent commit a look it computed itself. - --from <file.cube> now validates the ingested LUT for lut/grade types and rejects an invalid/oversized cube (no partial write) — the escape hatch for a LUT the agent generated with its own code. - SKILL.md: hard rule to never read a .cube body into context (~size^3 lines, zero legible signal) — inspect via grade-compare (see it) or cube-validate (ok/size), read the manifest description for meaning; plus both authoring paths and the parametric-vs-film-stock ceiling note. Verified: media-use 116/116, lint + format clean; smokes — --params builds a valid frozen cube, grade --params returns a lut block, bad JSON and an oversized --from cube are both rejected with no stray file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): grade-compare validates referenced LUTs, warns on no-op cells, caps candidates Bug-bash follow-ups — grade-compare silently accepted bad input: - Validate LUT *content*, not just existence: each referenced .cube is parsed with core's parseCubeLut (now exported from @hyperframes/core) and rejected with a per-cell error ("LUT for \"<label>\" is not a valid .cube: ..."). A file that exists but isn't a valid cube no longer renders a silent no-op cell. - Warn on inactive cells: a grading that normalizes to inactive (e.g. a malformed {lut:12345}) emits a stderr warning naming the cell; the auto-prepended "original" baseline is intentionally inactive and stays silent. stdout remains valid JSON. - Cap candidates at 16 (excluding baseline): over-cap input renders the first N and reports {truncated:true, total:M} on stdout + a stderr note — no silent drop, no unbounded giant sheet. Verified: grade-compare 10/10; non-cube LUT → clear error; {lut:12345} → warning + ok; 20 cells → cells=17 truncated total=20; valid runs unchanged. Lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(cli): general `hyperframes compare` visual-variant primitive Generalize grade-compare's "render N variants → one labeled sheet → the agent looks and picks" loop into a standalone command that works on ANY variation (font, layout, motion, grade, whole compositions) — the tool never needs to know what differs. - `hyperframes compare <path...> [--at <sec>] [--labels a,b,c] [--out] [--cols] [--json]`: renders each agent-authored composition variant through the real runtime (captureCompositionFrame) and stitches one labeled comparison sheet + JSON ({ok, sheet, rendered, variants, truncated?/total?}). 2+ paths required; caps at 16 with loud truncation. It presents, it does not judge — choosing is the caller's job. - Factored the shared "render a labeled set → contact sheet" path so compare, grade-compare, and snapshot all sit on it (no duplication). grade-compare is now the first color-specific specialization of this primitive. - New pathArgs util + contactSheet test; hyperframes-cli SKILL documents compare as the agent's "see your own renders and choose" primitive. Verified: 26/26 across compare + grade-compare + snapshot + contactSheet (no regressions); compare renders 3 variants into one visibly-distinct labeled sheet; 2+-path error path clean; lint/format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): green the skills CI — skip ffmpeg tests when absent, oxfmt markdown The "Test: skills" CI job runs bare `node --test` with no ffmpeg on PATH (by design — skills tests are meant to be node-builtin-only). The grade-analyzer + smart-grade tests shell to ffmpeg and were failing there with ENOENT. Guard them to skip when ffmpeg isn't on PATH; they still run locally / where it is. Also oxfmt README.md + hyperframes/media-use SKILL.md (the whole-repo `oxfmt --check .` Format job caught markdown left unformatted by the rebase conflict resolution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(ci): skip core-conformance test when tsx is unavailable The "Test: skills" CI job installs no deps, so the normalizeHfColorGrading conformance test (which imports core's TS via `node --import tsx`) failed there. Guard it to skip when tsx can't resolve; runs locally / in the deps-installed Test job. Completes the skills-CI greening (the ffmpeg guards handled the rest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(cli): escape grade-compare src double-quotes (CodeQL XSS) + Windows-safe compare test - grade-compare built `<img src="...">` (double-quoted) with the single-quote escaper, leaving `"` unescaped — a `"` in the frame path could break out (CodeQL: incomplete HTML attribute sanitization). Use escapeXml for src. - compare label test hard-coded POSIX paths that can't match on Windows; assert the derived labels (the subject); path resolution is covered elsewhere. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * refactor(media-use): generate LUT library from params (drop committed .cube files) The 3 bundled .cube files were 733 lines each (2,199 total) and were themselves buildCube output — pure repo bloat. Replace with compact per-look params in luts/index.json, generated on resolve; add an optional `url` for future scanned LUTs to be CDN-hosted + downloaded on demand (freezeUrl) instead of committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * feat(media-use): serve library LUTs from CDN on-demand (static.heygen.ai/luts), params fallback Looks now carry a CDN `url` (hosted at s3://heygen-public/luts → static.heygen.ai/luts/<id>.cube); resolve downloads + validates + freezes on demand, like bgm/image. `params` stays as the deterministic offline fallback (--local-only, or if the download fails), so resolution is never blocked on the network. Provider prefers url, falls back to params. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv * fix(media-use): address #2041 review — atomic LUT writes, compare telemetry, follow-ups - Atomic .cube writes: library provider (url + params) and the parametric generator now write to a .tmp path, validate, then rename, so a crash can never orphan an invalid .cube at the final path (was validate-after-write). - track("media_use_resolve") now emits provenance.via (url/params-fallback/params). - grade-compare + compare: --timeout flag (was hardcoded 5000) and a media_use_compare event (cells, truncated, total, render_ready_timed_out); openSettledCompositionPage now surfaces the render-ready timeout. - compare staging skips node_modules/.git; --for gets an upfront existence check. - Rec.709 luma comment; HYPERFRAMES_ANALYZE_TIMEOUT_MS override; measured note uses basename; LUT s3 hosting moved from index.json into luts/README.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
348 lines
11 KiB
JavaScript
348 lines
11 KiB
JavaScript
import { strict as assert } from "node:assert";
|
||
import { mkdtempSync, rmSync, readFileSync, writeFileSync, mkdirSync, existsSync } from "node:fs";
|
||
import { join } from "node:path";
|
||
import { tmpdir } from "node:os";
|
||
import {
|
||
readManifest,
|
||
appendRecord,
|
||
findByPrompt,
|
||
findByEntity,
|
||
nextId,
|
||
allocateId,
|
||
normalizePrompt,
|
||
manifestPath,
|
||
mediaDir,
|
||
typeDirPath,
|
||
typeSubdir,
|
||
} from "./manifest.mjs";
|
||
import { regenerateIndex, generateIndexContent } from "./index-gen.mjs";
|
||
import {
|
||
contentHash,
|
||
cachePut,
|
||
cacheGet,
|
||
cacheGetByEntity,
|
||
importFromCache,
|
||
promote,
|
||
} from "./cache.mjs";
|
||
|
||
let tmp;
|
||
|
||
function setup() {
|
||
tmp = mkdtempSync(join(tmpdir(), "mu-test-"));
|
||
}
|
||
|
||
function cleanup() {
|
||
if (tmp) rmSync(tmp, { recursive: true, force: true });
|
||
}
|
||
|
||
function makeRecord(overrides = {}) {
|
||
return {
|
||
id: "bgm_001",
|
||
type: "bgm",
|
||
path: ".media/audio/bgm/bgm_001.wav",
|
||
source: "search",
|
||
description: "soft minimal ambient",
|
||
duration: 11,
|
||
provenance: { provider: "heygen.audio.sounds", prompt: "subtle tech" },
|
||
...overrides,
|
||
};
|
||
}
|
||
|
||
function runTests() {
|
||
const tests = [];
|
||
|
||
function test(name, fn) {
|
||
tests.push({ name, fn });
|
||
}
|
||
|
||
// --- manifest.mjs ---
|
||
|
||
test("readManifest returns empty array when no manifest exists", () => {
|
||
setup();
|
||
const result = readManifest(tmp);
|
||
assert.deepStrictEqual(result, []);
|
||
cleanup();
|
||
});
|
||
|
||
test("appendRecord writes valid JSONL and readManifest parses it back", () => {
|
||
setup();
|
||
const record = makeRecord();
|
||
appendRecord(tmp, record);
|
||
const records = readManifest(tmp);
|
||
assert.equal(records.length, 1);
|
||
assert.deepStrictEqual(records[0], record);
|
||
cleanup();
|
||
});
|
||
|
||
test("appendRecord creates .media/ and type subdirs on first write", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord());
|
||
assert.ok(existsSync(mediaDir(tmp)));
|
||
assert.ok(existsSync(typeDirPath(tmp, "bgm")));
|
||
cleanup();
|
||
});
|
||
|
||
test("lut and grade artifacts use the shared .media/luts subdir", () => {
|
||
assert.equal(typeSubdir("lut"), "luts");
|
||
assert.equal(typeSubdir("grade"), "luts");
|
||
|
||
setup();
|
||
const allocated = allocateId(tmp, "lut", ".cube");
|
||
assert.equal(allocated.localPath, ".media/luts/lut_001.cube");
|
||
assert.ok(existsSync(join(tmp, allocated.localPath)));
|
||
cleanup();
|
||
});
|
||
|
||
test("appendRecord appends multiple records", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
|
||
appendRecord(tmp, makeRecord({ id: "bgm_002", provenance: { prompt: "energetic" } }));
|
||
const records = readManifest(tmp);
|
||
assert.equal(records.length, 2);
|
||
assert.equal(records[0].id, "bgm_001");
|
||
assert.equal(records[1].id, "bgm_002");
|
||
cleanup();
|
||
});
|
||
|
||
test("findByPrompt returns exact-match record", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord());
|
||
const found = findByPrompt(tmp, "subtle tech", "bgm");
|
||
assert.ok(found);
|
||
assert.equal(found.id, "bgm_001");
|
||
cleanup();
|
||
});
|
||
|
||
test("findByPrompt returns null on miss", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord());
|
||
assert.equal(findByPrompt(tmp, "nonexistent", "bgm"), null);
|
||
cleanup();
|
||
});
|
||
|
||
test("findByPrompt filters by type", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ type: "sfx" }));
|
||
assert.equal(findByPrompt(tmp, "subtle tech", "bgm"), null);
|
||
assert.ok(findByPrompt(tmp, "subtle tech", "sfx"));
|
||
cleanup();
|
||
});
|
||
|
||
test("findByPrompt matches across case and whitespace variants", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ provenance: { provider: "x", prompt: "calm ambient piano" } }));
|
||
assert.ok(findByPrompt(tmp, "Calm Ambient Piano", "bgm"), "case-insensitive");
|
||
assert.ok(findByPrompt(tmp, " calm ambient piano ", "bgm"), "whitespace-insensitive");
|
||
assert.equal(findByPrompt(tmp, "calm ambient guitar", "bgm"), null, "still a real miss");
|
||
cleanup();
|
||
});
|
||
|
||
test("normalizePrompt trims, lowercases, collapses whitespace", () => {
|
||
assert.equal(normalizePrompt(" Upbeat Tech Launch "), "upbeat tech launch");
|
||
assert.equal(normalizePrompt(null), "");
|
||
});
|
||
|
||
test("allocateId reserves the id on disk so a pre-append caller can't reuse it (MU-23)", () => {
|
||
setup();
|
||
const a = allocateId(tmp, "bgm", ".wav");
|
||
assert.equal(a.id, "bgm_001");
|
||
assert.ok(existsSync(join(tmp, a.localPath)), "placeholder reserved on disk");
|
||
// Second allocation BEFORE any manifest append (the download window) must not
|
||
// hand back bgm_001 again, even with a different extension.
|
||
const b = allocateId(tmp, "bgm", ".mp3");
|
||
assert.equal(b.id, "bgm_002");
|
||
assert.notEqual(a.localPath, b.localPath);
|
||
// Lock file is released (not left behind).
|
||
assert.ok(!existsSync(join(tmp, ".media", ".lock")), "lock released");
|
||
cleanup();
|
||
});
|
||
|
||
test("allocateId continues past the highest manifest id", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ id: "bgm_005" }));
|
||
assert.equal(allocateId(tmp, "bgm", ".wav").id, "bgm_006");
|
||
cleanup();
|
||
});
|
||
|
||
test("findByEntity matches case-insensitively", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ entity: "GitHub", type: "icon" }));
|
||
assert.ok(findByEntity(tmp, "github"));
|
||
assert.ok(findByEntity(tmp, "GITHUB"));
|
||
assert.equal(findByEntity(tmp, "gitlab"), null);
|
||
cleanup();
|
||
});
|
||
|
||
test("nextId generates sequential ids", () => {
|
||
setup();
|
||
assert.equal(nextId(tmp, "bgm"), "bgm_001");
|
||
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
|
||
assert.equal(nextId(tmp, "bgm"), "bgm_002");
|
||
appendRecord(tmp, makeRecord({ id: "bgm_002" }));
|
||
assert.equal(nextId(tmp, "bgm"), "bgm_003");
|
||
cleanup();
|
||
});
|
||
|
||
// --- index-gen.mjs ---
|
||
|
||
test("regenerateIndex produces plain-column table", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord());
|
||
regenerateIndex(tmp);
|
||
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
|
||
assert.ok(content.includes("# .media · 1 asset"));
|
||
assert.ok(content.includes("bgm_001"));
|
||
assert.ok(content.includes("soft minimal ambient"));
|
||
assert.ok(content.includes("11s"));
|
||
cleanup();
|
||
});
|
||
|
||
test("regenerateIndex handles empty manifest", () => {
|
||
setup();
|
||
mkdirSync(join(tmp, ".media"), { recursive: true });
|
||
writeFileSync(manifestPath(tmp), "");
|
||
regenerateIndex(tmp);
|
||
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
|
||
assert.ok(content.includes("# .media · 0 assets"));
|
||
cleanup();
|
||
});
|
||
|
||
test("generateIndexContent includes dims for images", () => {
|
||
const records = [
|
||
makeRecord({ id: "img_001", type: "image", width: 1920, height: 1080, duration: null }),
|
||
];
|
||
const content = generateIndexContent(records);
|
||
assert.ok(content.includes("1920×1080"));
|
||
assert.ok(content.includes("img_001"));
|
||
});
|
||
|
||
test("regenerateIndex matches manifest content after multiple writes", () => {
|
||
setup();
|
||
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
|
||
appendRecord(
|
||
tmp,
|
||
makeRecord({ id: "sfx_001", type: "sfx", description: "whoosh", duration: 3 }),
|
||
);
|
||
regenerateIndex(tmp);
|
||
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
|
||
assert.ok(content.includes("# .media · 2 assets"));
|
||
assert.ok(content.includes("bgm_001"));
|
||
assert.ok(content.includes("sfx_001"));
|
||
assert.ok(content.includes("whoosh"));
|
||
cleanup();
|
||
});
|
||
|
||
// --- cache.mjs ---
|
||
|
||
test("cacheGet returns null when cache is empty", () => {
|
||
const result = cacheGet("nonexistent prompt", "bgm");
|
||
assert.equal(result, null);
|
||
});
|
||
|
||
test("cachePut + cacheGet round-trip", () => {
|
||
setup();
|
||
const filePath = join(tmp, "test.wav");
|
||
writeFileSync(filePath, "fake audio bytes for testing");
|
||
const record = makeRecord({ provenance: { prompt: "cache test" } });
|
||
|
||
const { sha } = cachePut(filePath, record);
|
||
assert.ok(sha);
|
||
assert.equal(sha.length, 64);
|
||
|
||
const found = cacheGet("cache test", "bgm");
|
||
assert.ok(found);
|
||
assert.equal(found.reusable, true);
|
||
assert.equal(found.sha, sha);
|
||
|
||
// cross-project reuse must survive trivial prompt variation, not just
|
||
// byte-identical intents (the whole point of normalizePrompt).
|
||
assert.ok(cacheGet(" Cache Test ", "bgm"), "cacheGet is case/whitespace-insensitive");
|
||
cleanup();
|
||
});
|
||
|
||
test("cacheGetByEntity finds cached asset", () => {
|
||
setup();
|
||
const filePath = join(tmp, "logo.png");
|
||
writeFileSync(filePath, "fake png bytes");
|
||
const record = makeRecord({
|
||
type: "icon",
|
||
entity: "TestCorp",
|
||
provenance: { prompt: "TestCorp logo" },
|
||
});
|
||
|
||
cachePut(filePath, record);
|
||
const found = cacheGetByEntity("testcorp");
|
||
assert.ok(found);
|
||
assert.equal(found.entity, "TestCorp");
|
||
cleanup();
|
||
});
|
||
|
||
test("contentHash is deterministic", () => {
|
||
setup();
|
||
const filePath = join(tmp, "det.bin");
|
||
writeFileSync(filePath, "deterministic content");
|
||
const h1 = contentHash(filePath);
|
||
const h2 = contentHash(filePath);
|
||
assert.equal(h1, h2);
|
||
cleanup();
|
||
});
|
||
|
||
test("promote copies project asset to global cache", () => {
|
||
setup();
|
||
const record = makeRecord();
|
||
appendRecord(tmp, record);
|
||
const filePath = join(tmp, record.path);
|
||
mkdirSync(join(filePath, ".."), { recursive: true });
|
||
writeFileSync(filePath, "promotable audio data");
|
||
|
||
const { sha } = promote(tmp, "bgm_001");
|
||
assert.ok(sha);
|
||
|
||
const cached = cacheGet("subtle tech", "bgm");
|
||
assert.ok(cached);
|
||
assert.equal(cached.sha, sha);
|
||
cleanup();
|
||
});
|
||
|
||
test("importFromCache copies cached file into project", () => {
|
||
setup();
|
||
const filePath = join(tmp, "source.wav");
|
||
writeFileSync(filePath, "importable audio");
|
||
const record = makeRecord({ provenance: { prompt: "import test" } });
|
||
const { sha } = cachePut(filePath, record);
|
||
|
||
const cached = cacheGet("import test", "bgm");
|
||
const projectDir = mkdtempSync(join(tmpdir(), "mu-import-"));
|
||
const imported = importFromCache(cached, projectDir, "bgm_001", ".media/audio/bgm/bgm_001.wav");
|
||
|
||
assert.ok(imported);
|
||
assert.equal(imported.id, "bgm_001");
|
||
assert.equal(imported.provenance.imported_from, sha);
|
||
assert.ok(existsSync(join(projectDir, ".media/audio/bgm/bgm_001.wav")));
|
||
|
||
rmSync(projectDir, { recursive: true, force: true });
|
||
cleanup();
|
||
});
|
||
|
||
// --- run ---
|
||
|
||
let passed = 0;
|
||
let failed = 0;
|
||
for (const { name, fn } of tests) {
|
||
try {
|
||
fn();
|
||
passed++;
|
||
console.log(` \x1b[32m✓\x1b[0m ${name}`);
|
||
} catch (err) {
|
||
failed++;
|
||
console.log(` \x1b[31m✗\x1b[0m ${name}`);
|
||
console.log(` ${err.message}`);
|
||
}
|
||
}
|
||
console.log(`\n${passed} passed, ${failed} failed`);
|
||
if (failed > 0) process.exit(1);
|
||
}
|
||
|
||
console.log("media-use · manifest/index/cache tests\n");
|
||
runTests();
|