Files
hyperframes/packages/cli/src/capture/contactSheet.ts
T
Miguel ÁngelandClaude Opus 4.8 57b3c78987 feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare + compare (#2041)
* feat(media-use): color grading — grade/lut resolve, smart-grade, grade-compare CLI

Add color grading to media-use as first-class resolve types plus a faithful
comparison command. All local, offline, deterministic — no model, no GPU.

- resolve -t grade / -t lut: produce a data-color-grading block (or a frozen
  .cube). Look cascade: core preset (no file) -> bundled .cube library ->
  parametric buildCube. Emitted .cube is Rec.709 and validated against core's
  colorLuts constraints (LUT_3D_SIZE <= 64) before it is frozen.
- smart grade (grade --for <media>): ffmpeg signalstats -> adjust suggestion
  (exposure / contrast / white balance), surfaced with the measured evidence on
  stderr as a starting point; never auto-applied.
- hyperframes grade-compare: renders N candidate grades onto a reference frame
  through the real runtime shader into one labeled comparison PNG, so an agent
  picks a look without opening Studio. Prepends an "original" baseline cell by
  default (--no-baseline to omit). Shares the headless-capture pipeline with
  snapshot via capture/captureCompositionFrame.
- media-use SKILL: proactive "media opportunity pass" guidance (grounded
  signal -> offer, ask once, surface don't mutate).

Verified: media-use 116/116, grade-compare 7/7, snapshot 9/9, lint + format
clean, full build green, comparison renders end to end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* test(cli): narrow grade-compare baseline assertion off unknown-typed grading

Assert the whole cell via toEqual instead of reaching into .grading.preset /
.grading.lut on the unknown-typed field, keeping the test typecheck-clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* feat(media-use): agent-authored LUTs via --params + validate --from cube; never-read-.cube guardrail

- resolve -t lut / -t grade --params '<json>': build a parametric .cube from
  explicit params (bypassing the intent cascade), validate, and freeze in one
  step. --intent becomes the optional description. Lets an agent commit a look
  it computed itself.
- --from <file.cube> now validates the ingested LUT for lut/grade types and
  rejects an invalid/oversized cube (no partial write) — the escape hatch for a
  LUT the agent generated with its own code.
- SKILL.md: hard rule to never read a .cube body into context (~size^3 lines,
  zero legible signal) — inspect via grade-compare (see it) or cube-validate
  (ok/size), read the manifest description for meaning; plus both authoring
  paths and the parametric-vs-film-stock ceiling note.

Verified: media-use 116/116, lint + format clean; smokes — --params builds a
valid frozen cube, grade --params returns a lut block, bad JSON and an oversized
--from cube are both rejected with no stray file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* fix(cli): grade-compare validates referenced LUTs, warns on no-op cells, caps candidates

Bug-bash follow-ups — grade-compare silently accepted bad input:

- Validate LUT *content*, not just existence: each referenced .cube is parsed
  with core's parseCubeLut (now exported from @hyperframes/core) and rejected
  with a per-cell error ("LUT for \"<label>\" is not a valid .cube: ..."). A
  file that exists but isn't a valid cube no longer renders a silent no-op cell.
- Warn on inactive cells: a grading that normalizes to inactive (e.g. a
  malformed {lut:12345}) emits a stderr warning naming the cell; the
  auto-prepended "original" baseline is intentionally inactive and stays silent.
  stdout remains valid JSON.
- Cap candidates at 16 (excluding baseline): over-cap input renders the first N
  and reports {truncated:true, total:M} on stdout + a stderr note — no silent
  drop, no unbounded giant sheet.

Verified: grade-compare 10/10; non-cube LUT → clear error; {lut:12345} → warning
+ ok; 20 cells → cells=17 truncated total=20; valid runs unchanged. Lint/format
clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* feat(cli): general `hyperframes compare` visual-variant primitive

Generalize grade-compare's "render N variants → one labeled sheet → the agent
looks and picks" loop into a standalone command that works on ANY variation
(font, layout, motion, grade, whole compositions) — the tool never needs to
know what differs.

- `hyperframes compare <path...> [--at <sec>] [--labels a,b,c] [--out] [--cols]
  [--json]`: renders each agent-authored composition variant through the real
  runtime (captureCompositionFrame) and stitches one labeled comparison sheet +
  JSON ({ok, sheet, rendered, variants, truncated?/total?}). 2+ paths required;
  caps at 16 with loud truncation. It presents, it does not judge — choosing is
  the caller's job.
- Factored the shared "render a labeled set → contact sheet" path so compare,
  grade-compare, and snapshot all sit on it (no duplication). grade-compare is
  now the first color-specific specialization of this primitive.
- New pathArgs util + contactSheet test; hyperframes-cli SKILL documents compare
  as the agent's "see your own renders and choose" primitive.

Verified: 26/26 across compare + grade-compare + snapshot + contactSheet (no
regressions); compare renders 3 variants into one visibly-distinct labeled
sheet; 2+-path error path clean; lint/format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* fix(ci): green the skills CI — skip ffmpeg tests when absent, oxfmt markdown

The "Test: skills" CI job runs bare `node --test` with no ffmpeg on PATH (by
design — skills tests are meant to be node-builtin-only). The grade-analyzer +
smart-grade tests shell to ffmpeg and were failing there with ENOENT. Guard
them to skip when ffmpeg isn't on PATH; they still run locally / where it is.

Also oxfmt README.md + hyperframes/media-use SKILL.md (the whole-repo
`oxfmt --check .` Format job caught markdown left unformatted by the rebase
conflict resolution).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* fix(ci): skip core-conformance test when tsx is unavailable

The "Test: skills" CI job installs no deps, so the normalizeHfColorGrading
conformance test (which imports core's TS via `node --import tsx`) failed there.
Guard it to skip when tsx can't resolve; runs locally / in the deps-installed
Test job. Completes the skills-CI greening (the ffmpeg guards handled the rest).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* fix(cli): escape grade-compare src double-quotes (CodeQL XSS) + Windows-safe compare test

- grade-compare built `<img src="...">` (double-quoted) with the single-quote
  escaper, leaving `"` unescaped — a `"` in the frame path could break out
  (CodeQL: incomplete HTML attribute sanitization). Use escapeXml for src.
- compare label test hard-coded POSIX paths that can't match on Windows; assert
  the derived labels (the subject); path resolution is covered elsewhere.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* refactor(media-use): generate LUT library from params (drop committed .cube files)

The 3 bundled .cube files were 733 lines each (2,199 total) and were themselves
buildCube output — pure repo bloat. Replace with compact per-look params in
luts/index.json, generated on resolve; add an optional `url` for future scanned
LUTs to be CDN-hosted + downloaded on demand (freezeUrl) instead of committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* feat(media-use): serve library LUTs from CDN on-demand (static.heygen.ai/luts), params fallback

Looks now carry a CDN `url` (hosted at s3://heygen-public/luts → static.heygen.ai/luts/<id>.cube);
resolve downloads + validates + freezes on demand, like bgm/image. `params` stays
as the deterministic offline fallback (--local-only, or if the download fails), so
resolution is never blocked on the network. Provider prefers url, falls back to params.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

* fix(media-use): address #2041 review — atomic LUT writes, compare telemetry, follow-ups

- Atomic .cube writes: library provider (url + params) and the parametric
  generator now write to a .tmp path, validate, then rename, so a crash can
  never orphan an invalid .cube at the final path (was validate-after-write).
- track("media_use_resolve") now emits provenance.via (url/params-fallback/params).
- grade-compare + compare: --timeout flag (was hardcoded 5000) and a
  media_use_compare event (cells, truncated, total, render_ready_timed_out);
  openSettledCompositionPage now surfaces the render-ready timeout.
- compare staging skips node_modules/.git; --for gets an upfront existence check.
- Rec.709 luma comment; HYPERFRAMES_ANALYZE_TIMEOUT_MS override; measured note
  uses basename; LUT s3 hosting moved from index.json into luts/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H5k87mPZ4d6yiFwcWSb8Vv

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 22:20:16 -04:00

353 lines
10 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Generate labeled contact sheet grids from images.
*
* Stitches images into a numbered grid with cell labels.
* Saves 50-65% tokens vs. AI agents reading images individually.
*/
import sharp from "sharp";
import { readdirSync, readFileSync, writeFileSync, unlinkSync, existsSync } from "node:fs";
import { join, extname, basename, dirname } from "node:path";
interface ContactSheetOptions {
cols?: number;
maxImages?: number;
padding?: number;
labelMode?: "index" | "filename" | "custom";
labels?: string[];
quality?: number;
/** Target width per cell in pixels (default: 600) */
cellWidth?: number;
}
/**
* Create a contact sheet from a list of image paths.
* Returns the output file path, or null if no images.
*/
export async function createContactSheet(
imagePaths: string[],
outputPath: string,
opts: ContactSheetOptions = {},
): Promise<string | null> {
const {
cols = 3,
maxImages = 16,
padding = 4,
labelMode = "index",
labels,
quality = 88,
cellWidth = 600,
} = opts;
const files = imagePaths.slice(0, maxImages);
if (files.length === 0) return null;
// Read first image to determine aspect ratio
const firstMeta = await sharp(files[0]!).metadata();
const srcW = firstMeta.width || 1920;
const srcH = firstMeta.height || 1080;
// Scale to target cell width, maintain aspect ratio
const scale = cellWidth / srcW;
const cellW = cellWidth;
const cellH = Math.round(srcH * scale);
const rows = Math.ceil(files.length / cols);
const labelH = 26;
const totalW = cols * cellW + (cols + 1) * padding;
const totalH = rows * (cellH + labelH) + (rows + 1) * padding;
const overlays: sharp.OverlayOptions[] = [];
for (let i = 0; i < files.length; i++) {
const col = i % cols;
const row = Math.floor(i / cols);
const x = padding + col * (cellW + padding);
const y = padding + row * (cellH + labelH + padding);
// Resize image to cell size — contain keeps full image visible (no cropping)
const resized = await sharp(files[i]!)
.resize(cellW, cellH, { fit: "contain", background: { r: 26, g: 26, b: 26 } })
.toBuffer();
overlays.push({ input: resized, left: x, top: y + labelH });
// Label text
let labelText = `${i + 1}`;
if (labelMode === "filename") {
labelText = `${i + 1}. ${basename(files[i]!).replace(extname(files[i]!), "")}`;
} else if (labelMode === "custom" && labels?.[i]) {
labelText = `${i + 1}. ${labels[i]}`;
}
// Truncate label to fit cell
if (labelText.length > 60) labelText = labelText.slice(0, 57) + "...";
const labelSvg = Buffer.from(
`<svg width="${cellW}" height="${labelH}">` +
`<rect width="${cellW}" height="${labelH}" fill="#1a1a1a"/>` +
`<text x="8" y="18" font-family="Arial,Helvetica,sans-serif" font-size="13" font-weight="bold" fill="#ffffff">${escapeXml(labelText)}</text>` +
`</svg>`,
);
overlays.push({ input: labelSvg, left: x, top: y });
}
const sheet = sharp({
create: {
width: totalW,
height: totalH,
channels: 3,
background: { r: 26, g: 26, b: 26 },
},
}).composite(overlays);
if (extname(outputPath).toLowerCase() === ".png") {
await sheet.png().toFile(outputPath);
} else {
await sheet.jpeg({ quality }).toFile(outputPath);
}
return outputPath;
}
function escapeXml(s: string): string {
return s
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&apos;");
}
/**
* Split imagePaths into pages of `pageSize`, write one contact sheet per page.
* Output files: basePath → base-1.jpg, base-2.jpg, ...
* Returns the list of written file paths (empty if no images).
*/
async function createContactSheetPages(
imagePaths: string[],
outputBasePath: string,
opts: ContactSheetOptions & { pageSize?: number } = {},
labelOffset = 0,
customLabels?: string[],
): Promise<string[]> {
if (imagePaths.length === 0) return [];
const { pageSize = imagePaths.length, ...sheetOpts } = opts;
const ext = outputBasePath.match(/\.[^.]+$/)?.[0] ?? ".jpg";
const base = outputBasePath.slice(0, -ext.length);
const pages = Math.ceil(imagePaths.length / pageSize);
const results: string[] = [];
for (let p = 0; p < pages; p++) {
const chunk = imagePaths.slice(p * pageSize, (p + 1) * pageSize);
const chunkLabels = customLabels?.slice(p * pageSize, (p + 1) * pageSize);
const outPath = pages === 1 ? outputBasePath : `${base}-${p + 1}${ext}`;
const labelsForChunk = chunkLabels
? { labelMode: "custom" as const, labels: chunkLabels }
: sheetOpts.labelMode === "filename"
? { labelMode: "filename" as const }
: { labelMode: "index" as const };
const written = await createContactSheet(chunk, outPath, {
...sheetOpts,
...labelsForChunk,
maxImages: chunk.length,
});
if (written) results.push(written);
void labelOffset; // used by callers that pre-compute labels
}
return results;
}
/**
* Contact sheet for scroll screenshots. Paginated — all screenshots covered.
* Labels: "1. 0% scroll", "2. 23% scroll", etc.
* Returns array of written file paths.
*/
export async function createScrollContactSheet(
screenshotsDir: string,
outputPath: string,
): Promise<string[]> {
if (!existsSync(screenshotsDir)) return [];
const scrollFiles = readdirSync(screenshotsDir)
.filter((f) => f.startsWith("scroll-") && f.endsWith(".png"))
.sort();
if (scrollFiles.length === 0) return [];
const paths = scrollFiles.map((f) => join(screenshotsDir, f));
const labels = scrollFiles.map((f) => {
const m = f.match(/scroll-(\d+)\.png/);
return m ? `${m[1]}% scroll` : f;
});
// 3 cols max for readability; 9 per page (3×3) so cells stay large enough to read
return createContactSheetPages(
paths,
outputPath,
{ cols: 3, cellWidth: 600, pageSize: 9 },
0,
labels,
);
}
/**
* Contact sheet for snapshot frames. All frames covered across pages.
* Labels: "1. 1.0s", "2. 3.0s", etc.
* Returns array of written file paths.
*/
export async function createSnapshotContactSheet(
snapshotsDir: string,
outputPath: string,
): Promise<string[]> {
if (!existsSync(snapshotsDir)) return [];
const snapshotFiles = readdirSync(snapshotsDir)
.filter((f) => f.startsWith("frame-") && f.endsWith(".png"))
.sort();
if (snapshotFiles.length === 0) return [];
const paths = snapshotFiles.map((f) => join(snapshotsDir, f));
const labels = snapshotFiles.map((f) => {
const m = f.match(/at-([\d.]+)s/);
return m ? `${m[1]}s` : f;
});
// 3 cols, 9 per page (3×3)
return createContactSheetPages(
paths,
outputPath,
{ cols: 3, cellWidth: 600, pageSize: 9 },
0,
labels,
);
}
/**
* Contact sheet for captured assets. Paginated — all assets covered.
* Labels: "1. filename"
* Returns array of written file paths.
*/
export async function createAssetContactSheet(
assetsDir: string,
outputPath: string,
): Promise<string[]> {
if (!existsSync(assetsDir)) return [];
const imageExts = new Set([".png", ".jpg", ".jpeg", ".webp"]);
const assetFiles = readdirSync(assetsDir)
.filter((f) => imageExts.has(extname(f).toLowerCase()) && !f.includes("contact-sheet"))
.sort();
if (assetFiles.length === 0) return [];
const paths = assetFiles.map((f) => join(assetsDir, f));
// 4 cols, 12 per page (4×3) — covers all assets across as many pages as needed
return createContactSheetPages(paths, outputPath, {
cols: 4,
cellWidth: 480,
labelMode: "filename",
pageSize: 12,
});
}
/**
* Contact sheet for SVGs — renders each SVG to a thumbnail PNG, then grids them.
* Sharp supports SVG input natively, so no browser needed.
* Labels: "1. filename"
*
* Accepts one or two directories: the primary svgs/ subdir and optionally the
* parent assets/ root (for external SVGs downloaded as <img src="*.svg">).
* Files are deduplicated by basename so duplicates across dirs are collapsed.
*/
// fallow-ignore-next-line complexity
export async function createSvgContactSheet(
svgsDir: string,
outputPath: string,
assetsRootDir?: string,
): Promise<string[]> {
const dirsToScan = [svgsDir, assetsRootDir].filter(
(d): d is string => d !== undefined && existsSync(d),
);
if (dirsToScan.length === 0) return [];
const seen = new Set<string>();
const svgPaths: string[] = [];
for (const dir of dirsToScan) {
for (const f of readdirSync(dir)
.filter((f) => f.endsWith(".svg"))
.sort()) {
if (!seen.has(f)) {
seen.add(f);
svgPaths.push(join(dir, f));
}
}
}
if (svgPaths.length === 0) return [];
const svgFileNames = svgPaths.map((p) => p.split("/").pop()!);
// Render ALL SVGs to PNG thumbnails first, then paginate the sheets
const thumbSize = 200;
const tmpDir = dirname(outputPath);
const tmpPaths: string[] = [];
const labels: string[] = [];
for (let i = 0; i < svgPaths.length; i++) {
const svgPath = svgPaths[i]!;
const tmpPath = join(tmpDir, `.thumb-${i}.png`);
try {
const svgBuf = readFileSync(svgPath);
const thumb = await sharp(svgBuf)
.resize(thumbSize, thumbSize, {
fit: "contain",
background: { r: 245, g: 245, b: 245, alpha: 1 },
})
.flatten({ background: { r: 245, g: 245, b: 245 } })
.png()
.toBuffer();
writeFileSync(tmpPath, thumb);
tmpPaths.push(tmpPath);
labels.push(svgFileNames[i]!.replace(".svg", ""));
} catch {
// SVG might be malformed — skip
}
}
if (tmpPaths.length === 0) return [];
// 5 cols, 15 per page (5×3) — all SVGs covered across pages
let results: string[] = [];
try {
results = await createContactSheetPages(
tmpPaths,
outputPath,
{
cols: 5,
cellWidth: thumbSize,
pageSize: 15,
},
0,
labels,
);
} finally {
for (const tmp of tmpPaths) {
try {
unlinkSync(tmp);
} catch {
/* best effort */
}
}
}
return results;
}