ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-02 12:08:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2753 from heygen-com/feat/media-treatment-agent

Browse Source 
feat(cli): add agent-first media treatment workflow
This commit is contained in:
Ular Kimsanov
2026-07-24 19:33:31 -07:00
committed by GitHub
parent e710a1686f 4582881d00
commit 3978b8e8c9
28 changed files with 3402 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/guides/color-grading.mdx
+160 -6
View File
@@ -1,6 +1,6 @@
---
title: Color Grading
description: "Apply real-time color grading, presets, LUTs, vignette, grain, blur, and pixelate to video and image media in Studio and final renders."
description: "Apply real-time color grading, presets, LUTs, finishing, and media effects to video and image media in Studio and final renders."
---
HyperFrames Studio can color grade project-local `<video>` and `<img>` media directly in the preview. The same `data-color-grading` settings are used by the render pipeline, so the exported video should match the look you preview.
@@ -15,8 +15,9 @@ This is a lightweight media color tool for generated videos, uploaded footage, s
| Manual controls | Supported | Exposure, contrast, highlights, shadows, white point, black point, warmth, tint, vibrance, saturation. |
| Presets | Supported | Named HyperFrames presets backed by shader settings, not bundled third-party LUT packs. |
| Custom LUT upload | Supported | Project-local 3D `.cube` LUT files with strength control. |
| Finishing | Supported | Vignette and grain, with advanced settings behind the settings icon. |
| Effects | Supported | Blur and pixelate on the selected media surface. |
| Finish | Supported | Vignette and grain, with advanced settings behind the settings icon. |
| Studio Effects panel | Supported | Essentials, Retro & Glitch, Print, and Art effects share the same media shader and persisted `effects` object. |
| Studio Overlays panel | Supported | Inserts selected Registry overlays at the media timing as ordinary timeline layers. |
| Before preview | Supported | Hold the compare button to temporarily show the ungraded media. |
| Render parity | Supported | The render pipeline redraws the color-grading shader after video-frame injection. |
@@ -36,6 +37,7 @@ Studio labels `whites`, `blacks`, and `temperature` as White Point, Black Point,
| Rec.709 creative LUTs | Yes | Best LUT path today. Use project-local 3D `.cube` files. |
| Camera conversion LUTs | Partial | Technically accepted if they are supported 3D `.cube` files, but correctness depends on the source footage matching the LUT's expected input color space. |
| Full-scene grading including text/DOM | Not yet | Color Grading is media-only. Captions, text, SVG, and regular DOM overlays stay unchanged. |
| Face/region-only grading or privacy | Not yet | Realtime effects process the whole selected `<img>` or `<video>`. Isolate the region into a separate cropped/masked media layer or use an external segmentation/tracking tool first. |
| Remote media URLs | Partial | WebGL pixel processing requires compatible CORS headers. Project-local assets are the reliable path. |
| Professional ACES/OCIO/HDR finishing | Not yet | Future render/color-management work, not this Studio shader path. |
@@ -70,8 +72,24 @@ Color grading is stored on media elements as `data-color-grading`:
"grainRoughness":0.55
},
"effects":{
"blur":0.08
"blur":0.08,
"chromaBleed":0,
"tapeDamage":0,
"tapeTracking":0,
"tapeNoise":1,
"tapeSpeed":0.5,
"filmArtifacts":0,
"halftone":0,
"halftoneSize":0,
"twoInkPrint":0,
"twoInkPrintSize":0,
"ascii":0,
"asciiSize":0.066,
"asciiInvert":0,
"dither":0,
"ditherSize":0.25
},
"palette":["#0b0d0d","#eee9db"],
"colorSpace":"rec709"
}'
></video>
@@ -79,6 +97,26 @@ Color grading is stored on media elements as `data-color-grading`:
The runtime creates a sibling WebGL canvas for the media element, samples the current video or image frame, applies shader uniforms, then hides the native media only after a shader frame is ready.
## Studio Workflow
Select a real `<img>` or `<video>` element in Studio to open the media tools:
1. **Presets** shows every built-in starting point in a responsive preview
grid. Hover to preview it on the selected media, click to apply it, or choose
**Original** to return to the neutral preset.
2. **Adjust** provides tonal and color correction.
3. **Effects** groups configurable shader controls under Essentials, Retro &
Glitch, Print, and Art.
4. **Finish** provides vignette and grain.
5. **Custom LUT** accepts a project-local 3D `.cube` file.
6. **Overlays** installs Camcorder HUD, Editorial Flash, Organic Light Leak,
or Freeze-Frame Cutout from the existing Registry.
An overlay inserted from this panel starts with the selected media, uses its
duration, and is placed on the next visual track when one is known. It then
behaves like any other composition layer: edit or remove it through Layers and
Timeline. The left Catalog remains the browse-all Registry surface.
<Note>
Color Grading is intentionally **media-only**. It applies to `<video>` and `<img>` sources. Captions, text, divs, SVG, and UI graphics remain ungraded unless you render them into media first.
</Note>
@@ -116,8 +154,24 @@ Project-local media is the safest path. Remote media must be served with compati
},
"effects": {
"blur": 0,
"pixelate": 0
"pixelate": 0,
"chromaBleed": 0,
"tapeDamage": 0,
"tapeTracking": 0,
"tapeNoise": 1,
"tapeSpeed": 0.5,
"filmArtifacts": 0,
"halftone": 0,
"halftoneSize": 0,
"twoInkPrint": 0,
"twoInkPrintSize": 0,
"ascii": 0,
"asciiSize": 0.066,
"asciiInvert": 0,
"dither": 0,
"ditherSize": 0.25
},
"palette": ["#0b0d0d", "#eee9db"],
"lut": {
"src": "assets/luts/look.cube",
"intensity": 0.75
@@ -128,6 +182,88 @@ Project-local media is the safest path. Remote media must be served with compati
Omit `enabled`; the presence of `data-color-grading` implies that grading is active. All numeric controls are clamped by the runtime. The current color grading path is Rec.709/sRGB-oriented and assumes browser-decoded media frames.
`chromaBleed` is a bounded `0` to `1` treatment primitive that horizontally
softens chroma detail while preserving the center sample's luma. It is useful
inside restrained creator-camera/camcorder treatments; it is not VHS, CRT, RGB
split, tracking noise, or a complete camera emulation.
Most effect amounts and settings are normalized from `0` to `1`. Bloom supports
up to `3`, bloom radius uses pixels, and enum controls use the integer choices
shown in Studio:
- `tapeDamage` adds deterministic horizontal time-base instability, lower luma
bandwidth, restrained ghosting, noise, sparse dropouts, and bottom-edge head
switching. `tapeTracking` adds bounded moving horizontal tracking tears,
`tapeNoise` scales tape noise and row jitter, and `tapeSpeed` controls their
deterministic motion (`0.5` is normal speed). These three subordinate
controls do nothing without `tapeDamage`. A complete analog-tape treatment
may also use restrained `chromaBleed`, scanlines, RGB separation, and rare
row tears; none of these controls adds CRT curvature.
- `filmArtifacts` adds deterministic sparse dust and short scratches. Combine
it with existing grain, vignette, color, and seek-safe GSAP gate weave for an
8mm treatment. It does not change the frame by itself when set to `0`.
- `halftone` blends in a four-angle CMYK print screen. `halftoneSize` controls
its resolution-aware dot-cell size from fine to coarse. The channel angles
and edge response use fixed print-oriented defaults to keep authored output
consistent across agents.
- `twoInkPrint` maps warm midtones and deep/cool shadows to fixed original
vermilion and teal spot screens with a dark overprint on warm paper.
`twoInkPrintSize` controls its resolution-aware screen size. Do not combine
it with `halftone` or describe it as a named commercial print process.
- `ascii` converts the selected media to a procedural 5x7 glyph field.
`asciiSize` controls cell size and `asciiInvert` switches the light/dark ink
polarity. It uses the first and last colors from `palette`.
- `dither` applies a temporally stable ordered 4x4 Bayer dither.
`ditherSize` controls cell size and all `palette` colors participate in the
result. This is ordered dithering, not Floyd-Steinberg or another sequential
error-diffusion algorithm.
- `bloom` extracts bright pixels and runs a bounded half-resolution separable
blur. `bloomRadius` controls its radius.
- `monoScreen` provides configurable mono print shapes, angle, spread, invert,
and palette controls.
- `scanlines`, `crtCurvature`, and `chromaticAberration` provide display
geometry and channel treatments with their related settings.
- `digitalGlitch` provides deterministic line tears, blocks, displacement,
selective pixelation, channel split, opacity, and speed controls.
- `engraving`, `crosshatch`, and `kuwahara` provide stylized art treatments with
their calibrated settings. Kuwahara uses bounded half-float intermediate
targets when the browser supports them and otherwise reports unavailable.
`palette` accepts two to six exact `#RRGGBB` colors in authored order. Use
dark-to-light order for normal luminance mapping; intentionally reverse the
array for an inverted result. The runtime validates and lowercases colors but
does not reorder them. ASCII, dither, mono screen, engraving, and crosshatch use
it; a palette or subordinate setting alone does not allocate an effect. Omit it
for the family default.
When effects are combined, HyperFrames evaluates them in one fixed,
deterministic order: source framing and multipass blur/Kuwahara preparation;
chromatic and digital glitch; primary color grading and LUT blended by global
intensity; grain and film
artifacts; mono/engraving/crosshatch/halftone/two-ink/dither/ASCII; bloom,
scanlines, vignette, and CRT display masking; then before/after comparison.
Effects cannot currently be reordered. The fixed
pipeline keeps Studio, playback, seeking, and final render behavior
predictable.
Studio exposes these controls in the selected media element's **Effects**
accordion. Agents should choose one primary intent through the `media-use`
skill, then either seed from a source-aware treatment recipe or inspect the
canonical toolbox and assemble a bespoke combination:
```bash
hyperframes media-treatment --capabilities --json
hyperframes media-treatment --capability kuwahara --json
```
The first command reports a concise overview of every capability family. The
focused query reports one family's or effect's controls, calibrated apply
payload, render lane, palette support, and seek-safe animation paths. Use
`--all` only for exhaustive tooling. Recipes are tested shortcuts, not the
complete allowed surface. Persist the final combined payload with the same
`hyperframes media-treatment` command; unknown keys are rejected before the
composition is changed.
## Custom LUTs
HyperFrames supports project-local 3D `.cube` LUT files:
@@ -157,6 +293,15 @@ Color Grading is part of the media runtime, so render uses the same settings as
For 4K output, use the existing [4K Rendering](/guides/4k-rendering) workflow. Color Grading can run at 4K when the composition/render surface is 4K, but a 1080p source video does not become sharper just because the final render is 4K.
Performance follows the total number of treated pixels and the selected render
lane, not only the number of media elements. Several tiled videos can be
cheaper than several overlapping full-frame videos. Blur, Bloom, and Kuwahara
use multipass rendering. When more than two full-frame multipass-treated media
elements are visible together, verify continuous playback on the target
machine and simplify or pre-render the stack if frames drop. HyperFrames does
not impose a universal hard cap because GPU and decoder capacity varies by
device.
For HDR output, use the existing [HDR Rendering](/guides/hdr) workflow. Color Grading currently warns on detected HDR media, but the grading controls themselves are not HDR-aware.
When grading a video, animate opacity on a wrapper element instead of directly on the `<video>` element. The runtime hides the native media and draws the graded result through a sibling canvas, so wrapper opacity preserves preview/render parity.
@@ -168,7 +313,16 @@ When grading a video, animate opacity on a wrapper element instead of directly o
| Make uploaded footage look cleaner | Color Grading preset + adjust controls |
| Use a look from another editor | Custom 3D `.cube` LUT |
| Add polish to a product shot | Vignette, subtle grain, contrast, vibrance |
| Blur or pixelate selected media | Effects inside Color Grading |
| Blur or pixelate selected media | Effects panel |
| Add restrained camcorder chroma softness | Effects panel or `effects.chromaBleed` through an agent |
| Build an analog VHS treatment | `effects.tapeDamage` + bounded tracking/noise/speed + restrained chroma, scanline, row-tear, grain, and color settings |
| Build an 8mm home-movie treatment | `effects.filmArtifacts` + grain/vignette/color + seek-safe GSAP weave |
| Build a print/editorial treatment | `effects.halftone` + `effects.halftoneSize` |
| Build a two-spot editorial print | `effects.twoInkPrint` + `effects.twoInkPrintSize` |
| Build a terminal/editorial character treatment | `effects.ascii` + `effects.asciiSize` + an optional two-color `palette` |
| Build a restrained multi-color pixel treatment | `effects.dither` + `effects.ditherSize` + a two-to-six-color `palette` |
| Add an organic light leak | Install the finite `organic-light-leak-overlay` Registry block |
| Build a freeze-frame cutout | Existing background removal + the `freeze-frame-cutout` Registry overlay block + host GSAP |
| Make a presenter float over graphics | Existing [Remove Background](/guides/remove-background) workflow |
| Put text behind a presenter | Existing `remove-background --background-output` workflow |
| Render HDR delivery files | Existing [HDR Rendering](/guides/hdr) workflow |
packages/cli/src/cli.commands.test.ts
+6
View File
@@ -35,6 +35,12 @@ describe("CLI command registration", () => {
);
});
it("registers media-treatment as the only treatment authoring command", () => {
const loaders = commandLoaderBlock();
expect(loaders).toContain('"media-treatment"');
expect(loaders).not.toContain('"color-grading"');
});
// A command actively reconciling skills (`skills check`/`skills update`)
// must not also nudge the user to go reconcile skills — that nudge is
// either redundant (it just ran) or misleading (a stale cached count from
packages/cli/src/cli.ts
+2
View File
@@ -152,6 +152,8 @@ const commandLoaders = {
events: () => import("./commands/events.js").then((m) => m.default),
validate: () => import("./commands/validate.js").then((m) => m.default),
snapshot: () => import("./commands/snapshot.js").then((m) => m.default),
"media-treatment": () =>
import("./commands/media-treatment.js").then((m) => m.mediaTreatmentCommand),
"grade-compare": () => import("./commands/grade-compare.js").then((m) => m.default),
compare: () => import("./commands/compare.js").then((m) => m.default),
capture: () => import("./commands/capture.js").then((m) => m.default),
packages/cli/src/commands/coreSkillContent.test.ts
+53
View File
@@ -73,3 +73,56 @@ describe("media-use TTS documentation", () => {
expect(captions).toContain("heygen-tts.mjs");
});
});
describe("media treatment routing documentation", () => {
it("routes vague composition-media feedback to the canonical workflow", () => {
const router = read("skills", "hyperframes", "SKILL.md");
const mediaUse = read("skills", "media-use", "SKILL.md");
const treatments = read("skills", "media-use", "references", "media-treatments.md");
expect(router).toContain("dark/flat/boring footage");
expect(router).toContain("`/media-use`");
expect(mediaUse).toContain("references/media-treatments.md");
expect(mediaUse).toContain("`hyperframes media-treatment`");
expect(treatments).toContain("Persist pixel settings with `hyperframes media-treatment`");
expect(treatments).toContain("apply to the entire selected real `<img>` or");
expect(treatments).toContain("external segmentation/tracking tool");
});
it("keeps discovery progressive and verification visual", () => {
const treatments = read("skills", "media-use", "references", "media-treatments.md");
const recipes = read("skills", "media-use", "references", "media-treatment-recipes.md");
expect(treatments).toContain("hyperframes media-treatment --capabilities --json");
expect(treatments).toContain("--capability <id>");
expect(treatments).toContain("Recipes are optional macros");
expect(recipes).toContain("optional tested seeds");
expect(treatments).toContain("hyperframes add <name> --dir <project>");
expect(treatments).toContain("snapshots/treatment-before/contact-sheet.jpg");
expect(treatments).toMatch(/Do not report visual\s+quality from command success alone/);
});
it("indexes calibrated treatment recipes without making them mandatory", () => {
const treatments = read("skills", "media-use", "references", "media-treatments.md");
const recipes = read("skills", "media-use", "references", "media-treatment-recipes.md");
for (const heading of [
"Monochrome Screen Print",
"Engraved Illustration",
"Crosshatched Sketch",
"CRT Display",
]) {
expect(treatments).toContain(`\`${heading}\``);
expect(recipes).toContain(`## ${heading}`);
}
});
it("places the media-treatment discovery gate in new project instructions", () => {
for (const file of ["AGENTS.md", "CLAUDE.md"]) {
const template = read("packages", "cli", "src", "templates", "_shared", file);
expect(template).toContain("Changing how real footage or images look or reveal?");
expect(template).toContain("Load `/media-use`");
expect(template).toContain("do not improvise equivalent CSS/SVG filters or overlays");
}
});
});
packages/cli/src/commands/media-treatment.test.ts
+271
View File
@@ -0,0 +1,271 @@
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runCommand } from "citty";
import { describe, expect, it, vi } from "vitest";
import {
HF_COLOR_GRADING_ACTIVE_EFFECT_KEYS,
getHfColorGradingCapabilities,
} from "@hyperframes/core";
import {
applyMediaTreatmentToHtml,
getMediaTreatmentCapabilityDetail,
getMediaTreatmentCapabilityOverview,
mediaTreatmentCommand,
} from "./media-treatment.js";
import { CliRuntimeError } from "../utils/commandResult.js";
const VIDEO = `<!doctype html><html><body><video id="hero" src="hero.mp4"></video></body></html>`;
describe("applyMediaTreatmentToHtml", () => {
it("provides a concise first-hop overview of the complete treatment surface", () => {
const overview = getMediaTreatmentCapabilityOverview();
expect(overview.families.find(({ id }) => id === "correction")).not.toHaveProperty("items");
expect(overview.families.find(({ id }) => id === "art")).not.toHaveProperty("items");
expect(overview.families.find(({ id }) => id === "overlays")).toMatchObject({
owner: "registry",
});
expect(overview.families.some(({ id }) => id === "looks" || id === "treatments")).toBe(false);
const discoveredEffects = ["essentials", "retro-glitch", "print", "art"].flatMap((family) => {
const detail = getMediaTreatmentCapabilityDetail(family);
return (
typeof detail === "object" &&
detail !== null &&
"effects" in detail &&
Array.isArray(detail.effects)
? detail.effects
: []
).map((effect) =>
typeof effect === "object" && effect && "id" in effect ? effect.id : null,
);
});
expect(discoveredEffects.sort()).toEqual([...HF_COLOR_GRADING_ACTIVE_EFFECT_KEYS].sort());
expect(JSON.stringify(overview).length).toBeLessThan(3_000);
});
it("returns focused controls and apply data for one capability", () => {
expect(getMediaTreatmentCapabilityDetail("kuwahara")).toMatchObject({
id: "kuwahara",
family: "art",
renderLane: "multipass",
apply: { effects: { kuwahara: 1 } },
animation: {
property: expect.objectContaining({ path: "effects.kuwahara" }),
initial: expect.stringContaining("--hf-color-grading-kuwahara"),
tween: expect.stringContaining("timeline.to"),
},
});
expect(getMediaTreatmentCapabilityDetail("retro-glitch")).toMatchObject({
id: "retro-glitch",
effects: expect.arrayContaining([expect.objectContaining({ id: "chromaBleed" })]),
});
expect(getMediaTreatmentCapabilityDetail("deep-sea")).toMatchObject({
id: "deep-sea",
apply: { palette: expect.arrayContaining(["#0a1628"]) },
});
expect(getMediaTreatmentCapabilityDetail("exposure")).toMatchObject({
id: "exposure",
family: "correction",
animation: {
property: expect.objectContaining({ path: "adjust.exposure" }),
},
});
expect(getMediaTreatmentCapabilityDetail("vignette")).toMatchObject({
id: "vignette",
family: "finishing",
control: expect.objectContaining({ key: "vignette" }),
});
});
it("rejects unknown capability lookups", () => {
expect(() => getMediaTreatmentCapabilityDetail("make-it-cinematic")).toThrow(
/Unknown media-treatment capability/,
);
expect(() => getMediaTreatmentCapabilityDetail("__proto__")).toThrow(
/Unknown media-treatment capability/,
);
});
it("exposes enough canonical metadata to assemble a custom treatment", () => {
const capabilities = getHfColorGradingCapabilities();
expect(capabilities.targetTags).toEqual(["img", "video"]);
expect(capabilities.effects.find(({ key }) => key === "kuwahara")?.apply).toMatchObject({
kuwahara: 1,
kuwaharaRadius: 1 / 7,
});
expect(capabilities.animatable.find(({ path }) => path === "effects.blur")?.name).toBe(
"--hf-color-grading-blur",
);
});
it("normalizes and persists a grading payload on real media", () => {
const result = applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: { preset: "warm-daylight", intensity: 0.8 },
});
expect(result.changed).toBe(true);
expect(result.tag).toBe("video");
expect(result.value).toContain('"preset":"warm-daylight"');
expect(result.value).toContain('"intensity":0.8');
expect(result.html).toContain("data-color-grading=");
});
it("merges a validated patch and reports the stored before and after payloads", () => {
const initial = applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: {
adjust: { exposure: 0.1 },
details: { grain: 0.2 },
},
});
const patched = applyMediaTreatmentToHtml(initial.html, {
selector: "#hero",
grading: { adjust: { shadows: 0.08 } },
});
expect(patched.before).toMatchObject({
adjust: { exposure: 0.1 },
details: { grain: 0.2 },
});
expect(patched.after).toMatchObject({
adjust: { exposure: 0.1, shadows: 0.08 },
details: { grain: 0.2 },
});
const repeated = applyMediaTreatmentToHtml(patched.html, {
selector: "#hero",
grading: { adjust: { shadows: 0.08 } },
});
expect(repeated.changed).toBe(false);
expect(repeated.html).toBe(patched.html);
expect(repeated.after).toEqual(repeated.before);
});
it("preserves unresolved variable references for runtime resolution", () => {
const wholeGrade = applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: "$interviewGrade",
});
expect(wholeGrade.value).toBe("$interviewGrade");
const nested = applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: { adjust: { exposure: "$interviewExposure" } },
});
expect(JSON.parse(nested.value ?? "{}")).toMatchObject({
adjust: { exposure: "$interviewExposure" },
});
const storedVariable = VIDEO.replace(" src=", ` data-color-grading="$interviewGrade" src=`);
expect(() =>
applyMediaTreatmentToHtml(storedVariable, {
selector: "#hero",
grading: { adjust: { exposure: 0.1 } },
}),
).toThrow(/Cannot merge.*unresolved whole-grade variable/);
});
it("requires an unambiguous media target", () => {
const source = `<img class="media" src="a.png"><img class="media" src="b.png">`;
expect(() =>
applyMediaTreatmentToHtml(source, { selector: ".media", grading: { preset: "neutral" } }),
).toThrow(/matched 2 elements/);
const result = applyMediaTreatmentToHtml(source, {
selector: ".media",
selectorIndex: 1,
grading: { preset: "warm-daylight" },
});
expect((result.html.match(/data-color-grading/g) ?? []).length).toBe(1);
});
it("persists grading inside composition templates", () => {
const source = `<template><video id="hero" src="hero.mp4"></video></template>`;
const result = applyMediaTreatmentToHtml(source, {
selector: "#hero",
grading: { preset: "warm-daylight" },
});
expect(result.changed).toBe(true);
expect(result.html).toContain("data-color-grading=");
});
it("rejects non-media elements", () => {
expect(() =>
applyMediaTreatmentToHtml(`<div id="hero"></div>`, {
selector: "#hero",
grading: { preset: "warm-daylight" },
}),
).toThrow(/requires an <img> or <video>/);
});
it("rejects unknown keys instead of silently dropping agent mistakes", () => {
expect(() =>
applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: { adjustments: { exposure: -0.45 }, effects: { dither: 1 } },
}),
).toThrow(/grading.*adjustments/i);
expect(() =>
applyMediaTreatmentToHtml(VIDEO, {
selector: "#hero",
grading: { effects: { dithering: 1 } },
}),
).toThrow(/effects.*dithering/i);
});
it("requires --apply for --grading while keeping --clear explicit", async () => {
const project = mkdtempSync(join(tmpdir(), "hf-media-treatment-"));
const file = join(project, "index.html");
const grading = '{"adjust":{"exposure":0.1}}';
const log = vi.spyOn(console, "log").mockImplementation(() => undefined);
const error = vi.spyOn(console, "error").mockImplementation(() => undefined);
writeFileSync(file, VIDEO);
try {
await expect(
runCommand(mediaTreatmentCommand, {
rawArgs: ["--project", project, "--selector", "#hero", "--grading", grading],
}),
).rejects.toThrow(CliRuntimeError);
expect(error).toHaveBeenLastCalledWith(expect.stringContaining("--grading requires --apply"));
expect(readFileSync(file, "utf8")).toBe(VIDEO);
await runCommand(mediaTreatmentCommand, {
rawArgs: ["--project", project, "--selector", "#hero", "--grading", grading, "--apply"],
});
expect(readFileSync(file, "utf8")).toContain("data-color-grading");
await runCommand(mediaTreatmentCommand, {
rawArgs: ["--project", project, "--selector", "#hero", "--clear"],
});
expect(readFileSync(file, "utf8")).not.toContain("data-color-grading");
} finally {
log.mockRestore();
error.mockRestore();
rmSync(project, { recursive: true, force: true });
}
});
it("clears both explicit and normalized no-op grading", () => {
const graded = VIDEO.replace(" src=", ` data-color-grading='{"preset":"warm-daylight"}' src=`);
expect(
applyMediaTreatmentToHtml(graded, { selector: "#hero", clear: true }).html,
).not.toContain("data-color-grading");
expect(
applyMediaTreatmentToHtml(graded, { selector: "#hero", grading: { preset: "neutral" } }).html,
).not.toContain("data-color-grading");
});
it("does not report or serialize a no-op clear because unrelated HTML formatting differs", () => {
const source = `<!doctype html><html><head><meta charset="utf-8" /></head><body><video id="hero" src="hero.mp4"></video></body></html>`;
const result = applyMediaTreatmentToHtml(source, { selector: "#hero", clear: true });
expect(result.changed).toBe(false);
expect(result.html).toBe(source);
});
});
packages/cli/src/commands/media-treatment.ts
+537
View File
@@ -0,0 +1,537 @@
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { relative, resolve } from "node:path";
import {
HF_COLOR_GRADING_ATTR,
getHfColorGradingCapabilities,
isHfColorGradingActive,
isPathInside,
normalizeHfColorGrading,
serializeHfColorGrading,
} from "@hyperframes/core";
import {
isColorGradingVariableRef,
validateColorGradingContract,
} from "@hyperframes/parsers/color-grading-contract";
import { patchElementInHtml } from "@hyperframes/studio-server/source-mutation";
import { defineCommand } from "citty";
import { parseHTML } from "linkedom";
import type { Example } from "./_examples.js";
import { c } from "../ui/colors.js";
import { failCommand } from "../utils/commandResult.js";
import { normalizeErrorMessage } from "../utils/errorMessage.js";
import { readOptionalString } from "../utils/pathArgs.js";
import { resolveProject } from "../utils/project.js";
import { withMeta } from "../utils/updateCheck.js";
export function getMediaTreatmentCapabilityOverview() {
const capabilities = getHfColorGradingCapabilities();
const family = (id: string, label: string, description: string) => ({
id,
label,
description,
});
return {
version: capabilities.version,
targetTags: capabilities.targetTags,
colorSpace: capabilities.colorSpace,
families: [
family("correction", "Adjust", "Fix exposure, tonal balance, color casts, and saturation."),
family(
"presets",
"Presets",
"Apply a tested starting point, then tune only when the source or intent requires it.",
),
family("finishing", "Finish", "Shape vignette and deterministic film grain."),
...capabilities.effectFamilies,
family(
"palettes",
"Palettes",
"Reusable two-to-six-color palettes for compatible art effects.",
),
family(
"animation",
"Animation",
"Seek-safe CSS properties that registered GSAP timelines may animate.",
),
family("lut", "Custom LUT", "Apply a user-owned 3D .cube LUT."),
{
id: "overlays",
label: "Overlays",
description: "Install authored HUD, light-leak, flash, or freeze-frame overlay blocks.",
owner: "registry",
},
],
discovery: {
detail: "Use --capability <family-or-item> for exact controls and examples.",
full: "Use --all only for tooling or exhaustive inspection.",
},
};
}
export function getMediaTreatmentCapabilityDetail(id: string): unknown {
const capabilities = getHfColorGradingCapabilities();
const animation = (path: string) => {
const property = capabilities.animatable.find((candidate) => candidate.path === path);
if (!property) return null;
return {
property,
initial: `style="${property.name}: <start>"`,
tween: `timeline.to("<selector>", { "${property.name}": <end>, duration: <seconds> })`,
rules: [
"Author the initial value inline on the media element.",
"Use finite keyframes on a paused timeline registered in window.__timelines.",
"Do not use a frame-zero set, timers, random values, or onUpdate callbacks.",
],
};
};
const effect = capabilities.effects.find(({ key }) => key === id);
if (effect) {
return {
id,
family: effect.family,
label: effect.label,
description: effect.description,
renderLane: effect.renderLane,
supportsPalette: effect.supportsPalette,
apply: { effects: effect.apply },
controls: effect.controls,
animation: animation(`effects.${id}`),
};
}
const effectFamily = capabilities.effectFamilies.find((family) => family.id === id);
if (effectFamily) {
return {
...effectFamily,
effects: capabilities.effects
.filter((candidate) => candidate.family === id)
.map((candidate) => ({
id: candidate.key,
label: candidate.label,
description: candidate.description,
renderLane: candidate.renderLane,
supportsPalette: candidate.supportsPalette,
animatable: capabilities.animatable.some(
({ path }) => path === `effects.${candidate.key}`,
),
})),
};
}
const adjustment = capabilities.adjustments.find(({ key }) => key === id);
if (adjustment) {
return {
id,
family: "correction",
description: `Adjust ${id} within the canonical correction range.`,
control: adjustment,
apply: { adjust: { [id]: adjustment.identity } },
animation: animation(`adjust.${id}`),
};
}
const finishing = capabilities.finishing.find(({ key }) => key === id);
if (finishing) {
return {
id,
family: "finishing",
description: `Adjust ${id} within the canonical finishing range.`,
control: finishing,
apply: { details: { [id]: finishing.identity } },
};
}
const preset = capabilities.presets.find((candidate) => candidate.id === id);
if (preset) {
return {
id: preset.id,
label: preset.label,
description: "Tested built-in media preset.",
apply: { preset: preset.id, intensity: preset.intensity },
};
}
const palette = capabilities.palettes.find((candidate) => candidate.id === id);
if (palette) return { ...palette, apply: { palette: palette.colors } };
const details = {
correction: {
id,
description: "Fix exposure, tonal balance, color casts, and saturation.",
controls: capabilities.adjustments,
},
presets: {
id,
description: "Tested starting points for color and stylized media effects.",
presets: capabilities.presets,
},
finishing: {
id,
description: "Vignette and deterministic film-grain controls.",
controls: capabilities.finishing,
},
palettes: {
id,
description: "Named palettes plus the custom palette contract.",
contract: capabilities.palette,
palettes: capabilities.palettes,
},
animation: {
id,
description: "Seek-safe CSS properties for registered GSAP timelines.",
properties: capabilities.animatable,
},
lut: {
id,
description: "User-owned 3D .cube LUT support.",
contract: capabilities.lut,
},
overlays: {
id,
description: "Authored overlay blocks owned by the HyperFrames Registry.",
discover: "hyperframes catalog",
apply: "hyperframes add <overlay> --dir <project> --no-clipboard --json",
},
} as const;
const detail = Object.hasOwn(details, id) ? Reflect.get(details, id) : undefined;
if (detail) return detail;
throw new Error(`Unknown media-treatment capability: ${id}`);
}
export const examples: Example[] = [
[
"Discover the complete treatment surface without loading every control",
`hyperframes media-treatment --capabilities --json`,
],
[
"Inspect one relevant effect in detail",
`hyperframes media-treatment --capability kuwahara --json`,
],
[
"Inspect the exhaustive machine-readable catalog",
`hyperframes media-treatment --capabilities --all --json`,
],
[
"Apply a resolved treatment to one media element",
`hyperframes media-treatment --selector '#hero' --grading '{"preset":"skin-soft","intensity":0.6}' --apply`,
],
[
"Preview the exact mutation without writing",
`hyperframes media-treatment --file compositions/scene.html --selector 'video' --grading '{"preset":"warm-daylight"}' --apply --dry-run --json`,
],
["Remove a treatment", `hyperframes media-treatment --selector '#hero' --clear`],
];
interface ApplyMediaTreatmentOptions {
selector: string;
selectorIndex?: number;
grading?: unknown;
clear?: boolean;
}
interface ApplyMediaTreatmentResult {
html: string;
changed: boolean;
tag: "img" | "video";
value: string | null;
before: unknown;
after: unknown;
}
function parseSourceDocument(source: string): Document {
if (/<!doctype|<html[\s>]/i.test(source)) return parseHTML(source).document;
return parseHTML(`<!DOCTYPE html><html><body>${source}</body></html>`).document;
}
function assertKnownGradingShape(value: unknown): void {
const issue = validateColorGradingContract(value)[0];
if (issue) {
const hint = issue.hint ? ` ${issue.hint}` : "";
throw new Error(`Invalid color-grading ${issue.path}: ${issue.message}.${hint}`);
}
}
function containsColorGradingVariableRef(value: unknown): boolean {
if (isColorGradingVariableRef(value)) return true;
if (Array.isArray(value)) return value.some(containsColorGradingVariableRef);
if (typeof value !== "object" || value === null) return false;
return Object.values(value).some(containsColorGradingVariableRef);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function parseStoredGrading(raw: string | null): unknown {
if (raw === null) return null;
try {
return JSON.parse(raw);
} catch {
return raw;
}
}
function mergeGradingPatch(current: unknown, patch: unknown): unknown {
if (!isRecord(current) || !isRecord(patch)) return patch;
const merged = { ...current };
for (const [key, value] of Object.entries(patch)) {
merged[key] =
isRecord(value) && isRecord(merged[key]) ? mergeGradingPatch(merged[key], value) : value;
}
return merged;
}
function serializeGradingPatch(before: unknown, patch: unknown): string | null {
assertKnownGradingShape(patch);
if (isColorGradingVariableRef(before) && isRecord(patch)) {
throw new Error("Cannot merge a grading patch into an unresolved whole-grade variable");
}
const current =
typeof before === "string" && !isColorGradingVariableRef(before) ? { preset: before } : before;
const grading = mergeGradingPatch(current, patch);
assertKnownGradingShape(grading);
if (containsColorGradingVariableRef(grading)) {
return typeof grading === "string" ? grading.trim() : JSON.stringify(grading);
}
const normalized = normalizeHfColorGrading(grading);
if (!normalized) throw new Error("--grading must be valid HyperFrames color-grading JSON");
return isHfColorGradingActive(normalized) ? serializeHfColorGrading(normalized) : null;
}
function queryIncludingTemplates(root: Document | Element, selector: string): Element[] {
const matches = Array.from(root.querySelectorAll(selector));
if (matches.length > 0) return matches;
for (const template of root.querySelectorAll("template")) {
const nested = queryIncludingTemplates(template, selector);
if (nested.length > 0) return nested;
}
return [];
}
function selectMediaElement(
source: string,
selector: string,
selectorIndex?: number,
): { element: Element; selectorIndex: number; tag: "img" | "video" } {
const document = parseSourceDocument(source);
let matches: Element[];
try {
matches = queryIncludingTemplates(document, selector);
} catch {
throw new Error(`Invalid selector: ${selector}`);
}
if (matches.length === 0) throw new Error(`Selector did not match: ${selector}`);
if (selectorIndex === undefined && matches.length > 1) {
throw new Error(
`Selector matched ${matches.length} elements; use a unique selector or --selector-index`,
);
}
const resolvedIndex = selectorIndex ?? 0;
const element = matches[resolvedIndex];
if (!element) {
throw new Error(`--selector-index ${resolvedIndex} is outside ${matches.length} matches`);
}
const tag = element.tagName.toLowerCase();
if (tag !== "img" && tag !== "video") {
throw new Error(`Color grading requires an <img> or <video>; selector matched <${tag}>`);
}
return { element, selectorIndex: resolvedIndex, tag };
}
export function applyMediaTreatmentToHtml(
source: string,
options: ApplyMediaTreatmentOptions,
): ApplyMediaTreatmentResult {
const { element, selectorIndex, tag } = selectMediaElement(
source,
options.selector,
options.selectorIndex,
);
const before = parseStoredGrading(element.getAttribute(HF_COLOR_GRADING_ATTR));
const value = options.clear ? null : serializeGradingPatch(before, options.grading);
const changed = element.getAttribute(HF_COLOR_GRADING_ATTR) !== value;
const after = parseStoredGrading(value);
if (!changed) return { html: source, changed: false, tag, value, before, after };
const patched = patchElementInHtml(source, { selector: options.selector, selectorIndex }, [
{ type: "attribute", property: HF_COLOR_GRADING_ATTR, value },
]);
if (!patched.matched) throw new Error(`Could not persist selector: ${options.selector}`);
return { html: patched.html, changed: true, tag, value, before, after };
}
function parseSelectorIndex(raw: string | undefined): number | undefined {
if (raw === undefined) return undefined;
const value = Number(raw);
if (!Number.isInteger(value) || value < 0) {
throw new Error("--selector-index must be a non-negative integer");
}
return value;
}
function parseGrading(raw: string | undefined, apply: boolean, clear: boolean): unknown {
if (clear) {
if (raw !== undefined || apply) {
throw new Error("Use either --apply with --grading or --clear, not both");
}
return undefined;
}
if (!apply) {
if (raw !== undefined) throw new Error("--grading requires --apply");
throw new Error("Use --apply with --grading <json> or --clear");
}
if (raw === undefined) throw new Error("--apply requires --grading <json>");
try {
return JSON.parse(raw);
} catch (error) {
throw new Error(`Could not parse --grading JSON: ${normalizeErrorMessage(error)}`);
}
}
function mutationVerb(action: "apply" | "clear", changed: boolean, dryRun: boolean): string {
if (dryRun) return `Would ${action}`;
if (!changed) return action === "apply" ? "Already applied" : "Already clear";
return action === "apply" ? "Applied" : "Cleared";
}
export const mediaTreatmentCommand = defineCommand({
meta: {
name: "media-treatment",
description: "Discover, apply, or clear deterministic media treatments",
},
args: {
capabilities: {
type: "boolean",
description: "Print a concise agent-readable capability overview",
default: false,
},
capability: {
type: "string",
description: "Inspect one family, control, effect, preset, or palette",
},
all: {
type: "boolean",
description: "Print the exhaustive capability catalog",
default: false,
},
project: { type: "string", description: "Project directory (default: cwd)" },
file: {
type: "string",
description: "Composition file relative to project (default: index.html)",
},
selector: {
type: "string",
description: "Unique CSS selector for one <img> or <video>",
},
"selector-index": {
type: "string",
description: "Zero-based match index when the selector is not unique",
},
grading: { type: "string", description: "Canonical color-grading JSON patch" },
apply: {
type: "boolean",
description: "Apply the validated grading patch (explicit agent form)",
default: false,
},
clear: { type: "boolean", description: "Remove color grading from the target", default: false },
"dry-run": {
type: "boolean",
description: "Validate and report without writing",
default: false,
},
json: { type: "boolean", description: "Output an agent-friendly JSON result", default: false },
},
run({ args }) {
const runCapabilityQuery = () => {
const capability = readOptionalString(args.capability);
const hasMutationOption = [
readOptionalString(args.selector),
readOptionalString(args.grading),
args.clear,
args["dry-run"],
args.apply,
].some(Boolean);
if (hasMutationOption) {
throw new Error("--capabilities cannot be combined with mutation options");
}
if (args.all === true && capability) {
throw new Error("Use either --all or --capability, not both");
}
let capabilities: unknown = getMediaTreatmentCapabilityOverview();
if (args.all === true) capabilities = getHfColorGradingCapabilities();
else if (capability) capabilities = getMediaTreatmentCapabilityDetail(capability);
console.log(JSON.stringify(withMeta({ ok: true, capabilities }), null, 2));
};
const resolveMutationFile = () => {
const project = resolveProject(readOptionalString(args.project));
const fileArg = readOptionalString(args.file) ?? "index.html";
const filePath = resolve(project.dir, fileArg);
if (!isPathInside(filePath, project.dir) || !filePath.toLowerCase().endsWith(".html")) {
throw new Error("--file must be an HTML file inside the project");
}
if (!existsSync(filePath)) throw new Error(`Composition file not found: ${fileArg}`);
return { project, filePath };
};
const prepareMutation = () => {
const { project, filePath } = resolveMutationFile();
const selector = readOptionalString(args.selector);
if (!selector) throw new Error("--selector is required");
const clear = args.clear === true;
const apply = args.apply === true;
const selectorIndex = parseSelectorIndex(readOptionalString(args["selector-index"]));
const result = applyMediaTreatmentToHtml(readFileSync(filePath, "utf8"), {
selector,
selectorIndex,
grading: parseGrading(readOptionalString(args.grading), apply, clear),
clear,
});
const dryRun = args["dry-run"] === true;
if (result.changed && !dryRun) writeFileSync(filePath, result.html);
const action: "clear" | "apply" = result.value === null ? "clear" : "apply";
return {
action,
result,
selector,
payload: {
ok: true,
action,
file: relative(project.dir, filePath) || "index.html",
selector,
selectorIndex: selectorIndex ?? 0,
tag: result.tag,
changed: result.changed,
dryRun,
before: result.before,
after: result.after,
},
};
};
try {
if (args.capabilities === true || readOptionalString(args.capability) || args.all === true) {
return runCapabilityQuery();
}
const { action, result, selector, payload } = prepareMutation();
if (args.json === true) {
console.log(JSON.stringify(withMeta(payload), null, 2));
} else {
const verb = mutationVerb(action, result.changed, payload.dryRun);
console.log(`${c.success("◇")} ${verb} media treatment on ${c.accent(selector)}`);
}
} catch (error) {
const message = normalizeErrorMessage(error);
if (args.json === true) console.log(JSON.stringify(withMeta({ ok: false, error: message })));
else console.error(`${c.error("✗")} ${message}`);
failCommand();
}
},
});
packages/cli/src/help.ts
+4
View File
@@ -42,6 +42,10 @@ const GROUPS: Group[] = [
["inspect", "Inspect rendered visual layout across the timeline"],
["keyframes", "Inspect keyframes and render onion-shot diagnostics"],
["snapshot", "Capture key frames as PNG screenshots for visual verification"],
[
"media-treatment",
"Discover, apply, or clear deterministic media treatments on one media element",
],
[
"grade-compare",
"Render candidate color grades onto a reference frame as one labeled comparison PNG",
packages/cli/src/templates/_shared/AGENTS.md
+2
View File
@@ -20,6 +20,8 @@
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-keyframes`, `/hyperframes-creative`, `/hyperframes-cli`, `/media-use`, `/hyperframes-registry`, `/figma`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
**Changing how real footage or images look or reveal?** Load `/media-use` and its media-treatment policy before editing, even when the request only says dark, flat, boring, retro, private, or “make the reveal cooler.” Use canonical media treatments and seek-safe motion; do not improvise equivalent CSS/SVG filters or overlays.
> **Tailwind v4 projects** (`hyperframes init --tailwind`): see `/hyperframes-core` → `references/tailwind.md`.
> **Skill missing or stale?** Run `npx hyperframes skills update <name>` to install/refresh
packages/cli/src/templates/_shared/CLAUDE.md
+2
View File
@@ -20,6 +20,8 @@
The domain skills (`/hyperframes-core`, `/hyperframes-animation`, `/hyperframes-keyframes`, `/hyperframes-creative`, `/hyperframes-cli`, `/media-use`, `/hyperframes-registry`, `/figma`) and the full capability map live inside `/hyperframes` — it is the single source of truth for which skill handles which intent.
**Changing how real footage or images look or reveal?** Load `/media-use` and its media-treatment policy before editing, even when the request only says dark, flat, boring, retro, private, or “make the reveal cooler.” Use canonical media treatments and seek-safe motion; do not improvise equivalent CSS/SVG filters or overlays.
> **Tailwind v4 projects** (`hyperframes init --tailwind`): see `/hyperframes-core` → `references/tailwind.md`.
> **Skill missing or stale?** Run `npx hyperframes skills update <name>` to install/refresh
packages/core/src/colorGrading.ts
+123
View File
@@ -575,6 +575,127 @@ export const HF_COLOR_GRADING_ACTIVE_EFFECT_KEYS = [
export type HfColorGradingActiveEffectKey = (typeof HF_COLOR_GRADING_ACTIVE_EFFECT_KEYS)[number];
const EFFECT_FAMILIES = [
{
id: "essentials",
label: "Essentials",
description: "Common optical, focus, and privacy-oriented primitives.",
},
{
id: "retro-glitch",
label: "Retro & Glitch",
description: "Tape, film, CRT, channel separation, and digital disruption.",
},
{
id: "print",
label: "Print",
description: "Halftone, limited-ink, dither, and screen-print rendering.",
},
{
id: "art",
label: "Art",
description: "ASCII, engraved, crosshatched, and painterly rendering.",
},
] as const;
type EffectFamilyId = (typeof EFFECT_FAMILIES)[number]["id"];
const EFFECT_METADATA = {
blur: {
family: "essentials",
label: "Blur",
description: "Defocus or soften a full media layer.",
},
pixelate: {
family: "essentials",
label: "Pixelate",
description: "Turn a media layer into a block mosaic.",
},
bloom: {
family: "essentials",
label: "Bloom",
description: "Add thresholded glow around bright image regions.",
},
chromaBleed: {
family: "retro-glitch",
label: "Chroma Softening",
description: "Smear color like low-bandwidth video.",
},
tapeDamage: {
family: "retro-glitch",
label: "Tape Damage",
description: "Add moving tracking errors and tape noise.",
},
filmArtifacts: {
family: "retro-glitch",
label: "Film Artifacts",
description: "Add deterministic dust and film wear.",
},
scanlines: {
family: "retro-glitch",
label: "Scanlines",
description: "Overlay configurable horizontal display lines.",
},
crtCurvature: {
family: "retro-glitch",
label: "CRT Curvature",
description: "Warp media toward curved CRT geometry.",
},
chromaticAberration: {
family: "retro-glitch",
label: "Channel Separation",
description: "Offset color channels by angle.",
},
digitalGlitch: {
family: "retro-glitch",
label: "Digital Glitch",
description: "Compose tears, blocks, and color splits.",
},
halftone: {
family: "print",
label: "Halftone",
description: "Render source color through a print-dot raster.",
},
twoInkPrint: {
family: "print",
label: "Two-Ink Print",
description: "Reduce media to a two-ink print treatment.",
},
dither: {
family: "print",
label: "Ordered Dither",
description: "Quantize media into an ordered limited palette.",
},
monoScreen: {
family: "print",
label: "Mono Screen",
description: "Build monochrome dot, shape, or line artwork.",
},
ascii: {
family: "art",
label: "ASCII",
description: "Render media as configurable procedural glyph cells.",
},
engraving: {
family: "art",
label: "Engraving",
description: "Translate luminance into directional engraved lines.",
},
crosshatch: {
family: "art",
label: "Crosshatch",
description: "Translate media into layered hand-hatched lines.",
},
kuwahara: {
family: "art",
label: "Kuwahara Paint",
description: "Apply edge-preserving painterly smoothing.",
},
} as const satisfies Record<
HfColorGradingActiveEffectKey,
{ family: EffectFamilyId; label: string; description: string }
>;
export const HF_COLOR_GRADING_EFFECT_PRESETS = HF_COLOR_GRADING_PRESETS.filter((preset) =>
HF_COLOR_GRADING_ACTIVE_EFFECT_KEYS.some((key) => preset.effects[key] > 0.0001),
);
@@ -714,6 +835,7 @@ export function getHfColorGradingCapabilities() {
identity: ADJUST_ZERO[key],
...ADJUST_LIMITS[key],
})),
effectFamilies: EFFECT_FAMILIES,
finishing: HF_COLOR_GRADING_DETAIL_KEYS.map((key) => ({
key,
identity: DETAIL_DEFAULTS[key],
@@ -723,6 +845,7 @@ export function getHfColorGradingCapabilities() {
const apply = HF_COLOR_GRADING_EFFECT_APPLY_DEFAULTS[key];
return {
key,
...EFFECT_METADATA[key],
apply: { ...apply },
supportsPalette: PALETTE_EFFECT_KEYS.has(key),
renderLane: MULTIPASS_EFFECT_KEYS.has(key) ? "multipass" : "single-pass",
skills-manifest.json
+5 -5
View File
@@ -14,11 +14,11 @@
"files": 2
},
"general-video": {
"hash": "3303742225924078",
"hash": "e8c377a79bd57a69",
"files": 4
},
"hyperframes": {
"hash": "ab268ec1eca15ae9",
"hash": "02b48855a5321e48",
"files": 17
},
"hyperframes-animation": {
@@ -26,7 +26,7 @@
"files": 121
},
"hyperframes-cli": {
"hash": "de00375f053c05e8",
"hash": "a4db0693ff88e760",
"files": 11
},
"hyperframes-core": {
@@ -46,8 +46,8 @@
"files": 10
},
"media-use": {
"hash": "6b94dc295e2d6ba0",
"files": 145
"hash": "1234dcf993bbe1fc",
"files": 151
},
"motion-graphics": {
"hash": "da65c1864debfe11",
skills/general-video/SKILL.md
+1 -1
View File
@@ -20,7 +20,7 @@ A successful no-op means the skill is current. Surface an update failure instead
## 1. Apply cross-cutting source adapters
- **Media:** For any audio, image, icon, logo, voice, grade, LUT, caption, or media-operation need, load `/media-use` and follow its adoption, resolution, provider, provenance, and reuse contracts. Before the first authenticated provider action, run `npx hyperframes auth status` and relay its output verbatim. If signed out, apply the gate in `../hyperframes-core/references/brief-contract.md`: collaborative waits for sign-in or an explicit offline choice; autonomous states the status and continues through an available offline provider. Surface a blocker when no offline provider can satisfy a required capability. Local adoption alone does not require an auth gate.
- **Media:** For any audio, image, icon, logo, voice, grade, LUT, treatment/effect, caption, or media-operation need, load `/media-use` and follow its adoption, resolution, provider, provenance, and reuse contracts. Vague footage feedback and named styles use its `media-treatments.md` policy before editing; do not improvise supported media effects with CSS/SVG/opacity. Before the first authenticated provider action, run `npx hyperframes auth status` and relay its output verbatim. If signed out, apply the gate in `../hyperframes-core/references/brief-contract.md`: collaborative waits for sign-in or an explicit offline choice; autonomous states the status and continues through an available offline provider. Surface a blocker when no offline provider can satisfy a required capability. Local adoption alone does not require an auth gate.
- **Figma:** If any input is a `figma.com` URL, run `/figma` first. Build from its exported assets, tokens, components, or storyboard frames. Do not use raw Figma connector calls because they skip SVG sanitization, media provenance, and brand-token binding.
These adapters do not change the workflow selected by `/hyperframes`.
skills/hyperframes-cli/references/compare-and-batch.md
+1 -1
View File
@@ -48,7 +48,7 @@ npx hyperframes grade-compare \
`grades.json` is an array of labeled HyperFrames grading blocks:
```json
[{ "label": "warm", "grading": { "temperature": 0.2, "contrast": 0.1 } }]
[{ "label": "warm", "grading": { "adjust": { "temperature": 0.2, "contrast": 0.1 } } }]
```
Or compare explicit LUT files:
skills/hyperframes/SKILL.md
+10
View File
@@ -96,4 +96,14 @@ Use the bare name without `/`. If the command fails, surface the error; do not r
| Registry blocks and components | `/hyperframes-registry` |
| Figma assets, tokens, components, or storyboard frames as reconstructed motion | `/figma` |
Broad feedback about how photographic media looks or behaves also routes to
`/media-use`, even when the user never says “color grading” or “effect”: fix
dark/flat/boring footage, stylize a clip, hide a face, or improve a media
reveal. Load its media-treatment policy before editing; do not substitute a
generic LUT, CSS filter/overlay, or opacity tween for an existing canonical
treatment primitive. Keep text/layout/motion-only edits in their owning domain.
During a build with important photographic media, include one grounded
media-polish scan in the final quality pass; leaving suitable media unchanged is
a valid result.
Domain skills never take ownership of the end-to-end deliverable. Load only what the active workflow needs.
skills/hyperframes/references/capability-menu.md
+17 -17
View File
@@ -6,23 +6,23 @@ One list, three readers. The **pitch round** (`pitch-round.md`) speaks it before
Each row's last column reads **home → entry → what you get**: the owning skill, the exact doc or command to start from, and the artifact that comes back.
| Capability | Say it to the user as… | Home → entry → what you get |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Design spec (`frame.md`)** — one file that locks palette, type, and layout feel; every frame obeys it (the video-first sibling of a web `design.md`) | "a design system for the video — colors and typography stay consistent" | `/hyperframes-creative` → `references/design-spec.md` (what a spec is + resolution order); presets: `frame-presets/<name>/` (always installed — browsing needs no borrow); applying machinery `build-frame.mjs` (home: `/faceless-explainer`, `/product-launch-video`) → `frame.md` at the project root. How to ask: **§ The design ask** below |
| **Website capture** — headless-Chrome crawl of a real site: screenshots, brand tokens, assets | "I can capture your site and build from its real look" | CLI → `npx hyperframes capture <URL> -o ./capture` → `capture/` (screenshots, extracted tokens/text/assets); doctrine: `/product-launch-video` |
| **Beat analysis & audio-reactive motion** — a deterministic beat / energy map of a track; cuts land on the grid, elements pulse with the music | "if there's music, I can cut the video to its beat — and make elements move with it" | grid: `/music-to-video` → `scripts/analyze-beatgrid.py` → `audiomap.json` (beats, energy, sections); element reaction: `/hyperframes-creative` → `references/audio-reactive.md` + `scripts/extract-audio-data.py` |
| **Motion blueprints** — proven scene shapes (reveals, counters, charts, dioramas) picked per beat | "each scene gets a proven motion treatment, not improvised movement" | `/hyperframes-animation` → `blueprints-index.md` + `rules-index.md` → per-beat `blueprint:` ids the build reads |
| **Voice, music, SFX, images, logos, color grades** — generated or resolved from the catalog, frozen into the project | "narration in a voice you pick, background music, sound effects, brand logos — and a cinematic grade on images" | `/media-use` → `scripts/resolve.mjs` (one verb, incl. `--type grade`) + the audio engine `audio/scripts/audio.mjs` → frozen local files + `.media/manifest.jsonl` records / `audio_meta.json` |
| **Generative video** — an AI presenter delivers the script; a still photo becomes a talking clip; a finished video gets dubbed into another language | "an AI presenter can read your script on camera; I can animate a photo into a talking clip, or dub the video" | `/media-use` → `references/operations.md` § Generate: video (`heygen video create` / `video-translate`; OAuth free allowance where eligible) → an mp4 clip adopted into `assets/` + manifest record |
| **Transcription & captions** — word-timed transcripts; styled caption skins on the finished video | "accurate captions, styled to match" | `/media-use` → `scripts/transcribe.mjs` → word-timed transcript; caption machinery `captions.mjs` (home: `/faceless-explainer`) → the caption track |
| **Cut footage by its transcript** — trim a clip by choosing sentences, not timecodes | "I can trim your clip by picking the sentences to keep" | `/media-use` → `scripts/transcript-cut.mjs` → the trimmed clip + updated transcript |
| **Designed overlays on user footage** — kinetic titles, lower-thirds, data callouts synced to what's said | "your own clip can carry designed titles and info bars, timed to the speech" | the **whole ask** = the `/talking-head-recut` route — route there, don't rebuild it; **one scene inside a bigger piece** = `/hyperframes-animation` lower-third / callout blueprints + `/talking-head-recut`'s safe-zone thinking → overlay comps on the footage track |
| **Real map scenes** — a genuine basemap with located pins, routes, or a flight path | "for places and journeys — a real map, not a drawing of one" | `/motion-graphics` → `grounding/locate.mjs` (geocode) + `categories/maps/` (incl. `bake-basemap.mjs`) → a deterministic baked basemap + located pins |
| **Figma import** — assets, brand tokens, components, storyboard frames read as motion states | "if the design lives in Figma, I can build from it directly" | `/figma` → REST/CLI import (+ MCP for Motion/shaders) → sanitized SVGs, `var()`-bound brand tokens, frames-as-states |
| **Registry blocks** — 50+ installable scene compositions (data charts, device mockups, quote cards…) | "ready-made scenes we can drop in and restyle" | `/hyperframes-registry` → `npx hyperframes add <block>` → an installed sub-composition (wiring: `references/wiring-blocks.md`) |
| **Scene transitions** — cuts, crossfades, wipes, WebGL shader transitions between scenes | "how one scene hands off to the next — up to full shader wipes" | `/hyperframes-animation` → `transitions/overview.md` then `transitions/catalog.md`; assembly `transitions.mjs` (home: `/faceless-explainer`) → injected handoffs in the index |
| **User media on the timeline** — the user's own images / clips staged and woven into frames | "your own footage, screenshots, or photos placed into the video" | staging `stage-assets.mjs` (home: `/music-to-video`, `/product-launch-video`); adoption: `/media-use` `--adopt` → files in `assets/` + manifest records |
| **Publish to a stable link** — the finished piece on a public URL; re-publishing updates the same link | "when it's done I can publish it to a link you can share — updates keep the same URL" | `/hyperframes-cli` → `references/preview-render.md` (`npx hyperframes publish`) → a stable public URL |
| Capability | Say it to the user as… | Home → entry → what you get |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Design spec (`frame.md`)** — one file that locks palette, type, and layout feel; every frame obeys it (the video-first sibling of a web `design.md`) | "a design system for the video — colors and typography stay consistent" | `/hyperframes-creative` → `references/design-spec.md` (what a spec is + resolution order); presets: `frame-presets/<name>/` (always installed — browsing needs no borrow); applying machinery `build-frame.mjs` (home: `/faceless-explainer`, `/product-launch-video`) → `frame.md` at the project root. How to ask: **§ The design ask** below |
| **Website capture** — headless-Chrome crawl of a real site: screenshots, brand tokens, assets | "I can capture your site and build from its real look" | CLI → `npx hyperframes capture <URL> -o ./capture` → `capture/` (screenshots, extracted tokens/text/assets); doctrine: `/product-launch-video` |
| **Beat analysis & audio-reactive motion** — a deterministic beat / energy map of a track; cuts land on the grid, elements pulse with the music | "if there's music, I can cut the video to its beat — and make elements move with it" | grid: `/music-to-video` → `scripts/analyze-beatgrid.py` → `audiomap.json` (beats, energy, sections); element reaction: `/hyperframes-creative` → `references/audio-reactive.md` + `scripts/extract-audio-data.py` |
| **Motion blueprints** — proven scene shapes (reveals, counters, charts, dioramas) picked per beat | "each scene gets a proven motion treatment, not improvised movement" | `/hyperframes-animation` → `blueprints-index.md` + `rules-index.md` → per-beat `blueprint:` ids the build reads |
| **Voice, music, SFX, images, logos, media treatments** — resolved media plus source-aware color, effects, privacy, reveals, and justified overlays | "narration, music, sound effects, real assets — and footage polished or stylized to fit the story" | `/media-use` → deterministic resolve/operate tools plus `references/media-treatments.md`; shader pixels persist through `hyperframes media-treatment`, optional overlays come from Registry, and finite motion uses the host GSAP timeline |
| **Generative video** — an AI presenter delivers the script; a still photo becomes a talking clip; a finished video gets dubbed into another language | "an AI presenter can read your script on camera; I can animate a photo into a talking clip, or dub the video" | `/media-use` → `references/operations.md` § Generate: video (`heygen video create` / `video-translate`; OAuth free allowance where eligible) → an mp4 clip adopted into `assets/` + manifest record |
| **Transcription & captions** — word-timed transcripts; styled caption skins on the finished video | "accurate captions, styled to match" | `/media-use` → `scripts/transcribe.mjs` → word-timed transcript; caption machinery `captions.mjs` (home: `/faceless-explainer`) → the caption track |
| **Cut footage by its transcript** — trim a clip by choosing sentences, not timecodes | "I can trim your clip by picking the sentences to keep" | `/media-use` → `scripts/transcript-cut.mjs` → the trimmed clip + updated transcript |
| **Designed overlays on user footage** — kinetic titles, lower-thirds, data callouts synced to what's said | "your own clip can carry designed titles and info bars, timed to the speech" | the **whole ask** = the `/talking-head-recut` route — route there, don't rebuild it; **one scene inside a bigger piece** = `/hyperframes-animation` lower-third / callout blueprints + `/talking-head-recut`'s safe-zone thinking → overlay comps on the footage track |
| **Real map scenes** — a genuine basemap with located pins, routes, or a flight path | "for places and journeys — a real map, not a drawing of one" | `/motion-graphics` → `grounding/locate.mjs` (geocode) + `categories/maps/` (incl. `bake-basemap.mjs`) → a deterministic baked basemap + located pins |
| **Figma import** — assets, brand tokens, components, storyboard frames read as motion states | "if the design lives in Figma, I can build from it directly" | `/figma` → REST/CLI import (+ MCP for Motion/shaders) → sanitized SVGs, `var()`-bound brand tokens, frames-as-states |
| **Registry blocks** — 50+ installable scene compositions (data charts, device mockups, quote cards…) | "ready-made scenes we can drop in and restyle" | `/hyperframes-registry` → `npx hyperframes add <block>` → an installed sub-composition (wiring: `references/wiring-blocks.md`) |
| **Scene transitions** — cuts, crossfades, wipes, WebGL shader transitions between scenes | "how one scene hands off to the next — up to full shader wipes" | `/hyperframes-animation` → `transitions/overview.md` then `transitions/catalog.md`; assembly `transitions.mjs` (home: `/faceless-explainer`) → injected handoffs in the index |
| **User media on the timeline** — the user's own images / clips staged and woven into frames | "your own footage, screenshots, or photos placed into the video" | staging `stage-assets.mjs` (home: `/music-to-video`, `/product-launch-video`); adoption: `/media-use` `--adopt` → files in `assets/` + manifest records |
| **Publish to a stable link** — the finished piece on a public URL; re-publishing updates the same link | "when it's done I can publish it to a link you can share — updates keep the same URL" | `/hyperframes-cli` → `references/preview-render.md` (`npx hyperframes publish`) → a stable public URL |
Offer, don't unload: the intent layer recommends the one or two rows the confirmed concept itself calls for, states each as one plain-language line traced to the brief, and asks once — the route-filtered slice on request, the full table for the companion.
skills/media-use/SKILL.md
+56 -20
View File
@@ -1,6 +1,6 @@
---
name: media-use
description: Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, voice, color grade, or LUT into a frozen local file or paste-ready block + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Keeps search noise on disk, hands the agent one path or block. Use for any audio, image, icon, logo, voiceover, caption, color-grading, or media-asset need.
description: Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, voice, color grade, or LUT into a frozen local file or paste-ready block + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Also use for vague feedback that real footage looks dark, flat, boring, should feel retro/camcorder/print/ASCII, needs privacy, or needs a media reveal.
---
# media-use
@@ -25,36 +25,72 @@ Returns one line: `resolved <id> → <path> (<type>, <metadata>)`. All search no
| `icon` | icons, symbols (transparent) |
| `logo` | official brand marks (svgl → simple-icons → GitHub avatar → favicon; never redrawn) |
| `voice` | TTS voiceover (HeyGen free-usage path; optional local Kokoro) |
| `grade` | paste-ready HyperFrames `data-color-grading` block |
| `lut` | reusable validated `.cube` file |
| `grade` | measured correction candidate; broad polish/stylization follows Media Treatments |
| `lut` | user-provided or explicitly chosen reusable validated `.cube` file |
Before resolving fresh, list reusable candidates with `--candidates` and judge fit yourself — reuse rules, all flags, ingest (`--from`), and adopt are in `references/resolve.md`.
## Treat broad visual feedback as media intent
When a user explicitly asks to fix, polish, stylize, obscure, emphasize, or
reveal photographic media, read `references/media-treatments.md` even if they
do not name color grading or an effect. Inspect the real `<img>`/`<video>`,
choose one primary intent, then use deterministic persistence and verification.
Use a matching recipe as an optional tested seed, or inspect
`hyperframes media-treatment --capabilities --json`, then request one relevant
family/effect with `--capability <id>` and assemble a custom treatment from
canonical controls. Never load `--all` for ordinary authoring. A treatment may
compose correction, a preset, finishing, compatible shader effects, supported
keyframes, and optional Registry overlays. Add only source-justified bounded
tuning and compatible parts, never effects merely to make the result look more
sophisticated. Persist the final combined payload with
`hyperframes media-treatment`.
Use one progressively escalating workflow. For video, inspect one labeled
early/middle/late contact sheet rather than reading frames separately. Apply one
candidate and inspect one after-sheet for ordinary correction or polish.
Escalate to individual frames or moving draft evidence only when the result is
ambiguous, temporal, stylized, LUT-based, HDR/LOG-sensitive, private, or
brand-critical.
For ordinary correction or polish, persist the final treatment's
preset/adjustment JSON.
Do not generate a `.cube` LUT merely to encode exposure, shadows, contrast, or
warmth. Use a LUT only when the user supplies one or the selected treatment
explicitly owns one. `resolve --type grade --for ... --analyze` is measurement
evidence, not permission to replace the chosen treatment with a generated LUT.
Do not recreate supported vignette, grain, blur, pixelate, color, or treatment
effects with CSS/SVG overlays; that bypasses Studio controls and the canonical
preview/render shader path.
## Be proactive — run a media opportunity pass
The human usually can't tell which media would lift the piece. You can. When you build or review a composition, do **one** grounded scan and then **ask once** — don't silently add, and don't nag per asset.
Surface an opportunity only when a concrete signal is present:
| Signal detected | Offer |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| On-screen text / a script with no voiceover | TTS voiceover (audio engine) |
| Emoji or a `<div>` styled as an icon | resolve real `icon`s |
| Image that is a placeholder, tiny, or upscaled-looking | a better `image` (and/or upscale — see `references/operations.md`) |
| Hard scene cuts / transitions with no sound | transition `sfx` |
| A piece over ~10s with no music bed | `bgm` |
| Footage that reads under/over-exposed or color-cast | a corrective `grade` (analyze with `grade --for`, preview with `hyperframes grade-compare`) |
| Signal detected | Offer |
| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| On-screen text / a script with no voiceover | TTS voiceover (audio engine) |
| Emoji or a `<div>` styled as an icon | resolve real `icon`s |
| Image that is a placeholder, tiny, or upscaled-looking | a better `image` (and/or upscale — see `references/operations.md`) |
| Hard scene cuts / transitions with no sound | transition `sfx` |
| A piece over ~10s with no music bed | `bgm` |
| Footage that reads under/over-exposed or color-cast | a corrective `grade` (inspect with `grade --for --analyze`, preview with `hyperframes grade-compare`) |
| Photographic media that feels visually flat or off-topic | one specific source-appropriate preset or custom treatment, with the intended target named |
| A meaningful media entrance/reveal that feels static | one supported seek-safe treatment animation; preserve color unless the request also justifies a preset |
Rules that keep this a help, not nagware: **grounded, not generic** (no signal → no suggestion); **opinionated + concrete** (propose the specific fix with defaults chosen — the human approves **all / some / none**); **once per project** (one consolidated ask; respect "leave it"); **surface, never silently mutate** (color grades especially: propose and preview — a gray-world "correction" ruins an intentional sunset or neon look).
## Where to look — read only the file your task needs
| Task | Read |
| ------------------------------------------------------------------------- | ------------------------------- |
| resolve / reuse / adopt / ingest, flags, cascade, inventory | `references/resolve.md` |
| color grading, LUTs, smart grade (`--for`), grade-compare | `references/grading.md` |
| voiceover / TTS, music, SFX, captions, transcription (audio engine) | `references/audio.md` |
| cut / reframe / transform existing media, HEVC proxies, avatar video | `references/operations.md` |
| install + auth, provider table, RAM ladders, `--local-only`, `--provider` | `references/setup-providers.md` |
| remembered preferences + frozen recipes (user memory) | `references/memory.md` |
| ownership matrix, usage stats, telemetry, privacy (maintainer-facing) | `references/meta.md` |
| Task | Read |
| ------------------------------------------------------------------------- | -------------------------------- |
| resolve / reuse / adopt / ingest, flags, cascade, inventory | `references/resolve.md` |
| color grading, LUTs, smart grade (`--for`), grade-compare | `references/grading.md` |
| voiceover / TTS, music, SFX, captions, transcription (audio engine) | `references/audio.md` |
| cut / reframe / transform existing media, exact error diffusion, HEVC | `references/operations.md` |
| source-aware creative treatments, realtime effects, overlays, reveals | `references/media-treatments.md` |
| install + auth, provider table, RAM ladders, `--local-only`, `--provider` | `references/setup-providers.md` |
| remembered preferences + frozen recipes (user memory) | `references/memory.md` |
| ownership matrix, usage stats, telemetry, privacy (maintainer-facing) | `references/meta.md` |
skills/media-use/references/grading.md
+88 -8
View File
@@ -1,6 +1,21 @@
# Color grading — grade blocks and LUTs
Use `grade` when you need the actual HyperFrames `data-color-grading` value to paste onto an `<img>` or `<video>`. Core presets and params-backed library looks resolve locally; future CDN-backed library looks require network unless already frozen:
Use `grade` when you need a canonical HyperFrames grading/effects payload for
an `<img>` or `<video>`. Core presets and params-backed LUT entries resolve
locally; future CDN-backed LUT entries require network unless already
frozen. Persist a decided payload with the CLI rather than editing HTML by
hand:
For a vague but explicit polish request, do not jump directly from intent to a
preset name. Read `media-treatments.md`, choose a treatment whose subject and
avoid rules match the actual media, apply its conservative base with only
justified bounded tuning, then complete its visual verification steps. A named
owned treatment uses the exact preset/payload in its recipe; do not run the
generic grade/LUT resolver first.
Stop here and use that treatment workflow for requests such as retro, old home
video, camcorder, film, print, ASCII, glitch, privacy, or a media reveal. Do not
assemble those from a generic LUT plus handmade CSS vignette/grain/opacity.
**Never `cat`/read a `.cube` file into context.** A 3D LUT is ~size^3 lines of raw numbers (33^3 ≈ 36k lines at the default size). It bloats context and carries zero human/agent-legible signal. To understand or choose a LUT, use `hyperframes grade-compare` to see it rendered, or `cube-validate.mjs` for a one-line `{ok,size}` check. Read `.media/index.md` or `luts/index.json` for the description. Never read the LUT body itself.
@@ -17,7 +32,16 @@ Preset-first output uses the core runtime vocabulary and does not freeze a file:
}
```
Paste it as an attribute value after JSON string escaping:
Apply that payload to one unambiguous real media element:
```bash
hyperframes media-treatment --project . --file index.html \
--selector '#hero' \
--grading '{"preset":"warm-daylight","intensity":1}' --apply --json
```
Use `--dry-run` before writing when scope is uncertain and `--clear` to remove
the treatment. The low-level persisted result is still normal HTML:
```html
<video
@@ -27,7 +51,45 @@ Paste it as an attribute value after JSON string escaping:
></video>
```
Looks beyond the preset vocabulary freeze a validated `.cube` under `.media/luts/` and return a block that references it:
Direct attribute authoring is a fallback for environments where the CLI is not
available, not the primary agent workflow.
To build a treatment that is not already represented by a recipe, inspect the
canonical toolbox first:
```bash
hyperframes media-treatment --capabilities --json
```
It reports valid adjustment, finishing, effect, palette, preset, and animated
property ranges from Core. Compose one nested payload and pass it back through
`hyperframes media-treatment`; the command rejects unknown keys before mutation.
Do not generate or hand-edit a LUT merely to combine controls already owned by
the realtime shader.
For seek-safe effect motion, animate only the runtime-supported CSS properties
on that same real media element with its registered paused GSAP timeline:
| CSS property | Range |
| ---------------------------------- | ------- |
| `--hf-color-grading-intensity` | 0 to 1 |
| `--hf-color-grading-lut-intensity` | 0 to 1 |
| `--hf-color-grading-exposure` | -2 to 2 |
| `--hf-color-grading-blur` | 0 to 1 |
| `--hf-color-grading-bloom` | 0 to 3 |
| `--hf-color-grading-kuwahara` | 0 to 1 |
| `--hf-color-grading-pixelate` | 0 to 1 |
| `--hf-color-grading-ascii` | 0 to 1 |
| `--hf-color-grading-dither` | 0 to 1 |
Author the initial value directly in the media element's inline `style`, then
use finite `tl.to()` keyframes. Do not use a frame-zero `tl.set()`, CSS
animation clocks, timers, random values, or `onUpdate` callbacks. The static
`data-color-grading` payload remains the fallback and source of the other
controls.
For a reusable color transform beyond the preset vocabulary, freeze a validated
`.cube` under `.media/luts/` and return a block that references it:
```bash
node <SKILL_DIR>/scripts/resolve.mjs --type grade --intent "teal orange blockbuster" --project . --json
@@ -59,17 +121,35 @@ For a LUT generated by your own script, ingest it with `--from`; media-use valid
node <SKILL_DIR>/scripts/resolve.mjs --type lut --from custom.cube --project .
```
Parametric math (`buildCube`) cannot reproduce real film stocks or emulsion looks. Use a CDN-backed scanned `.cube` entry or ingest a real scanned `.cube` for those.
Parametric math (`buildCube`) cannot reproduce real film stocks or emulsion
transforms. Use a CDN-backed scanned `.cube` entry or ingest a real scanned
`.cube` for those.
For visual selection, list reusable looks with `resolve --type grade --candidates`, write the promising entries to a `grades.json`, run `hyperframes grade-compare --for <frame> --grades grades.json`, then commit the winner with `resolve -t grade` as the final `data-color-grading` block.
For visual selection, list reusable LUT candidates with
`resolve --type grade --candidates`, write the promising entries to a
`grades.json`, run
`hyperframes grade-compare --for <frame> --grades grades.json`, then commit the
winner with `resolve -t grade` as the final `data-color-grading` block.
Smart grade is `grade --for <media>`. It runs local `ffmpeg`/`ffprobe` signalstats, merges a bounded `adjust` suggestion into the returned block, and prints the measured evidence to stderr. Stdout remains valid JSON under `--json`; the suggestion is a starting point for the agent to tune, not an automatic neutralization of intentional color.
Use `grade --for <media> --analyze` when you only need side-effect-free
`ffmpeg`/`ffprobe` signalstats evidence. It returns a bounded `adjust`
suggestion without writing a manifest record. The suggestion is a starting
point for visual review, not an automatic neutralization of intentional color.
```bash
node <SKILL_DIR>/scripts/resolve.mjs --type grade --intent "warm cinematic" --for ./frame.png --project . --json
node <SKILL_DIR>/scripts/resolve.mjs --type grade --for ./frame.png --analyze --project . --json
```
Library looks live in `luts/index.json`. Each entry keeps `id`, `description`, `tags`, and `intensity`, then supplies either compact `params` for on-demand `buildCube(params)` generation or a direct CDN `url` for future scanned `.cube` files. Do not commit generated `.cube` bodies; resolve validates generated or downloaded cubes as it freezes them under `.media/luts/`.
Without `--analyze`, `grade --for` merges measured values into the resolved
grade and records that candidate in `.media`; use that form only when you
intend to keep the candidate.
Library LUT entries live in `luts/index.json`. Each entry keeps `id`,
`description`, `tags`, and `intensity`, then supplies either compact `params`
for on-demand `buildCube(params)` generation or a direct CDN `url` for future
scanned `.cube` files. Do not commit generated `.cube` bodies; resolve
validates generated or downloaded cubes as it freezes them under
`.media/luts/`.
```bash
node skills/media-use/scripts/resolve.mjs --type lut --intent "teal orange blockbuster" --project . --json
skills/media-use/references/media-treatment-recipes.md
+929
View File
@@ -0,0 +1,929 @@
# Media treatment recipes
These are optional tested seeds, not the complete capability surface. Read the
shared policy and choose one relevant section through `media-treatments.md`.
Agents may modify or combine a seed with compatible canonical controls after
inspecting the media, or assemble a bespoke payload from
`hyperframes media-treatment --capabilities --json` when no seed fits.
## Natural Portrait
Use for a talking head, interview, presenter, or people-focused photo whose
intended result is natural, polished, and restrained.
Do not use when the face is incidental or tiny, the source is intentionally
neon/monochrome/strongly stylized, or the requested result is beauty retouching.
This treatment changes the whole frame; it is not a face mask or skin-smoothing
effect.
Inspect face exposure, highlight retention, shadow detail, white balance, and
whether the existing look is intentional. Signalstats do not detect faces or
creative intent.
### Base payload
Start here, then tune only when the sampled frames justify it:
```json
{ "preset": "skin-soft", "intensity": 0.6 }
```
`skin-soft` is a global tonal/color preset whose vibrance math is reduced for
skin-like colors. It does not blur, retouch, segment, or track a face.
### Bounded tuning
Adjustment values are absolute values in the final payload, not deltas added to
the preset. Keep changes inside these conservative ranges unless the user asks
for a stylized result:
| Property | Natural Portrait range |
| ----------- | ---------------------- |
| intensity | 0.45 to 0.75 |
| exposure | -0.06 to 0.14 |
| contrast | -0.05 to 0.08 |
| highlights | -0.18 to -0.04 |
| shadows | 0.04 to 0.18 |
| whites | -0.10 to 0.04 |
| blacks | -0.06 to 0.06 |
| temperature | -0.05 to 0.10 |
| tint | -0.03 to 0.05 |
| vibrance | 0 to 0.06 |
| saturation | -0.04 to 0.06 |
Leave grain, blur, and pixelate at zero. A vignette is optional at `0` to
`0.05` only when it improves subject focus without looking like an effect.
Manual controls must stay inside their schema section; they are never
top-level keys. A tuned Natural Portrait payload looks like this:
```json
{
"preset": "skin-soft",
"intensity": 0.58,
"adjust": {
"highlights": -0.08,
"shadows": 0.08,
"temperature": 0.02,
"vibrance": 0.02
},
"details": { "vignette": 0.03 }
}
```
Use the same nested shape in `grade-compare` candidate files. `adjust` owns
tonal/color controls, `details` owns vignette/grain, and `effects` owns blur,
pixelate, chroma bleed, and the advanced treatment primitives below.
During the common comparison, reject any result that makes skin implausible,
loses highlight detail, flattens or desaturates dark skin, or casts clothing and
background colors accidentally.
## Product Polish
Use for photographed or filmed physical products when the goal is clean,
accurate, dimensional presentation. Protect product color, material texture,
label readability, specular highlights, and intentional lighting.
Do not use this treatment for literal app/site screenshots or screen captures;
follow UI Fidelity below. Do not neutralize a lifestyle scene's deliberate
ambient color, and do not infer exact brand-color correction without a neutral
reference or known product color.
Inspect the product separately from its background. Check white balance, label
legibility, surface texture, highlight clipping, shadow detail, white point,
and black point. Statistics cannot identify a white package, metallic
highlight, amber glass, or intentional warm light.
### Base payload
Compare this restrained correction against the untouched source:
```json
{
"intensity": 0.7,
"adjust": {
"exposure": 0.01,
"contrast": 0.06,
"highlights": -0.1,
"shadows": 0.04,
"whites": 0.02,
"blacks": -0.03,
"vibrance": 0.03,
"saturation": 0.02
}
}
```
This is a comparison starting point, not an instruction to change an already
finished source. If the original has accurate color, clean endpoints, and good
texture, leave the pixels unchanged and polish through framing or motion.
### Bounded tuning
| Property | Product Polish range |
| ----------- | -------------------- |
| intensity | 0.45 to 0.8 |
| exposure | -0.08 to 0.1 |
| contrast | 0 to 0.1 |
| highlights | -0.16 to 0 |
| shadows | 0 to 0.12 |
| whites | -0.08 to 0.05 |
| blacks | -0.06 to 0.04 |
| temperature | -0.05 to 0.05 |
| tint | -0.03 to 0.03 |
| vibrance | 0 to 0.06 |
| saturation | -0.04 to 0.05 |
Temperature and tint stay at zero unless the frames show a plausible cast.
Leave grain, vignette, blur, and pixelate at zero for catalog/e-commerce media.
For a lifestyle product shot, a vignette up to `0.04` is acceptable only when
it improves focus without changing the product itself.
During the common comparison, reject any result that clips white packaging,
muddies black products, shifts a known brand color, hides texture, or makes
labels harder to read. Report when preserving the original was the deliberate
decision.
## UI Fidelity
Use for literal app, website, dashboard, terminal, slide, or screen-recording
pixels whose colors and readability are part of the product being shown.
The default payload is **none**: do not add `data-color-grading`. Global color
changes affect brand colors, status colors, charts, screenshots, and tiny text
together, so even a tasteful photographic look can make the demonstration less
truthful.
Polish UI footage with crop, scale, pacing, cursor emphasis, surrounding DOM
overlays, or seek-safe motion outside the captured pixels. If the user
explicitly asks for a stylized UI look, preview it against the original and
state that exact UI color is no longer preserved. If a camera filmed a screen,
correct only a demonstrated capture cast or exposure issue and still verify
text and brand colors across representative frames.
## Film Memory
Use when the story explicitly calls for a warm memory, restrained flashback,
personal archive, or film-like recollection. This is not the default meaning of
"cinematic", and it is not scanned-film-stock emulation.
Do not use for literal UI, product catalog media, technical demonstrations, or
footage whose accurate current-day color is important. Use a separate camcorder
treatment for VHS/REC language. Do not add dust, scratches, light leaks, film
burns, or halation unless an owned component is available and the requested
story actually benefits from it.
Check that the source has enough highlight and shadow detail to tolerate a
faded treatment, and confirm nostalgia or temporal separation belongs in the
story. Compare the full moving treatment, not only a still preset card.
### Static pixel base
Start with this owned shader recipe:
```json
{
"preset": "vintage-wash",
"intensity": 0.6,
"details": {
"vignette": 0.12,
"grain": 0.12,
"grainSize": 0.2,
"grainRoughness": 0.6
}
}
```
Keep the static values inside these ranges:
| Property | Film Memory range |
| -------------- | ----------------- |
| intensity | 0.5 to 0.75 |
| vignette | 0.08 to 0.16 |
| grain | 0.08 to 0.16 |
| grainSize | 0.16 to 0.24 |
| grainRoughness | 0.5 to 0.7 |
The stronger end can flatten dark skin, black clothing, or already-faded
footage. Compare against the source and lower strength when it does.
### Temporal character
Use the existing registered paused GSAP timeline on the same media element:
- author `--hf-color-grading-exposure: 0` in the media element's inline
`style`;
- move it through a finite irregular sequence within `-0.03` to `0.03`, using
gentle `sine.inOut` segments around `0.45` to `0.8` seconds;
- for gate weave, keep `x`/`y` within `0.15%` of the shorter composition edge,
rotation within `0.03` degrees, and scale between `1.005` and `1.01` to
protect the frame edges;
- return close to the starting exposure and transform at the treatment end.
Do not use randomness, infinite CSS keyframes, timers, or `onUpdate`. Flicker
is a gentle exposure pulse, not a flash. Weave is slight mechanical drift, not
handheld shake.
Also run focused keyframe diagnostics and seek directly to the final-minus-frame
position. Reject brightness pumping, distracting drift, clipped edges, or
skin/detail loss. Report the motion ranges and describe this as an HF
film-memory treatment, not camera-stock emulation. If motion reads as an effect
before it reads as a memory, reduce or remove it.
## Creator Camcorder
Use when the story explicitly calls for a creator-camera recording, consumer
camcorder memory, or restrained digital-video character. This treatment is a
modern camcorder language, not VHS restoration, CRT simulation, surveillance,
or a promise to reproduce a specific camera model.
Do not apply it to literal UI, product catalog media, tiny media tiles, or
already compressed footage that has distracting color bleed. Do not add a REC
HUD merely because the source contains a person talking; the camera-device
language must support the story or the user's requested style.
Check skin, saturated edges, fine text, source compression, and whether the
source already has a deliberate camera look. Reject softened chroma that
damages labels, graphics, or identifying product color. Judge chroma softness
and grain in motion, not one still.
### Static pixel base
Start with the proven shader payload below, then tune only inside the bounded
ranges when representative frames justify it:
```json
{
"intensity": 0.72,
"adjust": {
"contrast": 0.08,
"highlights": -0.05,
"shadows": 0.02,
"whites": 0.03,
"blacks": -0.04,
"temperature": -0.03,
"tint": -0.015,
"vibrance": -0.03,
"saturation": -0.06
},
"details": {
"vignette": 0.06,
"grain": 0.08,
"grainSize": 0.18,
"grainRoughness": 0.58
},
"effects": { "chromaBleed": 0.55 }
}
```
| Property | Creator Camcorder range |
| -------------- | ----------------------- |
| intensity | 0.55 to 0.8 |
| contrast | 0.03 to 0.1 |
| highlights | -0.1 to 0 |
| shadows | 0 to 0.06 |
| whites | 0 to 0.05 |
| blacks | -0.08 to -0.01 |
| temperature | -0.06 to 0.04 |
| tint | -0.03 to 0.02 |
| vibrance | -0.06 to 0.02 |
| saturation | -0.12 to -0.02 |
| vignette | 0.03 to 0.1 |
| grain | 0.04 to 0.12 |
| grainSize | 0.14 to 0.24 |
| grainRoughness | 0.45 to 0.7 |
| chromaBleed | 0.35 to 0.7 |
Leave blur and pixelate at zero. Square pixels, scanlines, RGB splitting, and
tracking noise are different visual languages and are not defaults for this
treatment.
### Optional camera HUD
When the narrative benefits from explicit recording-device language, install
the Registry overlay block:
```bash
npx hyperframes add camcorder-hud --no-clipboard
```
Insert the printed `data-composition-src` host over the intended media range.
Edit the displayed date/time/mode/counter in
`compositions/camcorder-hud.html`. The block's paused GSAP timeline derives
its counter and REC blink from composition time, so play, scrub, and render
agree. Keep the HUD finite and scoped to the shot.
The HUD is an optional authored overlay. The pixel payload remains useful
without it, and the HUD alone is not evidence that the footage was treated.
### Optional source-to-camera reveal
Global grading intensity fades only primary correction and LUT output; it does
not fade the independent camcorder effects. For a visible source-to-camera
mode change, use two synchronized media layers and a finite opacity crossfade
from untreated to treated footage. Fade the HUD in on that same paused GSAP
timeline. Do not animate shader state with callbacks or an independent clock.
Also verify HUD placement and framing in each aspect ratio the project supports.
Report whether the HUD was used and describe this as an HF camcorder treatment,
not camera/VHS emulation. If an effect artifact is more noticeable than the
subject, reduce chroma bleed/grain or keep the source unchanged.
## VHS Playback
Use when the story explicitly calls for analog home-video tape, a dated archive,
or a visibly degraded VHS playback. This treatment is not Creator Camcorder,
generic pixelation, CRT display simulation, or a default retro look.
Do not use for literal UI, product catalog media, small text, clean modern
creator footage, or any source whose identifying color/detail must remain exact.
Inspect high-contrast vertical edges, faces, saturated objects, and the bottom
of the frame in motion. Analog damage must support the story without making the
subject hard to read.
### Pixel payload
Start with the complete proven combination, not `tapeDamage` alone:
```json
{
"intensity": 1,
"adjust": { "contrast": -0.04, "saturation": -0.08 },
"details": {
"grain": 0.16,
"grainSize": 0.12,
"grainRoughness": 0.72
},
"effects": {
"tapeDamage": 0.82,
"tapeTracking": 0.85,
"tapeNoise": 0.3,
"tapeSpeed": 0.5,
"chromaBleed": 0.5,
"chromaticAberration": 0.18,
"chromaticAngle": 0,
"scanlines": 0.35,
"scanlineCount": 0.17,
"scanlineSoftness": 1,
"digitalGlitch": 0.32,
"digitalGlitchColorSplit": 0,
"digitalGlitchLineTear": 0.08,
"digitalGlitchPixelate": 0,
"digitalGlitchBlockAmount": 0,
"digitalGlitchBlockDisplacement": 0,
"digitalGlitchBlockOpacity": 0,
"digitalGlitchSpeed": 0.5
}
}
```
| Property | VHS Playback range |
| --------------------- | ------------------ |
| intensity | 0.75 to 1 |
| contrast | -0.1 to 0 |
| saturation | -0.16 to 0 |
| grain | 0.08 to 0.18 |
| grainSize | 0.08 to 0.18 |
| grainRoughness | 0.55 to 0.8 |
| tapeDamage | 0.65 to 0.9 |
| tapeTracking | 0.5 to 0.9 |
| tapeNoise | 0.15 to 0.45 |
| tapeSpeed | 0.35 to 0.65 |
| chromaBleed | 0.35 to 0.65 |
| chromaticAberration | 0.08 to 0.22 |
| scanlines | 0.2 to 0.4 |
| scanlineCount | 0.14 to 0.2 |
| digitalGlitch | 0.2 to 0.4 |
| digitalGlitchLineTear | 0.04 to 0.1 |
`tapeDamage` owns deterministic horizontal line jitter, slow time-base wobble,
bottom-edge head switching, luma bandwidth loss, restrained ghosting, noise,
and sparse dropouts. Its subordinate tracking/noise/speed controls add bounded
moving tape tears and control their signal character without introducing a new
clock. `chromaBleed` separately reduces horizontal chroma detail. The restrained
scanline and chromatic settings supply the remaining tape-playback character.
The digital stage is used only for rare horizontal row tears: keep its color
split, pixelation, block displacement, block opacity, and corruption values at
zero. Leave blur, CRT curvature, generic pixelation, and a camera HUD off.
These values are an original HyperFrames recipe calibrated on the same public
Orange Cat source used for the external visual reference. They are not copied
shader code or a claim of pixel-identical output from the external reference. The scanline count is
mapped to the reference's approximately 127-cycle primary line pattern; the HF
tracking math stays bounded in media pixels and uses the composition clock.
The shader damage evolves from the existing deterministic media time, so it
needs no CSS loop or private timeline. Global grading intensity does not fade
tape damage or other independent effects. If the story requires a finite
source-to-tape reveal, crossfade synchronized untreated and treated media layers
on the host's paused GSAP timeline. During the common workflow, inspect dense
consecutive frames and reject hard edge tearing, face
smearing, frozen noise, square blocks, blank borders, or a bottom disturbance
that competes with the subject.
## 8mm Home Movie
Use for personal archive, family-memory, childhood, travel-memory, or explicit
small-gauge home-movie language. This is stronger and more materially film-like
than Film Memory, but it is still an owned HyperFrames treatment rather than a
claim to reproduce a named film stock, camera, or laboratory process.
Do not use for literal UI, technical demonstrations, catalog products, clean
interviews, or footage where dust/scratches would imply false provenance. Check
skin, highlights, dark clothing, and frame edges before applying it.
### Pixel payload
```json
{
"preset": "vintage-wash",
"intensity": 0.72,
"details": {
"vignette": 0.28,
"vignetteMidpoint": 0.54,
"vignetteFeather": 0.72,
"grain": 0.34,
"grainSize": 0.18,
"grainRoughness": 0.72
},
"effects": { "filmArtifacts": 0.62 }
}
```
| Property | 8mm Home Movie range |
| -------------- | -------------------- |
| intensity | 0.6 to 0.8 |
| vignette | 0.18 to 0.34 |
| grain | 0.22 to 0.42 |
| grainSize | 0.12 to 0.24 |
| grainRoughness | 0.6 to 0.8 |
| filmArtifacts | 0.35 to 0.7 |
`filmArtifacts` owns only deterministic sparse dust and short scratches. The
existing preset/details own color, vignette, and grain; the host's paused GSAP
timeline owns optional gate weave. Keep weave within `0.15%` of the shorter
composition edge, rotation within `0.03` degrees, and scale between `1.005` and
`1.015`. Use finite `sine.inOut` segments around `0.6` to `1` second, return
near the starting transform, and never use randomness, timers, `onUpdate`, or
an infinite CSS animation.
Reject a result when dust is constantly visible, scratches persist unnaturally,
the frame pumps, weave exposes an edge, highlights turn muddy, or the material
artifacts are more noticeable than the memory. For a subtler nostalgic result,
use Film Memory instead.
## Editorial Halftone
Use for print/editorial transitions, poster frames, comic/newsprint language,
stylized product or portrait beats, and graphic sequences where visible ink
screening is the point. This is a real four-angle CMYK raster treatment, not a
dotted DOM overlay.
Do not use on literal UI, dense text, tiny labels, footage that must remain
photorealistic, or a long talking-head segment unless the user explicitly asks
for strong print stylization. Preserve text/captions as ungraded DOM above the
media whenever they must stay readable.
### Pixel payload
```json
{
"intensity": 1,
"adjust": { "contrast": 0.04, "saturation": 0.04 },
"effects": { "halftone": 0.94, "halftoneSize": 0.36 }
}
```
| Property | Editorial Halftone range |
| ------------ | ------------------------ |
| intensity | 0.8 to 1 |
| contrast | -0.02 to 0.08 |
| saturation | -0.04 to 0.08 |
| halftone | 0.75 to 1 |
| halftoneSize | 0.15 to 0.55 |
The shader uses fixed C/M/Y/K screen angles of 15/75/0/45 degrees, separate ink
coverage, a warm paper base, and resolution-aware dot-cell sizing. Keep those
screen semantics fixed; tune only amount and size unless a future visual proof
justifies a broader schema. Judge the result at final output resolution because
browser zoom can misrepresent the screen. Reject unstable moire, unreadable
subjects, clipped ink detail, excessive dot size, or any treatment that looks
like a transparent dot texture laid over unchanged footage.
## Two-Ink Editorial Print
Use for poster frames, editorial portraits, music/social cutaways, zine
graphics, and bold print-led transitions where two visible spot inks are more
appropriate than photographic color. This is a fixed original HyperFrames
vermilion/teal treatment, not a claim to emulate a named printer, ink set, or
commercial print process.
Do not use for literal UI, brand-color-critical products, small labels, natural
talking heads, or media that must remain photorealistic. Keep captions and
graphics as normal DOM above the treated media.
### Pixel payload
```json
{
"intensity": 1,
"adjust": { "contrast": 0.08, "highlights": -0.06, "shadows": 0.04 },
"effects": { "twoInkPrint": 1, "twoInkPrintSize": 0.42 }
}
```
| Property | Two-Ink range |
| --------------- | ------------- |
| intensity | 0.8 to 1 |
| contrast | 0.02 to 0.1 |
| highlights | -0.1 to 0 |
| shadows | 0 to 0.08 |
| twoInkPrint | 0.8 to 1 |
| twoInkPrintSize | 0.18 to 0.55 |
The shader maps warm midtones to vermilion, deep/cool shadows to teal, and
shared dark coverage to a dark overprint on warm paper. It uses separate
15/75-degree screens, a subtle fixed registration offset, deterministic paper
texture, and resolution-aware dot sizing. Do not combine it with `halftone` or
a duotone LUT: that re-separates the result and defeats the two-ink contract.
Judge it at output resolution and across multiple frames. Reject missing second
ink, crushed faces, unstable moire, illegible silhouettes, or a result that
reads as a red tint with dots rather than two screened inks.
## Monochrome Screen Print
Use for graphic portrait beats, posterized social inserts, newspaper-like
screens, or a finite transition into visible monochrome cells. Keep captions
and typography as normal DOM above the treated media.
```json
{
"intensity": 1,
"effects": {
"monoScreen": 1,
"monoScreenSize": 0.35,
"monoScreenAngle": 0.25,
"monoScreenSpread": 0.3,
"monoScreenShape": 0,
"monoScreenInvert": 0
},
"palette": ["#111319", "#f2ecdc"]
}
```
Use `monoScreenShape` `0..4` for circle, square, diamond, triangle, or line.
Keep cell size within `0.15..0.55` and spread within `0.15..0.55`. Reject faces
that lose their silhouette, unstable moire, or cells too small to survive the
final encoded resolution.
## Engraved Illustration
Use for editorial portraits, historical/technical illustration, title-card
cutaways, or a source-to-line-art reveal. It is not routine correction and
should not be applied to literal UI or brand-color-critical product footage.
```json
{
"intensity": 1,
"effects": {
"engraving": 1,
"engravingSpacing": 0.4118,
"engravingMinThickness": 0.2,
"engravingMaxThickness": 0.4571,
"engravingAngle": 0.25,
"engravingContrast": 0.4667,
"engravingSharpness": 0.59,
"engravingWave": 0.2,
"engravingWaveFrequency": 0.2222
},
"palette": ["#101216", "#f3eddf"]
}
```
Preserve the calibrated base first. Tune spacing within `0.25..0.6`, contrast
within `0.3..0.65`, and wave within `0..0.35`. Reject squeezed framing, broken
contours, noisy flat backgrounds, or lines that flicker across moving frames.
## Crosshatched Sketch
Use for hand-rendered editorial beats, comic/documentary cutaways, and short
illustrative transformations where multiple line directions should preserve
the subject contour.
```json
{
"intensity": 1,
"effects": {
"crosshatch": 1,
"crosshatchSpacing": 0.28,
"crosshatchThickness": 0.25,
"crosshatchAngle": 0.25,
"crosshatchContrast": 0.3333,
"crosshatchEdges": 0.5,
"crosshatchLineWeight": 0,
"crosshatchWave": 0.33,
"crosshatchWaveFrequency": 0.2222
},
"palette": ["#101216", "#f3eddf"]
}
```
Tune spacing within `0.18..0.5`, edge detail within `0.3..0.7`, and wave within
`0.1..0.45`. Reject distorted aspect ratio, dense black fill that hides the
subject, or temporal shimmer stronger than the intended sketch language.
## CRT Display
Use when the media is intentionally shown as an older monitor, terminal, game
screen, or broadcast display. Curvature alone is geometry, not a complete CRT
treatment, so pair it with restrained scanlines and only slight channel
separation.
```json
{
"intensity": 1,
"effects": {
"crtCurvature": 0.2,
"scanlines": 0.35,
"scanlineCount": 0.17,
"scanlineSoftness": 1,
"chromaticAberration": 0.08,
"chromaticAngle": 0
}
}
```
Keep curvature within `0.08..0.28`, scanlines within `0.18..0.45`, and channel
separation within `0..0.12`. Reject excessive black corners, unreadable UI,
large color fringes, or applying the display language to ordinary footage when
the user only asked for correction.
## Procedural ASCII
Use for a deliberate terminal, code, data, surveillance, editorial, or
source-to-character reveal. This is a real shader-generated 5x7 glyph field,
not monospace text placed over unchanged footage.
Do not use as routine talking-head polish, on literal UI or dense text, or when
recognizing a face/product precisely matters. Keep captions and graphics as
normal DOM above the treated media.
Choose one of these proven starting points:
```json
{
"effects": { "ascii": 1, "asciiSize": 0.08, "asciiInvert": 1 },
"palette": ["#020605", "#38ff78"]
}
```
The first is **Terminal ASCII**: dark field, bright green glyphs, appropriate
for code/data/device language. For a warmer print-like **Editorial ASCII**, use:
```json
{
"effects": { "ascii": 1, "asciiSize": 0.066, "asciiInvert": 0 },
"palette": ["#0b0d0d", "#eee9db"]
}
```
Keep `ascii` between `0.75` and `1` for a fully readable treatment and
`asciiSize` between `0.04` and `0.15`. A finite reveal may author
`--hf-color-grading-ascii: 0` inline and tween it to `1` with the registered
paused GSAP timeline. Reject unstable cells, lost silhouette/face structure,
unreadable composition, or a palette that conflicts with the project.
## Ordered Palette Dither
Use for posterized social beats, music/editorial cutaways, pixel-art language,
or a finite source-to-palette reveal. The shader uses a stable 4x4 Bayer
threshold matrix and an explicit dark-to-light palette. Do not describe it as
Floyd-Steinberg, Atkinson, or another sequential error-diffusion process.
Do not use on literal UI, brand-color-critical products, tiny labels, or long
photorealistic sections. Start with one of these original palettes:
```json
{
"effects": { "dither": 1, "ditherSize": 0.25 },
"palette": ["#17121a", "#824c50", "#e09873", "#f7ddb1"]
}
```
The four-color option is **Warm Print**. For a louder social/music beat, use
the six-color **Electric Ink** palette:
```json
{
"effects": { "dither": 1, "ditherSize": 0.4 },
"palette": ["#080717", "#3c185f", "#7e2278", "#d9339f", "#ff6b66", "#aafae0"]
}
```
HyperFrames also owns these named ramps. The name is an authoring shortcut;
persist the listed colors through the existing `palette` array:
| Group | Palette ID | Ordered colors |
| ----------- | ---------------- | ---------------------------------------------------------------- |
| Classic | `noir` | `#000000`, `#ffffff` |
| Classic | `ink-paper` | `#1a1a2e`, `#f5f5dc` |
| Classic | `terminal` | `#001100`, `#00ff00` |
| Classic | `amber-glow` | `#1a0f00`, `#ffcc00` |
| Classic | `handheld-green` | `#0f380f`, `#306230`, `#8bac0f`, `#9bbc0f` |
| Mood | `golden-hour` | `#1a1205`, `#4a3510`, `#8b6914`, `#d4a017`, `#fff8dc` |
| Mood | `deep-sea` | `#0a1628`, `#1a3a5c`, `#2d6187`, `#5ba4c9`, `#a8dce8` |
| Mood | `arctic-night` | `#0a0a14`, `#1a2a4a`, `#3a5a8a`, `#6a9aca`, `#cae8ff` |
| Mood | `synthwave` | `#120458`, `#7b2cbf`, `#e040fb`, `#ff6ec7`, `#fff59d` |
| Mood | `vaporwave` | `#1a0a2e`, `#3d1a5c`, `#ff71ce`, `#01cdfe`, `#fffb96` |
| Mood | `forest` | `#1a2e1a`, `#2d4a2d`, `#4a7c4a`, `#7ab37a`, `#c8e6c8` |
| Mono | `sepia` | `#1a1610`, `#3d3020`, `#6b5a40`, `#a89070`, `#e8dcc8` |
| Mono | `blueprint` | `#001830`, `#003060`, `#0050a0`, `#0080e0`, `#e0f0ff` |
| HyperFrames | `warm-print` | `#17121a`, `#824c50`, `#e09873`, `#f7ddb1` |
| HyperFrames | `electric-ink` | `#080717`, `#3c185f`, `#7e2278`, `#d9339f`, `#ff6b66`, `#aafae0` |
Choose by inspected source and project language, not by palette name alone.
For example, `terminal` fits device/code language, `warm-print` fits editorial
print, and `synthwave` is an intentional stylization rather than generic polish.
`palette` must contain two to six exact `#RRGGBB` colors in authored order. Use
dark-to-light order for this treatment; the runtime validates colors but does
not reorder them, so reversing the array intentionally inverts the mapping.
Keep `dither` between `0.7` and `1` and `ditherSize` between `0.1` and `0.5`.
A finite reveal may author `--hf-color-grading-dither: 0` inline and tween it
to the chosen amount with GSAP. Judge the moving result at output resolution;
reject shimmer, lost subject structure, accidental muddy intermediate colors,
or a palette chosen without regard to the project's design language.
## Cached Error Diffusion
Use exact error diffusion for a deliberate 1-bit Macintosh, newspaper/print,
limited-palette game, or crunchy editorial treatment. It bakes a new image or
MP4 because every processed block depends on error from earlier blocks; it is
not a realtime shader setting.
Choose the algorithm by visible intent:
- `floyd-steinberg`: balanced default with organic fine texture.
- `atkinson`: higher-contrast, more open and distinctly early-Macintosh.
- `jarvis-judice-ninke`: smoother gradients with a wider 12-neighbor field.
- `stucki`: smooth, slightly sharper alternative to JJN.
- `burkes`: compact two-row texture.
- `sierra`, `sierra-lite`, `two-row-sierra`: progressively different
speed/texture tradeoffs; use only after comparing frames.
Run the exact processor and register its output through the existing media
ledger/cache:
```bash
node <SKILL_DIR>/scripts/dither.mjs \
--input .media/videos/video_001.mp4 \
--out .media/generated/video_001.atkinson.mp4 \
--algorithm atkinson \
--palette '#17121a,#824c50,#e09873,#f7ddb1' \
--point-size 3
node <SKILL_DIR>/scripts/resolve.mjs \
--from .media/generated/video_001.atkinson.mp4 --type video --project .
```
Use the registered output path on a real `<img>` or `<video>`. Keep text,
captions, logos, and interface graphics outside the processed media. For a
finite reveal, overlap the original and processed media with identical framing
and crossfade or wipe them using the registered paused GSAP timeline. Do not
label the realtime Bayer shader as Floyd-Steinberg/Atkinson, and do not process
PQ/HLG footage without an explicit SDR tone-map decision.
## Organic Light Leak
Use for one motivated memory beat, time shift, warm scene handoff, or tactile
transition. It is a finite deterministic CSS/GSAP overlay, not a looping
texture, generic flash, or film-stock emulation.
Install the Registry overlay block:
```bash
npx hyperframes add organic-light-leak-overlay --no-clipboard
```
Insert the printed `data-composition-src` host at the intended beat and keep
its duration finite. Its paused timeline owns one rise, peak, and complete
recovery and scales those phases to the placed duration. Inspect the source
before, at the brightest frame, and after recovery. Reject clipped faces, an
unmotivated warm wash, visible black from incorrect blend mode, or a leak that
conceals the subject longer than the transition needs.
## Freeze-Frame Cutout
Use for a social introduction, speaker emphasis, chapter punctuation, sports
or creator beat, or a scrapbook/editorial hold. This requires a real alpha
matte; decoration may not conceal a poor subject edge.
Extract the exact deterministic source frame first, then remove its background:
```bash
ffmpeg -ss <seconds> -i <source-video> -frames:v 1 -y .media/generated/freeze-source.png
npx hyperframes remove-background .media/generated/freeze-source.png \
-o .media/generated/freeze-cutout.png --json
npx hyperframes add freeze-frame-cutout --no-clipboard
```
Add the transparent result as a direct-root timed media layer and insert the
printed overlay block above the same time range. The block owns the paper,
tape, and flash; the host timeline only animates the real cutout:
```html
<img
id="hf-freeze-cutout"
class="clip"
src="./.media/generated/freeze-cutout.png"
alt=""
data-start="6"
data-duration="3"
data-track-index="20"
/>
```
```js
tl.fromTo(
"#hf-freeze-cutout",
{ y: 42, scale: 0.86, rotation: -2 },
{ y: 0, scale: 1, rotation: 0.4, duration: 0.5, ease: "back.out(1.35)" },
freezeAt,
);
```
Inspect the matte over both light and dark temporary plates before styling it.
Reject missing hair/fingers, background halos, a cutout that changes identity,
overly thick outline, exposed frame edges, or a flash that obscures the reveal.
If the matte is not acceptable, choose another frame or keep the original media.
## Social Flash / Editorial Reveal
Use this treatment for one meaningful high-energy cut, creator reveal, product
beat, or before/after handoff. It is not a default transition for every scene.
Avoid it for calm long-form footage, accessibility-sensitive contexts, already
clipped highlights, literal UI that must remain readable through the cut, or
any request for repeated strobing.
Inspect representative frames on both sides of the cut first. Grade each media
layer for its own subject using the appropriate contract above; the flash is
not a substitute for correction. For people, a restrained `skin-soft` payload
is a safe starting point. For literal UI, preserve the pixels and use only the
authored light/motion layers when they do not obscure required information.
Install the Registry overlay block:
```bash
npx hyperframes add editorial-flash-overlay --no-clipboard
```
Insert the printed `data-composition-src` host so the block's midpoint lands
on the cut. Its own paused timeline drives the finite flash. The host timeline
may coordinate outgoing and incoming media motion without reaching into the
block:
```js
tl.to(
"#outgoing-media",
{
scale: 1.035,
"--hf-color-grading-exposure": 0.82,
duration: 0.12,
ease: "power3.in",
},
cutAt - 0.16,
);
tl.fromTo(
"#incoming-media",
{ scale: 1.1 },
{ scale: 1, duration: 0.42, ease: "power3.out" },
cutAt,
);
tl.to(
"#incoming-media",
{
"--hf-color-grading-exposure": 0,
"--hf-color-grading-intensity": 0.58,
duration: 0.24,
ease: "power2.out",
},
cutAt,
);
```
When the shader steps are used, author
`--hf-color-grading-exposure: 0.72` and
`--hf-color-grading-intensity: 0` inline on the incoming media so a fresh seek
has the correct start state. Set the final intensity to the source-approved
value instead of copying `0.58` blindly. Skip the shader intensity step when
the incoming source should remain ungraded.
Keep the rise between roughly `0.035` and `0.055` seconds and the recovery
between `0.24` and `0.38` seconds. Default to one neutral/warm flash event,
never saturated red, never a looping strobe, and never more than one authored
flash inside a one-second treatment window. Verify frames immediately before,
at, and after the cut, then inspect moving playback and a rendered draft. The
peak must hide the cut; the recovery must reveal a correctly framed source with
no retained prior canvas, clipped face, or unexpected highlight damage.
skills/media-use/references/media-treatments.md
+199
View File
@@ -0,0 +1,199 @@
# Media treatments
A media treatment is a source-aware plan that composes existing HyperFrames
color, effect, timeline, and Registry primitives. It is not a second runtime
schema. Use this file to choose a primary direction. A matching recipe is an
optional tested seed; bespoke requests may assemble a validated treatment from
the canonical capability catalog.
## Permission and scope
- An explicit request such as "polish this", "make it look better", or "make it
fit the topic" delegates a conservative treatment. Apply it, verify it, and
report what changed.
- During an unsolicited opportunity scan, show or suggest the treatment first.
- Target meaningful photographic media. Skip text, SVG, logos, icons, UI
chrome, and intentionally stylized footage unless the user asks.
- Realtime grading and effects apply to the entire selected real `<img>` or
`<video>`. They do not isolate or track a face, plate, address, or other
region. For region-only work, first create a separate cropped/masked media
layer or use an external segmentation/tracking tool; never imply that a
whole-media Blur or Pixelate performed region isolation.
- The realtime treatment path is Rec.709/SDR. Do not silently process HDR, HLG,
PQ, or camera LOG sources through it.
## Classify the request
Choose the smallest lane that satisfies the request before choosing a recipe
or assembling a custom treatment:
| User intent | Lane |
| --------------------------------------------------- | ------------------------------------------ |
| too dark, flat, too warm, too many shadows | correction |
| polished, premium, warm, cinematic, fit the topic | preset or custom treatment |
| retro, print, ASCII, glitch, camcorder | shader effect or effect-bearing preset |
| obscure the whole selected media | privacy Blur or Pixelate |
| hide one face, plate, address, or screen region | separate crop/mask/asset or external tool |
| draw attention to media without changing its pixels | framing, motion, or optional overlay |
| reveal, focus, depixelate, fade the treatment | finite seek-safe treatment keyframes |
| REC HUD, light leak, flash, freeze-frame cutout | Registry overlay plus any justified pixels |
The lane identifies the primary reason for the change; it is not a one-feature
limit. A final treatment may combine correction, a preset, finishing, multiple
compatible shader effects, finite keyframed values, and optional overlays when
the inspected media and user intent justify the complete combination. Keep one
primary intent as the creative anchor so the result remains coherent and
deterministic.
Do not add a stylized Effect when correction solves the complaint. Do not
change color when the request is only temporal, and do not install an overlay
when the selected media alone communicates the result.
Match strength to intent. When the user explicitly names a bold look such as
VHS, glitch, ASCII, halftone, camcorder, print, or engraving, apply its
signature effects strongly enough to read unmistakably. The guard against
unrequested additions does not mean under-delivering an effect the user asked
for. Correction and polish stay restrained; named stylization must be obvious
in the after-frame.
### Translate vague feedback conservatively
| User feedback | First action | Add only when the frames justify it | Never infer |
| ------------------------------------- | ----------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ----------------------------------------------- |
| too many shadows and a bit boring | lift shadows/protect highlights, then compare one restrained source-appropriate preset | mild contrast or vibrance | retro texture, HUD, or palette effect |
| make the product footage feel premium | protect product color and labels, compare Product Polish | restrained vignette on lifestyle footage | a cinematic LUT or crushed blacks |
| make this reveal cooler | preserve color and animate one supported effect or treatment value | a short owned overlay block | unrelated whole-clip styling |
| make it feel like an old home video | compare 8mm and VHS language against the source | finite weave/flicker or justified HUD | that every old-video request means VHS |
| hide this face | explain that realtime effects are whole-media; isolate the region first or use an external tool | whole-media Blur/Pixelate only when the user accepts that scope | face tracking or masking that was not performed |
| keep the brand colors exact | leave UI/logo pixels unchanged; use framing and motion | demonstrated exposure-only correction | stylized preset, palette, or LUT |
When more than one lane could fit, generate at most two candidates and choose
from inspected before/after evidence. Ordinary correction or polish starts with
one candidate; a second candidate is an escalation, not the default. Do not
stack effects merely to make the answer look more sophisticated.
## Seed or assemble
Use the table when a tested recipe directly fits. Read only that recipe
section. Recipes are optional macros, not a closed list of allowed results.
| Intent or source | Recipe heading to read |
| -------------------------------------------------------- | ------------------------------------------------- |
| Talking head, interview, presenter, people-focused photo | `Natural Portrait` |
| Product footage, lifestyle footage, clean social polish | `Product Polish` |
| Screen capture, dashboard, website, app UI | `UI Fidelity` |
| Warm memory, restrained nostalgia | `Film Memory` |
| Creator/UGC handheld camera character | `Creator Camcorder` |
| Analog tape playback | `VHS Playback` |
| Small-gauge home-movie character | `8mm Home Movie` |
| Editorial dots or ink print | `Editorial Halftone` or `Two-Ink Editorial Print` |
| Monochrome dot/line screen print | `Monochrome Screen Print` |
| Engraved or hand-hatched illustration | `Engraved Illustration` or `Crosshatched Sketch` |
| Curved scanlined display | `CRT Display` |
| Glyph-based art | `Procedural ASCII` |
| Realtime palette quantization | `Ordered Palette Dither` |
| Exact historical error diffusion | `Cached Error Diffusion` |
| Finite warm flare layer | `Organic Light Leak` |
| Held-frame graphic interruption | `Freeze-Frame Cutout` |
| Short exposure-flash transition | `Social Flash / Editorial Reveal` |
Use `rg -n '^## <heading>$' <SKILL_DIR>/references/media-treatment-recipes.md`,
then read only from that heading to the next `##`. Do not load the entire
cookbook for one request.
When source intent is unclear, inspect the concise capability overview:
```bash
hyperframes media-treatment --capabilities --json
```
It lists the complete surface by family with one-line descriptions. Then load
only the family, effect, preset, or palette relevant to the inspected source:
```bash
hyperframes media-treatment --capability <id> --json
```
The focused result provides legal controls, recommended apply values, render
cost, palette support, and the exact animation contract when supported. Use
`--all` only for tooling/tests or a genuinely exhaustive audit. Compose one
nested payload from these existing parts. Recipes and catalog-built payloads
use the same renderer and persistence contract.
Treat `renderLane: "multipass"` as a cost signal. Blur, Bloom, and Kuwahara are
bounded but more expensive than single-pass effects; avoid stacking several of
them across many simultaneous media elements unless the composition needs it,
then verify playback and a draft render.
Cost follows treated pixel area as well as element count. More than two
simultaneously visible full-frame multipass media layers requires a continuous
playback check on the target machine; simplify or pre-render the stack if it
drops frames. Do not impose or claim a universal hard cap from one machine.
## Common workflow
1. Confirm the target is a real `<img>` or `<video>` and inspect source color
metadata.
2. For an image, read it once. For video, capture early/middle/late output as
one labeled sheet and read that one image:
```bash
hyperframes snapshot <project> --frames 3 --no-end --describe false \
--output snapshots/treatment-before
```
Read `snapshots/treatment-before/contact-sheet.jpg`; do not spend separate
model turns reading each frame unless the sheet exposes a specific problem.
Do not infer semantics from signal statistics alone.
3. Choose one primary lane. Use one matching recipe as a tested seed, or read
the overview and one focused capability detail when the request is bespoke.
A seed may be changed or combined with compatible catalog controls when the
contact sheet justifies it. Do not invent keys, exceed reported ranges, or
stack effects without a visual reason. Do not run the generic grade/LUT
resolver first; it adds irrelevant candidates and may download an unused
LUT. Use `grade --for --analyze` only when correction needs measured signal
evidence.
4. Persist pixel settings with `hyperframes media-treatment`; it validates and
merges a patch into the existing nested `data-color-grading` contract. Use registered
GSAP only for supported animated values and Registry overlay blocks only
for authored dressing.
```bash
hyperframes media-treatment --selector "<unique selector>" \
--grading '<nested JSON patch>' --apply --json
```
For a temporal reveal, use the focused capability result's `animation`
contract. If it is `null`, the capability is static. Author the starting CSS
property inline on the real media element and return temporary treatment
values to neutral so the finished shot preserves its existing pixels.
Prefer that bounded media animation first; if an overlay is justified,
install the owned Registry block instead of recreating it with bespoke
overlay markup.
For correction and ordinary polish, keep those values as editable
preset/adjustment JSON; do not generate a LUT for controls the realtime shader
already owns.
Use the canonical `details`/`effects` fields for vignette, grain, blur,
pixelate, and related primitives. Do not duplicate them with CSS filters,
SVG turbulence, opacity, or decorative DOM overlays.
5. When the treatment calls for an overlay, install that named block with
`hyperframes add <name> --dir <project> --no-clipboard --json`, inspect its
returned `data-composition-src` host, and place it once using the block's
timing contract. Check for the installed file and host element
ID before insertion; never duplicate an existing overlay block. This is one
treatment workflow: do not make the user discover Catalog or separately ask
for the recipe's justified overlay.
6. For ordinary correction/polish, capture one after-sheet with the same three
timestamps under `snapshots/treatment-after`, compare it to the before-sheet,
and stop when the result is clearly better. Run the normal project check;
do not encode a draft solely to prove a static correction.
7. Escalate only when evidence requires it. Read individual frames to diagnose a
specific visual problem. Preview and render moving evidence when judging
treatment keyframes, glitch/tape motion, overlays, playback smoothness, LUT
timing, or any other temporal behavior. HDR/LOG, privacy, and brand-sensitive
work also require the existing explicit caveats and stronger verification.
If the treatment is not clearly better, keep the source unchanged.
8. Report the selected media, primary intent, recipe seed if used, final
composed controls, optional overlays, and the frames/render that were
actually checked. Do not report visual quality from command success alone.
`resolve --type grade --for ... --analyze --json` provides deterministic
clipping and signal evidence, not subject recognition or automatic taste.
skills/media-use/references/operations.md
+34
View File
@@ -64,6 +64,40 @@ asset (it records provenance and auto-promotes to the global cache).
> `--from`. Add a thin `process` verb only if agents repeatedly fumble these
> recipes.
## Exact error-diffusion dither
Use the local processor when the requested look specifically calls for
Floyd-Steinberg, Atkinson/Macintosh, Jarvis-Judice-Ninke, Stucki, Burkes, or a
Sierra variant. These are sequential error-diffusion algorithms, not the
realtime Bayer `effects.dither` shader.
```bash
node <SKILL_DIR>/scripts/dither.mjs \
--input source.mp4 \
--out source.atkinson.mp4 \
--algorithm atkinson \
--palette '#0f380f,#306230,#8bac0f,#9bbc0f' \
--point-size 3
node <SKILL_DIR>/scripts/resolve.mjs \
--from source.atkinson.mp4 --type video --project .
```
Available algorithms: `floyd-steinberg`, `atkinson`,
`jarvis-judice-ninke`, `stucki`, `burkes`, `sierra`, `sierra-lite`, and
`two-row-sierra`. The default is balanced Floyd-Steinberg with a black/white
palette. Palettes contain 2-6 `#rrggbb` colors in authored dark-to-light order;
reversing the order intentionally inverts the mapping. `--point-size` controls
1-20px blocks; `--brightness` and `--contrast` accept 0.5-2; `--detail` accepts
0.1-1.
The processor supports ordinary SDR images and MP4 video, preserves video
audio, and emits BT.709 MP4. It rejects tagged PQ/HLG input rather than silently
tone-mapping it. To animate the transformation, keep the original and processed
files as two real media layers and use the seek-safe GSAP timeline to reveal or
crossfade between them. Use the realtime Bayer shader instead when the dither
amount itself must animate continuously.
## Transcription (default: Parakeet, better than whisper.cpp)
`transcribe.mjs` is the default local transcription path. It runs **NVIDIA
skills/media-use/scripts/dither.mjs
+328
View File
@@ -0,0 +1,328 @@
#!/usr/bin/env node
import { execFileSync, spawn } from "node:child_process";
import { once } from "node:events";
import { existsSync, mkdirSync, renameSync, rmSync } from "node:fs";
import { dirname, extname, resolve } from "node:path";
import { text } from "node:stream/consumers";
import { parseArgs } from "node:util";
import {
ERROR_DIFFUSION_ALGORITHMS,
applyErrorDiffusionRgba,
errorDiffusionBufferLength,
} from "./lib/error-diffusion.mjs";
const IMAGE_EXTENSIONS = new Set([".png", ".jpg", ".jpeg", ".webp", ".bmp", ".tif", ".tiff"]);
const OUTPUT_IMAGE_EXTENSIONS = new Set([".png", ".jpg", ".jpeg", ".webp"]);
const { values: args } = parseArgs({
options: {
input: { type: "string", short: "i" },
out: { type: "string", short: "o" },
algorithm: { type: "string", short: "a", default: "floyd-steinberg" },
palette: { type: "string", default: "#000000,#ffffff" },
"point-size": { type: "string", default: "3" },
brightness: { type: "string", default: "1" },
contrast: { type: "string", default: "1.2" },
detail: { type: "string", default: "1" },
json: { type: "boolean", default: false },
help: { type: "boolean", short: "h", default: false },
},
strict: true,
});
if (args.help) {
console.log(`media-use dither — exact cached error-diffusion for image or MP4 media
Usage:
node dither.mjs --input in.mp4 --out out.mp4 [options]
Options:
--algorithm, -a ${Object.keys(ERROR_DIFFUSION_ALGORITHMS).join(" | ")}
--palette 2-6 authored-order #rrggbb colors, comma-separated
--point-size Block size in pixels, 1-20 (default: 3)
--brightness 0.5-2 (default: 1)
--contrast 0.5-2 (default: 1.2)
--detail Diffused-error strength, 0.1-1 (default: 1)
--json Output JSON status
--help, -h Show this help
Video output uses the source average frame rate as CFR; VFR cadence is normalized.
After processing, register the output with:
node resolve.mjs --from <output> --type image|video`);
process.exit(0);
}
try {
const result = await run();
if (args.json) console.log(JSON.stringify({ ok: true, ...result }));
else {
console.log(`dithered ${result.input} -> ${result.out} (${result.algorithm})`);
console.log(`next: resolve --from ${result.out} --type ${result.type}`);
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (args.json) console.log(JSON.stringify({ ok: false, error: message }));
else console.error(`error: ${message}`);
process.exit(1);
}
async function run() {
if (!args.input || !args.out) throw new Error("--input and --out are required");
const inputPath = resolve(args.input);
const outPath = resolve(args.out);
if (!existsSync(inputPath)) throw new Error(`input file not found: ${inputPath}`);
if (inputPath === outPath) throw new Error("--out must differ from --input");
const metadata = probe(inputPath);
if (metadata.colorTransfer === "smpte2084" || metadata.colorTransfer === "arib-std-b67") {
throw new Error(
`HDR ${metadata.colorTransfer} input is not supported by the 8-bit SDR dither processor; tone-map to Rec.709 first`,
);
}
const options = {
algorithm: args.algorithm,
palette: args.palette.split(",").map((color) => color.trim()),
pointSize: Number(args["point-size"]),
brightness: Number(args.brightness),
contrast: Number(args.contrast),
detail: Number(args.detail),
};
// Validate before starting FFmpeg or creating an output file.
applyErrorDiffusionRgba(new Uint8ClampedArray(4), 1, 1, options, new Float32Array(3));
mkdirSync(dirname(outPath), { recursive: true });
const inputIsImage = IMAGE_EXTENSIONS.has(extname(inputPath).toLowerCase());
if (inputIsImage) {
if (!OUTPUT_IMAGE_EXTENSIONS.has(extname(outPath).toLowerCase())) {
throw new Error("image output must use .png, .jpg, .jpeg, or .webp");
}
processImage(inputPath, outPath, metadata, options);
} else {
if (extname(outPath).toLowerCase() !== ".mp4") throw new Error("video output must use .mp4");
await processVideo(inputPath, outPath, metadata, options);
}
return {
input: inputPath,
out: outPath,
type: inputIsImage ? "image" : "video",
algorithm: options.algorithm,
palette: options.palette,
point_size: options.pointSize,
brightness: options.brightness,
contrast: options.contrast,
detail: options.detail,
};
}
function probe(filePath) {
const raw = execFileSync(
"ffprobe",
["-v", "error", "-print_format", "json", "-show_streams", "-show_format", filePath],
{ encoding: "utf8", timeout: 10_000 },
);
const parsed = JSON.parse(raw);
const video = parsed.streams?.find((stream) => stream.codec_type === "video");
if (!video?.width || !video?.height)
throw new Error(`no readable video/image stream: ${filePath}`);
const fps = usableFrameRate(video.avg_frame_rate) ?? usableFrameRate(video.r_frame_rate) ?? "30";
return {
width: video.width,
height: video.height,
fps,
colorTransfer: video.color_transfer || "",
};
}
function processImage(inputPath, outPath, metadata, options) {
const frameBytes = metadata.width * metadata.height * 4;
const rgba = execFileSync(
"ffmpeg",
[
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-i",
inputPath,
"-frames:v",
"1",
"-f",
"rawvideo",
"-pix_fmt",
"rgba",
"-",
],
{ maxBuffer: frameBytes + 1024 },
);
if (rgba.length !== frameBytes)
throw new Error(`decoded ${rgba.length} bytes; expected ${frameBytes}`);
applyErrorDiffusionRgba(rgba, metadata.width, metadata.height, options);
const temporary = temporaryOutput(outPath);
try {
execFileSync(
"ffmpeg",
[
"-y",
"-hide_banner",
"-loglevel",
"error",
"-f",
"rawvideo",
"-pix_fmt",
"rgba",
"-s:v",
`${metadata.width}x${metadata.height}`,
"-i",
"-",
"-frames:v",
"1",
temporary,
],
{ input: rgba, maxBuffer: frameBytes + 1024 },
);
renameSync(temporary, outPath);
} finally {
rmSync(temporary, { force: true });
}
}
async function processVideo(inputPath, outPath, metadata, options) {
const temporary = temporaryOutput(outPath);
const frameBytes = metadata.width * metadata.height * 4;
const keyframeInterval = String(Math.max(1, Math.round(frameRateNumber(metadata.fps))));
const errors = new Float32Array(
errorDiffusionBufferLength(metadata.width, metadata.height, options.pointSize),
);
const decoder = spawn("ffmpeg", [
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-i",
inputPath,
"-map",
"0:v:0",
"-f",
"rawvideo",
"-pix_fmt",
"rgba",
"-",
]);
const encoder = spawn("ffmpeg", [
"-y",
"-hide_banner",
"-loglevel",
"error",
"-f",
"rawvideo",
"-pix_fmt",
"rgba",
"-s:v",
`${metadata.width}x${metadata.height}`,
"-r",
metadata.fps,
"-i",
"-",
"-i",
inputPath,
"-map",
"0:v:0",
"-map",
"1:a?",
"-map_metadata",
"1",
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"18",
"-g",
keyframeInterval,
"-keyint_min",
keyframeInterval,
"-sc_threshold",
"0",
"-pix_fmt",
"yuv420p",
"-x264-params",
"colorprim=bt709:transfer=bt709:colormatrix=bt709",
"-color_primaries:v",
"bt709",
"-color_trc:v",
"bt709",
"-colorspace:v",
"bt709",
"-color_range",
"tv",
"-c:a",
"aac",
"-b:a",
"192k",
"-shortest",
"-movflags",
"+faststart",
temporary,
]);
const decoderError = text(decoder.stderr);
const encoderError = text(encoder.stderr);
const decoderDone = once(decoder, "close").then(([code]) => code ?? 1);
const encoderDone = once(encoder, "close").then(([code]) => code ?? 1);
try {
const frame = Buffer.allocUnsafe(frameBytes);
let frameOffset = 0;
for await (const chunk of decoder.stdout) {
let chunkOffset = 0;
while (chunkOffset < chunk.length) {
const length = Math.min(frameBytes - frameOffset, chunk.length - chunkOffset);
chunk.copy(frame, frameOffset, chunkOffset, chunkOffset + length);
chunkOffset += length;
frameOffset += length;
if (frameOffset !== frameBytes) continue;
applyErrorDiffusionRgba(frame, metadata.width, metadata.height, options, errors);
await writeFrame(encoder.stdin, frame);
frameOffset = 0;
}
}
if (frameOffset)
throw new Error(`decoder returned a partial RGBA frame (${frameOffset} bytes)`);
encoder.stdin.end();
const [decoderCode, encoderCode] = await Promise.all([decoderDone, encoderDone]);
if (decoderCode !== 0) throw new Error(`FFmpeg decode failed: ${(await decoderError).trim()}`);
if (encoderCode !== 0) throw new Error(`FFmpeg encode failed: ${(await encoderError).trim()}`);
renameSync(temporary, outPath);
} catch (error) {
decoder.kill("SIGKILL");
encoder.kill("SIGKILL");
throw error;
} finally {
rmSync(temporary, { force: true });
}
}
function writeFrame(stream, frame) {
return new Promise((resolveWrite, reject) => {
stream.write(frame, (error) => (error ? reject(error) : resolveWrite()));
});
}
function usableFrameRate(value) {
if (!value || value === "0/0") return null;
const number = frameRateNumber(value);
return Number.isFinite(number) && number > 0 ? value : null;
}
function frameRateNumber(value) {
const [numerator, denominator = "1"] = value.split("/");
return Number(numerator) / Number(denominator);
}
function temporaryOutput(outPath) {
const extension = extname(outPath);
return `${outPath.slice(0, -extension.length)}.part-${process.pid}${extension}`;
}
skills/media-use/scripts/dither.test.mjs
+175
View File
@@ -0,0 +1,175 @@
import assert from "node:assert/strict";
import { execFileSync, spawnSync } from "node:child_process";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { fileURLToPath } from "node:url";
const SCRIPT = fileURLToPath(new URL("./dither.mjs", import.meta.url));
const HAS_FFMPEG =
spawnSync("ffmpeg", ["-version"], { stdio: "ignore" }).status === 0 &&
spawnSync("ffprobe", ["-version"], { stdio: "ignore" }).status === 0;
function fixture() {
const dir = mkdtempSync(join(tmpdir(), "media-use-dither-"));
return { dir, cleanup: () => rmSync(dir, { recursive: true, force: true }) };
}
function ffmpeg(args) {
execFileSync("ffmpeg", ["-y", "-hide_banner", "-loglevel", "error", ...args]);
}
function run(args) {
return spawnSync(process.execPath, [SCRIPT, ...args, "--json"], { encoding: "utf8" });
}
test("processes an image into the requested ordered palette", { skip: !HAS_FFMPEG }, (t) => {
const { dir, cleanup } = fixture();
t.after(cleanup);
const input = join(dir, "source.png");
const output = join(dir, "dithered.png");
ffmpeg(["-f", "lavfi", "-i", "testsrc2=size=16x12:rate=1", "-frames:v", "1", input]);
const result = run([
"--input",
input,
"--out",
output,
"--algorithm",
"floyd-steinberg",
"--palette",
"#000000,#ffffff",
"--point-size",
"3",
]);
assert.equal(result.status, 0, result.stderr || result.stdout);
assert.equal(JSON.parse(result.stdout).type, "image");
const rgb = execFileSync("ffmpeg", [
"-hide_banner",
"-loglevel",
"error",
"-i",
output,
"-frames:v",
"1",
"-f",
"rawvideo",
"-pix_fmt",
"rgb24",
"-",
]);
for (let index = 0; index < rgb.length; index += 3) {
const value = rgb[index];
assert.ok(value === 0 || value === 255);
assert.equal(rgb[index + 1], value);
assert.equal(rgb[index + 2], value);
}
});
test("processes moving MP4 frames, audio, and BT.709 metadata", { skip: !HAS_FFMPEG }, (t) => {
const { dir, cleanup } = fixture();
t.after(cleanup);
const input = join(dir, "source.mp4");
const output = join(dir, "dithered.mp4");
ffmpeg([
"-f",
"lavfi",
"-i",
"testsrc2=size=16x12:rate=3:duration=2",
"-f",
"lavfi",
"-i",
"sine=frequency=440:duration=3",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p",
"-c:a",
"aac",
input,
]);
const result = run([
"--input",
input,
"--out",
output,
"--algorithm",
"atkinson",
"--palette",
"#0f380f,#306230,#8bac0f,#9bbc0f",
]);
assert.equal(result.status, 0, result.stderr || result.stdout);
const probe = JSON.parse(
execFileSync(
"ffprobe",
["-v", "error", "-print_format", "json", "-show_streams", "-show_format", output],
{
encoding: "utf8",
},
),
);
const video = probe.streams.find((stream) => stream.codec_type === "video");
const audio = probe.streams.find((stream) => stream.codec_type === "audio");
assert.equal(video.width, 16);
assert.equal(video.height, 12);
assert.equal(video.nb_frames, "6");
assert.equal(video.color_space, "bt709");
assert.equal(video.color_transfer, "bt709");
assert.equal(video.color_primaries, "bt709");
assert.equal(video.color_range, "tv");
assert.equal(audio.codec_name, "aac");
assert.ok(Number(probe.format.duration) < 2.5, "audio must not outlive processed video");
const keyframes = execFileSync(
"ffprobe",
[
"-v",
"error",
"-select_streams",
"v:0",
"-skip_frame",
"nokey",
"-show_entries",
"frame=best_effort_timestamp_time",
"-of",
"csv=p=0",
output,
],
{ encoding: "utf8" },
)
.trim()
.split(/\s+/)
.map((value) => Number.parseFloat(value));
assert.deepEqual(keyframes, [0, 1]);
});
test("rejects tagged PQ or HLG instead of silently producing SDR", { skip: !HAS_FFMPEG }, (t) => {
const { dir, cleanup } = fixture();
t.after(cleanup);
const input = join(dir, "hlg.mp4");
ffmpeg([
"-f",
"lavfi",
"-i",
"color=c=white:size=16x12:rate=1:duration=1",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p10le",
"-x264-params",
"colorprim=bt2020:transfer=arib-std-b67:colormatrix=bt2020nc",
"-color_primaries:v",
"bt2020",
"-color_trc:v",
"arib-std-b67",
"-colorspace:v",
"bt2020nc",
input,
]);
const result = run(["--input", input, "--out", join(dir, "wrong.mp4")]);
assert.equal(result.status, 1);
assert.match(JSON.parse(result.stdout).error, /HDR arib-std-b67 input is not supported/);
});
skills/media-use/scripts/lib/error-diffusion.mjs
+230
View File
@@ -0,0 +1,230 @@
export const ERROR_DIFFUSION_ALGORITHMS = {
"floyd-steinberg": {
kernel: [
[1, 0, 7],
[-1, 1, 3],
[0, 1, 5],
[1, 1, 1],
],
divisor: 16,
},
atkinson: {
kernel: [
[1, 0, 1],
[2, 0, 1],
[-1, 1, 1],
[0, 1, 1],
[1, 1, 1],
[0, 2, 1],
],
divisor: 8,
},
"jarvis-judice-ninke": {
kernel: [
[1, 0, 7],
[2, 0, 5],
[-2, 1, 3],
[-1, 1, 5],
[0, 1, 7],
[1, 1, 5],
[2, 1, 3],
[-2, 2, 1],
[-1, 2, 3],
[0, 2, 5],
[1, 2, 3],
[2, 2, 1],
],
divisor: 48,
},
stucki: {
kernel: [
[1, 0, 8],
[2, 0, 4],
[-2, 1, 2],
[-1, 1, 4],
[0, 1, 8],
[1, 1, 4],
[2, 1, 2],
[-2, 2, 1],
[-1, 2, 2],
[0, 2, 4],
[1, 2, 2],
[2, 2, 1],
],
divisor: 42,
},
burkes: {
kernel: [
[1, 0, 8],
[2, 0, 4],
[-2, 1, 2],
[-1, 1, 4],
[0, 1, 8],
[1, 1, 4],
[2, 1, 2],
],
divisor: 32,
},
sierra: {
kernel: [
[1, 0, 5],
[2, 0, 3],
[-2, 1, 2],
[-1, 1, 4],
[0, 1, 5],
[1, 1, 4],
[2, 1, 2],
[-1, 2, 2],
[0, 2, 3],
[1, 2, 2],
],
divisor: 32,
},
"sierra-lite": {
kernel: [
[1, 0, 2],
[-1, 1, 1],
[0, 1, 1],
],
divisor: 4,
},
"two-row-sierra": {
kernel: [
[1, 0, 4],
[2, 0, 3],
[-2, 1, 1],
[-1, 1, 2],
[0, 1, 3],
[1, 1, 2],
[2, 1, 1],
],
divisor: 16,
},
};
const DEFAULTS = {
algorithm: "floyd-steinberg",
brightness: 1,
contrast: 1.2,
detail: 1,
palette: ["#000000", "#ffffff"],
pointSize: 3,
};
export function errorDiffusionBufferLength(width, height, pointSize) {
return Math.ceil(width / pointSize) * Math.ceil(height / pointSize) * 3;
}
export function applyErrorDiffusionRgba(data, width, height, options = {}, errorBuffer) {
if (!Number.isInteger(width) || width < 1 || !Number.isInteger(height) || height < 1) {
throw new Error("width and height must be positive integers");
}
if (!data || data.length !== width * height * 4) {
throw new Error(`RGBA data must contain ${width * height * 4} bytes`);
}
const algorithm = options.algorithm ?? DEFAULTS.algorithm;
const diffusion = ERROR_DIFFUSION_ALGORITHMS[algorithm];
if (!diffusion) throw new Error(`unknown error-diffusion algorithm: ${algorithm}`);
const pointSize = integerInRange(options.pointSize ?? DEFAULTS.pointSize, 1, 20, "pointSize");
const brightness = numberInRange(options.brightness ?? DEFAULTS.brightness, 0.5, 2, "brightness");
const contrast = numberInRange(options.contrast ?? DEFAULTS.contrast, 0.5, 2, "contrast");
const detail = numberInRange(options.detail ?? DEFAULTS.detail, 0.1, 1, "detail");
const palette = parsePalette(options.palette ?? DEFAULTS.palette);
const blockColumns = Math.ceil(width / pointSize);
const blockRows = Math.ceil(height / pointSize);
const errorLength = errorDiffusionBufferLength(width, height, pointSize);
const errors = errorBuffer ?? new Float32Array(errorLength);
if (!(errors instanceof Float32Array) || errors.length !== errorLength) {
throw new Error(`errorBuffer must be a Float32Array of length ${errorLength}`);
}
errors.fill(0);
const centerOffset = Math.floor(pointSize / 2);
for (let blockRow = 0; blockRow < blockRows; blockRow++) {
const blockY = blockRow * pointSize;
for (let blockColumn = 0; blockColumn < blockColumns; blockColumn++) {
const blockX = blockColumn * pointSize;
const centerX = Math.min(blockX + centerOffset, width - 1);
const centerY = Math.min(blockY + centerOffset, height - 1);
const rgbaIndex = (centerY * width + centerX) * 4;
const errorIndex = (blockRow * blockColumns + blockColumn) * 3;
const red = correctedChannel(data[rgbaIndex], errors[errorIndex], brightness, contrast);
const green = correctedChannel(
data[rgbaIndex + 1],
errors[errorIndex + 1],
brightness,
contrast,
);
const blue = correctedChannel(
data[rgbaIndex + 2],
errors[errorIndex + 2],
brightness,
contrast,
);
const luminance = 0.299 * red + 0.587 * green + 0.114 * blue;
const output = palette[Math.min(palette.length - 1, Math.floor(luminance * palette.length))];
for (let y = blockY; y < Math.min(blockY + pointSize, height); y++) {
for (let x = blockX; x < Math.min(blockX + pointSize, width); x++) {
const outputIndex = (y * width + x) * 4;
data[outputIndex] = Math.round(output[0] * 255);
data[outputIndex + 1] = Math.round(output[1] * 255);
data[outputIndex + 2] = Math.round(output[2] * 255);
}
}
for (const [dx, dy, weight] of diffusion.kernel) {
const targetColumn = blockColumn + dx;
const targetRow = blockRow + dy;
if (
targetColumn < 0 ||
targetColumn >= blockColumns ||
targetRow < 0 ||
targetRow >= blockRows
) {
continue;
}
const target = (targetRow * blockColumns + targetColumn) * 3;
const scale = (weight / diffusion.divisor) * detail;
errors[target] += (red - output[0]) * scale;
errors[target + 1] += (green - output[1]) * scale;
errors[target + 2] += (blue - output[2]) * scale;
}
}
}
return data;
}
function correctedChannel(byte, error, brightness, contrast) {
return Math.min(1, Math.max(0, ((byte / 255 - 0.5) * contrast + 0.5) * brightness + error));
}
function parsePalette(colors) {
if (!Array.isArray(colors) || colors.length < 2 || colors.length > 6) {
throw new Error("palette must contain 2 to 6 colors");
}
return colors.map((color) => {
const match = /^#([0-9a-f]{6})$/i.exec(color);
if (!match) throw new Error(`palette color must use #rrggbb: ${color}`);
const value = Number.parseInt(match[1], 16);
return [(value >> 16) / 255, ((value >> 8) & 255) / 255, (value & 255) / 255];
});
}
function numberInRange(value, min, max, name) {
const number = Number(value);
if (!Number.isFinite(number) || number < min || number > max) {
throw new Error(`${name} must be between ${min} and ${max}`);
}
return number;
}
function integerInRange(value, min, max, name) {
const number = Number(value);
if (!Number.isInteger(number) || number < min || number > max) {
throw new Error(`${name} must be an integer between ${min} and ${max}`);
}
return number;
}
skills/media-use/scripts/lib/error-diffusion.test.mjs
+102
View File
@@ -0,0 +1,102 @@
import assert from "node:assert/strict";
import test from "node:test";
import { ERROR_DIFFUSION_ALGORITHMS, applyErrorDiffusionRgba } from "./error-diffusion.mjs";
const EXPECTED_GRADIENTS = {
"floyd-steinberg": "00000101/00010101/00100101/00010111/01010101/01011011",
atkinson: "00000011/00001100/00010011/00010111/01001101/00111011",
"jarvis-judice-ninke": "00000011/00001011/00011001/00101111/00100111/01011011",
stucki: "00000011/00010101/00010110/00101011/00101101/01010111",
burkes: "00000101/00010011/00010110/00101011/01010111/00101011",
sierra: "00000011/00010101/00010110/00100111/00110111/00101101",
"sierra-lite": "00000101/00010101/00100101/00010110/01010111/01010101",
"two-row-sierra": "00000101/00010011/00010110/00101011/00101101/01011011",
};
test("exposes the eight article error-diffusion algorithms", () => {
assert.deepEqual(Object.keys(ERROR_DIFFUSION_ALGORITHMS), Object.keys(EXPECTED_GRADIENTS));
});
test("matches deterministic golden patterns for every diffusion kernel", () => {
const width = 8;
const height = 6;
const source = new Uint8ClampedArray(width * height * 4);
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
const value = Math.round((255 * (x + y * 0.7)) / (width - 1 + (height - 1) * 0.7));
const offset = (y * width + x) * 4;
source[offset] = value;
source[offset + 1] = Math.round(value * 0.8);
source[offset + 2] = Math.round(value * 0.55);
source[offset + 3] = 17 + x + y;
}
}
for (const [algorithm, expected] of Object.entries(EXPECTED_GRADIENTS)) {
const output = source.slice();
applyErrorDiffusionRgba(output, width, height, {
algorithm,
brightness: 1,
contrast: 1,
detail: 1,
palette: ["#000000", "#ffffff"],
pointSize: 1,
});
const rows = [];
for (let y = 0; y < height; y++) {
let row = "";
for (let x = 0; x < width; x++) row += output[(y * width + x) * 4] ? "1" : "0";
rows.push(row);
}
assert.equal(rows.join("/"), expected, algorithm);
for (let i = 0; i < width * height; i++) assert.equal(output[i * 4 + 3], source[i * 4 + 3]);
}
});
test("fills point-size blocks from their center sample", () => {
const data = new Uint8ClampedArray([
0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 255, 255, 255, 6, 0, 0, 0, 7, 255,
255, 255, 8,
]);
applyErrorDiffusionRgba(
data,
4,
2,
{
algorithm: "floyd-steinberg",
brightness: 1,
contrast: 1,
detail: 1,
palette: ["#000000", "#ffffff"],
pointSize: 2,
},
new Float32Array(6),
);
assert.deepEqual(
[...data],
[
255, 255, 255, 1, 255, 255, 255, 2, 255, 255, 255, 3, 255, 255, 255, 4, 255, 255, 255, 5, 255,
255, 255, 6, 255, 255, 255, 7, 255, 255, 255, 8,
],
);
});
test("preserves authored palette order and validates the public contract", () => {
const reversed = new Uint8ClampedArray([0, 0, 0, 255]);
applyErrorDiffusionRgba(reversed, 1, 1, {
palette: ["#ffffff", "#000000"],
});
assert.deepEqual([...reversed], [255, 255, 255, 255]);
assert.throws(
() => applyErrorDiffusionRgba(new Uint8ClampedArray(4), 1, 1, { palette: ["#000000"] }),
/2 to 6 colors/,
);
assert.throws(
() =>
applyErrorDiffusionRgba(new Uint8ClampedArray(4), 1, 1, {
algorithm: "ordered-bayer",
}),
/unknown error-diffusion algorithm/,
);
});
skills/media-use/scripts/lib/lut-preset-provider.mjs
+12 -4
View File
@@ -27,6 +27,11 @@ export const RESOLVABLE_PRESET_IDS = [
"soft-boost",
"bright-pop",
"deep-contrast",
"creator-camcorder",
"vhs-playback",
"home-movie-8mm",
"editorial-halftone",
"two-ink-print",
];
const PRESET_SYNONYMS = {
@@ -76,6 +81,11 @@ const PRESET_SYNONYMS = {
"clean colorful",
],
"deep-contrast": ["deep contrast", "high contrast punchy", "punchy contrast", "bold contrast"],
"creator-camcorder": ["creator camcorder", "creator video", "ugc camera", "handheld creator"],
"vhs-playback": ["vhs playback", "vhs tape", "analog tape", "degraded tape"],
"home-movie-8mm": ["8mm home movie", "8mm film", "family film", "small gauge film"],
"editorial-halftone": ["editorial halftone", "halftone", "print dots", "newsprint"],
"two-ink-print": ["two ink print", "two ink editorial", "duotone print", "poster print"],
};
function presetCandidates() {
@@ -122,10 +132,8 @@ export function matchColorLook(intent) {
.trim()
.toLowerCase()
.replace(/\s+/g, " ");
for (const candidate of presetCandidates()) {
if (candidate.preset === normalized) {
return { kind: "preset", preset: candidate.preset, score: 99 };
}
if (RESOLVABLE_PRESET_IDS.includes(normalized)) {
return { kind: "preset", preset: normalized, score: 99 };
}
const candidates = [...presetCandidates(), ...libraryCandidates()]
skills/media-use/scripts/lib/lut-preset-provider.test.mjs
+5
View File
@@ -42,6 +42,11 @@ test("removed preset phrases resolve to surviving looks", () => {
assert.equal(matchColorLook("cool clean").preset, "clean-studio");
});
test("complete-filter intent aliases resolve deterministically", () => {
assert.equal(matchColorLook("analog tape").preset, "vhs-playback");
assert.equal(matchColorLook("creator video").preset, "creator-camcorder");
});
test("library look freezes a validated cube from params offline (--local-only)", async () => {
const projectDir = mkdtempSync(join(tmpdir(), "mu-lut-provider-"));
try {
skills/media-use/scripts/resolve.mjs
+22
View File
@@ -79,6 +79,7 @@ const { values: args } = parseArgs({
from: { type: "string" },
params: { type: "string" },
for: { type: "string" },
analyze: { type: "boolean", default: false },
"local-only": { type: "boolean", default: false },
provider: { type: "string" },
"avatar-id": { type: "string" },
@@ -115,6 +116,7 @@ Options:
--params <json> Build an explicit parametric LUT (lut/grade only)
--for <media> Analyze a local image/video and add measured grade adjust
suggestions (grade only)
--analyze Return --for grade evidence without recording a candidate
--local-only Offline: skip every network provider
--provider Force one generator (e.g. codex, mflux, kokoro, heygen)
--avatar-id Override the default avatar for heygen.video generation
@@ -198,6 +200,26 @@ if (args.from) {
process.exit(0);
}
if (args.analyze) {
if (type !== "grade" || !args.for) {
console.error("error: --analyze requires --type grade and --for <media>");
process.exit(2);
}
const mediaPath = resolve(args.for);
if (!existsSync(mediaPath)) {
console.error(`error: --for file not found: ${mediaPath}`);
process.exit(2);
}
const analysis = analyzeMediaGrade(mediaPath);
if (args.json) {
console.log(JSON.stringify({ ok: true, type: "grade-analysis", ...analysis }));
} else {
console.log(formatMeasuredNote(mediaPath, analysis.measured));
console.log(`suggested adjust: ${JSON.stringify(analysis.adjust)}`);
}
process.exit(0);
}
// Recipes: folder-based named bundles resolved by entity name — no providers,
// no content hashing (an evolving versioned bundle, not an immutable file).
// Delegates to lib/recipe-store.mjs the way grade/lut delegate to resolveColor;
skills/media-use/scripts/resolve.test.mjs
+28
View File
@@ -553,6 +553,7 @@ test("--help exits 0", () => {
assert.ok(out.includes("media-use resolve"));
assert.ok(out.includes("--type"));
assert.ok(out.includes("--for"));
assert.ok(out.includes("--analyze"));
assert.ok(out.includes("--from"));
assert.ok(out.includes("--local-only"));
assert.ok(out.includes("--stats"));
@@ -765,6 +766,33 @@ test("smart grade merges measured adjust and keeps stdout valid JSON", () => {
cleanup();
});
test("grade analysis returns evidence without recording a candidate", () => {
if (!HAS_FFMPEG) {
console.log(" (skipped: ffmpeg not on PATH)");
return;
}
setup();
const frame = makeFrame(tmp, "under.png", "0x202020");
const proc = spawnResolve([
"--type",
"grade",
"--for",
frame,
"--analyze",
"--project",
tmp,
"--json",
]);
assert.equal(proc.status, 0, proc.stderr);
const parsed = JSON.parse(proc.stdout);
assert.equal(parsed.ok, true);
assert.equal(parsed.type, "grade-analysis");
assert.ok(parsed.adjust.exposure > 0, "under-exposed frame should suggest lift");
assert.ok(parsed.measured.frames > 0);
assert.equal(readManifest(tmp).length, 0);
cleanup();
});
test("emitted grading block survives the core normalizeHfColorGrading contract", () => {
if (!CAN_TSX) {
console.log(" (skipped: tsx / core source unavailable)");