feat(skill): w2h-verify script + Step 6 DoD wiring — tooling-based enforcement

Three rounds of text-based enforcement plateaued. A third agent debrief
showed the same failures: 9% asset usage (vs ≥30% floor), shader
transitions declared in STORYBOARD.md but not in shipping index.html,
SFX timestamps drifted up to 12.4s, animation-map skipped, MP4 not
rendered, honest-disclosure section omitted from final summary.

The pattern is clear: language-only enforcement is selectively
interpretable by the agent under ship pressure. Move enforcement into
tooling — facts the agent can't fudge.

**New script: `skills/website-to-hyperframes/scripts/w2h-verify.mjs`**

Pure file-analysis script (no shell spawns). Computes six checks and
prints a PASS/FAIL/INFO table:

1. Asset usage — assets referenced in compositions ÷ assets captured;
   target ≥30%. Tested against videos/huly-v3: caught 6/74 = 8% FAIL.
2. Shader transitions consistency — STORYBOARD.md-declared shaders vs
   index.html. Longest-name matching to avoid substring false positives
   (cross-warp-morph not double-counted as cross-warp). Tested: caught
   6 declared / 1 present / 5 missing.
3. SFX timestamp drift — parses STORYBOARD.md table rows for
   `sfx/X.mp3` + time-with-`s`, parses index.html <audio data-start>,
   flags drift >0.5s. Tested: caught 12.4s drift on click.mp3 that
   the agent debrief didn't even mention.
4. animation-map.json existence — explicit file check.
5. Rendered MP4 existence — scans project root, output/, renders/.
6. Required artifacts — STORYBOARD.md, DESIGN.md, SCRIPT.md, index.html
   all present.

Exit code: 0 (all pass) or 1 (one or more fail). The script's output
becomes the Step 6 deliverable — paste verbatim into the user-facing
summary.

**Skill update: `step-6-validate.md`**

Adds `w2h-verify report` to the DoD checklist with the rule: paste the
FULL output verbatim into the final summary. Cherry-picking rows,
substituting adjectives for percentages, or omitting FAIL lines is
explicitly forbidden. If a row says FAIL, either fix it and re-run
until PASS or include the FAIL line verbatim in "What I did NOT
verify" with a one-sentence reason.

Test run against the project that prompted this:

```
SUMMARY: 1 PASS · 4 FAIL · 1 INFO
- Asset usage: FAIL 6/74 (8%) target ≥30%
- Shader transitions: FAIL 6 declared, 1 present, 5 missing
- SFX timestamps: FAIL 3 drifted >0.5s (max 12.4s)
- animation-map.json: FAIL missing
- Rendered MP4: INFO no .mp4 found
- Required artifacts: PASS
```

The agent could selectively ignore "the WCAG warnings are false
positives." The agent cannot selectively ignore a line that says
`6/74 (8%) — target ≥30%`.

2 files changed (+ 1 new script, ~330 lines). Format checks pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
ukimsanov
2026-05-22 00:45:42 -07:00
co-authored by Claude Opus 4.7
parent 5594a8286c
commit 190f1ec71a
2 changed files with 395 additions and 0 deletions
@@ -18,10 +18,46 @@ Score each item 15. If any item scores below 3, fix it before continuing. **D
[ ] Brand assets actually visible → for each beat, name which captured SVG / illustration / screenshot is on screen and at what timestamp. If a beat shows zero captured assets, justify why.
[ ] Audio duration matches video ±0.5s → paste both numbers
[ ] animation-map.json generated → run `node <repo-root>/skills/hyperframes/scripts/animation-map.mjs <project-dir>`; confirm every beat has events listed and no bbox/flag warnings
[ ] w2h-verify report → run `node <repo-root>/skills/website-to-hyperframes/scripts/w2h-verify.mjs <project-dir>`; paste the FULL output (every row, every percent) verbatim into your final user-facing summary — see "w2h-verify — the source of truth" below
[ ] Audio + motion verification done → see "Audio + motion verification" below; played the full preview, confirmed SFX lands at storyboard timestamps
[ ] Critic sub-agent run → paste its single biggest quality gap finding, verbatim
```
### w2h-verify — the source of truth
The skill ran for months on agents reading "REQUIRED" and skipping anyway. The verify script ends that — it computes facts the agent cannot fudge:
- **Asset usage %** (assets referenced in compositions ÷ assets captured)
- **Shader transitions consistency** (shaders declared in STORYBOARD.md vs shaders present in index.html)
- **SFX timestamp drift** (storyboard `t=X.Xs` vs index.html `data-start=X.X`)
- **animation-map.json existence**
- **Rendered MP4 existence**
- **Required artifacts present** (STORYBOARD.md, DESIGN.md, SCRIPT.md, index.html)
Run it as the LAST gate in your DoD pass, after fixing everything else:
```bash
node <repo-root>/skills/website-to-hyperframes/scripts/w2h-verify.mjs <project-dir>
```
(Locate the repo root from a project subdirectory: `find / -path '*/skills/website-to-hyperframes/scripts/w2h-verify.mjs' -maxdepth 12 2>/dev/null | head -1`.)
**The script's output is the deliverable.** Paste the entire report — the table, the percentages, the FAIL lines — verbatim into your final user-facing summary, in the "What I verified" / "What I did NOT verify" section. The user will read it directly. You don't get to summarize, simplify, or omit rows.
**If any row says FAIL:**
- Either fix the underlying issue and re-run until the row says PASS
- Or include the FAIL row verbatim in your final summary's "What I did NOT verify" section with a one-sentence explanation of why you chose not to fix it
**Forbidden:**
- Hand-writing your own verification summary that doesn't match the script's output
- Cherry-picking which rows to include
- Replacing percentages with adjectives ("most assets used" instead of "8%")
- Running the script, seeing FAIL, and not mentioning it
The script's exit code is 0 (all pass) or 1 (one or more fail). If you ship with exit=1, the user knows from the report exactly what they're getting.
### Per-beat file read
This is what verification means now: you open each `compositions/beat-N.html` and read it top-to-bottom against DESIGN.md and STORYBOARD.md. Step 5 already required this once before advancing here — repeat it here as the final check, in case fixes during Step 5 introduced new problems.
@@ -0,0 +1,359 @@
#!/usr/bin/env node
// w2h-verify.mjs — verification report for a website-to-hyperframes project.
//
// Outputs facts the agent cannot fudge: asset usage %, storyboard-vs-build
// consistency, artifact existence. Result becomes the Step 6 deliverable —
// paste verbatim into the final user-facing summary.
//
// This script does PURE FILE ANALYSIS — it does not spawn lint/inspect.
// Run those separately via the CLI and include their summaries alongside.
//
// Usage:
// node skills/website-to-hyperframes/scripts/w2h-verify.mjs <project-dir>
//
// Exit codes:
// 0 = all gates pass
// 1 = one or more gates failed (final summary must disclose)
// 2 = script error (project-dir not found, etc.)
import { readFile, readdir } from "node:fs/promises";
import { join, resolve, basename } from "node:path";
import { existsSync } from "node:fs";
const PROJECT_DIR = resolve(process.argv[2] || ".");
// Thresholds — adjust here, not by interpretation.
const ASSET_USAGE_MIN_PCT = 30;
const SFX_DRIFT_TOLERANCE_S = 0.5;
const ASSET_EXTS = new Set([".jpg", ".jpeg", ".png", ".svg", ".webp", ".gif", ".mp4", ".webm"]);
const ASSET_EXCLUDE_PATTERNS = [/favicon/i, /apple-touch-icon/i];
const SHADER_NAMES = [
"cross-warp-morph",
"cross-warp",
"cinematic-zoom",
"gravitational-lens",
"glitch",
"light-leak",
"flash-through-white",
"whip-pan",
"domain-warp",
"thermal-bloom",
"swirl",
"ridged-noise",
"sdf-reveal",
"chromatic-aberration",
"ripple",
];
// ─── Main ────────────────────────────────────────────────────────────────────
async function main() {
if (!existsSync(PROJECT_DIR)) {
console.error(`✗ Project directory not found: ${PROJECT_DIR}`);
process.exit(2);
}
const results = [];
results.push(await checkAssetUsage());
results.push(await checkShaderTransitionsConsistency());
results.push(await checkSfxTimestampConsistency());
results.push(await checkAnimationMapExists());
results.push(await checkMp4Exists());
results.push(await checkStoryboardArtifactExists());
printReport(results);
const anyFail = results.some((r) => r.status === "FAIL");
process.exit(anyFail ? 1 : 0);
}
// ─── Checks ──────────────────────────────────────────────────────────────────
async function checkAssetUsage() {
const captureDir = join(PROJECT_DIR, "capture", "assets");
if (!existsSync(captureDir)) {
return {
name: "Asset usage",
status: "INFO",
detail: "No capture/assets/ directory — capture may not have run",
};
}
const captured = await listAssetFiles(captureDir);
const compositionsDir = join(PROJECT_DIR, "compositions");
const referencedSet = new Set();
const filesToScan = [join(PROJECT_DIR, "index.html")];
if (existsSync(compositionsDir)) {
const compFiles = (await readdir(compositionsDir)).filter((f) => f.endsWith(".html"));
for (const f of compFiles) filesToScan.push(join(compositionsDir, f));
}
for (const f of filesToScan) {
if (!existsSync(f)) continue;
const content = await readFile(f, "utf-8");
for (const asset of captured) {
if (content.includes(asset.path) || content.includes(basename(asset.path))) {
referencedSet.add(asset.path);
}
}
}
const total = captured.length;
const used = referencedSet.size;
const pct = total > 0 ? Math.round((used * 100) / total) : 0;
const pass = pct >= ASSET_USAGE_MIN_PCT;
return {
name: "Asset usage",
status: pass ? "PASS" : "FAIL",
detail: `${used}/${total} (${pct}%) — target ≥${ASSET_USAGE_MIN_PCT}%`,
extra: pass
? null
: `Unused brand assets are sitting in capture/assets/. The brand isn't visually present at ${pct}%.`,
};
}
async function checkShaderTransitionsConsistency() {
const storyboardPath = join(PROJECT_DIR, "STORYBOARD.md");
const indexPath = join(PROJECT_DIR, "index.html");
if (!existsSync(storyboardPath) || !existsSync(indexPath)) {
return {
name: "Shader transitions",
status: "INFO",
detail: "STORYBOARD.md or index.html missing — cannot check",
};
}
const storyboard = await readFile(storyboardPath, "utf-8");
const index = await readFile(indexPath, "utf-8");
// Match longest names first so "cross-warp-morph" doesn't double-count as "cross-warp" too.
const sortedNames = [...SHADER_NAMES].sort((a, b) => b.length - a.length);
let sbScratch = storyboard;
const declared = [];
for (const name of sortedNames) {
if (sbScratch.includes(name)) {
declared.push(name);
sbScratch = sbScratch.split(name).join(""); // strip all matches before next probe
}
}
if (declared.length === 0) {
return {
name: "Shader transitions",
status: "PASS",
detail: "STORYBOARD declared none — no shader transitions expected",
};
}
const present = declared.filter((name) => index.includes(name));
const missing = declared.filter((name) => !index.includes(name));
const pass = missing.length === 0;
return {
name: "Shader transitions",
status: pass ? "PASS" : "FAIL",
detail: `STORYBOARD declared ${declared.length}, ${present.length} present in index.html, ${missing.length} missing`,
extra: pass
? null
: `Missing from build: ${missing.join(", ")}. STORYBOARD.md and index.html disagree.`,
};
}
async function checkSfxTimestampConsistency() {
const storyboardPath = join(PROJECT_DIR, "STORYBOARD.md");
const indexPath = join(PROJECT_DIR, "index.html");
if (!existsSync(storyboardPath) || !existsSync(indexPath)) {
return { name: "SFX timestamps", status: "INFO", detail: "STORYBOARD.md or index.html missing" };
}
const storyboard = await readFile(storyboardPath, "utf-8");
const index = await readFile(indexPath, "utf-8");
const sfxRefs = [];
for (const line of storyboard.split("\n")) {
const fileMatch = line.match(/sfx\/([\w-]+\.mp3)/);
const timeMatch = line.match(/\|\s*(\d+(?:\.\d+)?)s?\s*\|/);
if (fileMatch && timeMatch) {
sfxRefs.push({ file: fileMatch[1], storyboardT: parseFloat(timeMatch[1]) });
}
}
if (sfxRefs.length === 0) {
return { name: "SFX timestamps", status: "INFO", detail: "No SFX entries detected in STORYBOARD.md" };
}
const indexSfx = new Map();
const audioRegex =
/<audio[^>]*src=["'](?:[^"']*\/)?sfx\/([\w-]+\.mp3)["'][^>]*?data-start=["']([0-9.]+)["']/g;
let m;
while ((m = audioRegex.exec(index)) !== null) {
indexSfx.set(m[1], parseFloat(m[2]));
}
const drifts = [];
const missing = [];
for (const ref of sfxRefs) {
if (!indexSfx.has(ref.file)) {
missing.push(ref.file);
continue;
}
const indexT = indexSfx.get(ref.file);
const drift = Math.abs(indexT - ref.storyboardT);
if (drift > SFX_DRIFT_TOLERANCE_S) {
drifts.push({ file: ref.file, storyboardT: ref.storyboardT, indexT, drift });
}
}
const pass = missing.length === 0 && drifts.length === 0;
return {
name: "SFX timestamps",
status: pass ? "PASS" : "FAIL",
detail: `${sfxRefs.length} SFX in STORYBOARD · ${indexSfx.size} in index.html · ${missing.length} missing · ${drifts.length} drifted >${SFX_DRIFT_TOLERANCE_S}s`,
extra: pass
? null
: [
...missing.map((f) => `MISSING in index.html: ${f}`),
...drifts.map(
(d) =>
`DRIFT: ${d.file} storyboard=${d.storyboardT}s index=${d.indexT}s drift=${d.drift.toFixed(2)}s`,
),
].join("\n "),
};
}
async function checkAnimationMapExists() {
const path = join(PROJECT_DIR, "animation-map.json");
if (existsSync(path)) {
return { name: "animation-map.json", status: "PASS", detail: "exists" };
}
return {
name: "animation-map.json",
status: "FAIL",
detail: "missing — run `node <repo>/skills/hyperframes/scripts/animation-map.mjs <project-dir>`",
};
}
async function checkMp4Exists() {
const candidates = [PROJECT_DIR, join(PROJECT_DIR, "output"), join(PROJECT_DIR, "renders")];
for (const dir of candidates) {
if (!existsSync(dir)) continue;
try {
const files = await readdir(dir);
if (files.some((f) => f.endsWith(".mp4"))) {
return {
name: "Rendered MP4",
status: "PASS",
detail: `found .mp4 in ${dir.replace(PROJECT_DIR, ".")}`,
};
}
} catch {
/* ignore */
}
}
return {
name: "Rendered MP4",
status: "INFO",
detail:
"no .mp4 found — preview-only delivery; if claiming verified motion, render is required (Path 2 of audio+motion verification)",
};
}
async function checkStoryboardArtifactExists() {
const required = ["STORYBOARD.md", "DESIGN.md", "SCRIPT.md", "index.html"];
const missing = required.filter((f) => !existsSync(join(PROJECT_DIR, f)));
if (missing.length === 0) {
return { name: "Required artifacts", status: "PASS", detail: required.join(", ") };
}
return {
name: "Required artifacts",
status: "FAIL",
detail: `missing: ${missing.join(", ")}`,
};
}
// ─── Helpers ─────────────────────────────────────────────────────────────────
async function listAssetFiles(dir) {
const out = [];
async function walk(d) {
let entries;
try {
entries = await readdir(d, { withFileTypes: true });
} catch {
return;
}
for (const e of entries) {
const full = join(d, e.name);
if (e.isDirectory()) {
await walk(full);
continue;
}
const ext = e.name.slice(e.name.lastIndexOf(".")).toLowerCase();
if (!ASSET_EXTS.has(ext)) continue;
if (ASSET_EXCLUDE_PATTERNS.some((re) => re.test(e.name))) continue;
const rel = full.replace(PROJECT_DIR + "/", "");
out.push({ path: rel, name: e.name });
}
}
await walk(dir);
return out;
}
// ─── Report ──────────────────────────────────────────────────────────────────
function printReport(results) {
const cols = { name: 26, status: 8, detail: 60 };
const line = "─".repeat(cols.name + cols.status + cols.detail + 6);
console.log("");
console.log(`w2h-verify · ${PROJECT_DIR}`);
console.log(line);
console.log(
"Check".padEnd(cols.name) + " │ " + "Status".padEnd(cols.status) + " │ " + "Detail",
);
console.log(line);
for (const r of results) {
const symbol = r.status === "PASS" ? "✓" : r.status === "FAIL" ? "✗" : "·";
console.log(
r.name.padEnd(cols.name) +
" │ " +
`${symbol} ${r.status}`.padEnd(cols.status) +
" │ " +
(r.detail || ""),
);
if (r.extra) {
console.log(
"".padEnd(cols.name) +
" │ " +
"".padEnd(cols.status) +
" │ " +
r.extra.split("\n").join("\n" + " ".repeat(cols.name + cols.status + 6 + 4)),
);
}
}
console.log(line);
const pass = results.filter((r) => r.status === "PASS").length;
const fail = results.filter((r) => r.status === "FAIL").length;
const info = results.filter((r) => r.status === "INFO").length;
console.log(`SUMMARY: ${pass} PASS · ${fail} FAIL · ${info} INFO`);
if (fail > 0) {
console.log("");
console.log("Step 6 NOT done. Fix FAIL items, OR include this report verbatim in your final");
console.log("summary's \"What I did NOT verify\" section so the user knows what's broken.");
} else {
console.log("");
console.log("All gates pass. Paste this report into your final user-facing summary as evidence.");
}
console.log("");
}
main().catch((e) => {
console.error("w2h-verify script error:", e);
process.exit(2);
});