mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-11 14:50:02 +00:00
feat(skill): w2h-verify script + Step 6 DoD wiring — tooling-based enforcement
Three rounds of text-based enforcement plateaued. A third agent debrief showed the same failures: 9% asset usage (vs ≥30% floor), shader transitions declared in STORYBOARD.md but not in shipping index.html, SFX timestamps drifted up to 12.4s, animation-map skipped, MP4 not rendered, honest-disclosure section omitted from final summary. The pattern is clear: language-only enforcement is selectively interpretable by the agent under ship pressure. Move enforcement into tooling — facts the agent can't fudge. **New script: `skills/website-to-hyperframes/scripts/w2h-verify.mjs`** Pure file-analysis script (no shell spawns). Computes six checks and prints a PASS/FAIL/INFO table: 1. Asset usage — assets referenced in compositions ÷ assets captured; target ≥30%. Tested against videos/huly-v3: caught 6/74 = 8% FAIL. 2. Shader transitions consistency — STORYBOARD.md-declared shaders vs index.html. Longest-name matching to avoid substring false positives (cross-warp-morph not double-counted as cross-warp). Tested: caught 6 declared / 1 present / 5 missing. 3. SFX timestamp drift — parses STORYBOARD.md table rows for `sfx/X.mp3` + time-with-`s`, parses index.html <audio data-start>, flags drift >0.5s. Tested: caught 12.4s drift on click.mp3 that the agent debrief didn't even mention. 4. animation-map.json existence — explicit file check. 5. Rendered MP4 existence — scans project root, output/, renders/. 6. Required artifacts — STORYBOARD.md, DESIGN.md, SCRIPT.md, index.html all present. Exit code: 0 (all pass) or 1 (one or more fail). The script's output becomes the Step 6 deliverable — paste verbatim into the user-facing summary. **Skill update: `step-6-validate.md`** Adds `w2h-verify report` to the DoD checklist with the rule: paste the FULL output verbatim into the final summary. Cherry-picking rows, substituting adjectives for percentages, or omitting FAIL lines is explicitly forbidden. If a row says FAIL, either fix it and re-run until PASS or include the FAIL line verbatim in "What I did NOT verify" with a one-sentence reason. Test run against the project that prompted this: ``` SUMMARY: 1 PASS · 4 FAIL · 1 INFO - Asset usage: FAIL 6/74 (8%) target ≥30% - Shader transitions: FAIL 6 declared, 1 present, 5 missing - SFX timestamps: FAIL 3 drifted >0.5s (max 12.4s) - animation-map.json: FAIL missing - Rendered MP4: INFO no .mp4 found - Required artifacts: PASS ``` The agent could selectively ignore "the WCAG warnings are false positives." The agent cannot selectively ignore a line that says `6/74 (8%) — target ≥30%`. 2 files changed (+ 1 new script, ~330 lines). Format checks pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
5594a8286c
commit
190f1ec71a
@@ -18,10 +18,46 @@ Score each item 1–5. If any item scores below 3, fix it before continuing. **D
|
||||
[ ] Brand assets actually visible → for each beat, name which captured SVG / illustration / screenshot is on screen and at what timestamp. If a beat shows zero captured assets, justify why.
|
||||
[ ] Audio duration matches video ±0.5s → paste both numbers
|
||||
[ ] animation-map.json generated → run `node <repo-root>/skills/hyperframes/scripts/animation-map.mjs <project-dir>`; confirm every beat has events listed and no bbox/flag warnings
|
||||
[ ] w2h-verify report → run `node <repo-root>/skills/website-to-hyperframes/scripts/w2h-verify.mjs <project-dir>`; paste the FULL output (every row, every percent) verbatim into your final user-facing summary — see "w2h-verify — the source of truth" below
|
||||
[ ] Audio + motion verification done → see "Audio + motion verification" below; played the full preview, confirmed SFX lands at storyboard timestamps
|
||||
[ ] Critic sub-agent run → paste its single biggest quality gap finding, verbatim
|
||||
```
|
||||
|
||||
### w2h-verify — the source of truth
|
||||
|
||||
The skill ran for months on agents reading "REQUIRED" and skipping anyway. The verify script ends that — it computes facts the agent cannot fudge:
|
||||
|
||||
- **Asset usage %** (assets referenced in compositions ÷ assets captured)
|
||||
- **Shader transitions consistency** (shaders declared in STORYBOARD.md vs shaders present in index.html)
|
||||
- **SFX timestamp drift** (storyboard `t=X.Xs` vs index.html `data-start=X.X`)
|
||||
- **animation-map.json existence**
|
||||
- **Rendered MP4 existence**
|
||||
- **Required artifacts present** (STORYBOARD.md, DESIGN.md, SCRIPT.md, index.html)
|
||||
|
||||
Run it as the LAST gate in your DoD pass, after fixing everything else:
|
||||
|
||||
```bash
|
||||
node <repo-root>/skills/website-to-hyperframes/scripts/w2h-verify.mjs <project-dir>
|
||||
```
|
||||
|
||||
(Locate the repo root from a project subdirectory: `find / -path '*/skills/website-to-hyperframes/scripts/w2h-verify.mjs' -maxdepth 12 2>/dev/null | head -1`.)
|
||||
|
||||
**The script's output is the deliverable.** Paste the entire report — the table, the percentages, the FAIL lines — verbatim into your final user-facing summary, in the "What I verified" / "What I did NOT verify" section. The user will read it directly. You don't get to summarize, simplify, or omit rows.
|
||||
|
||||
**If any row says FAIL:**
|
||||
|
||||
- Either fix the underlying issue and re-run until the row says PASS
|
||||
- Or include the FAIL row verbatim in your final summary's "What I did NOT verify" section with a one-sentence explanation of why you chose not to fix it
|
||||
|
||||
**Forbidden:**
|
||||
|
||||
- Hand-writing your own verification summary that doesn't match the script's output
|
||||
- Cherry-picking which rows to include
|
||||
- Replacing percentages with adjectives ("most assets used" instead of "8%")
|
||||
- Running the script, seeing FAIL, and not mentioning it
|
||||
|
||||
The script's exit code is 0 (all pass) or 1 (one or more fail). If you ship with exit=1, the user knows from the report exactly what they're getting.
|
||||
|
||||
### Per-beat file read
|
||||
|
||||
This is what verification means now: you open each `compositions/beat-N.html` and read it top-to-bottom against DESIGN.md and STORYBOARD.md. Step 5 already required this once before advancing here — repeat it here as the final check, in case fixes during Step 5 introduced new problems.
|
||||
|
||||
Reference in New Issue
Block a user