ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-02 20:18:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(skill): w2h enforcement edits — close the 9 shirking patterns from agent debrief

Browse Source 
After the audit-fix commit (e47bc6c6) landed clarity fixes, a real test run
revealed that the skill still got skipped at the gates that matter most.
The agent's honest debrief listed 9 distinct patterns where it judged
"deliver fast" over "verify what the skill said to verify."

This commit forces evidence at each of those gates so silent skips are no
longer possible without lying — at which point the gate fails by design.

**Patterns 1, 3, 8 — step-5-build.md**

- Pattern 1: "Read each beat HTML top-to-bottom" gate now requires a
  structured evidence block per beat with quoted CSS hex codes, headline
  font-size, captured asset paths, GSAP first/last events, and SFX trigger.
  "I read it and it looks fine" / "the sub-agent confirmed" are not
  acceptable. Snapshots are 3 frames out of 300+ in motion.
- Pattern 3: SFX timestamp computation rule. Every data-start MUST be
  computed (beat-local + beat global start = global timestamp), not
  estimated by eye. The agent typed `data-start="6.0"` for a storyboard
  moment at 5.0s — a 1-second drift, not a rounding error.
- Pattern 8: Recurring sub-agent workarounds must be surfaced under
  "Tooling issues encountered" — burying them means the next session
  hits the same bug.

**Patterns 2, 6, 7, 9 — step-6-validate.md**

- Pattern 2: WCAG contrast warnings now require per-warning verification
  with quoted validator output and opacity check at the sampled timestamp.
  Blanket dismissal as "mostly transition-window false positives" is
  explicitly forbidden.
- Pattern 6: animation-map.json check added to the DoD checklist —
  runs `skills/hyperframes/scripts/animation-map.mjs` and confirms
  per-beat event coverage.
- Pattern 7: Audio + motion verification is now a separate DoD item from
  snapshot verification. Snapshots are silent stills; you must actually
  play the preview and confirm SFX lands at storyboard timestamps within
  ±0.1s. CLI-only sessions must explicitly disclose this as deferred.
- Pattern 9: Honest disclosure section added — final user-facing summary
  MUST end with "What I verified" and "What I did NOT verify" blocks.
  "Looks great, ready to ship" with no disclosure now fails the gate.

**Patterns 4, 5 — beat-builder-guide.md**

- Pattern 4: Sub-agent FLAG protocol. Required phrasing for non-blocking
  issues is concrete and actionable with line numbers. Forbidden phrasing:
  "if X feels too long, you could...", "consider tweaking...", "might
  want to...". Main agent must address each FLAG or write a rejection.
- Pattern 5: Spec ambiguity escalation. If the storyboard names a
  transition without establishing the start state ("Row 1 transitions
  blue → orange" but Row 1's initial color isn't specified), sub-agent
  MUST flag it and ask for confirmation rather than guess. Picking an
  interpretation silently means the build "looks fine" while diverging
  from intent.

3 files changed, +170/-21 (net +149 lines of enforcement language).
Format checks pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
ukimsanov
2026-05-21 20:12:46 -07:00
co-authored by Claude Opus 4.7
parent e47bc6c6db
commit 18444decde
3 changed files with 169 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/website-to-hyperframes/references/beat-builder-guide.md
+41
View File
@@ -115,6 +115,47 @@ After lint passes, snapshots are taken, and you've fixed every issue you saw —
So in your report, name the hex codes you used, the captured asset paths you placed, the headline `font-size`, and the GSAP timeline's last `tl.fromTo(...)` timestamp. Brief, concrete, true. If anything diverges from DESIGN.md or the storyboard, say so explicitly — the main agent can decide whether to accept the divergence or send you back to fix it. Surprises caught at this hand-off cost minutes; surprises caught at Step 6 cost iterations.
### FLAG protocol — required phrasing for non-blocking issues
When you find any of these, surface them as **FLAGS** in your report, not as conditional suggestions:
- Visual states that briefly look broken (empty containers, hanging elements, gap moments)
- Spec ambiguities you had to resolve by guessing
- Linter bugs you worked around
- Tween values you changed from the spec because they wouldn't fit
**Forbidden phrasing:** "if the X feels too long, you could...", "consider tweaking Y", "might want to..."
**Required phrasing — concrete, actionable, with line numbers:**
```
FLAG: at beat-local t=1.2s the doc card is visible but its inner content is still
opacity 0 — a 0.4s empty-panel window.
RECOMMENDED FIX: pull title typewriter from 1.6s → 1.4s
in compositions/beat-5-name.html line 234.
```
The main agent MUST EITHER apply each FLAG's fix OR write a one-sentence rejection with reason. Silently dropping a FLAG is a verification failure that gets caught at Step 6 (or worse, in the user's preview).
### Spec ambiguity — escalate, don't paper over
If STORYBOARD.md gives you a transition or transformation but doesn't establish the **start** state, do NOT guess. Examples of ambiguity worth flagging:
- "Row 1 transitions from Huly Blue to Huly Orange at 3.5s" — but Row 1's initial color isn't specified
- "Headline grows" — but the start size isn't specified
- "Cards slide in" — but the off-screen position isn't specified
- "Subhead appears after the headline" — but exact timing offset isn't specified
**Required action:** FLAG the ambiguity in your report verbatim:
```
FLAG: STORYBOARD.md beat 3 says "Row 1 transitions blue → orange at 3.5s" but
Row 1's initial color is not specified anywhere. I interpreted Row 1 starts
blue and tweened to orange. CONFIRM or correct.
```
The main agent then confirms or corrects before Step 6 advances. Picking an interpretation silently means the build looks "fine" while diverging from intent — and the user only notices in motion.
---
## Continuous motion — the most important rule
skills/website-to-hyperframes/references/step-5-build.md
+58 -18
View File
@@ -391,28 +391,68 @@ For every `.html` file in `compositions/`, confirm that `index.html` has a `data
## 5. Read each beat HTML top-to-bottom — REQUIRED gate before Step 6
**Do not declare Step 5 complete on sub-agents' word.** Earlier sessions had sub-agents reply "looks good, 0 errors" and the main agent trusted them — that's how videos shipped with mismatched colors, missing logos, headlines too small to read. Close the trust path by opening every file the sub-agent produced.
**This gate is non-skippable.** "I read it and it looks fine", "the sub-agent confirmed", "the snapshots look right" are NOT acceptable. Snapshots are 3 frames out of 300+ in motion — they hide everything that goes wrong between them.
**Why this gate exists:** Earlier sessions had sub-agents reply "looks good, 0 errors" and the main agent trusted them — that's how videos shipped with mismatched colors, missing logos, headlines too small to read, and SFX firing 1 second late because the agent typed timestamps "by eye" instead of computing them.
For each `compositions/beat-N.html`:
1. **Open the file and read it top-to-bottom.** Not a glance. Not a grep. Read the `<style>` block, then the markup, then the `<script>` block. Understand what's actually there.
2. **Cross-check against DESIGN.md:**
- Does the `--bg` / primary background hex from DESIGN.md appear in the CSS or inline styles?
- Does the accent hex appear (if this beat uses an accent)?
- Are fonts the ones DESIGN.md specified? If `@font-face` is declared, does the path match a real file under `capture/assets/fonts/` or a published `@fontsource/*` import?
- Is the headline `font-size` ≥80px?
3. **Cross-check against STORYBOARD.md (this beat's section):**
- Are the captured assets the storyboard called for actually referenced in the HTML (`<img src=...>`, inline SVG, `background-image: url(...)`, etc.)? Open the asset paths and confirm the files exist.
- Does the GSAP timeline cover the full beat duration, not just the first 1-2 seconds of entrance tweens? Look for events spread across the `BEAT` constant.
- Does the shot framing/camera move described in the storyboard show up in the GSAP code (scale/x/y/yPercent transforms with meaningful magnitudes)?
4. **Check the technical gates inline:**
- `data-composition-id` on the root div matches the `window.__timelines["..."]` key in the script
- `data-width` and `data-height` match the host div in index.html
- The script is INSIDE the `<template>`, not after `</template>`
- No bare `gsap.to(...)`, no `Math.random()`, no `repeat: -1`
5. **Open each frame in `snapshots/beat-N/`** and confirm visually that the entrance, hold, and exit moments look like what the storyboard described. If `snapshots/descriptions.md` exists, read Gemini's per-frame analysis of this beat in particular.
1. **Open the file and read it top-to-bottom.** Not a glance. Not a grep. Read the `<style>` block, then the markup, then the `<script>` block.
2. **Fill in this evidence block — every line, with quoted values from the file:**
**Anything off — fix it inline (small CSS / GSAP correction) or re-dispatch the sub-agent with the specific problem quoted.** Do not move to Step 6 until every beat has been read top-to-bottom and the cross-checks pass.
```
Beat N: compositions/beat-N-NAME.html
BG color in CSS: <hex from line Y> ← quote the exact line
Accent color in CSS: <hex from line Z> ← quote the exact line
Headline font-size: <px from line> (≥80? yes/no)
Headline font-family: <stack from line> (matches DESIGN.md? yes/no)
@font-face src paths: <list> (each path exists? yes/no)
Captured assets used: <full list of paths from <img src=>, inline SVG ids, background-image url()>
Storyboard called for: <list from STORYBOARD.md beat N>
Assets match storyboard? yes/no — if no, specify the gap
GSAP first event: tl.X("...", {...}, <t>) beat-local t=<num>
GSAP last event: tl.X("...", {...}, <t>) beat-local t=<num>
Beat duration: <N>s (events span full duration? yes/no)
SFX trigger: <element> data-start=<num>
Storyboard SFX line: "<quote the line>" → expected t=<num>
SFX timestamp matches? yes/no — if no, specify the drift
Technical gates: data-composition-id matches window.__timelines key? yes/no
script INSIDE <template>? yes/no
no Math.random / no repeat:-1 / no bare gsap.to? yes/no
VERDICT: PASS / FIX (specify exactly what)
```
If you cannot fill any line (e.g., "I see no SFX trigger" or "headline font-family not specified"), that IS a finding — fix or escalate, don't paper over.
3. **Open each frame in `snapshots/beat-N/`** and confirm visually that entrance/hold/exit match the storyboard.
**Anything off — fix it inline (small CSS/GSAP correction) or re-dispatch the sub-agent with the specific problem quoted.** Do not move to Step 6 until every beat has its evidence block filled and PASS.
### SFX timestamp computation — compute, don't eyeball
Every SFX `data-start` value MUST be computed from STORYBOARD.md, not estimated visually.
For each SFX entry:
1. Storyboard names beat-local time (e.g. "Beat 2 at 1.2s into the beat").
2. Get the beat's global start time from beat ordering (e.g. Beat 1: 0–3.5s → Beat 2 starts at 3.5s globally).
3. Add beat-local + global start: `3.5 + 1.2 = 4.7s`.
4. Write `data-start="4.7"` in index.html.
**Forbidden:** writing `data-start="<approximate visual moment>"` by reading the storyboard and estimating by eye. The evidence block above MUST quote both the storyboard SFX line and the index.html `data-start` line — and confirm they match within ±0.05s. A 1-second drift is not a rounding error; it's a build failure.
### Surface recurring sub-agent workarounds to the user
When 2+ sub-agents independently report the same workaround (e.g., "I had to base64-encode the data URI because the linter false-positives on inline SVG"), that's a tooling bug worth surfacing. List these in your Step 5 final report under "Tooling issues encountered" even if each instance was resolved. Format:
```
TOOLING ISSUES (worth filing):
- 3 sub-agents (beats 1, 4, 6) hit `root_missing_composition_id` false positive on
inline SVG data URI in CSS. Workarounds: base64 (beat 1), removed overlay (beats 4, 6).
Worth filing as a regression against packages/core/src/lint.
```
Burying recurring workarounds means the next session hits the same bug and works around it again. Don't.
### Brand-defaults check (whole-video, after every beat passes its own read)
skills/website-to-hyperframes/references/step-6-validate.md
+70 -2
View File
@@ -17,6 +17,8 @@ Score each item 1–5. If any item scores below 3, fix it before continuing. **D
[ ] No mid-video dark frames → state explicitly which frames (if any) are dark and why
[ ] Brand assets actually visible → for each beat, name which captured SVG / illustration / screenshot is on screen and at what timestamp. If a beat shows zero captured assets, justify why.
[ ] Audio duration matches video ±0.5s → paste both numbers
[ ] animation-map.json generated → run `node <repo-root>/skills/hyperframes/scripts/animation-map.mjs <project-dir>`; confirm every beat has events listed and no bbox/flag warnings
[ ] Audio + motion verification done → see "Audio + motion verification" below; played the full preview, confirmed SFX lands at storyboard timestamps
[ ] Critic sub-agent run → paste its single biggest quality gap finding, verbatim
```
@@ -62,9 +64,24 @@ Some are style suggestions you can safely ignore:
- **Deprecated attributes** (data-layer, data-end) — still work, just not preferred
- **Dense tracks** — informational, not a bug
**WCAG contrast false positives** — the validator samples text colors at fixed timestamps. Elements that are at `opacity: 0` (pre-entrance) or mid-fade at those sample timestamps get measured against the background as if they were fully visible, which produces spurious contrast failures. Before changing a color to clear a WCAG warning, verify visually that the element is actually unreadable when on-screen at full opacity. If it's only flagged for pre-entrance / exit moments, the warning is a sampling artifact, not a real failure. Bumping the color to "fix" these false positives changes the brand identity for no real benefit.
**WCAG contrast warnings — per-warning verification, not blanket dismissal.**
Don't blindly ignore 158 warnings. Don't blindly fix all of them either. Read them.
The validator samples text colors at fixed timestamps. Elements at `opacity: 0` (pre-entrance) or mid-fade get measured as if fully visible — real false positives exist. BUT this is a per-warning judgment, not a blanket excuse.
**For EACH warning the validator emits, paste this block in your verdict:**
```
Warning N: <quote the validator output verbatim>
Element: <selector>
Sampled timestamp: t=<n>
At t=<n>, is this element on-screen at full opacity? (yes/no — confirm by viewing snapshot at that timestamp)
Verdict: REAL ISSUE / SAMPLING ARTIFACT (justify in one sentence)
Action: <hex change at line N> OR NONE because <reason>
```
**Forbidden:** writing "the N warnings are mostly transition-window false positives" without per-warning evidence. That phrasing alone fails the gate. The validator does not report 158 warnings as a group — it reports them individually, and you verify them individually.
Don't blindly ignore. Don't blindly fix. Verify each.
## Visual Verification (snapshot)
@@ -156,6 +173,28 @@ If you cannot find any problems and want to score everything 4–5, you are not
Read every score. Fix anything below 3 before showing the user. If the CTA scores below 3, fix the CTA. Do not rationalize low scores as "the user can decide."
## Audio + motion verification — separate from snapshot verification
Snapshots are silent stills. They prove what FRAMES look like, but they do NOT prove:
- That audio fires at all
- That SFX lands exactly when the visual moment lands (target: ±0.1s, hard limit: ±0.5s)
- That narration syncs to per-beat content
- That transitions feel right in motion
After snapshot DoD items pass, **play the preview Studio URL in a browser** (or via Playwright if available). Don't scrub — actually play it from start to end at 1.0× speed. Then verify:
```
[ ] Played full video front-to-back at 1.0× — actually played, not scrubbed
[ ] For each SFX in STORYBOARD.md: sound lands at the visual moment within ±0.1s
(Beat N SFX `<file>`: storyboard says t=<x>s → heard at t=<y>s → drift <z>s)
[ ] Narration delivers the right line per beat (no off-by-one or missing lines)
[ ] No moments where audio is present but visual is unintentionally mid-transition
[ ] Audio audible and not clipped/peaked
```
**If you cannot play the preview** (no Playwright in this session, CLI-only environment), state this explicitly: "Audio + motion verification deferred to user — no preview tool available in this session." That is honest disclosure. **Silently skipping this check while presenting a "looks good" verdict fails the gate.**
## Preview (always do this)
Always start the preview so the user can see and scrub through the project:
@@ -172,6 +211,35 @@ http://localhost:<port>/#project/<project-name>
Use the actual port and project name from the preview command output. Do NOT present `index.html` as the project link — that's the source file. The user-facing project is the running Studio preview.
### Honest disclosure — REQUIRED in your final summary
Your final message to the user MUST end with these two sections, even if everything passed. Both sections appear AFTER the preview URL, BEFORE you stop talking.
```
**What I verified:**
- <one bullet per DoD item that passed, with the actual evidence cited inline>
(e.g. "Lint: zero errors — output pasted above")
(e.g. "Per-beat read: 7/7 beats PASS, evidence blocks above")
(e.g. "WCAG: 3 warnings flagged, all 3 verified as sampling artifacts — see verdicts above")
**What I did NOT verify (spot-check these):**
- <one bullet per item you skipped, deferred, or could not complete — and why>
(e.g. "Audio + motion verification deferred — no Playwright in this session. SFX timing is computed but unconfirmed in playback.")
(e.g. "animation-map.json skipped — script not found at expected path; manually confirmed timeline coverage in per-beat reads instead.")
(e.g. "Beat 5 has a 0.4s window where the doc card is visible but contents are still opacity 0 — sub-agent flagged it, I chose not to fix because it was below my threshold; worth your eye.")
```
The user reads this section to know what to spot-check.
**UNACCEPTABLE final summaries:**
- "Looks great, ready to ship" (no disclosure)
- "All checks pass" (when one was actually skipped)
- "Sub-agents confirmed everything" (delegating trust without verifying)
- Omitting the "What I did NOT verify" section because you happened to verify everything (still include it — write "None" if true, but the section header must appear).
Lying or omitting here is worse than skipping a check honestly. A short user spot-check beats a hidden broken video every time.
## Render (on-demand only)
**Do NOT render automatically.** Preview is the delivery — the user scrubs, spots tweaks, and you iterate. Rendering takes minutes per pass and is wasted if the user wants changes.