fix(cli): sample real pixels behind hidden text for contrast-audit

## What

Fixes five reported false-positive/false-negative patterns in the WCAG contrast audit (`hyperframes validate --contrast`):

1. **SVG fill vs. text color** — foreground read from CSS `color` instead of SVG `fill`.
2. **Cross-component color bleed** — background estimate bleeds into a neighboring panel/layer.
3. **Backdrop-filter glass text** — background estimate misses the blur/tint and reads the raw backdrop.
4. **Partially-overlapping translucent decoration** — a decorative shape inside or partly touching the text's bbox goes undetected.
5. **Solid-fill pill/button** — investigated, did **not** reproduce; already handled correctly by the existing own-background ancestor walk. Not touched.

## Why

The audit estimated an element's background two ways:
- foreground: always `getComputedStyle(el).color` — wrong for SVG `<text>`/`<tspan>`, which is painted via `fill`, an independent CSS property.
- background: a 4px pixel ring sampled just **outside** the text's bounding box, with a fallback to an ancestor's opaque `background-color` for solid pills/buttons.

The ring is a proximity heuristic. It's wrong whenever what's immediately outside the text differs from what's actually behind it:
- text near the edge of its own panel, with a differently-colored sibling panel/layer just past the bbox — the ring samples the neighbor.
- a `backdrop-filter: blur()` glass panel sized only a couple pixels larger than the text — the ring exits the panel into the raw, unblurred, untinted backdrop.
- a translucent decoration that only partially overlaps the ring, or sits entirely **inside** the bbox — invisible to the ring regardless of size.

## How

**SVG fill (#1):** elements inside an `<svg>` (`el.ownerSVGElement`) now prefer the computed `fill` when it resolves to a solid `rgb()`/`rgba()` color, falling back to `color` for paint values that aren't a plain color (`none`, `context-fill`, gradient/pattern refs).

**Cross-comp bleed / glass blur / partial decoration (#2–#4):** replaced the ring-sampling + own-background-ancestor-walk heuristic with a two-phase capture:
1. `__contrastAuditPrepare()` walks the DOM, computes each candidate's foreground (unchanged logic from #1), and **hides that element's own text paint** (`color`/`fill` → `transparent`, layout-neutral — no reflow).
2. The caller takes **one** screenshot with the glyphs invisible (same number of screenshots as before — just moved after the hide instead of before it).
3. `__contrastAuditFinish(imgBase64, time, candidates)` restores the original paint immediately, then samples the **real composited pixels directly inside each element's own bbox** — no proximity heuristic needed, since these are the exact pixels that were behind the glyphs.

This is a real architectural change to `contrast-audit.browser.js`'s calling contract (single `__contrastAudit` → `__contrastAuditPrepare`/`__contrastAuditFinish`), with `validate.ts`'s `runContrastAudit` updated to match, including a try/finally restore-safety-net so a mid-loop screenshot/decode failure can't leave a later sample auditing a page with stale hidden text.

Mirrored the identical change in `skills/hyperframes-creative/scripts/contrast-report.mjs`, which duplicates the same DOM-walk/sampling logic (not just the WCAG math). There, the **visible** frame for the human-facing overlay image still comes from the producer's normal `captureFrameToBuffer` path (unchanged); only the **background-sampling** capture is a plain `session.page.screenshot()` taken after hiding text — deliberately bypassing `captureFrameToBuffer`, whose static-frame dedup cache knows nothing about the DOM mutation and would hand back a stale pre-mutation buffer.

**Solid-fill pill (#5):** reproduced a rounded pill/button with a busy page background outside it. The existing own-background ancestor walk already resolves the pill's declared `background-color` correctly regardless of the rounded corners — confirmed via repro, both before and after this change report the identical (correct) result. No fix needed; left untouched, and this case is covered by the new architecture too (would give the same right answer even without the ancestor-walk fallback).

Added `packages/cli/src/commands/contrast-sample.ts` (mirroring the existing `contrast-bg.ts`/`contrast-fg.ts` pattern) hosting the pure sample-rect/grid-point computation, unit tested — the browser-injected scripts can't import it directly, so it's kept in sync by hand, same convention as the rest of this file.

## Test plan

- [x] Unit tests: `contrast-fg.test.ts` (SVG fill resolution), `contrast-sample.test.ts` (sample-rect clamping/degenerate cases), plus the full `packages/cli` suite (1424 tests) passes, including an updated `layout-audit.browser.test.ts` case that called the old single-function `__contrastAudit` API directly.
- [x] Manual verification — standalone `puppeteer-core` harness against real `chrome-headless-shell`, one minimal HTML fixture per pattern, comparing the audit's reported ratio/verdict against a hand-constructed ground truth:
  - **SVG fill**: `fill:white` / no `color` on black bg → before: `fg=rgb(0,0,0)` ratio `1:1` (false FAIL); after: `fg=rgb(255,255,255)` ratio `21:1` (correct PASS).
  - **Cross-comp bleed**: text on a black sibling highlight box 2px larger than the text, white page bg outside it → before: `bg=rgb(255,255,255)` ratio `1.23:1` (false FAIL); after: `bg=rgb(0,0,0)` ratio `17.14:1` (correct PASS).
  - **Glass blur**: black text on an 18%-white-tinted `backdrop-filter: blur(14px)` panel over a yellow/blue gradient, panel only ~2px larger than the text → before: `bg=rgb(0,64,255)` (raw gradient color, blur/tint completely missed) ratio `3.18:1` (false FAIL); after: `bg=rgb(159,160,165)` (correct blurred/tinted blend) ratio `8.05:1` (correct PASS).
  - **Partial decoration**: text 92%-covered by a translucent white badge on a dark bg → before: `bg=rgb(16,16,16)` (ring never touches the badge, which sits entirely inside the bbox) ratio `17.45:1` (false PASS); after: `bg=rgb(171,171,171)` (correctly detects the badge) ratio `2.11:1` (correct FAIL).
  - **Solid pill sanity**: unaffected — `bg=rgb(10,10,10)` ratio `19.8:1` before and after.
- [x] End-to-end: ran the actual `hyperframes validate --contrast` CLI command (via `tsx src/cli.ts`) against a real scaffolded project containing all 4 patterns simultaneously — only the genuinely-failing case (the 92%-covered decoration) is reported (`1.09:1`, need `3:1`); the cross-comp-bleed, glass-blur, and solid-pill cases are correctly silent. A second vanilla scaffold with plain white-on-dark text produces zero false positives.
- [x] `oxlint`, `oxfmt --check`, and `tsc --noEmit` all pass on the changed files.
This commit is contained in:
Miguel Ángel
2026-07-09 18:01:48 -04:00
committed by GitHub
parent dff27cb1df
commit 1614dd3e5a
10 changed files with 887 additions and 184 deletions
@@ -2,25 +2,59 @@
// Loaded as a raw string and injected via page.addScriptTag to avoid
// esbuild mangling (page.evaluate serializes functions; __name helpers break).
//
// NOTE: WCAG math (relLum, wcagRatio, parseColor, median) is duplicated in
// skills/hyperframes/scripts/contrast-report.mjs — keep in sync.
// Two-phase API — see packages/cli/src/commands/validate.ts's
// runContrastAudit() for the calling contract:
//
// 1. window.__contrastAuditPrepare() walks the DOM for text-bearing
// elements, computes each one's foreground paint (CSS `color`, or SVG
// `fill` for SVG text — fill and color are independent CSS properties
// in SVG), and HIDES that element's own text paint (color/fill set to
// transparent). Hiding is layout-neutral — it doesn't reflow anything —
// so the caller can screenshot the frame right after with the glyphs
// invisible but everything else unchanged. Returns the candidate list
// (selector, text, fg, bbox, font metrics).
// 2. Caller takes ONE screenshot (page.screenshot()) — same number of
// screenshots as before, just moved to after prepare() instead of
// before it.
// 3. window.__contrastAuditFinish(imgBase64, time, candidates) restores
// the original paint FIRST (so a slow/failed decode can't leave the
// page's text stuck invisible), then decodes the screenshot and, for
// each candidate, samples the real composited pixels directly INSIDE
// its own bounding box for the true background.
//
// Why sample inside the box instead of the 4px ring just outside it (the
// previous approach): the ring is a proximity heuristic that's wrong
// whenever what's immediately outside the text differs from what's actually
// behind it — a neighboring panel/component just past the text's edge, a
// rounded pill/button whose corner falls inside the ring, a
// backdrop-filter-blurred glass panel sized only a couple pixels larger
// than the text, or a translucent decoration that only partially overlaps
// the ring (or sits entirely inside the bbox, never touching the ring at
// all). Hiding the glyphs and sampling their own box side-steps all of
// that: it reads the exact pixels that were behind them.
//
// window.__contrastAuditRestoreIfPending() is a safety net: if the caller's
// screenshot or finish() call throws between prepare() and finish(), calling
// this restores any still-hidden paint so the next sample in the loop
// doesn't audit a page with stale invisible text. It's a no-op after a
// normal finish() call.
//
// NOTE: this logic (DOM-walk, foreground/paint-hide, background sampling)
// plus the pure WCAG math (relLum, wcagRatio, median) is duplicated in
// skills/hyperframes-creative/scripts/contrast-report.mjs — keep in sync.
// The pure "which rect to sample" decision is also mirrored in
// contrast-sample.ts (unit-tested there since this file can't import).
/* eslint-disable */
window.__contrastAudit = async function (imgBase64, time) {
function relLum(r, g, b) {
function ch(v) {
var s = v / 255;
return s <= 0.03928 ? s / 12.92 : Math.pow((s + 0.055) / 1.055, 2.4);
}
return 0.2126 * ch(r) + 0.7152 * ch(g) + 0.0722 * ch(b);
}
function wcagRatio(r1, g1, b1, r2, g2, b2) {
var l1 = relLum(r1, g1, b1),
l2 = relLum(r2, g2, b2);
var hi = l1 > l2 ? l1 : l2,
lo = l1 > l2 ? l2 : l1;
return (hi + 0.05) / (lo + 0.05);
window.__contrastAuditPrepare = function () {
// SVG text (<text>, <tspan>, <textPath>) is painted via the `fill`
// property, not `color` — the two are independent CSS properties in SVG.
// A page can set `fill` (inline style, `fill` attribute, or a CSS rule)
// without ever touching `color`, in which case getComputedStyle(el).color
// resolves to the inherited/initial value (often black) and does not
// reflect what's actually rendered on screen.
function isSvgTextElement(el) {
return !!el.ownerSVGElement;
}
function parseColor(c) {
@@ -32,19 +66,32 @@ window.__contrastAudit = async function (imgBase64, time) {
return [p[0], p[1], p[2], p[3] != null ? p[3] : 1];
}
// Like parseColor, but returns null instead of defaulting to black when the
// value isn't a solid rgb()/rgba() color — e.g. SVG paint keywords such as
// "none"/"context-fill", or a gradient/pattern reference like
// 'url("#grad")'. Callers should fall back to another source of truth
// rather than trust a fabricated black.
function tryParseSolidColor(c) {
var m = c.match(/rgba?\(([^)]+)\)/);
if (!m) return null;
var p = m[1].split(",").map(function (s) {
return parseFloat(s.trim());
});
if (
p.some(function (v) {
return isNaN(v);
})
)
return null;
return [p[0], p[1], p[2], p[3] != null ? p[3] : 1];
}
function selectorOf(el) {
if (el.id) return "#" + el.id;
var cls = Array.from(el.classList).slice(0, 2).join(".");
return cls ? el.tagName.toLowerCase() + "." + cls : el.tagName.toLowerCase();
}
function median(arr) {
var s = arr.slice().sort(function (a, b) {
return a - b;
});
return s[Math.floor(s.length / 2)];
}
function hasClipPath(el) {
for (var ce = el; ce; ce = ce.parentElement) {
var cp = getComputedStyle(ce).clipPath;
@@ -84,28 +131,15 @@ window.__contrastAudit = async function (imgBase64, time) {
return !paintsAnyProbePoint(el, rect);
}
// Decode screenshot into canvas pixel data
var img = new Image();
await new Promise(function (resolve) {
img.onload = resolve;
img.onerror = function () {
resolve();
};
img.src = "data:image/png;base64," + imgBase64;
});
if (!img.naturalWidth) return [];
var canvas = document.createElement("canvas");
canvas.width = img.naturalWidth || 1920;
canvas.height = img.naturalHeight || 1080;
var ctx = canvas.getContext("2d");
if (!ctx) return [];
ctx.drawImage(img, 0, 0);
var px = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
var w = canvas.width;
var h = canvas.height;
// Walk DOM for text elements
var out = [];
var restores = [];
// Registered BEFORE the walk starts (not after it finishes) and pushed to
// incrementally as each element is hidden: if getComputedStyle/
// getBoundingClientRect/etc. throws partway through the walk (e.g. on a
// detached or otherwise pathological element), everything hidden before
// the throw is still reachable for restore instead of leaking hidden
// indefinitely.
window.__contrastAuditRestores = restores;
var walker = document.createTreeWalker(document.body, NodeFilter.SHOW_ELEMENT);
var node;
while ((node = walker.nextNode())) {
@@ -148,85 +182,199 @@ window.__contrastAudit = async function (imgBase64, time) {
if (ancHidden) continue;
var rect = el.getBoundingClientRect();
if (rect.width < 8 || rect.height < 8) continue;
if (rect.right <= 0 || rect.bottom <= 0 || rect.left >= w || rect.top >= h) continue;
if (rect.right <= 0 || rect.bottom <= 0) continue;
if (isClippedAway(el, rect)) continue;
var fg = parseColor(cs.color);
// For SVG text, `fill` is the paint that's actually rendered; `color` is
// frequently just the inherited/initial value and unrelated to what's on
// screen. Only trust `fill` when it resolves to a solid color — "none",
// "context-fill", and gradient/pattern refs (url(#...)) fall back to
// `color` rather than crashing parseColor or reporting a fabricated
// black.
var isSvgText = isSvgTextElement(el);
var fg = isSvgText ? tryParseSolidColor(cs.fill) || parseColor(cs.color) : parseColor(cs.color);
if (fg[3] <= 0.01) continue;
// Prefer an opaque own/ancestor background-color over the pixel ring. A
// caption/CTA that paints its OWN solid background (a pill, a button, a
// card) composites its text over THAT color, not over whatever surrounds
// the box. Sampling the ring there measured the text against the scene
// behind the element (often a dark photo) and reported false ~1:1 warnings
// for perfectly readable CTAs. Walk up until a fully-opaque background-color
// is found; stop and defer to the ring on the first background-image (text
// over real pixels). Keep this in sync with commands/contrast-bg.ts.
var ownBg = null;
for (var bn = el; bn && bn !== document.body; bn = bn.parentElement) {
var bcs = getComputedStyle(bn);
if (bcs.backgroundImage && bcs.backgroundImage !== "none") break;
var bc = parseColor(bcs.backgroundColor);
if (bc[3] >= 0.999) {
ownBg = [bc[0], bc[1], bc[2]];
break;
}
}
var fontSize = parseFloat(cs.fontSize);
var fontWeight = Number(cs.fontWeight) || 400;
var large = fontSize >= 24 || (fontSize >= 19 && fontWeight >= 700);
var bgR, bgG, bgB;
if (ownBg) {
bgR = ownBg[0];
bgG = ownBg[1];
bgB = ownBg[2];
} else {
// Sample 4px ring outside bbox for background color
var rr = [],
gg = [],
bb = [];
var x0 = Math.max(0, Math.floor(rect.x) - 4);
var x1 = Math.min(w - 1, Math.ceil(rect.x + rect.width) + 4);
var y0 = Math.max(0, Math.floor(rect.y) - 4);
var y1 = Math.min(h - 1, Math.ceil(rect.y + rect.height) + 4);
var sample = function (sx, sy) {
if (sx < 0 || sx >= w || sy < 0 || sy >= h) return;
var idx = (sy * w + sx) * 4;
// Hide this element's OWN text paint so the caller's next screenshot
// reveals the true pixels behind the glyphs. Layout-neutral: color/fill
// never affect box geometry. `!important` beats any non-!important rule
// that might otherwise win on specificity; we restore the exact prior
// inline value (or remove the property entirely) afterward.
//
// A `transition` on color/fill would otherwise animate this hide instead
// of applying it instantly — the caller's screenshot lands in the same
// task-queue gap as the transition's own frames, so it can catch a
// partially-transparent (still partly the original color) glyph instead
// of a fully hidden one, contaminating the background sample. Force
// `transition: none` alongside color/fill so the hide is atomic, and
// restore it alongside them.
var origTransition = el.style.getPropertyValue("transition");
var origTransitionPriority = el.style.getPropertyPriority("transition");
el.style.setProperty("transition", "none", "important");
var origColor = el.style.getPropertyValue("color");
var origColorPriority = el.style.getPropertyPriority("color");
el.style.setProperty("color", "transparent", "important");
var origFill = null,
origFillPriority = null;
if (isSvgText) {
origFill = el.style.getPropertyValue("fill");
origFillPriority = el.style.getPropertyPriority("fill");
el.style.setProperty("fill", "transparent", "important");
}
restores.push({
el: el,
origTransition: origTransition,
origTransitionPriority: origTransitionPriority,
origColor: origColor,
origColorPriority: origColorPriority,
origFill: origFill,
origFillPriority: origFillPriority,
isSvgText: isSvgText,
});
out.push({
selector: selectorOf(el),
text: (el.textContent || "").trim().slice(0, 50),
fg: fg,
fontSize: fontSize,
fontWeight: fontWeight,
large: large,
bbox: { x: rect.x, y: rect.y, w: rect.width, h: rect.height },
});
}
return out;
};
function __contrastAuditRestoreAll() {
var restores = window.__contrastAuditRestores;
if (!restores) return;
for (var i = 0; i < restores.length; i++) {
var r = restores[i];
if (r.origColor) r.el.style.setProperty("color", r.origColor, r.origColorPriority);
else r.el.style.removeProperty("color");
if (r.isSvgText) {
if (r.origFill) r.el.style.setProperty("fill", r.origFill, r.origFillPriority);
else r.el.style.removeProperty("fill");
}
if (r.origTransition)
r.el.style.setProperty("transition", r.origTransition, r.origTransitionPriority);
else r.el.style.removeProperty("transition");
}
window.__contrastAuditRestores = null;
}
// Safety net for the caller: if the screenshot or finish() call throws
// between prepare() and finish(), call this to restore any still-hidden
// paint so the next sample in the loop isn't auditing a page with stale
// invisible text. No-op if finish() already ran normally.
window.__contrastAuditRestoreIfPending = function () {
__contrastAuditRestoreAll();
};
window.__contrastAuditFinish = async function (imgBase64, time, candidates) {
function relLum(r, g, b) {
function ch(v) {
var s = v / 255;
return s <= 0.03928 ? s / 12.92 : Math.pow((s + 0.055) / 1.055, 2.4);
}
return 0.2126 * ch(r) + 0.7152 * ch(g) + 0.0722 * ch(b);
}
function wcagRatio(r1, g1, b1, r2, g2, b2) {
var l1 = relLum(r1, g1, b1),
l2 = relLum(r2, g2, b2);
var hi = l1 > l2 ? l1 : l2,
lo = l1 > l2 ? l2 : l1;
return (hi + 0.05) / (lo + 0.05);
}
function median(arr) {
var s = arr.slice().sort(function (a, b) {
return a - b;
});
return s[Math.floor(s.length / 2)];
}
// Restore original paint first — we already have the screenshot, and we
// never want a decode failure below to leave the page's text invisible.
__contrastAuditRestoreAll();
var img = new Image();
await new Promise(function (resolve) {
img.onload = resolve;
img.onerror = function () {
resolve();
};
img.src = "data:image/png;base64," + imgBase64;
});
if (!img.naturalWidth) return [];
var canvas = document.createElement("canvas");
canvas.width = img.naturalWidth || 1920;
canvas.height = img.naturalHeight || 1080;
var ctx = canvas.getContext("2d");
if (!ctx) return [];
ctx.drawImage(img, 0, 0);
var px = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
var w = canvas.width;
var h = canvas.height;
var out = [];
for (var ci = 0; ci < candidates.length; ci++) {
var c = candidates[ci];
var bbox = c.bbox;
// Sample the element's OWN box (glyphs are hidden in this screenshot),
// inset 1px on each side to dodge anti-aliased edge pixels, clamped to
// the canvas. Mirrors contrast-sample.ts's computeSampleRect.
var x0 = Math.max(0, Math.round(bbox.x) + 1);
var x1 = Math.min(w - 1, Math.round(bbox.x + bbox.w) - 1);
var y0 = Math.max(0, Math.round(bbox.y) + 1);
var y1 = Math.min(h - 1, Math.round(bbox.y + bbox.h) - 1);
if (x1 <= x0 || y1 <= y0) continue;
// Bounded grid, not a full scan — dense enough to catch a
// partially-overlapping decoration without turning a wide caption bar
// into thousands of samples. Mirrors contrast-sample.ts's
// sampleGridPoints.
var stepX = Math.max(1, Math.floor((x1 - x0) / 12));
var stepY = Math.max(1, Math.floor((y1 - y0) / 6));
var rr = [],
gg = [],
bb = [];
for (var y = y0; y <= y1; y += stepY) {
for (var x = x0; x <= x1; x += stepX) {
var idx = (y * w + x) * 4;
rr.push(px[idx]);
gg.push(px[idx + 1]);
bb.push(px[idx + 2]);
};
for (var x = x0; x <= x1; x++) {
sample(x, y0);
sample(x, y1);
}
for (var y = y0; y <= y1; y++) {
sample(x0, y);
sample(x1, y);
}
if (rr.length === 0) continue;
bgR = median(rr);
bgG = median(gg);
bgB = median(bb);
}
if (rr.length === 0) continue;
// Composite foreground alpha over measured background
var bgR = median(rr),
bgG = median(gg),
bgB = median(bb);
// Composite foreground alpha over the measured background
var fg = c.fg;
var compR = Math.round(fg[0] * fg[3] + bgR * (1 - fg[3]));
var compG = Math.round(fg[1] * fg[3] + bgG * (1 - fg[3]));
var compB = Math.round(fg[2] * fg[3] + bgB * (1 - fg[3]));
var ratio = +wcagRatio(compR, compG, compB, bgR, bgG, bgB).toFixed(2);
var fontSize = parseFloat(cs.fontSize);
var fontWeight = Number(cs.fontWeight) || 400;
var large = fontSize >= 24 || (fontSize >= 19 && fontWeight >= 700);
out.push({
time: time,
selector: selectorOf(el),
text: (el.textContent || "").trim().slice(0, 50),
selector: c.selector,
text: c.text,
ratio: ratio,
wcagAA: large ? ratio >= 3 : ratio >= 4.5,
large: large,
wcagAA: c.large ? ratio >= 3 : ratio >= 4.5,
large: c.large,
fg: "rgb(" + compR + "," + compG + "," + compB + ")",
bg: "rgb(" + bgR + "," + bgG + "," + bgB + ")",
});