feat(cli): judge text behind a transparent video by its decoded alpha in check (#5657)

* feat(cli): judge text behind a transparent video by its decoded alpha in check

The layout audit treated every <video> as opaque over its whole box, so a
caption between footage and a remove-background cutout always failed
text_occluded and needed data-layout-allow-occlusion, which also hid a word
the subject's head fully covered.

check now draws the video's current (seeked) frame into one reused,
downscaled scratch canvas, once per sample and only for a video a text probe
lands under. A fully opaque frame keeps today's behaviour. A frame with
transparency is judged per rendered word against its alpha: partial cover is
an info finding carrying the most covered word's share, and a word 80% hidden
or two neighbouring words each half hidden is an error. A frame that cannot
be read falls back to opaque with a video_alpha_unread info note.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cli): serve the video-alpha fixtures from memory, not request paths

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): check finds words under narrow, mirrored or backed cutouts

Review follow-ups on #5657:
- A matte the probe walk passes over, even at clear-alpha points, now triggers
  the per-word check, so a feature between two probe columns (an arm, a mic
  stand) still fails the word it hides.
- Mirrored videos and images map viewport points through the flip; a rotated,
  skewed or 3D-transformed video falls back to opaque with video_alpha_unread.
- A video with an opaque CSS background keeps its whole box occluding.
- cli.mdx lists every unread cause and why a fully transparent frame is one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
WaterrrForever
2026-10-15 18:53:16 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 08ee1bd5ad
commit cea8aa8a7d
10 changed files with 1095 additions and 25 deletions
+1
View File
@@ -1441,6 +1441,7 @@ const LAYOUT_ISSUE_CODES: readonly LayoutIssueCode[] = [
"content_overlap",
"visibility_escapes_segment",
"text_occluded",
"video_alpha_unread",
"text_not_painted",
"caption_zone_collision",
"frame_out_of_frame",
+3 -2
View File
@@ -18,6 +18,7 @@ export type LayoutIssueCode =
| "content_overlap"
| "visibility_escapes_segment"
| "text_occluded"
| "video_alpha_unread"
| "text_not_painted"
| "caption_zone_collision"
| "frame_out_of_frame"
@@ -62,8 +63,8 @@ export interface LayoutIssue {
rect: LayoutRect;
containerRect?: LayoutRect;
overflow?: LayoutOverflow;
/** `text_occluded` only: approximate fraction (0-1) of the occlusion probe
* grid that hit an opaque occluder — see layout-audit.browser.js. */
/** `text_occluded` only (0-1): probe-grid share hit by an opaque occluder, or behind a
* transparent video the most covered word's share under its alpha (layout-audit.browser.js). */
coveredFraction?: number;
fixHint?: string;
}