mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
feat: website capture pipeline + 7-step video production skill (#284)
* feat(cli): add website capture with AI-powered DESIGN.md generation Adds `hyperframes capture <url>` command that extracts a complete design system from any website, producing AI-agent-ready output: - Full-page screenshot (lazy-load aware, nav at top) - AI-generated DESIGN.md via Claude API (colors, typography, elevation, components, do's/don'ts) with programmatic asset catalog (136+ assets with HTML context annotations like img[src], css url(), link[rel=preload]) - CSS-purged compositions (87% size reduction via PurgeCSS) - HTML-prettified compositions (one-tag-per-line for AI readability) - CLAUDE.md + .cursorrules auto-generated for AI agent instructions - Asset deduplication (srcset variants) and tracking pixel filtering * feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement - switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback - playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs) - replica refinement loop: generate, screenshot, compare, fix - extract inline svgs (50 max, 10kb each) to assets/svgs/ - extract visible text in dom order for content accuracy - detect js libraries (gsap, three.js, scrolltrigger) via globals - improved asset catalog grouping and naming - reverse-engineered aura system prompt documentation - comprehensive session handoff doc Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update session handoff with slack research findings - key finding: team already wants DESIGN.md integration (James, Bin, Vance) - skills quality matters enormously - must invoke /hyperframes-compose - eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide) - templates at templates/ need study before finalizing skill - session handoff updated with critical next steps * refactor(cli): simplify capture pipeline, remove replica generator * feat(capture): add Lottie detection and WebGL shader extraction Captures Lottie animations via network interception and WebGL shader source via gl.shaderSource hooking during site crawl. Updates website-to-hyperframes skill with asset planning guidance, Lottie/shader reading instructions, and stronger creative direction for scene planning. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(capture): clean pipeline + shader-first creative workflow Capture pipeline: - Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps) - Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md) - Remove dead canvas-to-image code in htmlExtractor (post canvas removal) - Parallelize image downloads (batches of 5 via Promise.allSettled) - Fix pre-existing TS error (match[1] guard in font downloader) - Default capture output to captures/<hostname> Skill creative overhaul: - Add shader transition selection to creative director step (Step 4) - Add shader wiring instructions to engineer step (Step 5) - Replace 4-line energy modifiers with visual vocabulary table - Strip rigid scene-by-scene templates from video-recipes.md - Strip example fill data from scene plan tables - Add "read transition refs before planning" instruction - Add creative ambition language ("how the hell did they make this") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add skill architecture redesign spec Comprehensive redesign of website-to-hyperframes skill and capture pipeline based on code review findings and Claude Code architecture research. Key changes: remove AI auto-generation, restructure skill into phases, embed shader boilerplate in scaffold, fix color format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add implementation plan for skill architecture redesign 13-task plan covering: capture pipeline cleanup (remove AI generation, fix colors to HEX, add asset descriptions, shader-ready scaffold), skill restructuring (4 phases with artifact gates), and compose skill Visual Identity Gate upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(capture): remove AI auto-generation and SDK dependencies * fix(capture): convert extracted colors to HEX format * refactor(capture): remove AI key path, add asset descriptions generator * refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions * feat(capture): pre-wire shader transitions in index.html scaffold * chore: remove duplicate visual-styles.md (canonical is in hyperframes/) * refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator * feat(skill): add Phase 1 understand reference * feat(skill): add Phase 2 design reference with full DESIGN.md schema * feat(skill): add Phase 3 creative direction reference * feat(skill): add Phase 4 build reference with inline shader example * feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md * docs: update CLAUDE.md skill references for phase-based workflow * fix: address code review findings - Remove orphaned `false` argument in generateAgentPrompt call (critical: was shifting hasLottie, hasShaders, catalogedAssets parameters) - Add HSL color handling in rgbToHex via temp element resolution - Remove build artifact commit section from phase-4-build.md - Fix __GSAP_TIMELINE reference to __timelines Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(capture): regex double-escape + simplify scaffold + fix asset descriptions - Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\() so browser receives valid regex patterns via page.evaluate() - Simplify index.html scaffold: scene slots + audio + timeline + comment pointing to shader-setup.md reference (no broken inline shader boilerplate) - Fix asset descriptions: use CatalogedAsset.contexts/notes instead of nonexistent htmlContext field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: code review — 16 bugs, 7-step skill rewrite, cleanup Code fixes: - snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero for --frames 1, port bind error handling, rAF-based render settle - index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini batch/rate-limit comments, fix video preview viewport y-coordinate - tokenExtractor.ts: remove dead seen[si] dedup code - gsap.ts: index ALL classes for inline-style transform conflict detection Skill architecture rewrite (4-phase → 7-step): - Replace phase-1 through phase-4 with step-1 through step-7 - Add techniques.md (10 visual techniques with code patterns) - Fix /hyperframes-compose → /hyperframes (skill doesn't exist) - Fix captures/arc-browser reference → shader-setup.md (file doesn't exist) - Fix step-7 hardcoded captures/stripe path - Document Gemini API free/paid rate limits in step-1 Cleanup: - CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step - .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast) - Delete old phase-*.md, animation-recreation.md, tts-integration.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove dev artifacts, research docs, wrong lockfiles Remove files that shouldn't ship in this PR: - docs/research/ (aura analysis, prompt catalogs) - docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes) - docs/superpowers/ planning and spec docs - pnpm-lock.yaml at root and cli (repo uses bun, not pnpm) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention Main PR #283 removed the full skills table from CLAUDE.md and moved it to AGENTS.md. Align with that decision: use main's slim dev-focused format, fix pnpm→bun references, add one-line /website-to-hyperframes pointer. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): add capture command to help groups The capture command was registered in cli.ts but missing from the help groups, so it wouldn't appear in `hyperframes --help`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format skill reference files (oxfmt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate bun.lock after rebase The lockfile was stale after rebasing onto main — bun install --frozen-lockfile failed in CI because new dependencies (google/genai, patchright, purgecss) weren't reflected in the lockfile. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review comments + improve capture quality Review fixes (16 comments from jrusso1020 + vanceingalls): - screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots - screenshotCapture: dynamic screenshot count based on page height (30% overlap) - snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard - htmlExtractor: fix code injection via parameterized evaluate - index.ts: video preview re-measures position after scroll, .env file loading - capture.ts: BLOCKED.md on timeout failures - gsap.ts: 5 inline-style lint tests added (all pass) - Remove Playwright, patchright deps; @google/genai to optionalDependencies - Gitignore: generic patterns instead of 20 hardcoded directories - Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance) Capture quality improvements (tested on 10+ websites): - Color extraction: canvas-based oklch/lab resolver, pixel sampling via elementFromPoint, broad sweep for accent colors, gradient/shadow extraction - Section detection: broadened selectors for div-based layouts, height cap to skip page-level wrappers, parent bg walkup for dark sites - Font downloads: cap 6 per family / 30 total (Cal.com: 306→30) - CTA detection: text pattern matching + nav context filtering - Heading text: innerText with whitespace normalization - Gemini captioning: maxOutputTokens 100→300, .env auto-loading - .env.example updated with GEMINI_API_KEY docs - TTS ranking: Kokoro first with Python 3.10+ note * fix: address PR review comments + improve capture quality Review round 2 fixes (jrusso1020 + vanceingalls): - verify/index.ts: add path traversal guard (relative + isAbsolute) - verify/index.ts: fix sections[i] undefined typecheck error (CI green) - index.ts: escape Lottie JSON with \u003c to prevent </script> breakout - step-4-storyboard: fix technique count contradiction (2-3 per beat, not across whole video) - step-6-build: perspective tilt uses gsap.set() instead of CSS transform (avoids GSAP overwrite conflict) - step-1-capture: reorder — command first, Gemini note after (zero-config is the default path, API key is optional enhancement) - step-7-validate: add tsx fallback for snapshot command - step-3-script: vary hook patterns, don't default to number every time - assetDownloader: exempt SVGs from 10KB minimum filter (company logos like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets) Note: adm-zip was NOT removed (reviewer #3) — it's still in packages/cli/package.json:30. The root package.json had patchright and purgecss removed, not adm-zip. Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms zero references in the entire codebase. The @anthropic-ai/sdk dependency was removed earlier in this branch. * refactor(capture): split index.ts (1175 to 566 lines) into modules Mechanical extraction, zero logic changes. New files: - mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots - contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions - scaffolding.ts (135 lines): .env loading, project scaffold generation Also fixes false-positive BLOCKED.md with structural Cloudflare detection. Tested on 20 websites, pre/post output identical. * chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss) The --split feature auto-generates compositions from captured HTML — a different approach from the /website-to-hyperframes skill workflow where agents build compositions from scratch using the storyboard. No skill file, no step reference, and no test session ever used --split. Removes 923 lines of unused code + purgecss dependency. Backed up to ~/Desktop/capture-split-backup/ for reference. * fix(security): add ssrf protection, lottie injection fix, oom guard - assetDownloader: add isPrivateUrl() guard blocking private IP ranges (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata endpoints, localhost, and non-HTTP schemes - mediaCapture: fix Lottie JSON injection by loading shell HTML first then passing animation data via parameterized page.evaluate() - index.ts: check Content-Length header before response.buffer() in Lottie network interception to avoid OOM on multi-GB responses * fix(capture): security fixes, timeout, sub-agent dispatch instructions Security (from miguel-heygen review): - assetDownloader: export isPrivateUrl() SSRF guard - htmlExtractor: add isPrivateUrl check before CSS fetch - mediaCapture: add isPrivateUrl check before Lottie fetch - mediaCapture: fix previewPage leak (try/finally) - mediaCapture: skip Lottie files > 2MB for preview (CDP limit) - contentExtractor: skip images > 4MB for Gemini captioning - index.ts: check Content-Length before response.buffer() (OOM guard) - snapshot.ts: register error handler before server.listen() Capture improvements: - Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare) - step-6-build: sub-agent dispatch template with explicit rules: pass file PATHS not contents, use local fonts not Google Fonts, verify ../assets/ references after each beat * fix(capture): catalog before DOM mutation, networkidle2, faster Gemini Critical: asset cataloger now runs BEFORE extractHtml which converts img src to data URLs. Framer sites like heykuba.com went from 2 to 78 images. - networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets) - Lazy-load wait: scroll to bottom, wait for img.complete - CSS background-image cataloging for Framer/Webflow - SVG naming: checks class, id, parent, inner text (not just aria-label) - Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img) - maxOutputTokens 300->500, descriptions sorted captioned-first - Remove tsx fallback from step-1 (reviewer nit, published CLI has it) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ular Kimsanov
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
ebc12f7dc9
commit
87f4c77e2f
@@ -0,0 +1,370 @@
|
||||
/**
|
||||
* Media capture helpers for the website capture pipeline.
|
||||
*
|
||||
* Handles Lottie animation preview rendering and video element manifest capture.
|
||||
*
|
||||
* All page.evaluate() calls use string expressions to avoid
|
||||
* tsx/esbuild __name injection (see esbuild issue #1031).
|
||||
*/
|
||||
|
||||
import type { Browser, Page } from "puppeteer-core";
|
||||
import { mkdirSync, writeFileSync, readdirSync, readFileSync, statSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { isPrivateUrl } from "./assetDownloader.js";
|
||||
|
||||
/** Discovered Lottie item from network interception or DOM scan. */
|
||||
export interface DiscoveredLottie {
|
||||
url: string;
|
||||
data?: unknown;
|
||||
dimensions?: { w: number; h: number };
|
||||
frameRate?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Download and save discovered Lottie animations to disk.
|
||||
*
|
||||
* Handles both plain JSON and dotLottie (.lottie ZIP) formats.
|
||||
* Deduplicates by content hash. Returns the count of saved files.
|
||||
*/
|
||||
export async function saveLottieAnimations(
|
||||
discoveredLotties: DiscoveredLottie[],
|
||||
lottieDir: string,
|
||||
): Promise<number> {
|
||||
let savedCount = 0;
|
||||
const savedHashes = new Set<string>(); // Deduplicate by content
|
||||
|
||||
for (let li = 0; li < discoveredLotties.length && li < 10; li++) {
|
||||
const lottieItem = discoveredLotties[li]!;
|
||||
try {
|
||||
let jsonData: string | undefined;
|
||||
|
||||
if (lottieItem.data) {
|
||||
// Already have the JSON data from network interception
|
||||
jsonData = JSON.stringify(lottieItem.data);
|
||||
} else if (lottieItem.url) {
|
||||
// SSRF guard — don't fetch private/internal URLs
|
||||
if (isPrivateUrl(lottieItem.url)) continue;
|
||||
// Download the file
|
||||
const res = await fetch(lottieItem.url, {
|
||||
signal: AbortSignal.timeout(10000),
|
||||
headers: { "User-Agent": "HyperFrames/1.0" },
|
||||
});
|
||||
if (!res.ok) continue;
|
||||
const buf = Buffer.from(await res.arrayBuffer());
|
||||
|
||||
if (lottieItem.url.endsWith(".lottie")) {
|
||||
// dotLottie is a ZIP — extract the animation JSON
|
||||
try {
|
||||
const AdmZip = (await import("adm-zip")).default;
|
||||
const zip = new AdmZip(buf);
|
||||
const entries = zip.getEntries();
|
||||
// Look for animation JSON in both v1 (animations/) and v2 (a/) paths
|
||||
const animEntry = entries.find(
|
||||
(e) =>
|
||||
(e.entryName.startsWith("a/") || e.entryName.startsWith("animations/")) &&
|
||||
e.entryName.endsWith(".json"),
|
||||
);
|
||||
if (animEntry) {
|
||||
jsonData = animEntry.getData().toString("utf-8");
|
||||
}
|
||||
} catch {
|
||||
// adm-zip not available or extraction failed — save raw .lottie
|
||||
const hash = buf.toString("base64").slice(0, 100);
|
||||
if (savedHashes.has(hash)) continue;
|
||||
savedHashes.add(hash);
|
||||
writeFileSync(join(lottieDir, `animation-${savedCount}.lottie`), buf);
|
||||
savedCount++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
// Plain JSON file
|
||||
jsonData = buf.toString("utf-8");
|
||||
}
|
||||
}
|
||||
|
||||
if (jsonData) {
|
||||
// Deduplicate by content hash (first 100 chars of stringified JSON)
|
||||
const hash = jsonData.slice(0, 200);
|
||||
if (savedHashes.has(hash)) continue;
|
||||
savedHashes.add(hash);
|
||||
|
||||
// Validate it's actually Lottie
|
||||
try {
|
||||
const parsed = JSON.parse(jsonData);
|
||||
if (!parsed.layers || !parsed.w) continue;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
writeFileSync(join(lottieDir, `animation-${savedCount}.json`), jsonData, "utf-8");
|
||||
savedCount++;
|
||||
}
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
return savedCount;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render preview thumbnails for saved Lottie animation JSON files.
|
||||
*
|
||||
* Opens each Lottie JSON in a headless Chrome page via lottie-web,
|
||||
* seeks to ~30% through the animation, and takes a transparent screenshot.
|
||||
* Writes a lottie-manifest.json with metadata + preview paths.
|
||||
*/
|
||||
export async function renderLottiePreviews(
|
||||
chromeBrowser: Browser,
|
||||
lottieDir: string,
|
||||
outputDir: string,
|
||||
): Promise<void> {
|
||||
const manifest: Array<{
|
||||
file: string;
|
||||
preview: string;
|
||||
name: string;
|
||||
width: number;
|
||||
height: number;
|
||||
duration: number;
|
||||
frameRate: number;
|
||||
layers: number;
|
||||
}> = [];
|
||||
const previewDir = join(lottieDir, "previews");
|
||||
mkdirSync(previewDir, { recursive: true });
|
||||
|
||||
for (const file of readdirSync(lottieDir)) {
|
||||
if (!file.endsWith(".json")) continue;
|
||||
try {
|
||||
const raw = JSON.parse(readFileSync(join(lottieDir, file), "utf-8"));
|
||||
const fr = raw.fr || 30;
|
||||
const dur = ((raw.op || 0) - (raw.ip || 0)) / fr;
|
||||
const previewName = file.replace(".json", "-preview.png");
|
||||
|
||||
// Render a mid-frame thumbnail using Puppeteer + lottie-web
|
||||
// Skip huge Lottie files for preview (CDP has a ~256MB message limit)
|
||||
const fileSize = statSync(join(lottieDir, file)).size;
|
||||
if (fileSize > 2_000_000) continue;
|
||||
|
||||
let previewPage;
|
||||
try {
|
||||
previewPage = await chromeBrowser.newPage();
|
||||
await previewPage.setViewport({ width: 400, height: 400 });
|
||||
const animData = JSON.parse(readFileSync(join(lottieDir, file), "utf-8"));
|
||||
const midFrame = Math.floor(((raw.op || 0) - (raw.ip || 0)) * 0.3);
|
||||
// Load the shell page first (no untrusted data in the HTML)
|
||||
await previewPage.setContent(
|
||||
`<!DOCTYPE html>
|
||||
<html><head>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/lottie-web/5.12.2/lottie.min.js"></script>
|
||||
<style>*{margin:0;padding:0;background:transparent}#c{width:400px;height:400px}</style>
|
||||
</head><body><div id="c"></div></body></html>`,
|
||||
{ waitUntil: "networkidle0", timeout: 10000 },
|
||||
);
|
||||
// Pass animation data safely via parameterized evaluate (no string interpolation)
|
||||
await previewPage.evaluate(
|
||||
(data: unknown, frame: number) => {
|
||||
const a = (window as any).lottie.loadAnimation({
|
||||
container: document.getElementById("c"),
|
||||
renderer: "svg",
|
||||
loop: false,
|
||||
autoplay: false,
|
||||
animationData: data,
|
||||
});
|
||||
a.addEventListener("DOMLoaded", () => {
|
||||
a.goToAndStop(frame, true);
|
||||
(window as any).__READY = true;
|
||||
});
|
||||
},
|
||||
animData,
|
||||
midFrame,
|
||||
);
|
||||
await previewPage
|
||||
.waitForFunction(() => (window as any).__READY === true, { timeout: 5000 })
|
||||
.catch(() => {});
|
||||
await previewPage.screenshot({
|
||||
path: join(previewDir, previewName),
|
||||
type: "png",
|
||||
omitBackground: true,
|
||||
});
|
||||
} catch {
|
||||
/* preview rendering failed — non-critical */
|
||||
} finally {
|
||||
await previewPage?.close().catch(() => {});
|
||||
}
|
||||
|
||||
manifest.push({
|
||||
file: `assets/lottie/${file}`,
|
||||
preview: `assets/lottie/previews/${previewName}`,
|
||||
name: raw.nm || file,
|
||||
width: raw.w || 0,
|
||||
height: raw.h || 0,
|
||||
duration: Math.round(dur * 10) / 10,
|
||||
frameRate: fr,
|
||||
layers: (raw.layers || []).length,
|
||||
});
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
if (manifest.length > 0) {
|
||||
writeFileSync(
|
||||
join(outputDir, "extracted", "lottie-manifest.json"),
|
||||
JSON.stringify(manifest, null, 2),
|
||||
"utf-8",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture video element manifest — screenshot each <video> element and
|
||||
* extract surrounding context (heading, caption, aria-label).
|
||||
*
|
||||
* Writes video-manifest.json and preview screenshots to assets/videos/previews/.
|
||||
*/
|
||||
export async function captureVideoManifest(
|
||||
page: Page,
|
||||
outputDir: string,
|
||||
progress: (stage: string, detail?: string) => void,
|
||||
): Promise<void> {
|
||||
const videoElements = (await page.evaluate(`(() => {
|
||||
var videos = Array.from(document.querySelectorAll('video'));
|
||||
return videos.map(function(v) {
|
||||
var src = v.src || v.currentSrc || (v.querySelector('source') ? v.querySelector('source').src : '');
|
||||
if (!src || !src.startsWith('http')) return null;
|
||||
|
||||
// Get bounding box for screenshot
|
||||
var rect = v.getBoundingClientRect();
|
||||
if (rect.width < 10 || rect.height < 10) return null;
|
||||
|
||||
// Nearest heading above the video
|
||||
var heading = '';
|
||||
var el = v;
|
||||
for (var i = 0; i < 8; i++) {
|
||||
el = el.parentElement;
|
||||
if (!el) break;
|
||||
var h = el.querySelector('h1,h2,h3,h4');
|
||||
if (h) { heading = h.textContent.trim().slice(0, 100); break; }
|
||||
}
|
||||
|
||||
// Nearest paragraph/caption text
|
||||
var caption = '';
|
||||
el = v;
|
||||
for (var j = 0; j < 5; j++) {
|
||||
el = el.parentElement;
|
||||
if (!el) break;
|
||||
var p = el.querySelector('p,figcaption,[class*="caption"],[class*="desc"]');
|
||||
if (p) { caption = p.textContent.trim().slice(0, 200); break; }
|
||||
}
|
||||
|
||||
// aria-label on video or wrapper
|
||||
var ariaLabel = v.getAttribute('aria-label') || v.getAttribute('title') || '';
|
||||
var wrapper = v.parentElement;
|
||||
if (!ariaLabel && wrapper) ariaLabel = wrapper.getAttribute('aria-label') || '';
|
||||
|
||||
return {
|
||||
src: src,
|
||||
width: Math.round(rect.width),
|
||||
height: Math.round(rect.height),
|
||||
top: Math.round(rect.top),
|
||||
left: Math.round(rect.left),
|
||||
heading: heading,
|
||||
caption: caption,
|
||||
ariaLabel: ariaLabel,
|
||||
filename: src.split('/').pop().split('?')[0],
|
||||
};
|
||||
}).filter(Boolean);
|
||||
})()`)) as Array<{
|
||||
src: string;
|
||||
width: number;
|
||||
height: number;
|
||||
top: number;
|
||||
left: number;
|
||||
heading: string;
|
||||
caption: string;
|
||||
ariaLabel: string;
|
||||
filename: string;
|
||||
}>;
|
||||
|
||||
// Deduplicate by src
|
||||
const seenSrcs = new Set<string>();
|
||||
const uniqueVideos = videoElements.filter((v) => {
|
||||
if (seenSrcs.has(v.src)) return false;
|
||||
seenSrcs.add(v.src);
|
||||
return true;
|
||||
});
|
||||
|
||||
if (uniqueVideos.length > 0) {
|
||||
const videoManifestDir = join(outputDir, "assets", "videos");
|
||||
mkdirSync(videoManifestDir, { recursive: true });
|
||||
const previewDir = join(videoManifestDir, "previews");
|
||||
mkdirSync(previewDir, { recursive: true });
|
||||
|
||||
const videoManifest: Array<{
|
||||
index: number;
|
||||
url: string;
|
||||
filename: string;
|
||||
width: number;
|
||||
height: number;
|
||||
heading: string;
|
||||
caption: string;
|
||||
ariaLabel: string;
|
||||
preview: string;
|
||||
}> = [];
|
||||
|
||||
for (let vi = 0; vi < uniqueVideos.length && vi < 20; vi++) {
|
||||
const v = uniqueVideos[vi]!;
|
||||
const previewName = `video-${vi}-preview.png`;
|
||||
const previewPath = join(previewDir, previewName);
|
||||
|
||||
// Screenshot the video element to get a visible frame
|
||||
try {
|
||||
// Scroll to the video element so it's in the viewport
|
||||
await page.evaluate(`window.scrollTo(0, ${Math.max(0, v.top - 100)})`);
|
||||
await new Promise((r) => setTimeout(r, 300));
|
||||
// Re-measure position after scroll (layout may have shifted)
|
||||
const rect = (await page.evaluate((fn) => {
|
||||
const vid = [...document.querySelectorAll("video")].find((x) =>
|
||||
(x.src || x.currentSrc || "").includes(fn),
|
||||
);
|
||||
if (!vid) return null;
|
||||
// Seek to 0.1s and wait for a frame to decode
|
||||
vid.currentTime = 0.1;
|
||||
return vid.getBoundingClientRect().toJSON();
|
||||
}, v.filename)) as { x: number; y: number; width: number; height: number } | null;
|
||||
if (!rect || rect.width < 10) continue;
|
||||
await new Promise((r) => setTimeout(r, 200)); // let decoder settle
|
||||
await page.screenshot({
|
||||
path: previewPath,
|
||||
clip: {
|
||||
x: Math.max(0, rect.x),
|
||||
y: Math.max(0, rect.y),
|
||||
width: Math.min(rect.width, 1920),
|
||||
height: Math.min(rect.height, 1080),
|
||||
},
|
||||
});
|
||||
} catch {
|
||||
/* preview failed — non-critical */
|
||||
}
|
||||
|
||||
videoManifest.push({
|
||||
index: vi,
|
||||
url: v.src,
|
||||
filename: v.filename,
|
||||
width: v.width,
|
||||
height: v.height,
|
||||
heading: v.heading,
|
||||
caption: v.caption,
|
||||
ariaLabel: v.ariaLabel,
|
||||
preview: `assets/videos/previews/${previewName}`,
|
||||
});
|
||||
}
|
||||
|
||||
if (videoManifest.length > 0) {
|
||||
writeFileSync(
|
||||
join(outputDir, "extracted", "video-manifest.json"),
|
||||
JSON.stringify(videoManifest, null, 2),
|
||||
"utf-8",
|
||||
);
|
||||
progress("design", `${videoManifest.length} video previews captured`);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user