mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-04 16:42:27 +00:00
807078c7cde9d5c8403588722d1cd9397c513a0d
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Xuanru Li
|
e786b78b33 |
feat(producer): renderStretch to re-time short compositions across longer scenes (#2676)
Linear: VA-1859 ## Problem For a `fit_to_scene` B-roll where the composition's intrinsic timeline (e.g. `data-duration=1.0s` → 30 frames) is shorter than the scene it fills (e.g. 4.8s narration), the producer renders only the intrinsic 30 frames and the downstream compositor frame-holds/PTS-stretches that fixed clip to the scene length. Spreading 30 unique frames over 4.8s starves motion to ~6 effective fps → a visibly choppy result. Root cause: the producer welds one `composition.duration` to both the frame count and the 1:1 seek mapping, with no notion of a target output length. ## Fix Add optional `renderStretch: number` (default `1.0` = no-op), `renderStretch = intrinsic / target`: - **Frame count** comes from the target: `outputDuration = intrinsic / renderStretch`, `totalFrames = outputDuration × fps` (`probeStage.ts`). `composition.duration` stays intrinsic (drives video/audio windows). - **Per-frame seek** is scaled: `time = (frameIndex / fps) × renderStretch`, so the N output frames map across `[0, intrinsic]` — a fresh frame per output frame. All seek sites go through a single shared `outputFrameToTimelineSeconds(frameIndex, fps, renderStretch)` helper (`core.types.ts`), consumed by every capture path so none can silently diverge: - parallel (`parallelCoordinator.ts`), `sdr_streaming` (`captureStreamingStage.ts` ×3), `sdr_disk` (`captureStage.ts`), HDR loops. - DrawElement + static self-verify (`frameCapture.ts`) — ground-truth seek uses the same mapping, so PSNR compares like-for-like (no spurious verification failure on stretched comps). - Distributed path: `renderStretch` threaded through `DistributedRenderConfig` → chunk workers, and **folded into the plan hash only when `!= 1`** so a pre-stretch cached plan is never reused. With `renderStretch = 1` (or omitted → `?? 1`): every seek is `×1.0` (IEEE-754 identity), frame counts unchanged, and the plan hash is byte-identical — a provable no-op. `player.ts` absolute-seek is untouched. ## Verify - typecheck (core + engine + producer): pass. lint/format/fallow/commitlint: pass. `planHash` + `renderRequest` unit suites: pass. - Adversarial self-review found + fixed three capture-path gaps (streaming, self-verify, distributed) before this revision. - **Not yet runtime-verified** on a real render — needs a fit_to_scene render at `renderStretch < 1` confirming N distinct frames over the target length (draft until then). Paired with experiment-framework#42766, which computes and forwards `renderStretch = hf intrinsic / scene duration`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Xuanru Li
|
4ad582606b |
feat(lint): flag relative-value second writers and tl.set initial hides (#2612)
## What Part 2 of the GSAP seek-safety rules (stacks on #2611): the two rules that touch existing catalog content and required reconciliation with an existing rule. - `gsap_relative_value_second_writer` (error) — a relative var value (`y: "-=15"`) on a property whose target has another writer **active at the relative tween's start**. The relative base is captured at tween init, which reads a different partial state per seek path: sequential seek inits it mid-entrance, a cold render worker inits it at the entrance's end state, and the element teleports at chunk boundaries (production case: all scene nodes jumping ~20px mid-scene). Writers that complete strictly before the start are safe (children render in start-time order within a seek pass — verified against gsap 3.15.0) and are not flagged; neither are single-writer relatives, `from()`/`fromTo()`, build-time `gsap.set`, or relative position parameters (`"+=0.5"`). Selector resolution bails on combinators and cross-composition scoping rather than guessing. Findings aggregate per tween pair and report the overlap window. - `gsap_timeline_set_initial_hide` (warning) — initial-state hiding via `tl.set(target, vars, 0)` on a paused timeline is not rendered while the playhead sits at exactly 0, so frame 0 shows the unhidden state (verified against gsap 3.15.0: opacity stays 1 after `tl.time(0)`, applies only past 0). Exempt when the target is already hidden by authored CSS/inline styles or a standalone `gsap.set()`, and only sets preceding every tween in source order qualify (mutated position variables resolve to their initial binding in the parser — outro hard-kills don't masquerade as position-0 sets). - Reconciliation: `gsap_fullscreen_overlay_starts_visible`'s fixHint previously recommended exactly the flagged `tl.set(sel, {opacity:0}, 0)` pattern; it now recommends authored CSS hiding or immediate `gsap.set()`. - Docs for the full rule family in `docs/packages/lint.mdx`. ## Corpus impact (the reason this is its own PR) These two rules are the ones that fire on repo-shipped content: - `gsap_relative_value_second_writer`: 4 errors in `gooey-metaball`, all genuine overlaps. Measured with gsap 3.15.0: ballD diverges **3.31 xPercent / 1.99 yPercent (~8px/5px at 240px ball size)** between sequential and cold seek — a permanent base offset that appears as a teleport at a chunk boundary. Real but modest; happy to fix the block in a follow-up (start the drift at the entrance's end, or use absolute `fromTo`). - `gsap_timeline_set_initial_hide`: 10 warnings across the catalog after the CSS-hidden exemption (down from 54 pre-narrowing); spot-checked as genuine frame-0 pops with no authored hide (e.g. `vfx-text-cursor` `#phrase-b`). Adversarially reviewed the same way as #2611 (393-composition corpus + gsap semantics experiments); FP classes fixed and locked as negative tests: precede-only second writers, descendant/cross-composition selector mis-joins, CSS-hidden re-assertions, mutated position variables. ## Tests Full `packages/lint` suite green at 440 tests including multi-composition roots; `tsc`, oxlint, fallow audit clean. |
||
|
Xuanru Li
|
f3d2100663 |
feat(lint): seek-order safety and SVG draw-on rules for GSAP timelines (#2611)
## What Five lint rules (plus one extended core pattern) for GSAP defect classes that pass every existing check but break rendered output — the narrow, corpus-clean half of what was originally one PR (split per review; part 2 with the two catalog-touching rules stacks on top as #2612). - `gsap_repeat_refresh_relative_value` (error) — `repeatRefresh: true` + relative value re-captures and accumulates per iteration; a cold seek into iteration N skips the accumulation (verified with gsap 3.15.0: sequential 47.5 vs cold 17.5). - `gsap_function_value_hazard` (error/warning) — function values that call a method on the first parameter (GSAP passes `(index, target, targets)` — the first param is a number, so `(el) => el.getTotalLength()` throws and aborts the seeked frame) or measure the DOM: transform-sensitive reads (`getBoundingClientRect`, `getComputedStyle`, `gsap.getProperty`) are errors; transform-invariant layout reads (`offsetWidth`, `getBBox`, ...) are warnings. Pure-index arithmetic, `gsap.utils.wrap/distribute`, dataset/attribute reads, and closures over build-time constants are exempt. - `gsap_callback_dom_measurement` (warning) — DOM layout measurement reachable from `tl.add()`/`tl.call()`/`eventCallback`/`onStart|onUpdate|...` via a two-hop named-function scan. The capture path seeks with `suppressEvents: false`, so callbacks re-fire on every seek and measured geometry is seek-order-dependent. `gsap.getProperty`-driven derived output (scramble/typewriter patterns) is exempt. - `svg_measure_before_path_d` (error/warning) — `getTotalLength()` on a `<path>` with no static `d`: error when no `d` assignment exists anywhere (returns 0 in Chrome, silently killing dash animations); warning when assignments exist only inside function bodies. Recognizes `setAttribute`, GSAP `attr: { d }`, and CSS `d: path()`. - `svg_drawon_css_dasharray_conflict` (error) — GSAP `strokeDasharray` on an element whose CSS declares a multi-component `stroke-dasharray`. GSAP merges per component, so `strokeDasharray: pathLength` computes to `"641.4px, 10px"` — the gap stays 10px, the hide-then-reveal hides only 10px, and the line stays visible all scene with a crawling notch. One of this repo's own producer fixtures has this exact bug (true positive from the corpus run). - `gsap.utils.random()` and `"random(...)"` string tween values added to `non_deterministic_code` (core) — each worker inits independently, so the same tween resolves different randoms across chunks. Both motivating production bugs (nodes teleporting at chunk boundaries; a draw-on line visible all scene) are minimally reproduced in the tests. ## Review hardening Two independent adversarial reviews ran before submission: a false-positive hunt over all 393 compositions in this repo plus 23 constructed adversarial snippets (with gsap 3.15.0 semantics experiments), and a maintainer-conventions pass. Fixed FP classes are locked in as negative tests: all-interpolation template ids, GSAP attr-plugin `d` writes, getProperty-driven callbacks, transform-invariant marquee reads. Corpus residue for these five rules: **1 error (a genuine dasharray bug in a producer fixture, happy to fix in a follow-up) and 2 warnings** (real layout reads in callbacks). ## Tests `packages/lint` green at this commit in isolation; `tsc`, oxlint, fallow audit clean. ## Notes for reviewers - All rules follow the file's conservative philosophy: anything not statically resolvable is skipped; false negatives over false positives. - Open question: should the cold-seek family gate on `HyperframeLinterOptions.distributed` (error when distributed, warning otherwise), following the `system_font_will_alias` precedent? Happy to wire either way. |
||
|
Xuanru Li
|
7f4eaeb568 |
feat(cli): coordinate-frame layout findings in check (#2354)
* feat(cli): coordinate-frame layout findings in check Four production compositions shipped with 100-600px layout drift, each a different coordinate-frame confusion the check graded info or missed entirely: viewport pixels written as container left/top, gsap x/y treated as absolute position, a -350px margin fighting flex centering, and stage-relative path coords drawn into a nested SVG. Three new layout findings close the class: - positioned_out_of_parent: an absolute/fixed element rendering mostly outside its positioning ancestor (warning) — the parent needs no overflow clipping, which is what let container_overflow miss it. - box_out_of_canvas: a painted panel breaching the canvas (warning) — text is canvas_overflow's, media is frame_out_of_frame's, painted boxes were nobody's. - connector_detached: a connector path whose endpoints land far from every anchorable element (warning) — measured coordinates drawn into an SVG with a different origin. canvas_overflow additionally promotes from info to warning when held across samples AND the breach exceeds 5% of the canvas. All three are persistence-tiered and respect data-layout-allow-overflow. Verified against the four incident compositions: every one now surfaces its drift as held warnings (previously: info or silence). * fix(cli): harden coordinate-frame findings against review false positives Reworks all three findings after two-lens review (adversarial FP hunt in real Chrome + maintainer pass): - escaped_container (was positioned_out_of_parent): uses offsetParent (transform-aware, skips fixed-as-canvas), exempts fully-detached callouts within an attachment allowance while still flagging touching-but-mostly-outside drift. - panel_out_of_canvas (was box_out_of_canvas): paint alone qualifies (flat solid panels were a false negative), fully off-canvas rects are parked entrances and stay silent, pointer-events:none marks decorative layers, hero-sized breaches warn while small bleeds stay info. - connector_detached: endpoints via getPointAtLength + getScreenCTM (viewBox, preserveAspectRatio, group transforms, every command type), defs/marker/clipPath subtrees skipped, word-boundary connector naming, containment tier limited to opaque non-ancestor targets (a text-bearing wrapper contains its own diagram's endpoints). - canvas_overflow promotion requires partial visibility — a fully off-canvas rect is a parked entrance, not drift. Verified: the four incident compositions still surface their drift as held warnings; the review's false-positive repros (fixed HUD, callout, parked entrance, corner bleed, marker arrowheads, g-transform and viewBox-scaled connectors) are clean at warning level. Docs and the CLI skill reference now describe the coordinate-frame findings. * fix(cli): panel ownership is geometric — direct-text panels were a silent false negative A painted panel whose direct text stays in-bounds while its box breaches the canvas produced neither finding: canvas_overflow measures the text range and panel_out_of_canvas skipped every own-text element. Skip the panel finding only when the element's own text ALSO breaches (that geometry belongs to canvas_overflow); pin the message/fixHint wording of all three findings with positive assertions; document the SVG-internal anchor blind spot. * fix(cli): classify panel decoration by paint kind, not pointer-events pointer-events:none exempted the framed-painting incident's gold frame layers — hero content that happens to disable hit-testing. Decoration is now gradient-only paint (spotlights, textures, vignettes); url() images, solid fills and borders are content regardless of pointer-events. * fix(cli): add fixHint to the test-local AuditIssue shape * fix(cli): gradient stops decide content vs decoration; ownership matches canvas_overflow's tolerance A gradient with any solid stop (alpha >= 0.6) is content — heroes and cards painted with linear-gradient were invisible under the blanket gradient exemption; all-translucent stops (spotlights, vignettes) stay decoration. The text-ownership check now uses the audit tolerance that canvas_overflow itself fires at, making the contract strict-mutex: any text breach past that tolerance cedes the element, so a shallow 20px text breach no longer double-reports. |
||
|
Xuanru LiandClaude Fable 5
|
3e3b37d37f |
fix(cli): occlusion-probe false positives — pointer-events blindness, low-alpha gradients, not-yet-entered text (#2357)
* fix(cli): three occlusion-probe false-positive sources in text_occluded - pointer-events:none text is invisible to elementFromPoint, so the probe always hit whatever paints beneath and misread visible text as buried; restore hit-testing on the element for the duration of the probe - a backgroundImage counted as opaque regardless of alpha, so a 4%-alpha grid/scrim gradient qualified as an occluder; gradients now occlude only when their colours reach alpha > 0.6 (url() images unchanged) - a visible container whose every text-bearing descendant is still at opacity 0 (entrance not started) was probed anyway; skip when no text ink is on screen Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): address review — document-wide hit-testing restore, gradient compositing, whitespace ink - restore hit-testing for ALL pointer-events:none elements during the text audit pass (not just the probed text): an occluder that itself carries pointer-events:none is invisible to elementFromPoint, which made truly buried text read as clean once the text alone became hittable - hasVisibleTextInk ignores whitespace-only text nodes (indented markup defeated the gate) and uses a 0.05 floor so mid-fade text keeps its persistence occurrences - hasOpaqueBackground composites gradient alpha with background-color (two 0.5-alpha layers paint at ~0.75); gradientMaxAlpha returns opaque for any colour function it cannot score (oklch/lab/...); percentage alpha values now parse as fractions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): walk the elementsFromPoint stack in occluderAt A transparent layer that becomes hittable (pointer-events restored) must not mask an opaque occluder painting beneath it — single-point elementFromPoint returned the transparent top and dropped two genuinely buried cases in the census acceptance run; the stack walk keeps 10/10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): composite stacked background-image layers when judging occluder opacity Two 0.5-alpha gradient layers paint at 0.75 combined; taking the max color-stop alpha across the whole declaration under-counted them and suppressed real text_occluded findings. Split layers at top-level commas (paren-aware), score each, composite as 1-prod(1-a_i). Also pins the 0.05 text-ink floor with a boundary test (review feedback on #2357). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): keep walking the occlusion stack past pair-specific exemptions sharedPreserve3d and isCrossSceneTransitionOverlap excuse one hit, not the whole probe; returning null let a transparent decorative layer in the text's 3D context mask a real occluder below it (review feedback). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Xuanru LiandClaude Opus 4.8
|
906c8d04f8 |
fix(cli): localize remote assets before validate so it matches render (#2001)
validate served the composition over a loopback origin and let headless
Chrome fetch remote <img crossorigin>/@font-face assets cross-origin, while
the render pipeline downloads them to disk first. Buckets whose CORS
allowlist omits the loopback origin then failed the CORS-mode request with a
false net::ERR_FAILED that never occurs in the real render, pushing authors
(and agent pipelines) to delete crossorigin — which disables WebGL
color-grading/shaders for that asset.
Reuse producer's localizeRemote{Media,Image,FontFace}Sources in validate,
downloading into a temp dir served as an extra static-server asset root
(project dir untouched, cleaned up after). validate now matches render.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Xuanru LiandClaude Opus 4.8
|
dd774b3692 |
feat(capture): extract gradient washes, glass panels, and nav CTAs (#1879)
The design-style extractor now captures a site's signature color grounds and materials that a flat background-color misses: - Capture gradient background-image + backdrop-filter on buttons/cards/nav. - backgrounds[]: dominant gradient / mesh washes ranked by on-screen area (includes ::before/::after glow orbs), chroma-weighted so a small vivid brand wash outranks a large neutral scrim. - glass[]: frosted-glass panels (backdrop-filter blur) with their raw translucent fill, border, radius, shadow — ranked by area. - nav CTA capture: keep filled buttons inside <nav> (a page's primary "Sign up" / "Start for free" CTA that the old nav-drop lost), including gradient-filled CTAs whose background-COLOR is transparent. - Dedup keys for buttons/cards now include gradient + glass so a gradient/frosted variant is not collapsed into its flat sibling. - Fix: a fully-transparent fill rgba(...,0) now reports "transparent" instead of #000000 — the old bug turned every transparent wrapper into a phantom black button/card. types: ComponentStyle gains backgroundImage/backdropFilter; DesignStyles gains backgrounds[] and glass[]. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Xuanru Li
|
8694424807 |
Merge pull request #1827 from heygen-com/feat/capture-component-extraction
feat(capture): extract chips/stat-cells/tabs, detect icon fonts, transparent grounds |
||
|
Xuanru LiandClaude Opus 4.8
|
6cc87312d4 |
feat(capture): extract chips/stat-cells/tabs, detect icon fonts, transparent grounds
designStyleExtractor now also extracts chip/pill/badge/tag, stat/metric cells, and tab components — by class-substring selector plus a shape fallback (small + fully rounded + short text) so hashed/utility class names (Tailwind, CSS-modules) are still caught. It also emits a "transparent" sentinel for fully-transparent (rgba(...,0)) grounds instead of collapsing them to #000000, so a transparent chip/tab/stat on a light-ground site no longer reads as solid black. fontMetadataExtractor now flags icon fonts (isIcon) by glyph coverage: a font is an icon font only when it BOTH lacks a real Latin alphabet (<26 of A-Za-z) AND is mostly (>50%) Private-Use-Area glyphs. The Latin gate matters — some text fonts pack thousands of PUA glyphs yet are plainly text (Apple SF Pro is ~81% PUA but ships a full alphabet; Descript's Booton ~50%); flagging by PUA ratio alone would strip a brand's real typeface. Measured icon fonts: "hushly" 63% PUA / 7 letters, Font Awesome 95% / 0 letters. Names alone can't identify icon fonts ("hushly", "swiper-icons"), hence the glyph-based test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Xuanru Li
|
602590b44d |
Merge pull request #1820 from heygen-com/fix/snapshot-remote-video-frames
fix(cli): snapshot renders remote http(s) <video> frames (not just local files) |