mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-01 19:42:03 +00:00
chore: remove plan docs and gitignore docs/plans/ (#1265)
Plan documents are working artifacts that shouldn't be committed. Removes three plans that were accidentally tracked and adds docs/plans/ to .gitignore to prevent future occurrences.
This commit is contained in:
Miguel Ángel
@@ -66,6 +66,9 @@ tmp/
|
||||
packages/core/src/generated/
|
||||
packages/producer/src/services/fontData.generated.ts
|
||||
|
||||
# Plan documents (working artifacts, not committed)
|
||||
docs/plans/
|
||||
|
||||
# Local proof / test artifacts
|
||||
qa-artifacts/
|
||||
my-video/
|
||||
|
||||
@@ -1,474 +0,0 @@
|
||||
---
|
||||
title: "feat: Preview canvas snap lines, alignment guides, and grid overlay"
|
||||
status: active
|
||||
created: 2026-06-04
|
||||
type: feat
|
||||
depth: deep
|
||||
---
|
||||
|
||||
# Preview Canvas Snap Lines, Alignment Guides & Grid Overlay
|
||||
|
||||
## Summary
|
||||
|
||||
Add Figma-quality snapping to the Studio preview canvas. When dragging or resizing elements, alignment guides appear showing edge-to-edge, center-to-center, and composition boundary alignment. Equidistance guides show equal spacing between elements. A toggleable grid overlay with snap-to-grid rounds out the system. The snap engine operates entirely in overlay coordinate space, keeping the existing gesture pipeline untouched except for a thin snap-adjustment hook.
|
||||
|
||||
---
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Studio's preview canvas currently supports drag, resize, and rotate gestures but offers no alignment assistance. Users must eyeball element positioning, making it impossible to precisely align elements to each other, to the composition center, or to a grid. Every professional NLE and design tool (Figma, After Effects, Motion, Premiere Pro) provides snap/alignment guides — their absence is a gap that makes Studio feel amateur for precision work.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- **R1.** During drag gestures, snap the dragged element's edges and center to the edges and center of all other visible elements on the canvas, the composition edges, and the composition center lines.
|
||||
- **R2.** During resize gestures, snap the resizing edge to the same targets as R1.
|
||||
- **R3.** Render vertical and horizontal snap guide lines across the full overlay when a snap is active. Lines appear only during the gesture and disappear on pointer-up.
|
||||
- **R4.** Show equidistance guides (Figma-style spacing indicators) when the dragged element's gap to one neighbor equals its gap to another neighbor on the same axis.
|
||||
- **R5.** Provide a toggleable grid overlay with configurable spacing. When snap-to-grid is enabled, element positions snap to grid intersections during drag/resize.
|
||||
- **R6.** Snapping is always-on by default. Holding a modifier key (Alt/Option) temporarily disables snapping during a gesture.
|
||||
- **R7.** Snap tolerance is defined in overlay-space pixels (default 6px) so it feels consistent regardless of zoom level.
|
||||
- **R8.** The snap engine must perform well with up to 80 elements on the canvas (the existing `maxItems` cap in `collectDomEditLayerItems`).
|
||||
- **R9.** Snap and grid preferences (enabled, grid spacing, grid visible, snap-to-grid) persist via `studioUiPreferences` (localStorage).
|
||||
- **R10.** Group drag snapping: when multiple elements are selected and dragged together, snap the group's bounding box edges/center, not individual members.
|
||||
|
||||
---
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
### KTD1. Snap computation in overlay coordinate space
|
||||
|
||||
All snap math operates on `OverlayRect` values — the already-computed screen-space rectangles used by `DomEditOverlay`. This avoids duplicating the complex iframe-to-overlay coordinate transformation. The snap engine receives the dragged element's current overlay rect and an array of target overlay rects, and returns adjusted dx/dy deltas plus active snap line positions.
|
||||
|
||||
**Rationale:** The overlay rects are already computed every RAF frame by `useDomEditOverlayRects`. They account for composition scaling, nested sub-compositions, and GSAP transforms. Working in this space means snap calculations are a simple 1D comparison on each axis.
|
||||
|
||||
### KTD2. Snap as a pure function injected into the gesture pipeline
|
||||
|
||||
The snap engine is a stateless pure function: `resolveSnapAdjustment(input) → { dx, dy, guides[] }`. It's called inside the existing `onPointerMove` handler in `useDomEditOverlayGestures.ts` after computing raw dx/dy but before applying them. No new hooks, no new state management — just a function call that adjusts the deltas.
|
||||
|
||||
**Rationale:** The gesture system uses refs and imperative DOM updates for performance (no React re-renders during drag). A pure function fits this pattern perfectly. It's also trivially testable.
|
||||
|
||||
### KTD3. Snap targets collected once at gesture start, not every frame
|
||||
|
||||
When a drag or resize gesture begins, collect all sibling element overlay rects into a flat array of snap edges. This array is stored on the `GestureState` and reused for every pointer-move event during the gesture. Elements don't move relative to each other during a single-element drag, so the snapshot is valid for the gesture's lifetime.
|
||||
|
||||
**Rationale:** Avoids querying the iframe DOM or recalculating overlay rects for non-dragged elements on every pointer-move. With 80 elements, this is the difference between O(1) per-frame work and O(n) DOM reads per frame.
|
||||
|
||||
### KTD4. Snap guide rendering via a dedicated React component with ref-driven updates
|
||||
|
||||
A `<SnapGuideOverlay>` component renders inside the `DomEditOverlay` div. During gestures, snap guide positions are written to a ref and flushed to DOM via direct style manipulation (same pattern as `boxRef` in the existing gesture code), bypassing React re-renders. On gesture end, the ref is cleared and the guides disappear.
|
||||
|
||||
**Rationale:** React state updates during drag cause frame drops. The existing codebase already uses this ref-driven pattern for the selection box — snap guides follow the same approach.
|
||||
|
||||
### KTD5. Grid overlay as a CSS background pattern
|
||||
|
||||
The grid overlay uses a CSS `background-image` with `repeating-linear-gradient` on the overlay div, not canvas or SVG. Grid lines are purely visual — snap-to-grid is computed in the snap engine using modular arithmetic, not by iterating grid lines.
|
||||
|
||||
**Rationale:** CSS gradients are GPU-composited and zero-JS-cost. They scale perfectly with the preview zoom because they're applied to the overlay which already matches the composition's visual size. No canvas/SVG management needed.
|
||||
|
||||
### KTD6. Equidistance detection via sorted-edge scan
|
||||
|
||||
To find equal spacing, sort all element rects by position on each axis, then scan adjacent pairs to find gaps. When the gap between the dragged element and neighbor A equals the gap between neighbor A and neighbor B, show a spacing indicator. This is O(n log n) per axis at gesture start (sorting) and O(n) per pointer-move (scanning the sorted list).
|
||||
|
||||
**Rationale:** Figma's equidistance guides are the feature's signature UX differentiator. The sorted-edge approach is the standard algorithm — simple, fast, and well-understood.
|
||||
|
||||
---
|
||||
|
||||
## High-Level Technical Design
|
||||
|
||||
### Snap Engine Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph "Gesture Start"
|
||||
A[Collect sibling OverlayRects] --> B[Extract snap edges array]
|
||||
C[Read composition bounds] --> B
|
||||
D[Read grid config] --> B
|
||||
end
|
||||
|
||||
subgraph "Every Pointer Move"
|
||||
E[Raw dx, dy from pointer] --> F[resolveSnapAdjustment]
|
||||
B --> F
|
||||
F --> G{Snap active?}
|
||||
G -->|yes| H[Adjusted dx, dy + guide lines]
|
||||
G -->|no| I[Original dx, dy, no guides]
|
||||
H --> J[Apply to element + render guides]
|
||||
I --> J
|
||||
end
|
||||
|
||||
subgraph "Gesture End"
|
||||
K[Clear snap state + hide guides]
|
||||
end
|
||||
```
|
||||
|
||||
### Snap Edge Model
|
||||
|
||||
Each snap target produces up to 5 edges per axis:
|
||||
|
||||
```
|
||||
Horizontal edges (for vertical snap lines): left, centerX, right
|
||||
Vertical edges (for horizontal snap lines): top, centerY, bottom
|
||||
```
|
||||
|
||||
The snap engine tests the dragged element's 3 edges against all target edges on each axis. The closest match within threshold wins. When multiple edges match at the same distance, all are shown as guides (Figma behavior).
|
||||
|
||||
### Component Architecture
|
||||
|
||||
```
|
||||
DomEditOverlay (existing)
|
||||
├── Selection box (existing)
|
||||
├── Resize handle (existing)
|
||||
├── Rotation handle (existing)
|
||||
├── SnapGuideOverlay (new) ← renders guide lines + spacing indicators
|
||||
└── GridOverlay (new) ← CSS background grid, always rendered when enabled
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
### In scope
|
||||
- Element-to-element snap (edges + centers) during drag and resize
|
||||
- Composition boundary snap (edges + center lines)
|
||||
- Grid overlay with configurable spacing
|
||||
- Snap-to-grid during drag and resize
|
||||
- Equidistance/spacing guides with pixel labels
|
||||
- Alt/Option hold-to-disable modifier
|
||||
- Preferences persistence in localStorage
|
||||
- Group drag snapping (bounding box)
|
||||
|
||||
### Deferred to Follow-Up Work
|
||||
- User-placed ruler guides (drag-from-ruler paradigm)
|
||||
- Safe area overlays (title-safe, action-safe)
|
||||
- Rule-of-thirds overlay
|
||||
- Snap during rotate gestures
|
||||
- Snap sound/haptic feedback
|
||||
- Keyboard arrow-key nudge with snap
|
||||
- Infinite edge extension (After Effects toggle)
|
||||
|
||||
### Outside this feature's scope
|
||||
- Timeline snapping (separate feature, different coordinate space)
|
||||
- Undo/redo integration (snap doesn't create new state — it adjusts existing gestures)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Units
|
||||
|
||||
### U1. Snap engine core — `snapEngine.ts`
|
||||
|
||||
**Goal:** Pure-function snap computation module with zero dependencies on React or DOM.
|
||||
|
||||
**Requirements:** R1, R2, R7
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/snapEngine.ts` (new)
|
||||
- `packages/studio/src/components/editor/snapEngine.test.ts` (new)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Define types:
|
||||
- `SnapEdge`: `{ position: number; source: 'element' | 'composition' | 'grid'; id: string }`
|
||||
- `SnapTarget`: `{ left, top, right, bottom, centerX, centerY, id }`
|
||||
- `SnapResult`: `{ dx: number; dy: number; guides: SnapGuide[]; spacingGuides: SpacingGuide[] }`
|
||||
- `SnapGuide`: `{ axis: 'x' | 'y'; position: number; from: number; to: number }`
|
||||
- `SpacingGuide`: `{ axis: 'x' | 'y'; position: number; size: number; from: number; to: number }`
|
||||
|
||||
Functions:
|
||||
- `extractSnapTargets(rects: OverlayRect[], ids: string[]): SnapTarget[]` — converts overlay rects to snap targets
|
||||
- `buildCompositionSnapTarget(compositionRect: OverlayRect): SnapTarget` — composition edges + center
|
||||
- `buildGridSnapEdges(compositionRect: OverlayRect, gridSpacing: number): { x: SnapEdge[]; y: SnapEdge[] }` — grid line positions
|
||||
- `resolveSnapAdjustment(input: { movingRect: OverlayRect; proposedDx: number; proposedDy: number; targets: SnapTarget[]; gridEdges?: { x: SnapEdge[]; y: SnapEdge[] }; threshold: number; disabled: boolean }): SnapResult` — the main entry point. Tests proposed position against all targets, returns adjusted deltas and guide positions.
|
||||
- `resolveResizeSnapAdjustment(input: { movingRect: OverlayRect; resizeEdge: 'right' | 'bottom'; proposedDx: number; proposedDy: number; targets: SnapTarget[]; gridEdges?: ...; threshold: number; disabled: boolean }): SnapResult` — resize variant that only snaps the active resize edge.
|
||||
- `resolveEquidistanceGuides(input: { movingRect: OverlayRect; targets: SnapTarget[]; threshold: number }): SpacingGuide[]` — scans for equal gaps between sorted elements.
|
||||
|
||||
The threshold operates in overlay pixels. At any zoom level, the overlay rect is already scaled, so a 6px threshold feels consistent.
|
||||
|
||||
**Patterns to follow:** Pure function style like `resolveDomEditResizeGesture` and `resolveDomEditRotationGesture` in `domEditOverlayGestures.ts`. Same input-object pattern, same deterministic output.
|
||||
|
||||
**Test scenarios:**
|
||||
- Dragging element left edge within 6px of target right edge → dx adjusted to align, guide emitted at aligned position
|
||||
- Dragging element center within threshold of composition center → snaps to center, two guide lines emitted (horizontal + vertical center)
|
||||
- Dragging element with no targets within threshold → returns original dx/dy, empty guides
|
||||
- Dragging element that matches multiple targets at same distance → all matching guides emitted
|
||||
- Resize right edge within threshold of target left edge → dx adjusted, guide emitted
|
||||
- Resize does not snap the non-resizing edges (top/left stay free during bottom-right resize)
|
||||
- Grid snap: position within threshold of grid line → snaps to grid line
|
||||
- Grid snap with element snap: element snap takes priority when both are within threshold
|
||||
- Equidistance: three elements in a row, dragging middle element to equal gap → spacing guide emitted with correct pixel size
|
||||
- Equidistance: no equal gaps → no spacing guides
|
||||
- Threshold of 0 → no snapping occurs
|
||||
- `disabled: true` → returns original dx/dy, empty guides
|
||||
- Large number of targets (80) → correct results (no performance regression)
|
||||
|
||||
### U2. Snap target collection at gesture start
|
||||
|
||||
**Goal:** Collect all sibling element overlay rects into a snap target array when a drag or resize gesture begins. Store on `GestureState` / `GroupGestureState`.
|
||||
|
||||
**Requirements:** R1, R2, R8, R10
|
||||
|
||||
**Dependencies:** U1
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/domEditOverlayGestures.ts` (modify — add `snapTargets` to state types)
|
||||
- `packages/studio/src/components/editor/domEditOverlayStartGesture.ts` (modify — collect targets on start)
|
||||
- `packages/studio/src/components/editor/snapTargetCollection.ts` (new — target collection logic)
|
||||
|
||||
**Approach:**
|
||||
|
||||
At gesture start in `startGesture()` and `startGroupDrag()`:
|
||||
1. Read the iframe content document
|
||||
2. Use `collectDomEditLayerItems()` (already exists, caps at 80) to get all editable elements
|
||||
3. For each element that is NOT the dragged element (or not in the drag group), compute its `toOverlayRect()`
|
||||
4. Convert to `SnapTarget[]` via `extractSnapTargets()`
|
||||
5. Also build the composition snap target from the iframe/root rect
|
||||
6. Store `{ snapTargets, compositionTarget }` on the gesture state
|
||||
|
||||
For group drag, exclude all group members from snap targets and use the group bounding box as the moving rect.
|
||||
|
||||
Read grid config from `studioUiPreferences` at gesture start. If snap-to-grid is enabled, pre-compute grid edges via `buildGridSnapEdges()` and store on gesture state.
|
||||
|
||||
**Patterns to follow:** The existing `startGesture` function in `domEditOverlayStartGesture.ts` already reads overlay rects and measures matrices. Target collection follows the same pattern — one-time setup cost at gesture start.
|
||||
|
||||
**Test scenarios:**
|
||||
- Starting a drag with 5 sibling elements → 5 snap targets collected (dragged element excluded)
|
||||
- Starting a drag with no siblings → only composition target present
|
||||
- Starting a group drag with 3 selected, 2 unselected → 2 snap targets collected
|
||||
- Elements in nested sub-compositions are included as targets (their overlay rects account for nesting)
|
||||
- Hidden elements (display:none, opacity:0) are excluded via existing `isElementVisibleForOverlay` check
|
||||
|
||||
### U3. Snap adjustment in drag gesture handler
|
||||
|
||||
**Goal:** Inject snap adjustment into the pointer-move handler for single and group drag gestures.
|
||||
|
||||
**Requirements:** R1, R6, R7, R10
|
||||
|
||||
**Dependencies:** U1, U2
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/useDomEditOverlayGestures.ts` (modify — call `resolveSnapAdjustment` in drag branch)
|
||||
|
||||
**Approach:**
|
||||
|
||||
In `onPointerMove`, inside the `g.kind === "drag"` branch:
|
||||
|
||||
1. Compute raw `dx`, `dy` from pointer delta (already done)
|
||||
2. Build the proposed overlay rect: `{ left: g.originLeft + dx, top: g.originTop + dy, ... }`
|
||||
3. Call `resolveSnapAdjustment({ movingRect: proposedRect, proposedDx: dx, proposedDy: dy, targets: g.snapTargets, gridEdges: g.gridEdges, threshold: SNAP_THRESHOLD_PX, disabled: e.altKey })`
|
||||
4. Use the returned adjusted `dx`, `dy` instead of the raw values for all downstream operations (overlay rect update, box style, `applyManualOffsetDragDraft`)
|
||||
5. Store the returned `guides` and `spacingGuides` on a ref for the `SnapGuideOverlay` to read
|
||||
|
||||
For group drag in the `groupG` branch, same pattern but using the group bounding box as the moving rect.
|
||||
|
||||
The Alt/Option key check (`e.altKey`) passes through as the `disabled` flag — when held, the snap engine returns raw deltas.
|
||||
|
||||
**Patterns to follow:** The existing rotation snap uses `e.shiftKey` to toggle 15-degree snapping — same modifier-key pattern.
|
||||
|
||||
**Test scenarios:**
|
||||
- Dragging element near sibling edge → position snaps, guide lines appear
|
||||
- Holding Alt while dragging → no snapping, free movement
|
||||
- Releasing Alt mid-drag → snapping re-engages
|
||||
- Dragging group near composition center → group bounding box snaps to center
|
||||
- Dragging with snap disabled in preferences → no snapping occurs
|
||||
- Pointer-up clears snap guides immediately
|
||||
|
||||
### U4. Snap adjustment in resize gesture handler
|
||||
|
||||
**Goal:** Inject snap adjustment into the pointer-move handler for resize gestures.
|
||||
|
||||
**Requirements:** R2, R6, R7
|
||||
|
||||
**Dependencies:** U1, U2
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/useDomEditOverlayGestures.ts` (modify — call `resolveResizeSnapAdjustment` in resize branch)
|
||||
|
||||
**Approach:**
|
||||
|
||||
In `onPointerMove`, inside the `else` (resize) branch:
|
||||
|
||||
1. After computing raw dx/dy, build a proposed rect with the new width/height
|
||||
2. Call `resolveResizeSnapAdjustment({ movingRect: proposedRect, resizeEdge: 'right' or 'bottom', proposedDx: dx, proposedDy: dy, targets: g.snapTargets, ... })`
|
||||
3. Use the adjusted dx/dy for the resize calculation
|
||||
4. Store guides on the ref for rendering
|
||||
|
||||
Only the actively resizing edges snap — the anchor edges (top-left) stay fixed.
|
||||
|
||||
**Patterns to follow:** Same injection point pattern as U3.
|
||||
|
||||
**Test scenarios:**
|
||||
- Resizing right edge near sibling left edge → width snaps, vertical guide appears
|
||||
- Resizing bottom edge near composition bottom → height snaps
|
||||
- Uniform resize (Shift held) with snap → snaps the dominant axis, aspect ratio maintained
|
||||
- Alt held during resize → no snapping
|
||||
|
||||
### U5. Snap guide overlay component — `SnapGuideOverlay.tsx`
|
||||
|
||||
**Goal:** Render snap guide lines and equidistance spacing indicators during active gestures.
|
||||
|
||||
**Requirements:** R3, R4
|
||||
|
||||
**Dependencies:** U1
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/SnapGuideOverlay.tsx` (new)
|
||||
- `packages/studio/src/components/editor/DomEditOverlay.tsx` (modify — add SnapGuideOverlay child)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Component structure:
|
||||
- Receives a ref (`snapGuidesRef`) that the gesture handler writes to on every pointer-move
|
||||
- Runs its own RAF loop (or piggybacks on the existing overlay RAF) to read the ref and update DOM
|
||||
- Renders guide lines as absolutely-positioned `<div>` elements: 1px wide/tall, full overlay extent, colored magenta/pink (#FF44CC at ~80% opacity — high contrast on both dark and light compositions, matches Figma convention)
|
||||
- Renders equidistance indicators as small spans between elements showing the pixel distance, styled with a semi-transparent background pill
|
||||
|
||||
Guide line DOM: pre-allocate a pool of 6 divs (3 per axis max — left/center/right or top/center/bottom). Hide unused ones with `display: none`. Update positions via `style.transform` for GPU compositing. No React re-renders during drag.
|
||||
|
||||
Spacing indicator DOM: pre-allocate a pool of 4 spacing indicator divs (2 per axis). Each shows a dashed line between elements with a centered pixel-count label.
|
||||
|
||||
On gesture end (pointer-up), hide all guide elements.
|
||||
|
||||
**Patterns to follow:** Same ref-driven DOM manipulation as `boxRef` in `DomEditOverlay.tsx`. The HUD overlay in `NLEPreview.tsx` uses a similar pattern (ref + direct style writes + opacity transitions).
|
||||
|
||||
**Test scenarios:**
|
||||
- Single vertical guide → 1px magenta line spanning full overlay height at the snap position
|
||||
- Multiple guides on same axis → all rendered simultaneously
|
||||
- Spacing guide between two elements → dashed connector with centered pixel label
|
||||
- Guides disappear immediately on pointer-up
|
||||
- Guides don't render when snap is disabled (Alt held)
|
||||
- Guides don't cause layout shift or affect pointer events (pointer-events: none)
|
||||
|
||||
### U6. Grid overlay component — `GridOverlay.tsx`
|
||||
|
||||
**Goal:** Toggleable grid overlay on the preview canvas with configurable spacing.
|
||||
|
||||
**Requirements:** R5, R9
|
||||
|
||||
**Dependencies:** None (can be built in parallel with U1-U5)
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/GridOverlay.tsx` (new)
|
||||
- `packages/studio/src/components/editor/DomEditOverlay.tsx` (modify — add GridOverlay child)
|
||||
- `packages/studio/src/utils/studioUiPreferences.ts` (modify — add snap/grid preferences)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Add to `StudioUiPreferences`:
|
||||
```
|
||||
snapEnabled?: boolean // default true
|
||||
gridVisible?: boolean // default false
|
||||
gridSpacing?: number // default 50 (composition pixels)
|
||||
snapToGrid?: boolean // default false
|
||||
```
|
||||
|
||||
`GridOverlay` component:
|
||||
- Absolutely positioned div covering the composition area within the overlay
|
||||
- Uses CSS `background-image` with `repeating-linear-gradient` for both axes
|
||||
- Grid line color: white at ~12% opacity (subtle, non-distracting on any background)
|
||||
- Grid spacing in composition pixels, scaled to overlay pixels via `editScaleX/Y` from the composition root rect
|
||||
- `pointer-events: none` so it doesn't interfere with gestures
|
||||
- Only renders when `gridVisible` is true
|
||||
|
||||
The grid scales with preview zoom because it's positioned within the stage/overlay coordinate space.
|
||||
|
||||
**Patterns to follow:** `StudioUiPreferences` read/write pattern in `studioUiPreferences.ts`. CSS gradient approach similar to design tools.
|
||||
|
||||
**Test scenarios:**
|
||||
- Grid visible → evenly spaced lines rendered across composition area
|
||||
- Grid hidden → no grid DOM present
|
||||
- Changing grid spacing → grid redraws with new spacing
|
||||
- Grid lines align with composition boundaries (first/last lines at edges)
|
||||
- Grid doesn't capture pointer events
|
||||
- Grid preferences survive page reload (localStorage)
|
||||
- Grid renders correctly at different zoom levels
|
||||
|
||||
### U7. Snap preferences UI — toolbar toggle
|
||||
|
||||
**Goal:** Add snap and grid toggle controls to the Studio preview toolbar.
|
||||
|
||||
**Requirements:** R6, R9
|
||||
|
||||
**Dependencies:** U6
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/SnapToolbar.tsx` (new)
|
||||
- `packages/studio/src/components/nle/NLEPreview.tsx` (modify — add toolbar)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Add a minimal toolbar overlay at the top-right of the preview viewport:
|
||||
- **Magnet icon** — toggles snap on/off (writes `snapEnabled` to preferences). Active state uses `studio-accent` color.
|
||||
- **Grid icon** — toggles grid visibility. Long-press or right-click opens a small popover for grid spacing input and snap-to-grid toggle.
|
||||
- Icons from Phosphor (already used in the project for toolbar icons).
|
||||
- Toolbar is semi-transparent, appears on hover over the preview area, stays visible during active gestures.
|
||||
|
||||
Keyboard shortcut: press `S` to toggle snap (After Effects convention), `G` to toggle grid. Only active when the preview/overlay has focus.
|
||||
|
||||
**Patterns to follow:** The zoom reset button in `NLEPreview.tsx` (positioned absolutely, semi-transparent, bottom-right) sets the visual pattern. The toolbar follows the same style but top-right.
|
||||
|
||||
**Test scenarios:**
|
||||
- Clicking magnet icon toggles snap on/off
|
||||
- Clicking grid icon toggles grid visibility
|
||||
- Grid spacing popover accepts numeric input, validates (min 10, max 500)
|
||||
- `S` key toggles snap when preview is focused
|
||||
- `G` key toggles grid when preview is focused
|
||||
- Preferences persist across page reload
|
||||
- Toolbar doesn't interfere with canvas gestures
|
||||
|
||||
### U8. Integration wiring and end-to-end behavior
|
||||
|
||||
**Goal:** Wire all pieces together, ensure correct behavior across the full gesture lifecycle, and handle edge cases.
|
||||
|
||||
**Requirements:** R1–R10 (all)
|
||||
|
||||
**Dependencies:** U1–U7
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/useDomEditOverlayGestures.ts` (finalize snap integration)
|
||||
- `packages/studio/src/components/editor/DomEditOverlay.tsx` (finalize component tree)
|
||||
- `packages/studio/src/components/editor/domEditOverlayStartGesture.ts` (finalize target collection)
|
||||
|
||||
**Approach:**
|
||||
|
||||
- Thread the `snapGuidesRef` from gesture handlers through to `SnapGuideOverlay`
|
||||
- Ensure pointer-up in all code paths (normal end, cancel, blocked-move threshold, escape) clears snap state
|
||||
- Ensure snap targets are recollected if selection changes mid-gesture (shouldn't happen, but defensive)
|
||||
- Verify group drag uses group bounding box, not individual member rects
|
||||
- Verify resize snap only adjusts the active edge
|
||||
- Test zoom + snap interaction: snap threshold should feel consistent at 50%, 100%, 200% zoom
|
||||
|
||||
**Patterns to follow:** The existing gesture lifecycle in `useDomEditOverlayGestures.ts` — gesture start sets up state, pointer-move reads it, pointer-up/cancel cleans it up.
|
||||
|
||||
**Test scenarios:**
|
||||
- End-to-end: drag element near sibling → snaps → guide appears → release → guide disappears → element stays at snapped position
|
||||
- End-to-end: resize element → right edge snaps to sibling → guide appears → release → element resized to snapped size
|
||||
- End-to-end: drag element to equal spacing between two others → spacing guide appears with distance label
|
||||
- Snap + grid: element snaps to grid line when snap-to-grid enabled, guide line shown at grid position
|
||||
- Element snap takes priority over grid snap when both are within threshold
|
||||
- Pointer cancel (e.g., right-click during drag) clears all snap state and guides
|
||||
- Drag at 200% zoom → snap threshold still feels like 6 visual pixels
|
||||
- Group drag with 3 elements → snaps group bounding box to composition center
|
||||
|
||||
---
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Snap target collection at gesture start adds latency (80 elements × `toOverlayRect` = ~80 BCR reads) | Profile on a composition with 80 elements. If > 2ms, cache rects from the existing RAF loop instead of re-reading. |
|
||||
| Guide line rendering causes frame drops during fast drag | Pre-allocated div pool with `transform` updates avoids layout/paint. `will-change: transform` on guide divs. |
|
||||
| CSS grid overlay doesn't align perfectly with composition pixels at fractional zoom | Use the same scaling math as `toOverlayRect` to compute grid line positions. Accept sub-pixel rounding at extreme zoom levels. |
|
||||
| Equidistance detection false positives (floating point gaps that are "almost equal") | Use a tolerance of 1 overlay pixel for gap equality comparison. |
|
||||
|
||||
---
|
||||
|
||||
## Sources & Research
|
||||
|
||||
- **Figma** — Smart guides, equidistance spacing indicators, snap-to-objects behavior. The equidistance pattern (purple gap labels) is the primary UX reference.
|
||||
- **After Effects** — Snap toggle (S key), edge extension, composition panel snapping model. Snap-always-on with toggle is the interaction model.
|
||||
- **Apple Motion** — Yellow dynamic guides, N key toggle, Command hold-to-disable.
|
||||
- **DaVinci Resolve** — Viewer guides, ruler-drag paradigm, safe area overlays (deferred).
|
||||
- **Premiere Pro** — Ctrl hold-to-enable (rejected — always-on is lower friction).
|
||||
- **Adobe Animate** — 18px snap tolerance as documented threshold reference.
|
||||
@@ -1,290 +0,0 @@
|
||||
---
|
||||
title: "fix: Pin fonts cross-platform — eliminate silent system-font fallback at render time"
|
||||
status: active
|
||||
type: fix
|
||||
created: 2026-06-07
|
||||
depth: Standard
|
||||
origin: null
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
System fonts (SF Mono, SF Pro, Menlo, Monaco, Consolas, Georgia, Verdana, etc.) silently fall back to generic families at render time because the deterministic font injector (`FONT_ALIASES`) has no mapping for them. The blank template and the `ensureFullDocument` CSS reset declare no `font-family` at all, so compositions start with browser defaults. Studio exposes macOS-only fonts (`COMMON_LOCAL_FONT_FAMILIES`) in its font picker with no warning that they break headless rendering. The fix expands `FONT_ALIASES` to cover all common system fonts across macOS/Windows/Linux, adds Inter + JetBrains Mono as defaults in templates and the CSS reset, upgrades the lint rule to catch unaliased system fonts, and annotates Studio's local font list so users see the render-time mapping.
|
||||
|
||||
---
|
||||
|
||||
## Problem Frame
|
||||
|
||||
When a composition author uses a system font like SF Mono or Menlo — either by typing it in CSS or selecting it in Studio's font picker — the deterministic font injector (`injectDeterministicFontFaces`) finds no match in `FONT_ALIASES`, passes the family name to Google Fonts, gets a 404 (it's not a Google Font), and silently leaves it unresolved. At render time in the Docker container, the font isn't installed, Chrome falls back to a generic monospace (DejaVu Sans Mono on Linux), and monospace alignment breaks because the metrics differ. The same pattern affects SF Pro, Menlo, Monaco, Consolas, Verdana, Tahoma, Calibri, Cambria, Georgia, Palatino, and other OS-bundled fonts.
|
||||
|
||||
A secondary issue: compositions scaffolded by `hyperframes init` or wrapped by `ensureFullDocument` have no default `font-family`, so any text element without an explicit font declaration renders in the browser's default serif (Times New Roman), which is never the intended look.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. Every common system font across macOS, Windows, and Linux must map to a bundled canonical font in `FONT_ALIASES` so it renders deterministically
|
||||
- R2. The blank template must declare `Inter` (sans-serif) and `JetBrains Mono` (monospace) as default fonts
|
||||
- R3. The `ensureFullDocument` CSS reset must include a default `font-family` so wrapped fragments don't fall back to Times New Roman
|
||||
- R4. The `TEXT_STYLES` constant must use `Inter` instead of `system-ui, -apple-system, sans-serif`
|
||||
- R5. The lint rule `font_family_without_font_face` must warn about system fonts that aren't in `FONT_ALIASES` — not just fonts missing `@font-face`
|
||||
- R6. The `PRODUCER_BUNDLED_FONTS` set in the lint rule must stay in sync with `FONT_ALIASES`
|
||||
- R7. Studio's local font list must annotate fonts that are aliased to a bundled font at render time, so users know what they'll get
|
||||
- R8. All existing tests must pass; new tests must cover the expanded alias map
|
||||
|
||||
---
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
**KTD-1. Alias mapping strategy — map to metrically closest bundled font, not to a "safe generic".**
|
||||
System sans-serif fonts (SF Pro, Calibri, Verdana, Segoe UI) map to Inter. System monospace fonts (SF Mono, Menlo, Monaco, Consolas, Lucida Console) map to JetBrains Mono. System serif fonts (Georgia, Palatino, Book Antiqua, Cambria) map to EB Garamond. This preserves the author's intent (category + approximate weight) while guaranteeing deterministic rendering. Rationale: mapping everything to Inter would destroy monospace alignment and serif aesthetics.
|
||||
|
||||
**KTD-2. Single source of truth for alias data — `FONT_ALIASES` in `deterministicFonts.ts` remains canonical; `PRODUCER_BUNDLED_FONTS` is generated from it.**
|
||||
Today these two structures are manually kept in sync. Rather than continuing that, export the alias keys from the producer and import them in the lint rule. This eliminates drift permanently.
|
||||
|
||||
**KTD-3. Default font in templates — Inter for body, JetBrains Mono for code.**
|
||||
These are already in `CANONICAL_FONTS` with embedded data URIs. No new font packages needed. The deterministic injector will auto-inject their `@font-face` rules when it sees them in the CSS.
|
||||
|
||||
**KTD-4. Studio annotation approach — add a `renderAlias` field to local font entries rather than removing them.**
|
||||
Users may genuinely want to preview with their local SF Pro. The annotation tells them "renders as Inter in video output" so they can make an informed choice. Removing the fonts would break the Studio experience for macOS users who work locally.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
### In Scope
|
||||
- Expanding `FONT_ALIASES` with ~30 additional system font mappings
|
||||
- Updating the blank template, `ensureFullDocument`, and `TEXT_STYLES` defaults
|
||||
- Making `PRODUCER_BUNDLED_FONTS` derive from the canonical alias map
|
||||
- Enhancing lint rules to catch system fonts
|
||||
- Studio font catalog annotation
|
||||
- Tests for all of the above
|
||||
|
||||
### Out of Scope (Non-Goals)
|
||||
- Font subsetting (noted in the codebase as "not yet implemented" — separate concern)
|
||||
- Adding new canonical fonts to the bundle (Inter, JetBrains Mono, EB Garamond already cover all needed categories)
|
||||
- CJK system font aliases (Noto Sans JP is already bundled; CJK system fonts like PingFang, Hiragino, MS Gothic are a different problem with different solutions)
|
||||
- Studio font picker UI redesign
|
||||
|
||||
### Deferred to Follow-Up Work
|
||||
- Auto-generating `PRODUCER_BUNDLED_FONTS` at build time from the producer package export (would require a build-order dependency change)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Units
|
||||
|
||||
### U1. Expand FONT_ALIASES with cross-platform system font mappings
|
||||
|
||||
**Goal:** Every common system font on macOS, Windows, and Linux resolves to a bundled canonical font.
|
||||
|
||||
**Requirements:** R1
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- `packages/producer/src/services/deterministicFonts.ts`
|
||||
- `packages/producer/src/services/deterministicFonts.test.ts`
|
||||
|
||||
**Approach:** Add entries to `FONT_ALIASES` for the following system fonts, grouped by target canonical:
|
||||
|
||||
*Maps to `inter` (sans-serif):*
|
||||
- SF Pro, SF Pro Display, SF Pro Text, SF Pro Rounded
|
||||
- Avenir, Avenir Next
|
||||
- Lucida Grande, Lucida Sans, Lucida Sans Unicode
|
||||
- Verdana, Tahoma, Trebuchet MS
|
||||
- Calibri, Candara, Corbel
|
||||
- Ubuntu (the system font, not the Google Font — same name, so the alias makes it deterministic)
|
||||
- Noto Sans, DejaVu Sans, Liberation Sans
|
||||
- Geneva, Optima
|
||||
|
||||
*Maps to `jetbrains-mono` (monospace):*
|
||||
- SF Mono
|
||||
- Menlo
|
||||
- Monaco
|
||||
- Consolas
|
||||
- Lucida Console, Lucida Sans Typewriter
|
||||
- Andale Mono
|
||||
- DejaVu Sans Mono, Liberation Mono
|
||||
- Ubuntu Mono (system variant)
|
||||
|
||||
*Maps to `eb-garamond` (serif):*
|
||||
- Georgia
|
||||
- Palatino, Palatino Linotype, Book Antiqua
|
||||
- Cambria
|
||||
- Times, Times New Roman
|
||||
- DejaVu Serif, Liberation Serif
|
||||
|
||||
Export the `FONT_ALIASES` keys as a `Set<string>` named `FONT_ALIAS_KEYS` so the lint rule can import it.
|
||||
|
||||
**Patterns to follow:** Existing alias entries at lines 156-188 of `deterministicFonts.ts`. All keys are lowercase.
|
||||
|
||||
**Test scenarios:**
|
||||
- Each new alias resolves to the expected canonical font key
|
||||
- SF Mono resolves to jetbrains-mono
|
||||
- SF Pro Display resolves to inter
|
||||
- Menlo resolves to jetbrains-mono
|
||||
- Consolas resolves to jetbrains-mono
|
||||
- Georgia resolves to eb-garamond
|
||||
- Times New Roman resolves to eb-garamond
|
||||
- Verdana resolves to inter
|
||||
- The existing aliases still resolve correctly (no regressions)
|
||||
|
||||
**Verification:** `bun run --cwd packages/producer test` passes. The exported `FONT_ALIAS_KEYS` set contains all new entries.
|
||||
|
||||
---
|
||||
|
||||
### U2. Update default fonts in templates and CSS reset
|
||||
|
||||
**Goal:** Compositions start with Inter and JetBrains Mono as default fonts, not browser defaults.
|
||||
|
||||
**Requirements:** R2, R3, R4
|
||||
|
||||
**Dependencies:** None (independent of U1 — the deterministic injector already handles Inter and JetBrains Mono)
|
||||
|
||||
**Files:**
|
||||
- `packages/cli/src/templates/blank/index.html`
|
||||
- `packages/core/src/templates/constants.ts`
|
||||
- `packages/producer/src/services/htmlCompiler.ts`
|
||||
- `packages/core/src/templates/base.test.ts`
|
||||
- `packages/producer/src/services/htmlCompiler.test.ts`
|
||||
|
||||
**Approach:**
|
||||
|
||||
1. **Blank template** — add a `<style>` block with:
|
||||
```
|
||||
body { font-family: "Inter", sans-serif; }
|
||||
code, pre, .monospace { font-family: "JetBrains Mono", monospace; }
|
||||
```
|
||||
|
||||
2. **`TEXT_STYLES` constant** — change `font-family: system-ui, -apple-system, sans-serif` to `font-family: "Inter", sans-serif`.
|
||||
|
||||
3. **`ensureFullDocument` CSS reset** — add `font-family:"Inter",sans-serif` to the `body` rule. The deterministic injector will see the `Inter` declaration and inject its `@font-face`.
|
||||
|
||||
**Patterns to follow:** The existing CSS reset style in `ensureFullDocument` at line 710.
|
||||
|
||||
**Test scenarios:**
|
||||
- `TEXT_STYLES` contains `"Inter"` and does not contain `system-ui` or `-apple-system`
|
||||
- The blank template HTML contains `font-family: "Inter"` in a style block
|
||||
- The blank template HTML contains `font-family: "JetBrains Mono"` for monospace
|
||||
- `ensureFullDocument` output contains `font-family:"Inter"` in the body style
|
||||
- Existing `base.test.ts` assertions still pass (update the font-family expectation)
|
||||
- Fragment wrapping produces a document whose body has `font-family:"Inter",sans-serif`
|
||||
|
||||
**Verification:** `bun run --cwd packages/core test` and `bun run --cwd packages/producer test` pass.
|
||||
|
||||
---
|
||||
|
||||
### U3. Sync PRODUCER_BUNDLED_FONTS with FONT_ALIASES via import
|
||||
|
||||
**Goal:** Eliminate manual drift between the alias map in the producer and the lint rule's bundled-font list.
|
||||
|
||||
**Requirements:** R6
|
||||
|
||||
**Dependencies:** U1 (needs the exported `FONT_ALIAS_KEYS`)
|
||||
|
||||
**Files:**
|
||||
- `packages/core/src/lint/rules/fonts.ts`
|
||||
- `packages/producer/src/services/deterministicFonts.ts` (export already added in U1)
|
||||
|
||||
**Approach:** Replace the hardcoded `PRODUCER_BUNDLED_FONTS` set in `fonts.ts` with an import from the producer package. The producer already exports `FONT_ALIAS_KEYS` (added in U1). The lint rule imports it and uses it directly.
|
||||
|
||||
Check if `@hyperframes/producer` is already a dependency of `@hyperframes/core`. If not, consider exporting the alias keys from `@hyperframes/core` instead (since the lint rules live in core) by moving the canonical alias list to a shared location in core and importing it in both the producer and the lint rule.
|
||||
|
||||
If a circular dependency would result, export the alias key list as a plain JSON-serializable array from a shared file in core that both packages import, keeping `deterministicFonts.ts` as the runtime consumer and the lint rule as the build-time consumer.
|
||||
|
||||
**Test scenarios:**
|
||||
- The lint rule's bundled font set matches the producer's alias keys exactly
|
||||
- Adding a new alias in `deterministicFonts.ts` automatically makes the lint rule recognize it (verified by the import chain, not a separate test)
|
||||
- Existing lint rule tests pass without modification (the recognized font set is a superset of the old one)
|
||||
|
||||
**Verification:** `bun run --cwd packages/core test` passes. Manually verify that `PRODUCER_BUNDLED_FONTS` is no longer a hardcoded set.
|
||||
|
||||
---
|
||||
|
||||
### U4. Enhance lint rule to warn about unaliased system fonts
|
||||
|
||||
**Goal:** The linter catches system-only fonts that will silently fall back at render time.
|
||||
|
||||
**Requirements:** R5
|
||||
|
||||
**Dependencies:** U3 (needs the synced bundled font set)
|
||||
|
||||
**Files:**
|
||||
- `packages/core/src/lint/rules/fonts.ts`
|
||||
- `packages/core/src/lint/rules/fonts.test.ts`
|
||||
|
||||
**Approach:** Add a new lint rule `system_font_will_alias` (severity: info) that fires when a composition uses a font family that exists in `FONT_ALIAS_KEYS` but is NOT the canonical name (i.e., it will be silently mapped). The message tells the user what it will render as: "Font 'SF Mono' will render as 'JetBrains Mono' in video output. Use 'JetBrains Mono' directly for consistent preview/render results."
|
||||
|
||||
This is distinct from the existing `font_family_without_font_face` rule (which warns about fonts that can't be resolved at all). The new rule covers fonts that CAN be resolved but will be substituted — an informational heads-up, not a warning.
|
||||
|
||||
Also add a `SYSTEM_FONT_FAMILIES` set (fonts that exist only as OS installations, not in Google Fonts) to distinguish them from Google Fonts in the `font_family_without_font_face` rule. A system font without an alias is a harder error than a Google Font without an alias (the Google Font can be fetched; the system font cannot).
|
||||
|
||||
**Patterns to follow:** Existing lint rules in `fonts.ts`.
|
||||
|
||||
**Test scenarios:**
|
||||
- A composition using `font-family: "SF Mono", monospace` triggers `system_font_will_alias` with message mentioning JetBrains Mono
|
||||
- A composition using `font-family: "Inter", sans-serif` does NOT trigger `system_font_will_alias` (Inter is the canonical name)
|
||||
- A composition using `font-family: "Helvetica Neue", sans-serif` triggers `system_font_will_alias` with message mentioning Inter
|
||||
- A composition using `font-family: "Comic Sans MS", sans-serif` with no `@font-face` triggers `font_family_without_font_face` (not aliased, not a Google Font)
|
||||
- A composition using `font-family: "Roboto", sans-serif` does NOT trigger any font rule (Roboto is a canonical bundled font)
|
||||
|
||||
**Verification:** `bun run --cwd packages/core test` passes.
|
||||
|
||||
---
|
||||
|
||||
### U5. Annotate Studio font catalog with render-time aliases
|
||||
|
||||
**Goal:** Studio users see what their local fonts will render as in video output.
|
||||
|
||||
**Requirements:** R7
|
||||
|
||||
**Dependencies:** U1 (needs the alias map data)
|
||||
|
||||
**Files:**
|
||||
- `packages/studio/src/components/editor/fontCatalog.ts`
|
||||
- `packages/studio/src/components/editor/propertyPanelHelpers.ts`
|
||||
|
||||
**Approach:** Add a `RENDER_ALIAS_MAP` to `fontCatalog.ts` that maps local font family names to their render-time canonical names. This is a static map (not imported from the producer, to avoid a build dependency) that covers the `COMMON_LOCAL_FONT_FAMILIES` entries. The property panel helpers use this map to display "(renders as Inter)" or "(renders as JetBrains Mono)" next to local font names in the font picker dropdown.
|
||||
|
||||
Update `COMMON_LOCAL_FONT_FAMILIES` to remove entries that are already in the Google Fonts list or the canonical bundled font list (no point listing "Arial" as a "local font" when Inter is the canonical and Arial is already aliased). Keep only the genuinely local fonts (TT Norms Pro, SF Pro Display, SF Pro Text, Avenir, Avenir Next, Menlo, Monaco) and add the render alias annotation.
|
||||
|
||||
**Patterns to follow:** Existing `COMMON_LOCAL_FONT_FAMILIES` structure and `sortFontOptions` in `propertyPanelHelpers.ts`.
|
||||
|
||||
**Test scenarios:**
|
||||
- `RENDER_ALIAS_MAP` maps "SF Pro Display" to "Inter"
|
||||
- `RENDER_ALIAS_MAP` maps "Menlo" to "JetBrains Mono"
|
||||
- `RENDER_ALIAS_MAP` maps "Monaco" to "JetBrains Mono"
|
||||
- `COMMON_LOCAL_FONT_FAMILIES` no longer contains fonts that are in the Google or canonical bundled lists (Arial, Courier New, Helvetica Neue)
|
||||
- Font picker helper produces annotation text for aliased local fonts
|
||||
|
||||
**Verification:** `bun run build` succeeds (Studio type-checks). Manual check: Studio font picker shows alias annotations.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **OQ-1.** Should the deterministic font injector log a warning when it resolves an alias (e.g., "SF Mono → JetBrains Mono")? Currently it silently substitutes. A warning would help debugging but could be noisy for compositions that intentionally use system font names. *Deferred to implementation — start silent, add opt-in verbose logging if needed.*
|
||||
|
||||
---
|
||||
|
||||
## System-Wide Impact
|
||||
|
||||
- **Existing compositions using system fonts** will now render with the aliased canonical font instead of an unpredictable fallback. This is a visual change but an intentional improvement — the previous behavior was already broken (wrong font rendered).
|
||||
- **New compositions** will start with Inter instead of no font. This only affects compositions that had no explicit font-family declarations.
|
||||
- **Lint output** will show new `system_font_will_alias` findings for compositions using aliased fonts. Severity is info, not warning — won't block CI.
|
||||
- **Docker/CI rendering** is unaffected (system fonts were already falling back; now they fall back to the correct aliased font instead of a random generic).
|
||||
|
||||
---
|
||||
|
||||
## Sources & Research
|
||||
|
||||
- `packages/producer/src/services/deterministicFonts.ts` — canonical font injection engine with FONT_ALIASES and CANONICAL_FONTS
|
||||
- `packages/core/src/lint/rules/fonts.ts` — lint rules with PRODUCER_BUNDLED_FONTS
|
||||
- `packages/core/src/templates/constants.ts` — TEXT_STYLES with system-ui default
|
||||
- `packages/cli/src/templates/blank/index.html` — blank template with no font declarations
|
||||
- `packages/producer/src/services/htmlCompiler.ts` — ensureFullDocument CSS reset with no font-family
|
||||
- `packages/studio/src/components/editor/fontCatalog.ts` — Studio font catalog with COMMON_LOCAL_FONT_FAMILIES
|
||||
- `packages/producer/src/services/render/planValidation.ts` — system font validation for distributed rendering
|
||||
- macOS system fonts: SF Pro, SF Mono, Menlo, Monaco, Avenir, Lucida Grande, Geneva, Optima, Palatino, Georgia, Courier
|
||||
- Windows system fonts: Segoe UI (already aliased), Calibri, Cambria, Consolas, Candara, Corbel, Verdana, Tahoma, Trebuchet MS, Georgia, Palatino Linotype, Book Antiqua, Lucida Console, Lucida Sans Unicode
|
||||
- Linux system fonts: DejaVu Sans/Serif/Mono, Liberation Sans/Serif/Mono, Noto Sans/Serif, Ubuntu (font family)
|
||||
@@ -1,314 +0,0 @@
|
||||
---
|
||||
title: "feat: Font resolution pipeline — make compositions self-contained by capturing local and remote fonts"
|
||||
status: active
|
||||
type: feat
|
||||
created: 2026-06-07
|
||||
depth: Standard
|
||||
origin: null
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Compositions that reference system fonts (SF Mono, Menlo, TT Norms Pro) or fonts hosted on non-Google CDNs currently fall through the deterministic font injector and render with unpredictable fallbacks. The alias map from the prior PR mitigates this for known system fonts, but it's lossy (Inter ≠ SF Pro metrics) and always incomplete. This plan adds a font resolution pipeline that captures the *actual font files* and embeds them, making compositions portable without aliasing. The alias map becomes the fallback for distributed renders where local font access doesn't exist.
|
||||
|
||||
---
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The deterministic font injector resolves fonts through two paths: bundled aliases (18 canonical fonts) and Google Fonts fetch. Anything outside those two paths is unresolved — the font silently falls back at render time. Three categories of fonts hit this gap:
|
||||
|
||||
1. **System fonts not in the alias map** — commercial or niche fonts like TT Norms Pro, Frutiger, Proxima Nova that are installed locally but don't exist in Google Fonts and aren't aliased
|
||||
2. **Fonts hosted on non-Google CDNs** — `<link>` tags pointing to S3, Cloudflare, or custom font servers whose stylesheets are never fetched and inlined by the compiler
|
||||
3. **Any new system font** — every time Apple, Microsoft, or a Linux distro ships a new system font, the alias map needs a manual update
|
||||
|
||||
A font resolution pipeline that captures font files from the local filesystem eliminates categories 1 and 3 entirely. Extending `localizeRemoteFontFaces` to handle external stylesheets (not just @font-face src URLs) eliminates category 2.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. The compiler must locate and embed local system font files (ttf/otf/woff2) as data URIs when a font-family isn't resolved by the bundled or Google Fonts paths
|
||||
- R2. System font location must work on macOS (system_profiler + directory scan), Windows (C:\Windows\Fonts), and Linux (fontconfig fc-match + directory scan)
|
||||
- R3. Font files must be compressed to woff2 before embedding to keep composition size reasonable (ttf → woff2 is typically 60-70% smaller)
|
||||
- R4. External `<link rel="stylesheet">` tags pointing to non-Google font CDNs must have their stylesheets fetched, @font-face rules extracted, and font URLs inlined as data URIs
|
||||
- R5. The resolution order must be: existing @font-face → bundled alias → Google Fonts → local system font → alias fallback → actionable error
|
||||
- R6. Studio must auto-import system fonts when selected in the picker so the font file is part of the project from composition time (not deferred to render time)
|
||||
- R7. All existing tests must pass; new tests must cover system font resolution, woff2 conversion, external stylesheet inlining, and the resolution order
|
||||
- R8. System font capture must be skipped in distributed/Lambda renders (no local filesystem) — the alias map serves as fallback there
|
||||
|
||||
---
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
**KTD-1. System font locator lives in `@hyperframes/core` as a shared module.**
|
||||
Both the compiler (producer) and the Studio API (core) need to locate font files. The Studio API's `GET /fonts` endpoint already scans OS font directories but only returns family names. The new module extends this to return file paths, keyed by normalized family name. The Studio API re-exports it; the producer imports it for the capture step.
|
||||
|
||||
**KTD-2. woff2 compression via the `wawoff2` npm package (WASM-based).**
|
||||
System fonts are typically ttf/otf (500KB-2MB per weight). Embedding raw ttf as data URIs would bloat compositions. `wawoff2` is a WASM port of Google's woff2 reference implementation — works cross-platform without native binaries, supports Node.js and Bun. Alternative considered: shelling out to `woff2_compress` (faster but requires system installation, fails on Lambda). `wawoff2` is the right tradeoff — zero system requirements, ~10ms per font file.
|
||||
|
||||
**KTD-3. System font capture is a new Path 3 inside `buildFontFaceCss`, after Google Fonts, before unresolved.**
|
||||
This ordering ensures bundled fonts (zero-latency, deterministic) and Google Fonts (cached, consistent) take priority. System font capture is the fallback for fonts that can't be found anywhere else. The capture step only runs when `allowSystemFontCapture` is true (set by the compiler but not by distributed render adapters).
|
||||
|
||||
**KTD-4. External stylesheet inlining extends `localizeRemoteFontFaces` rather than creating a new pipeline step.**
|
||||
The function already downloads remote font URLs from @font-face blocks. Extending it to also fetch external `<link rel="stylesheet">` content, extract @font-face rules, and inline their font URLs is a natural extension that keeps font localization in one place. Google Fonts `<link>` tags are excluded (the deterministic injector already handles those).
|
||||
|
||||
**KTD-5. Studio auto-import uses the server-side font locator via `GET /fonts/file` API.**
|
||||
The browser's `window.queryLocalFonts()` API already triggers auto-import for Local-source fonts. For System-source fonts (from `DEFAULT_FONT_FAMILIES`), a new API endpoint `GET /fonts/file?family=<name>` returns the font file binary, which the Studio can then import into the project. This bridges the gap where the browser API is unavailable (Safari, Firefox) or the font isn't in the browser's Local Font Access list.
|
||||
|
||||
---
|
||||
|
||||
## High-Level Technical Design
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[font-family declaration in HTML] --> B{Has @font-face?}
|
||||
B -- yes --> Z[Done — use existing]
|
||||
B -- no --> C{In FONT_ALIAS_MAP?}
|
||||
C -- yes --> D[Emit bundled data URI]
|
||||
D --> D1[Fill weight gaps from Google Fonts]
|
||||
C -- no --> E{On Google Fonts?}
|
||||
E -- yes --> F[Fetch + cache + embed as data URI]
|
||||
E -- no --> G{allowSystemFontCapture?}
|
||||
G -- yes --> H{Found on local filesystem?}
|
||||
H -- yes --> I[Read file → woff2 compress → embed as data URI]
|
||||
H -- no --> J{In FONT_ALIAS_MAP as alias?}
|
||||
G -- no --> J
|
||||
J -- yes --> K[Use aliased bundled font — fallback]
|
||||
J -- no --> L[Unresolved — actionable error]
|
||||
|
||||
style I fill:#2d6a4f,color:#fff
|
||||
style G fill:#264653,color:#fff
|
||||
style L fill:#9b2226,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
### In Scope
|
||||
- System font locator module (macOS, Windows, Linux)
|
||||
- woff2 compression integration
|
||||
- New resolution path in `buildFontFaceCss`
|
||||
- External stylesheet font inlining in `localizeRemoteFontFaces`
|
||||
- Studio font file API endpoint
|
||||
- Studio auto-import for system fonts
|
||||
- Tests for all of the above
|
||||
|
||||
### Out of Scope (Non-Goals)
|
||||
- Font subsetting (embedding only the glyphs used — separate optimization concern)
|
||||
- Variable font axis resolution (the capture step embeds the full file as-is)
|
||||
- CJK font capture (Noto CJK fonts are 15-20MB; embedding them as data URIs is impractical)
|
||||
- Redistributing commercially licensed fonts in CI artifacts (the capture happens on the author's machine; the composition is a rendering artifact)
|
||||
|
||||
### Deferred to Follow-Up Work
|
||||
- Font file caching across compositions (currently each composition embeds independently)
|
||||
- Consolidating the four diverging `GENERIC_FAMILIES` sets into one shared module
|
||||
- Runtime validation that all FONT_ALIAS_MAP values are valid CANONICAL_FONTS keys
|
||||
|
||||
---
|
||||
|
||||
## Implementation Units
|
||||
|
||||
### U1. System font locator module
|
||||
|
||||
**Goal:** Create a shared module that locates a font file on the local filesystem given a family name, returning the file path and format.
|
||||
|
||||
**Requirements:** R1, R2
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- `packages/core/src/fonts/systemFontLocator.ts` (create)
|
||||
- `packages/core/src/fonts/systemFontLocator.test.ts` (create)
|
||||
- `packages/core/package.json` (add subpath export `./fonts/system-locator`)
|
||||
|
||||
**Approach:** Extract and extend the directory scanning logic from `packages/core/src/studio-api/routes/fonts.ts`. The existing `fontDirectories()` and `collectFontsFromDir` scan directories and derive family names from filenames. The new module adds:
|
||||
|
||||
- A `locateSystemFont(family: string)` function that returns `{ path: string; format: "ttf" | "otf" | "woff2" | "woff" } | null`
|
||||
- On macOS: first try `system_profiler SPFontsDataType -json` to get authoritative file paths (the JSON includes `path` per typeface), fall back to directory scan
|
||||
- On Windows: scan `%WINDIR%\Fonts` matching by filename-derived family name
|
||||
- On Linux: first try `fc-match "<family>" --format="%{file}"` (fontconfig), fall back to directory scan
|
||||
- Match by normalized family name (case-insensitive, style-suffix-stripped)
|
||||
- Prefer woff2 > otf > ttf when multiple formats exist for the same family
|
||||
- Cache results in a module-level Map (same pattern as the Studio API's `cachedFonts`)
|
||||
|
||||
**Patterns to follow:** `fontDirectories()` and `collectFontsFromDir` in `packages/core/src/studio-api/routes/fonts.ts`. `normalizeFamilyName` in `deterministicFonts.ts`.
|
||||
|
||||
**Test scenarios:**
|
||||
- `locateSystemFont("Inter")` returns null on a system without Inter installed (or returns a path if installed — test should handle both)
|
||||
- `locateSystemFont("nonexistent-font-xyz")` returns null
|
||||
- `locateSystemFont` normalizes case: `"SF MONO"` matches the same as `"SF Mono"`
|
||||
- On macOS (CI), `locateSystemFont("Helvetica")` returns a valid path under `/System/Library/Fonts/`
|
||||
- The format detection correctly identifies `.ttf`, `.otf`, and `.woff2` extensions
|
||||
- Directory scan respects max depth of 2 (matching existing pattern)
|
||||
- Cache returns the same result on repeated calls without re-scanning
|
||||
|
||||
**Verification:** `bun run --cwd packages/core test` passes. Module exports are accessible via `@hyperframes/core/fonts/system-locator`.
|
||||
|
||||
---
|
||||
|
||||
### U2. woff2 compression capability
|
||||
|
||||
**Goal:** Add the ability to compress ttf/otf font files to woff2 format for efficient data URI embedding.
|
||||
|
||||
**Requirements:** R3
|
||||
|
||||
**Dependencies:** None (independent of U1)
|
||||
|
||||
**Files:**
|
||||
- `packages/producer/package.json` (add `wawoff2` dependency)
|
||||
- `packages/producer/src/services/fontCompression.ts` (create)
|
||||
- `packages/producer/src/services/fontCompression.test.ts` (create)
|
||||
|
||||
**Approach:** Add the `wawoff2` npm package (WASM-based woff2 compressor). Create a thin wrapper:
|
||||
|
||||
- `compressToWoff2(input: Buffer): Promise<Buffer>` — compress a ttf/otf buffer to woff2
|
||||
- `fontToDataUri(input: Buffer, originalFormat: string): Promise<string>` — compress if needed, then base64 encode as `data:font/woff2;base64,...`
|
||||
- If the input is already woff2, skip compression and just encode
|
||||
- Handle compression failures gracefully — fall back to embedding the raw format (`data:font/truetype;base64,...` for ttf) with a console warning
|
||||
|
||||
**Patterns to follow:** `fontDataUri()` in `deterministicFonts.ts` for the data URI encoding pattern.
|
||||
|
||||
**Test scenarios:**
|
||||
- `compressToWoff2` accepts a ttf Buffer and returns a smaller woff2 Buffer
|
||||
- `fontToDataUri` with a ttf input returns a `data:font/woff2;base64,...` string
|
||||
- `fontToDataUri` with a woff2 input returns a `data:font/woff2;base64,...` string without re-compression
|
||||
- `fontToDataUri` with an otf input returns a compressed woff2 data URI
|
||||
- Compression failure falls back to raw format data URI with a warning
|
||||
|
||||
**Verification:** `bun run --cwd packages/producer test` passes. `wawoff2` is listed in dependencies.
|
||||
|
||||
---
|
||||
|
||||
### U3. System font resolution in buildFontFaceCss
|
||||
|
||||
**Goal:** Add a new resolution path that locates and embeds local system fonts when the bundled and Google Fonts paths fail.
|
||||
|
||||
**Requirements:** R1, R5, R8
|
||||
|
||||
**Dependencies:** U1, U2
|
||||
|
||||
**Files:**
|
||||
- `packages/producer/src/services/deterministicFonts.ts` (modify `buildFontFaceCss` and `injectDeterministicFontFaces`)
|
||||
- `packages/producer/src/services/deterministicFonts.test.ts` (extend)
|
||||
|
||||
**Approach:** Add a Path 3 inside `buildFontFaceCss`, after the Google Fonts fetch (Path 2) returns empty:
|
||||
|
||||
1. Check `options.allowSystemFontCapture` — if false (distributed render), skip to unresolved
|
||||
2. Call `locateSystemFont(originalCaseFamily)` from the system font locator
|
||||
3. If found: read the font file, call `fontToDataUri` to compress and encode, emit a single `@font-face` rule with the data URI
|
||||
4. If not found: fall through to unresolved (existing behavior)
|
||||
|
||||
Add `allowSystemFontCapture: boolean` to `InjectDeterministicFontFacesOptions`. Default to `true` in `compileForRender` (local renders). Set to `false` in distributed render adapters (`packages/producer/src/services/render/` and `packages/aws-lambda/`).
|
||||
|
||||
**Patterns to follow:** The Google Fonts fetch path in `buildFontFaceCss` (Path 2). The `buildFontFaceRule` helper for emitting @font-face CSS.
|
||||
|
||||
**Test scenarios:**
|
||||
- With `allowSystemFontCapture: true` and a font that exists on the local system, the function emits a @font-face rule with a data URI
|
||||
- With `allowSystemFontCapture: false`, the system font path is skipped even if the font exists locally
|
||||
- A font already covered by @font-face is not re-captured
|
||||
- A font in FONT_ALIAS_MAP is resolved via the alias, not via system capture
|
||||
- A font available on Google Fonts is resolved via Google, not via system capture
|
||||
- A font not found anywhere (not aliased, not on Google, not on system) is added to the unresolved list
|
||||
- The emitted @font-face rule has `font-display: block` (matching existing rules)
|
||||
- The resolution order is verified: alias → Google → system → unresolved
|
||||
|
||||
**Verification:** `bun run --cwd packages/producer test` passes. Render a composition referencing a local-only font and confirm the @font-face rule appears in the compiled output.
|
||||
|
||||
---
|
||||
|
||||
### U4. External stylesheet font inlining
|
||||
|
||||
**Goal:** Fetch non-Google external `<link rel="stylesheet">` content, extract @font-face rules, and inline their font URLs as data URIs.
|
||||
|
||||
**Requirements:** R4
|
||||
|
||||
**Dependencies:** None (independent of U1-U3)
|
||||
|
||||
**Files:**
|
||||
- `packages/producer/src/services/htmlCompiler.ts` (modify `localizeRemoteFontFaces`)
|
||||
- `packages/producer/src/services/htmlCompiler.test.ts` (extend)
|
||||
|
||||
**Approach:** Extend `localizeRemoteFontFaces` to also handle external stylesheet `<link>` tags:
|
||||
|
||||
1. Before the existing @font-face URL scan, find all `<link rel="stylesheet" href="https://...">` tags in the HTML
|
||||
2. Exclude Google Fonts URLs (already handled by the deterministic injector)
|
||||
3. Fetch each external stylesheet's CSS content
|
||||
4. Extract @font-face blocks from the fetched CSS
|
||||
5. For each @font-face block, download the woff2/ttf URLs referenced in `src: url(...)`
|
||||
6. Rewrite the CSS with local paths (same as existing behavior for @font-face URLs)
|
||||
7. Inject the fetched @font-face rules into the HTML as a `<style>` block, replacing the `<link>` tag
|
||||
8. Handle fetch failures gracefully — keep the `<link>` tag if the stylesheet can't be fetched (network access at render time is the fallback)
|
||||
|
||||
**Patterns to follow:** The existing `downloadAndRewriteUrls` helper. The `REMOTE_FONTFACE_URL_RE` pattern for extracting URLs from @font-face blocks.
|
||||
|
||||
**Test scenarios:**
|
||||
- A `<link>` tag pointing to a non-Google font CDN is fetched, its @font-face rules are extracted, and font URLs are inlined
|
||||
- A `<link>` tag pointing to Google Fonts is left untouched (handled by deterministic injector)
|
||||
- A `<link>` tag that fails to fetch is kept in the HTML with a console warning
|
||||
- Multiple `<link>` tags are processed independently
|
||||
- The `<link>` tag is replaced with a `<style>` block containing the inlined @font-face rules
|
||||
- Non-font `<link>` tags (rel="icon", rel="preconnect") are not touched
|
||||
|
||||
**Verification:** `bun run --cwd packages/producer test` passes. A composition with a `<link>` tag to a non-Google font CDN produces inlined @font-face rules in the compiled output.
|
||||
|
||||
---
|
||||
|
||||
### U5. Studio font file API and auto-import
|
||||
|
||||
**Goal:** Add a server-side API endpoint that returns font file binary data, and wire Studio's font picker to auto-import system fonts on selection.
|
||||
|
||||
**Requirements:** R6
|
||||
|
||||
**Dependencies:** U1 (needs system font locator)
|
||||
|
||||
**Files:**
|
||||
- `packages/core/src/studio-api/routes/fonts.ts` (add `GET /fonts/file` route)
|
||||
- `packages/studio/src/components/editor/propertyPanelFont.tsx` (modify `commitFamily` for System-source fonts)
|
||||
- `packages/studio/src/components/editor/propertyPanelHelpers.ts` (no change expected)
|
||||
|
||||
**Approach:**
|
||||
|
||||
Server side: Add `GET /fonts/file?family=<name>` route that:
|
||||
1. Calls `locateSystemFont(family)` from the shared locator module
|
||||
2. If found: reads the file and returns it with appropriate Content-Type (`font/ttf`, `font/otf`, `font/woff2`) and Content-Disposition header
|
||||
3. If not found: returns 404
|
||||
|
||||
Client side: In `commitFamily`, when `option.source === "System"` and the font is not a CSS generic (sans-serif, monospace, etc.):
|
||||
1. Fetch `GET /fonts/file?family=<family>`
|
||||
2. If the response is OK: create a File object from the blob, pass to `onImportFonts([file])`, inject the @font-face stylesheet
|
||||
3. If 404 or error: fall through to existing behavior (just commit the font-family name)
|
||||
|
||||
This means system fonts selected in Studio become project-embedded files — the composition is self-contained before it ever reaches the compiler.
|
||||
|
||||
**Patterns to follow:** The existing `GET /fonts` and `GET /fonts/google` routes. The `importLocalFont` function in `propertyPanelFont.tsx` for the client-side import flow.
|
||||
|
||||
**Test scenarios:**
|
||||
- `GET /fonts/file?family=Helvetica` returns a font file with correct Content-Type on macOS
|
||||
- `GET /fonts/file?family=nonexistent` returns 404
|
||||
- `GET /fonts/file` without a family parameter returns 400
|
||||
- Studio: selecting a System-source font triggers a fetch to `/fonts/file` and imports the result
|
||||
- Studio: selecting a System-source font that returns 404 falls through to commit the name directly
|
||||
- Studio: selecting a CSS generic (sans-serif) does not trigger the font file fetch
|
||||
|
||||
**Verification:** `bun run build` succeeds. Studio font picker auto-imports system fonts on selection (manual verification).
|
||||
|
||||
---
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
- **woff2 compression adds ~2MB to the producer package** (WASM binary). Acceptable for a build-time tool.
|
||||
- **System font file paths vary by OS and version.** The locator uses multiple strategies (system_profiler, fontconfig, directory scan) to maximize coverage, but some fonts in unusual locations may not be found.
|
||||
- **Font licensing** — system fonts like SF Pro are licensed for use on Apple platforms. Embedding them in a video composition (a static artifact rendered on the author's machine) is standard fair use, but the capture step should not redistribute font files to other machines. The `allowSystemFontCapture: false` flag in distributed renders prevents this.
|
||||
- **Large font files** — some fonts (especially CJK families) are 15-20MB. CJK fonts are explicitly out of scope; a size cap (e.g., 5MB per font file) should be enforced to prevent accidentally embedding oversized files.
|
||||
|
||||
---
|
||||
|
||||
## Sources & Research
|
||||
|
||||
- `packages/core/src/studio-api/routes/fonts.ts` — existing OS font directory scanning
|
||||
- `packages/cli/src/capture/fontMetadataExtractor.ts` — fontkit-based font metadata extraction
|
||||
- `packages/producer/src/services/deterministicFonts.ts` — `injectDeterministicFontFaces` and `buildFontFaceCss` resolution pipeline
|
||||
- `packages/producer/src/services/htmlCompiler.ts` — `localizeRemoteFontFaces`, `promoteCssImportsToLinkTags`, `compileForRender` font operation sequence
|
||||
- `packages/studio/src/components/editor/propertyPanelFont.tsx` — Studio font picker with Local/System/Google sources
|
||||
- `wawoff2` npm package — WASM-based woff2 compression, cross-platform, no native dependencies
|
||||
Reference in New Issue
Block a user