Files
hyperframes/packages/studio/src/captions/components/CaptionOverlayUtils.ts
T
Vance IngallsandClaude Opus 5 254de3d1c4 fix(studio): captions UX — mode exit, undo, autosave surfacing, honest gating (#1968)
Caption-editing fixes from the studio UX review. This surface held five of
the thirteen criticals; the theme is that the editing UI shipped ahead of
its apply/persist pipeline, so several controls mutated an in-memory model
with no downstream effect, and the mode itself could never be exited.

Mode trap: caption edit mode auto-activated on detection and had no exit —
`setEditMode(false)` and `reset()` had zero call sites, so the caption
overlay replaced normal element editing for the rest of the session, even
after switching compositions. The store now resets on composition change
(flushing the last debounced edit first), an "Editing captions · Exit" pill
sits on the preview, and a re-enter button appears once dismissed.

Honest gating of dead surfaces: the Animation tab (31 presets ×
duration/ease/stagger/intensity) edited state that was never applied to
playback nor serialized — wiring it needs a CaptionOverride schema
extension in packages/core plus a runtime engine, so the tab is now visibly
disabled with an amber "isn't applied to playback or saved yet" notice
instead of silently discarding work. Timing edge-drags moved a block that
never changed playback and never saved; the handles are gone and the blocks
remain as select/seek targets. Double-click split desynced the overlay↔DOM
index mapping, so split is out until regeneration exists.

Undo: store-level undo/redo (cap 50, 800ms coalescing by edit target)
across all ten mutations, with ⌘Z/⇧⌘Z intercepted while caption mode is
active and reapplied to the live iframe. Previously ⌘Z reverted an
unrelated file edit while the bad caption drag persisted.

Autosave: save failures, including non-2xx, raise a persistent "not
saved — Retry" banner; the code's own comment called this a data-loss path
and it was telemetry-only. Debounced saves flush on unmount instead of
being discarded, `beforeunload` flushes and warns while pending, and
corrupt overrides JSON is distinguished from a missing file.

Input safety and a11y: arrow-key nudge no longer hijacks arrows inside
form inputs; numeric fields commit finite values only (typing "-" used to
inject NaN into gsap and persist null); "Mixed" shows on multi-select
divergence; Escape cancels an in-flight drag and restores the pre-drag
transform; ⌘A selects all; caption blocks are keyboard-selectable with a
playhead line and click-to-seek (CaptionTimeline's `onSeek` prop existed
but nothing passed it); 24px hit areas around the 8px handles; a hint when
no boxes are visible; visible input focus styles; tablist semantics.

Perf: the 66ms getBoundingClientRect polling loop is replaced with
event-driven updates (player-store subscription, preview messages,
ResizeObserver, rAF-coalesced); the interval now runs only during playback.

Reconciled against main: StudioPreviewArea.tsx was deleted by the Studio
revamp (#2291), so the mode pill, the sync-error banner and the re-enter
button move to its successor, nle/PreviewOverlays.tsx, and the caption
track's onSeek is wired in EditorShell. The per-keyframe
onChangeKeyframeEase change that also lived in that file is dropped:
main removed the prop, and #1967 now routes the diamond menu's ease action
to the focused-ease-segment editor instead.

Restacked onto main now that PRs 1962-1967 have squash-merged, so this
carries only its own changes.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 11:37:28 -07:00

276 lines
9.3 KiB
TypeScript

// DOM helpers for CaptionOverlay — word box reading, transform I/O, wrapper management
export interface WordBox {
segmentId: string;
groupId: string;
groupIndex: number;
wordIndex: number;
x: number;
y: number;
width: number;
height: number;
}
export function readWordBoxes(
iframe: HTMLIFrameElement,
model: {
groupOrder: string[];
groups: Map<string, { segmentIds: string[] }>;
},
overlayEl: HTMLElement,
): WordBox[] {
let doc: Document | null = null;
let win: Window | null = null;
try {
doc = iframe.contentDocument;
win = iframe.contentWindow;
} catch {
return [];
}
if (!doc || !win) return [];
const iframeDisplayRect = iframe.getBoundingClientRect();
const overlayRect = overlayEl.getBoundingClientRect();
const nativeW = parseFloat(iframe.style.width) || iframeDisplayRect.width;
const cssScale = iframeDisplayRect.width / nativeW;
const offsetX = iframeDisplayRect.left - overlayRect.left;
const offsetY = iframeDisplayRect.top - overlayRect.top;
const groupEls = doc.querySelectorAll<HTMLElement>(".caption-group");
const boxes: WordBox[] = [];
for (let gi = 0; gi < model.groupOrder.length; gi++) {
const groupId = model.groupOrder[gi];
const group = model.groups.get(groupId);
if (!group) continue;
const groupEl = groupEls[gi] as HTMLElement | undefined;
if (!groupEl) continue;
const computed = win.getComputedStyle(groupEl);
if (parseFloat(computed.opacity) <= 0.01 || computed.visibility === "hidden") continue;
const resolvedWordEls: HTMLElement[] = [];
for (const child of groupEl.children) {
const c = child as HTMLElement;
if (c.dataset.captionWrapper === "true") {
const inner = c.querySelector<HTMLElement>(":scope > span");
if (inner) resolvedWordEls.push(inner);
} else if (c.tagName === "SPAN") {
resolvedWordEls.push(c);
}
}
if (resolvedWordEls.length === 0 && groupEl.textContent?.trim()) {
const textNode = groupEl.childNodes[0];
if (textNode && textNode.nodeType === Node.TEXT_NODE) {
const words = (textNode.textContent || "").split(/\s+/).filter(Boolean);
const frag = doc.createDocumentFragment();
for (const word of words) {
const span = doc.createElement("span");
span.textContent = word + " ";
span.style.display = "inline";
frag.appendChild(span);
resolvedWordEls.push(span);
}
groupEl.replaceChild(frag, textNode);
} else {
const singleSpan = groupEl.querySelector<HTMLElement>(":scope > span");
if (singleSpan && singleSpan.textContent?.trim()) {
const words = singleSpan.textContent.split(/\s+/).filter(Boolean);
const frag = doc.createDocumentFragment();
for (const word of words) {
const span = doc.createElement("span");
span.textContent = word + " ";
span.style.display = "inline";
frag.appendChild(span);
resolvedWordEls.push(span);
}
singleSpan.replaceWith(frag);
}
}
}
for (let wi = 0; wi < group.segmentIds.length; wi++) {
const segId = group.segmentIds[wi];
const wordEl = resolvedWordEls[wi] as HTMLElement | undefined;
if (!wordEl) continue;
const rect = wordEl.getBoundingClientRect();
boxes.push({
segmentId: segId,
groupId,
groupIndex: gi,
wordIndex: wi,
x: rect.left * cssScale + offsetX,
y: rect.top * cssScale + offsetY,
width: rect.width * cssScale,
height: rect.height * cssScale,
});
}
}
return boxes;
}
export function getWordEl(
iframe: HTMLIFrameElement,
groupIndex: number,
wordIndex: number,
): HTMLElement | null {
let doc: Document | null = null;
try {
doc = iframe.contentDocument;
} catch {
return null;
}
if (!doc) return null;
const groupEl = doc.querySelectorAll<HTMLElement>(".caption-group")[groupIndex];
if (!groupEl) return null;
const wordEls: HTMLElement[] = [];
for (const child of groupEl.children) {
const el = child as HTMLElement;
if (el.dataset.captionWrapper === "true") {
const inner = el.querySelector<HTMLElement>(":scope > span");
if (inner) wordEls.push(inner);
} else if (el.tagName === "SPAN") {
wordEls.push(el);
}
}
return wordEls[wordIndex] ?? null;
}
/**
* Read GSAP's internal transform state for an element.
* GSAP stores transforms in its own cache, not in el.style.transform.
*/
export function readGsapTransform(
el: HTMLElement,
iframeWin: Window,
): { x: number; y: number; scale: number; rotation: number } {
const gsap = (
iframeWin as unknown as { gsap?: { getProperty?: (el: HTMLElement, prop: string) => number } }
).gsap;
if (gsap && gsap.getProperty) {
return {
x: gsap.getProperty(el, "x") || 0,
y: gsap.getProperty(el, "y") || 0,
scale: gsap.getProperty(el, "scale") || 1,
rotation: gsap.getProperty(el, "rotation") || 0,
};
}
const t = el.style.transform || "";
const scaleMatch = t.match(/scale\(([^)]+)\)/);
const rotMatch = t.match(/rotate\(([^)]+)deg\)/);
const txyMatch = t.match(/translate\(([^,]+)px,\s*([^)]+)px\)/);
return {
x: txyMatch ? parseFloat(txyMatch[1]) : 0,
y: txyMatch ? parseFloat(txyMatch[2]) : 0,
scale: scaleMatch ? parseFloat(scaleMatch[1]) : 1,
rotation: rotMatch ? parseFloat(rotMatch[1]) : 0,
};
}
/**
* Get or create an inline-block wrapper span around a word element.
* Transforms are applied to the wrapper so the word's GSAP animations are preserved.
*/
export function getOrCreateWrapper(el: HTMLElement): HTMLElement {
if (el.dataset.captionWrapper === "true") return el;
const parent = el.parentElement;
if (parent && parent.dataset.captionWrapper === "true") return parent;
const doc = el.ownerDocument;
const wrapper = doc.createElement("span");
wrapper.style.display = "inline-block";
wrapper.dataset.captionWrapper = "true";
el.parentNode?.insertBefore(wrapper, el);
wrapper.appendChild(el);
return wrapper;
}
/**
* Write transform values to a wrapper span around the word element.
*/
export function writeTransform(
el: HTMLElement,
iframeWin: Window,
x: number,
y: number,
scale: number,
rotation: number,
) {
const wrapper = getOrCreateWrapper(el);
const gsap = (
iframeWin as unknown as {
gsap?: { set?: (el: HTMLElement, props: Record<string, number>) => void };
}
).gsap;
if (gsap && gsap.set) {
gsap.set(wrapper, { x, y, scale, rotation });
} else {
wrapper.style.transform = `translate(${x.toFixed(1)}px, ${y.toFixed(1)}px) rotate(${rotation.toFixed(1)}deg) scale(${scale.toFixed(3)})`;
}
}
// ---------------------------------------------------------------------------
// Iframe registry — lets non-component code (undo/redo in useAppHotkeys)
// reapply a restored model's transforms to the live preview.
// ---------------------------------------------------------------------------
let registeredIframe: React.RefObject<HTMLIFrameElement | null> | null = null;
export function registerCaptionIframe(ref: React.RefObject<HTMLIFrameElement | null>): () => void {
registeredIframe = ref;
return () => {
if (registeredIframe === ref) registeredIframe = null;
};
}
/**
* True when the caption preview iframe is mounted AND visible. The caption
* store's isEditMode stays true while the preview is merely hidden (e.g.
* storyboard view), so hotkeys must not route ⌘Z to the caption stack unless
* the user can actually see the captions the undo would change.
*/
export function isCaptionPreviewVisible(): boolean {
const iframe = registeredIframe?.current;
return Boolean(iframe?.isConnected && iframe.offsetParent !== null);
}
/** Reapply every segment's transform from a (restored) model to the preview DOM. */
export function applyCaptionModelToIframe(model: {
groupOrder: string[];
groups: Map<string, { segmentIds: string[] }>;
segments: Map<string, { style: { x?: number; y?: number; scaleX?: number; rotation?: number } }>;
}): void {
const iframe = registeredIframe?.current;
if (!iframe) return;
let win: Window | null = null;
try {
win = iframe.contentWindow;
} catch {
return;
}
if (!win) return;
for (let gi = 0; gi < model.groupOrder.length; gi++) {
const group = model.groups.get(model.groupOrder[gi]);
if (!group) continue;
for (let wi = 0; wi < group.segmentIds.length; wi++) {
const seg = model.segments.get(group.segmentIds[wi]);
if (!seg) continue;
const wordEl = getWordEl(iframe, gi, wi);
if (!wordEl) continue;
const s = seg.style;
writeTransform(wordEl, win, s.x ?? 0, s.y ?? 0, s.scaleX ?? 1, s.rotation ?? 0);
}
}
}
/** Compute style deltas from the current wrapper transform — used by syncToStore in the overlay. */
export function computeTransformStyle(el: HTMLElement, iframeWin: Window): Record<string, number> {
const wrapper = getOrCreateWrapper(el);
const { x, y, scale, rotation } = readGsapTransform(wrapper, iframeWin);
const style: Record<string, number> = {};
if (Math.abs(x) > 0.5) style.x = x;
if (Math.abs(y) > 0.5) style.y = y;
if (Math.abs(scale - 1) > 0.001) {
style.scaleX = scale;
style.scaleY = scale;
}
if (Math.abs(rotation) > 0.1) style.rotation = rotation;
return style;
}