fix(studio): address caption designer PR feedback (#200)

* fix(studio): address caption designer PR feedback

Fixes from review comments on feature/caption-designer (#180):

- fix(generator): guard named colors in hexToRgba — "red", "transparent"
  no longer produce NaN rgba values
- fix(sync): log auto-save failures instead of silently swallowing them
- fix(sync): check res.ok before parsing caption-overrides response
- refactor(components): extract Section, Row, inputCls into shared.tsx
  to eliminate duplication between CaptionPropertyPanel and
  CaptionAnimationPanel
- fix(store): replace non-deterministic Date.now()+Math.random() ID with
  counter-based group IDs
- fix(store): read selectedGroupId from state param instead of get() to
  avoid stale reads in batched set() calls
- fix(overlay): remove cssScale multiplier from getBoundingClientRect
  coords — the browser already accounts for CSS transforms
- docs(parser): add comment explaining the lazy ];  regex assumption

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): address remaining caption designer feedback

Overlay: handle both per-word spans (generator output) and grouped text
nodes (existing templates). Wraps text nodes into individual spans on
demand so the overlay can target words in any caption format.

Property panel: add Typography (font, size, weight, spacing) and Color
(color, active, dim, opacity) sections alongside existing Position and
Transform controls.

Timeline: move caption timeline into a dedicated flex-shrink-0 section
below the main timeline tracks instead of inside the scrollable area.
Gives it fixed 60px height that's always visible.

Caption overrides: classify color tweens by comparing target color to
the dim baseline instead of relying on timeline position order. This
handles compositions with custom color tweens correctly.

App.tsx: remove polling interval, rely on runtime postMessage events
for caption detection. Add clarifying comment on why useEffect is
appropriate (external event subscription).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): restore cssScale in overlay coordinate conversion

getBoundingClientRect() on iframe-internal elements returns coordinates
in the iframe's native resolution (1920x1080), not the CSS-scaled
display size. The cssScale multiplier is needed to convert to parent
window coordinates. The earlier removal was incorrect — it only worked
at 1:1 scale.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): fix reversed scaling on left-side corner handles

Scale interaction used horizontal dx from start position, which goes
negative when dragging left handles outward. Now uses distance from box
center — dragging away from center increases scale regardless of which
corner handle is used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): make rotation respond to horizontal drag only

Rotation handle sits directly above the word, so atan2-based rotation
barely responds to left/right movement. Replace with linear horizontal
mapping: drag right = clockwise, drag left = counter-clockwise,
200px = 90 degrees. Vertical movement is ignored.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(studio): remove animation tab and typography/color from property panel

Keep only Position (X, Y) and Transform (Scale, Rotation) controls.
Remove tab switcher UI since there's only one view now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix oxfmt formatting in CLAUDE.md and captions skill docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-04-02 11:38:35 -07:00
committed by GitHub
co-authored by Claude Opus 4.6
parent d36c1785b9
commit 5e2781b459
15 changed files with 433 additions and 321 deletions
Submodule .claude/worktrees/agent-a692178b deleted from 1e4c101fb4
-1
View File
@@ -45,7 +45,6 @@ npx skills add greensock/gsap-skills # GSAP skills
Uses [vercel-labs/skills](https://github.com/vercel-labs/skills). Installs to Claude Code, Gemini CLI, and Codex CLI by default. Pass `-a <agent>` for other targets.
## Project Overview
Open-source video rendering framework: write HTML, render video.
+18 -13
View File
@@ -83,25 +83,30 @@ export function applyCaptionOverrides(): void {
if (override.fontWeight !== undefined) styleProps.fontWeight = override.fontWeight;
if (override.fontFamily !== undefined) styleProps.fontFamily = override.fontFamily;
// Replace color values in existing GSAP tweens by timeline order.
// For any word, color tweens follow: dim (setup) → active (spoken) → after.
// Sort by startTime and assign by position, not by content heuristics.
// Replace color values in existing GSAP tweens.
// Instead of relying on timeline position order (fragile if custom
// color tweens exist), we classify each tween by comparing its
// target color to the current computed color of the element.
// Tweens that match the current color are "dim" tweens; tweens
// with a different color are "active" tweens.
if (override.activeColor || override.dimColor) {
const allTweens = gsap.getTweensOf(el);
const colorTweens = allTweens
.filter((tw) => tw.vars.color !== undefined)
.sort((a, b) => a.startTime() - b.startTime());
for (let i = 0; i < colorTweens.length; i++) {
if (i === 0 && override.dimColor) {
// First color tween = dim setup
colorTweens[i].vars.color = override.dimColor;
} else if (i === 1 && override.activeColor) {
// Second color tween = active/spoken
colorTweens[i].vars.color = override.activeColor;
} else if (i >= 2 && override.dimColor) {
// Third+ = after/deactivate (use dim color)
colorTweens[i].vars.color = override.dimColor;
// Use the first tween's color as the dim baseline — if no tweens,
// fall back to computed style.
const dimBaseline = colorTweens.length > 0 ? String(colorTweens[0].vars.color) : "";
for (const tw of colorTweens) {
const tweenColor = String(tw.vars.color);
if (tweenColor === dimBaseline) {
// This tween targets the dim/inactive color
if (override.dimColor) tw.vars.color = override.dimColor;
} else {
// This tween targets the active/spoken color
if (override.activeColor) tw.vars.color = override.activeColor;
}
}
+18 -18
View File
@@ -65,20 +65,18 @@ export function StudioApp() {
const [rightWidth, setRightWidth] = useState(400);
const [leftCollapsed, setLeftCollapsed] = useState(false);
const [rightCollapsed, setRightCollapsed] = useState(true);
// Auto-enter caption edit mode when viewing a captions composition
// Auto-enter caption edit mode when the iframe contains .caption-group elements.
// Listens for the runtime's postMessage events (state/timeline) which fire after
// all compositions are loaded, then checks for caption groups.
// This is a subscription to external events (postMessage from runtime) — useEffect
// is appropriate here. The runtime fires "state"/"timeline" messages after all
// compositions load, which triggers caption detection.
// eslint-disable-next-line no-restricted-syntax
useEffect(() => {
if (!projectId) return;
let pollId: ReturnType<typeof setInterval> | null = null;
let activating = false;
const tryActivateCaptions = () => {
if (useCaptionStore.getState().isEditMode || activating) {
if (pollId) { clearInterval(pollId); pollId = null; }
return;
}
@@ -88,7 +86,9 @@ export function StudioApp() {
try {
doc = iframe?.contentDocument ?? null;
win = iframe?.contentWindow ?? null;
} catch { return; }
} catch {
return;
}
if (!doc || !win) return;
const groups = doc.querySelectorAll(".caption-group");
@@ -102,7 +102,8 @@ export function StudioApp() {
// Strategy 1: data-composition-src or data-composition-file attributes
const compHosts = doc.querySelectorAll("[data-composition-src], [data-composition-file]");
for (const host of compHosts) {
const src = host.getAttribute("data-composition-src") || host.getAttribute("data-composition-file");
const src =
host.getAttribute("data-composition-src") || host.getAttribute("data-composition-file");
if (src && src.includes("captions")) {
captionSrcPath = src;
break;
@@ -154,7 +155,9 @@ export function StudioApp() {
captionSync.loadOverrides();
})
.catch(() => {})
.finally(() => { activating = false; });
.finally(() => {
activating = false;
});
};
// Listen for runtime messages that signal composition loading is complete
@@ -168,14 +171,11 @@ export function StudioApp() {
window.addEventListener("message", handleMessage);
// Try immediately in case compositions are already loaded
tryActivateCaptions();
// Poll until captions are detected — sub-composition scripts run async
pollId = setInterval(tryActivateCaptions, 200);
return () => {
window.removeEventListener("message", handleMessage);
if (pollId) clearInterval(pollId);
};
}, [activeCompPath, projectId, compIdToSrc]);
}, [activeCompPath, projectId, compIdToSrc, captionSync]);
// Auto-expand right panel when a caption word is selected
// eslint-disable-next-line no-restricted-syntax
@@ -296,7 +296,6 @@ export function StudioApp() {
const previewIframeRef = useRef<HTMLIFrameElement | null>(null);
const consoleErrorsRef = useRef<LintFinding[]>([]);
// Listen for external file changes (user editing HTML outside the editor).
// In dev: use Vite HMR. In embedded/production: use SSE from /api/events.
useMountEffect(() => {
@@ -850,14 +849,15 @@ export function StudioApp() {
});
}}
previewOverlay={
captionEditMode ? (
<CaptionOverlay iframeRef={previewIframeRef} />
) : undefined
captionEditMode ? <CaptionOverlay iframeRef={previewIframeRef} /> : undefined
}
timelineFooter={
captionEditMode ? (
<div className="border-t border-neutral-800/30">
<div className="flex items-center gap-1.5 px-2 py-1">
<div
className="border-t border-neutral-800/30 flex-shrink-0"
style={{ height: 60 }}
>
<div className="flex items-center gap-1.5 px-2 py-0.5">
<span className="text-[9px] font-medium text-neutral-500 uppercase tracking-wider">
Captions
</span>
@@ -61,38 +61,7 @@ const EASE_PRESETS = [
"bounce.out",
];
// ---------------------------------------------------------------------------
// Shared input class (matches CaptionPropertyPanel)
// ---------------------------------------------------------------------------
const inputCls =
"w-full bg-neutral-900 border border-neutral-800 rounded px-1.5 py-0.5 text-2xs text-neutral-200 font-mono outline-none focus:border-neutral-600";
// ---------------------------------------------------------------------------
// Helper Components
// ---------------------------------------------------------------------------
function Section({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="mb-3">
<div className="flex items-center gap-1.5 mt-2 mb-1.5">
<span className="text-2xs font-medium text-neutral-500 uppercase tracking-wider">
{label}
</span>
</div>
<div className="space-y-1">{children}</div>
</div>
);
}
function Row({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="flex items-center gap-2">
<span className="text-2xs text-neutral-600 w-14 text-right flex-shrink-0">{label}</span>
<div className="flex-1 min-w-0">{children}</div>
</div>
);
}
import { Section, Row, inputCls } from "./shared";
// ---------------------------------------------------------------------------
// Animation phase controls
@@ -37,6 +37,10 @@ function readWordBoxes(
const iframeDisplayRect = iframe.getBoundingClientRect();
const overlayRect = overlayEl.getBoundingClientRect();
// The iframe renders at native resolution (e.g. 1920x1080) but is
// CSS-scaled to fit the viewport. getBoundingClientRect() on elements
// inside the iframe returns coordinates in the iframe's native space.
// Multiply by cssScale to convert to parent window coordinates.
const nativeW = parseFloat(iframe.style.width) || iframeDisplayRect.width;
const cssScale = iframeDisplayRect.width / nativeW;
const offsetX = iframeDisplayRect.left - overlayRect.left;
@@ -53,7 +57,9 @@ function readWordBoxes(
if (!groupEl) continue;
const computed = win.getComputedStyle(groupEl);
if (parseFloat(computed.opacity) <= 0.01 || computed.visibility === "hidden") continue;
// Find word spans — may be direct children or inside wrappers
// Find word elements — handles both per-word spans (generator output)
// and grouped text nodes (existing caption templates that use
// el.textContent = line.text instead of individual word spans).
const resolvedWordEls: HTMLElement[] = [];
for (const child of groupEl.children) {
const c = child as HTMLElement;
@@ -64,13 +70,49 @@ function readWordBoxes(
resolvedWordEls.push(c);
}
}
// Fallback: if no word spans found but group has text content,
// the template uses grouped text. Wrap each word in a span so
// the overlay can target them individually.
if (resolvedWordEls.length === 0 && groupEl.textContent?.trim()) {
const textNode = groupEl.childNodes[0];
if (textNode && textNode.nodeType === Node.TEXT_NODE) {
const words = (textNode.textContent || "").split(/\s+/).filter(Boolean);
const frag = doc.createDocumentFragment();
for (const word of words) {
const span = doc.createElement("span");
span.textContent = word + " ";
span.style.display = "inline";
frag.appendChild(span);
resolvedWordEls.push(span);
}
groupEl.replaceChild(frag, textNode);
} else {
// Single span child with all text (e.g. vignelli template)
const singleSpan = groupEl.querySelector<HTMLElement>(":scope > span");
if (singleSpan && singleSpan.textContent?.trim()) {
const words = singleSpan.textContent.split(/\s+/).filter(Boolean);
const frag = doc.createDocumentFragment();
for (const word of words) {
const span = doc.createElement("span");
span.textContent = word + " ";
span.style.display = "inline";
frag.appendChild(span);
resolvedWordEls.push(span);
}
singleSpan.replaceWith(frag);
}
}
}
for (let wi = 0; wi < group.segmentIds.length; wi++) {
const segId = group.segmentIds[wi];
const wordEl = resolvedWordEls[wi] as HTMLElement | undefined;
if (!wordEl) continue;
const rect = wordEl.getBoundingClientRect();
boxes.push({
segmentId: segId, groupId, groupIndex: gi, wordIndex: wi,
segmentId: segId,
groupId,
groupIndex: gi,
wordIndex: wi,
x: rect.left * cssScale + offsetX,
y: rect.top * cssScale + offsetY,
width: rect.width * cssScale,
@@ -81,9 +123,17 @@ function readWordBoxes(
return boxes;
}
function getWordEl(iframe: HTMLIFrameElement, groupIndex: number, wordIndex: number): HTMLElement | null {
function getWordEl(
iframe: HTMLIFrameElement,
groupIndex: number,
wordIndex: number,
): HTMLElement | null {
let doc: Document | null = null;
try { doc = iframe.contentDocument; } catch { return null; }
try {
doc = iframe.contentDocument;
} catch {
return null;
}
if (!doc) return null;
const groupEl = doc.querySelectorAll<HTMLElement>(".caption-group")[groupIndex];
if (!groupEl) return null;
@@ -108,8 +158,13 @@ function getWordEl(iframe: HTMLIFrameElement, groupIndex: number, wordIndex: num
* Read GSAP's internal transform state for an element.
* GSAP stores transforms in its own cache, not in el.style.transform.
*/
function readGsapTransform(el: HTMLElement, iframeWin: Window): { x: number; y: number; scale: number; rotation: number } {
const gsap = (iframeWin as unknown as { gsap?: { getProperty?: (el: HTMLElement, prop: string) => number } }).gsap;
function readGsapTransform(
el: HTMLElement,
iframeWin: Window,
): { x: number; y: number; scale: number; rotation: number } {
const gsap = (
iframeWin as unknown as { gsap?: { getProperty?: (el: HTMLElement, prop: string) => number } }
).gsap;
if (gsap && gsap.getProperty) {
return {
x: gsap.getProperty(el, "x") || 0,
@@ -155,9 +210,20 @@ function getOrCreateWrapper(el: HTMLElement): HTMLElement {
* Write transform values to a wrapper span around the word element.
* The word keeps its GSAP animations; the wrapper handles editor transforms.
*/
function writeTransform(el: HTMLElement, iframeWin: Window, x: number, y: number, scale: number, rotation: number) {
function writeTransform(
el: HTMLElement,
iframeWin: Window,
x: number,
y: number,
scale: number,
rotation: number,
) {
const wrapper = getOrCreateWrapper(el);
const gsap = (iframeWin as unknown as { gsap?: { set?: (el: HTMLElement, props: Record<string, number>) => void } }).gsap;
const gsap = (
iframeWin as unknown as {
gsap?: { set?: (el: HTMLElement, props: Record<string, number>) => void };
}
).gsap;
if (gsap && gsap.set) {
gsap.set(wrapper, { x, y, scale, rotation });
} else {
@@ -173,7 +239,10 @@ function syncToStore(segmentId: string, el: HTMLElement, iframeWin: Window) {
const style: Record<string, number> = {};
if (Math.abs(x) > 0.5) style.x = x;
if (Math.abs(y) > 0.5) style.y = y;
if (Math.abs(scale - 1) > 0.001) { style.scaleX = scale; style.scaleY = scale; }
if (Math.abs(scale - 1) > 0.001) {
style.scaleX = scale;
style.scaleY = scale;
}
if (Math.abs(rotation) > 0.1) style.rotation = rotation;
if (Object.keys(style).length > 0) {
useCaptionStore.getState().updateSegmentStyle(segmentId, style);
@@ -183,9 +252,7 @@ function syncToStore(segmentId: string, el: HTMLElement, iframeWin: Window) {
const HANDLE = 8;
const ROTATION_OFFSET = 20; // px above the selection box
export const CaptionOverlay = memo(function CaptionOverlay({
iframeRef,
}: CaptionOverlayProps) {
export const CaptionOverlay = memo(function CaptionOverlay({ iframeRef }: CaptionOverlayProps) {
const isEditMode = useCaptionStore((s) => s.isEditMode);
const model = useCaptionStore((s) => s.model);
const selectedSegmentIds = useCaptionStore((s) => s.selectedSegmentIds);
@@ -199,9 +266,38 @@ export const CaptionOverlay = memo(function CaptionOverlay({
// Interaction mode — only one active at a time
const interactionRef = useRef<
| { type: "move"; wordEl: HTMLElement; segmentId: string; startMX: number; startMY: number; origTX: number; origTY: number; origScale: number; origRotation: number }
| { type: "scale"; wordEl: HTMLElement; segmentId: string; startMX: number; startWidth: number; origTX: number; origTY: number; origScale: number; origRotation: number }
| { type: "rotate"; wordEl: HTMLElement; segmentId: string; centerX: number; centerY: number; startAngle: number; origTX: number; origTY: number; origRotation: number; origScale: number }
| {
type: "move";
wordEl: HTMLElement;
segmentId: string;
startMX: number;
startMY: number;
origTX: number;
origTY: number;
origScale: number;
origRotation: number;
}
| {
type: "scale";
wordEl: HTMLElement;
segmentId: string;
startMX: number;
startDxFromCenter: number;
origTX: number;
origTY: number;
origScale: number;
origRotation: number;
}
| {
type: "rotate";
wordEl: HTMLElement;
segmentId: string;
startMX: number;
origTX: number;
origTY: number;
origRotation: number;
origScale: number;
}
| null
>(null);
@@ -215,8 +311,13 @@ export const CaptionOverlay = memo(function CaptionOverlay({
if (!iframe || !m || !overlay) return;
const next = readWordBoxes(iframe, m, overlay);
// Skip state update if nothing changed (avoids re-render every 66ms)
if (next.length === prevBoxes.length &&
next.every((b, i) => Math.abs(b.x - prevBoxes[i].x) < 0.5 && Math.abs(b.y - prevBoxes[i].y) < 0.5)) return;
if (
next.length === prevBoxes.length &&
next.every(
(b, i) => Math.abs(b.x - prevBoxes[i].x) < 0.5 && Math.abs(b.y - prevBoxes[i].y) < 0.5,
)
)
return;
prevBoxes = next;
setWordBoxes(next);
};
@@ -273,7 +374,8 @@ export const CaptionOverlay = memo(function CaptionOverlay({
}, [iframeRef]);
// --- Move ---
const startMove = useCallback((groupIndex: number, wordIndex: number, segmentId: string, e: React.PointerEvent) => {
const startMove = useCallback(
(groupIndex: number, wordIndex: number, segmentId: string, e: React.PointerEvent) => {
e.stopPropagation();
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
const iframe = iframeRef.current;
@@ -283,15 +385,23 @@ export const CaptionOverlay = memo(function CaptionOverlay({
if (!wordEl || !win) return;
const state = readGsapTransform(getOrCreateWrapper(wordEl), win);
interactionRef.current = {
type: "move", wordEl, segmentId,
startMX: e.clientX, startMY: e.clientY,
origTX: state.x, origTY: state.y,
origScale: state.scale, origRotation: state.rotation,
type: "move",
wordEl,
segmentId,
startMX: e.clientX,
startMY: e.clientY,
origTX: state.x,
origTY: state.y,
origScale: state.scale,
origRotation: state.rotation,
};
}, [iframeRef]);
},
[iframeRef],
);
// --- Scale ---
const startScale = useCallback((groupIndex: number, wordIndex: number, segmentId: string, e: React.PointerEvent) => {
const startScale = useCallback(
(groupIndex: number, wordIndex: number, segmentId: string, e: React.PointerEvent) => {
e.stopPropagation();
e.preventDefault();
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
@@ -301,17 +411,30 @@ export const CaptionOverlay = memo(function CaptionOverlay({
const win = iframe.contentWindow;
if (!wordEl || !win) return;
const rect = wordEl.getBoundingClientRect();
const cssScale = getCssScale();
const boxCenterX =
rect.left * cssScale +
(iframeRef.current?.getBoundingClientRect().left ?? 0) +
(rect.width * cssScale) / 2;
const state = readGsapTransform(getOrCreateWrapper(wordEl), win);
interactionRef.current = {
type: "scale", wordEl, segmentId,
startMX: e.clientX, startWidth: rect.width,
origTX: state.x, origTY: state.y,
origScale: state.scale, origRotation: state.rotation,
type: "scale",
wordEl,
segmentId,
startMX: e.clientX,
startDxFromCenter: e.clientX - boxCenterX,
origTX: state.x,
origTY: state.y,
origScale: state.scale,
origRotation: state.rotation,
};
}, [iframeRef]);
},
[iframeRef, getCssScale],
);
// --- Rotate ---
const startRotate = useCallback((box: WordBox, e: React.PointerEvent) => {
const startRotate = useCallback(
(box: WordBox, e: React.PointerEvent) => {
e.stopPropagation();
e.preventDefault();
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
@@ -320,25 +443,33 @@ export const CaptionOverlay = memo(function CaptionOverlay({
const wordEl = getWordEl(iframe, box.groupIndex, box.wordIndex);
const win = iframe.contentWindow;
if (!wordEl || !win) return;
const cx = box.x + box.width / 2;
const cy = box.y + box.height / 2;
const startAngle = Math.atan2(e.clientY - cy, e.clientX - cx) * (180 / Math.PI);
const state = readGsapTransform(getOrCreateWrapper(wordEl), win);
interactionRef.current = {
type: "rotate", wordEl, segmentId: box.segmentId,
centerX: cx, centerY: cy,
startAngle, origTX: state.x, origTY: state.y,
origRotation: state.rotation, origScale: state.scale,
type: "rotate",
wordEl,
segmentId: box.segmentId,
startMX: e.clientX,
origTX: state.x,
origTY: state.y,
origRotation: state.rotation,
origScale: state.scale,
};
}, [iframeRef]);
},
[iframeRef],
);
/** Get iframe contentWindow, needed for gsap calls */
const getIframeWin = useCallback((): Window | null => {
try { return iframeRef.current?.contentWindow ?? null; } catch { return null; }
try {
return iframeRef.current?.contentWindow ?? null;
} catch {
return null;
}
}, [iframeRef]);
// --- Unified pointer move ---
const handlePointerMove = useCallback((e: React.PointerEvent) => {
const handlePointerMove = useCallback(
(e: React.PointerEvent) => {
const i = interactionRef.current;
if (!i) return;
const win = getIframeWin();
@@ -350,16 +481,24 @@ export const CaptionOverlay = memo(function CaptionOverlay({
const dy = (e.clientY - i.startMY) / cssScale;
writeTransform(i.wordEl, win, i.origTX + dx, i.origTY + dy, i.origScale, i.origRotation);
} else if (i.type === "scale") {
const dx = e.clientX - i.startMX;
const factor = 1 + dx / Math.max(i.startWidth, 50);
// Use distance from box center so dragging outward from ANY corner
// increases scale (not just right-side handles).
const cx = i.startMX - i.startDxFromCenter;
const startDist = Math.abs(i.startDxFromCenter);
const currentDist = Math.abs(e.clientX - cx);
const factor = startDist > 5 ? currentDist / startDist : 1;
const newScale = Math.max(0.1, i.origScale * factor);
writeTransform(i.wordEl, win, i.origTX, i.origTY, newScale, i.origRotation);
} else if (i.type === "rotate") {
const angle = Math.atan2(e.clientY - i.centerY, e.clientX - i.centerX) * (180 / Math.PI);
const delta = angle - i.startAngle;
// Horizontal drag maps to rotation: right = clockwise, left = counter-clockwise.
// 200px of horizontal movement = 90 degrees.
const dx = e.clientX - i.startMX;
const delta = (dx / 200) * 90;
writeTransform(i.wordEl, win, i.origTX, i.origTY, i.origScale, i.origRotation + delta);
}
}, [getCssScale, getIframeWin]);
},
[getCssScale, getIframeWin],
);
// --- Unified pointer up — sync back to store ---
const handlePointerUp = useCallback(() => {
@@ -371,9 +510,12 @@ export const CaptionOverlay = memo(function CaptionOverlay({
}
}, [getIframeWin]);
const handleBackgroundClick = useCallback((e: React.MouseEvent) => {
const handleBackgroundClick = useCallback(
(e: React.MouseEvent) => {
if (e.target === e.currentTarget) clearSelection();
}, [clearSelection]);
},
[clearSelection],
);
if (!isEditMode) return null;
@@ -397,11 +539,18 @@ export const CaptionOverlay = memo(function CaptionOverlay({
isSelected ? "ring-2 ring-studio-accent" : "hover:ring-1 hover:ring-white/30",
].join(" ")}
style={{
left: box.x, top: box.y, width: box.width, height: box.height,
left: box.x,
top: box.y,
width: box.width,
height: box.height,
cursor: isSelected ? "move" : "pointer",
touchAction: "none", borderRadius: 2,
touchAction: "none",
borderRadius: 2,
}}
onClick={(e) => {
e.stopPropagation();
selectSegment(box.segmentId, e.shiftKey);
}}
onClick={(e) => { e.stopPropagation(); selectSegment(box.segmentId, e.shiftKey); }}
onPointerDown={(e) => {
if (isSelected) startMove(box.groupIndex, box.wordIndex, box.segmentId, e);
}}
@@ -412,13 +561,16 @@ export const CaptionOverlay = memo(function CaptionOverlay({
<div
style={{
position: "absolute",
left: "50%", top: -ROTATION_OFFSET - HANDLE,
left: "50%",
top: -ROTATION_OFFSET - HANDLE,
marginLeft: -HANDLE / 2,
width: HANDLE, height: HANDLE,
width: HANDLE,
height: HANDLE,
borderRadius: "50%",
backgroundColor: "var(--hf-accent, #3CE6AC)",
border: "1px solid rgba(0,0,0,0.5)",
cursor: "grab", touchAction: "none",
cursor: "grab",
touchAction: "none",
}}
onPointerDown={(e) => startRotate(box, e)}
/>
@@ -426,11 +578,14 @@ export const CaptionOverlay = memo(function CaptionOverlay({
<div
style={{
position: "absolute",
left: "50%", top: -ROTATION_OFFSET,
width: 1, height: ROTATION_OFFSET,
left: "50%",
top: -ROTATION_OFFSET,
width: 1,
height: ROTATION_OFFSET,
marginLeft: -0.5,
backgroundColor: "var(--hf-accent, #3CE6AC)",
opacity: 0.5, pointerEvents: "none",
opacity: 0.5,
pointerEvents: "none",
}}
/>
{/* Scale handles — four corners */}
@@ -443,13 +598,18 @@ export const CaptionOverlay = memo(function CaptionOverlay({
<div
key={idx}
style={{
position: "absolute", ...pos,
width: HANDLE, height: HANDLE,
position: "absolute",
...pos,
width: HANDLE,
height: HANDLE,
backgroundColor: "var(--hf-accent, #3CE6AC)",
border: "1px solid rgba(0,0,0,0.5)",
borderRadius: 2, touchAction: "none",
borderRadius: 2,
touchAction: "none",
}}
onPointerDown={(e) => startScale(box.groupIndex, box.wordIndex, box.segmentId, e)}
onPointerDown={(e) =>
startScale(box.groupIndex, box.wordIndex, box.segmentId, e)
}
/>
))}
</>
@@ -1,36 +1,7 @@
import { memo, useCallback, useState } from "react";
import { memo, useCallback } from "react";
import { useCaptionStore } from "../store";
import type { CaptionStyle } from "../types";
import { CaptionAnimationPanel } from "./CaptionAnimationPanel";
// ---------------------------------------------------------------------------
// Helper Components
// ---------------------------------------------------------------------------
function Section({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="mb-3">
<div className="flex items-center gap-1.5 mt-2 mb-1.5">
<span className="text-2xs font-medium text-neutral-500 uppercase tracking-wider">
{label}
</span>
</div>
<div className="space-y-1">{children}</div>
</div>
);
}
function Row({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="flex items-center gap-2">
<span className="text-2xs text-neutral-600 w-14 text-right flex-shrink-0">{label}</span>
<div className="flex-1 min-w-0">{children}</div>
</div>
);
}
const inputCls =
"w-full bg-neutral-900 border border-neutral-800 rounded px-1.5 py-0.5 text-2xs text-neutral-200 font-mono outline-none focus:border-neutral-600";
import { Section, Row, inputCls } from "./shared";
// ---------------------------------------------------------------------------
// Main component
@@ -49,8 +20,6 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
const updateSelectedStyle = useCaptionStore((s) => s.updateSelectedStyle);
const updateGroupStyle = useCaptionStore((s) => s.updateGroupStyle);
const [activeTab, setActiveTab] = useState<"style" | "animation">("style");
// Resolve effective style for the first selected segment
const firstSegmentId = selectedSegmentIds.size > 0 ? [...selectedSegmentIds][0] : undefined;
const firstSegment = model?.segments.get(firstSegmentId ?? "");
@@ -76,7 +45,6 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
...segmentOverrides,
};
/**
* Apply a CSS style change to selected word elements in the iframe DOM in real time.
* Maps CaptionStyle property names to CSS properties.
@@ -112,9 +80,15 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
const c = child as HTMLElement;
if (c.dataset.captionWrapper === "true") {
const inner = c.querySelector<HTMLElement>(":scope > span");
if (inner && idx === wi) { targetEls.push(inner); break; }
if (inner && idx === wi) {
targetEls.push(inner);
break;
}
} else if (c.tagName === "SPAN") {
if (idx === wi) { targetEls.push(c); break; }
if (idx === wi) {
targetEls.push(c);
break;
}
}
idx++;
}
@@ -123,15 +97,23 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
}
// Apply transform updates via gsap.set on the WRAPPER (not the word span)
const hasTransform = updates.x !== undefined || updates.y !== undefined ||
updates.scaleX !== undefined || updates.scaleY !== undefined || updates.rotation !== undefined;
const hasTransform =
updates.x !== undefined ||
updates.y !== undefined ||
updates.scaleX !== undefined ||
updates.scaleY !== undefined ||
updates.rotation !== undefined;
if (hasTransform) {
try {
const iframeGsap = (iframeRef.current?.contentWindow as unknown as {
gsap?: { set: (el: HTMLElement, props: Record<string, unknown>) => void;
getProperty: (el: HTMLElement, prop: string) => number };
})?.gsap;
const iframeGsap = (
iframeRef.current?.contentWindow as unknown as {
gsap?: {
set: (el: HTMLElement, props: Record<string, unknown>) => void;
getProperty: (el: HTMLElement, prop: string) => number;
};
}
)?.gsap;
if (iframeGsap) {
for (const el of targetEls) {
// Get or create wrapper
@@ -156,7 +138,9 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
});
}
}
} catch { /* cross-origin */ }
} catch {
/* cross-origin */
}
}
},
[iframeRef, model, selectedSegmentIds],
@@ -194,53 +178,15 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
const scaleX = effectiveStyle.scaleX ?? 1;
// Count label
const countLabel = selectedSegmentIds.size === 1
? "1 word"
: `${selectedSegmentIds.size} words`;
const countLabel = selectedSegmentIds.size === 1 ? "1 word" : `${selectedSegmentIds.size} words`;
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="px-3 py-2 border-b border-neutral-800 flex-shrink-0">
<div className="flex items-center justify-between mb-1.5">
<span className="text-2xs text-neutral-500">
{countLabel}
</span>
</div>
{/* Tab switcher */}
<div className="flex gap-1">
<button
type="button"
onClick={() => setActiveTab("style")}
className={[
"flex-1 py-0.5 rounded text-2xs font-medium transition-colors",
activeTab === "style"
? "bg-studio-accent/20 text-studio-accent border border-studio-accent/50"
: "text-neutral-500 border border-neutral-800 hover:text-neutral-300 hover:border-neutral-600",
].join(" ")}
>
Style
</button>
<button
type="button"
onClick={() => setActiveTab("animation")}
className={[
"flex-1 py-0.5 rounded text-2xs font-medium transition-colors",
activeTab === "animation"
? "bg-studio-accent/20 text-studio-accent border border-studio-accent/50"
: "text-neutral-500 border border-neutral-800 hover:text-neutral-300 hover:border-neutral-600",
].join(" ")}
>
Animation
</button>
</div>
<span className="text-2xs text-neutral-500">{countLabel}</span>
</div>
{/* Animation tab */}
{activeTab === "animation" && <CaptionAnimationPanel />}
{/* Style tab — Transform only */}
{activeTab === "style" && (
<div className="flex-1 overflow-y-auto px-3 py-2">
<Section label="Position">
<Row label="X">
@@ -251,7 +197,6 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
className={inputCls}
/>
</Row>
<Row label="Y">
<input
type="number"
@@ -277,7 +222,6 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
className={inputCls}
/>
</Row>
<Row label="Rotation">
<input
type="number"
@@ -288,7 +232,6 @@ export const CaptionPropertyPanel = memo(function CaptionPropertyPanel({
</Row>
</Section>
</div>
)}
</div>
);
});
@@ -0,0 +1,26 @@
import type React from "react";
export const inputCls =
"w-full bg-neutral-900 border border-neutral-800 rounded px-1.5 py-0.5 text-2xs text-neutral-200 font-mono outline-none focus:border-neutral-600";
export function Section({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="mb-3">
<div className="flex items-center gap-1.5 mt-2 mb-1.5">
<span className="text-2xs font-medium text-neutral-500 uppercase tracking-wider">
{label}
</span>
</div>
<div className="space-y-1">{children}</div>
</div>
);
}
export function Row({ label, children }: { label: string; children: React.ReactNode }) {
return (
<div className="flex items-center gap-2">
<span className="text-2xs text-neutral-600 w-14 text-right flex-shrink-0">{label}</span>
<div className="flex-1 min-w-0">{children}</div>
</div>
);
}
@@ -233,6 +233,10 @@ function hexToRgba(color: string, opacity: number): string {
if (color.startsWith("rgb")) {
return color;
}
// Named colors and other non-hex values — return as-is
if (!color.startsWith("#")) {
return color;
}
// Try to parse hex
const hex = color.replace("#", "");
if (hex.length === 3 || hex.length === 6) {
@@ -74,10 +74,11 @@ export function useCaptionSync(projectId: string | null) {
const overrides = buildOverrides(state.model);
fetch(
`/api/projects/${pid}/files/${encodeURIComponent("caption-overrides.json")}`,
{ method: "PUT", headers: { "Content-Type": "text/plain" }, body: JSON.stringify(overrides, null, 2) },
).catch(() => {});
fetch(`/api/projects/${pid}/files/${encodeURIComponent("caption-overrides.json")}`, {
method: "PUT",
headers: { "Content-Type": "text/plain" },
body: JSON.stringify(overrides, null, 2),
}).catch((err) => console.warn("[captions] auto-save failed:", err));
}, []);
// Auto-save on model changes with 800ms debounce
@@ -140,7 +141,10 @@ export function useCaptionSync(projectId: string | null) {
const style: Partial<CaptionStyle> = { ...seg.style };
if (override.x !== undefined) style.x = override.x;
if (override.y !== undefined) style.y = override.y;
if (override.scale !== undefined) { style.scaleX = override.scale; style.scaleY = override.scale; }
if (override.scale !== undefined) {
style.scaleX = override.scale;
style.scaleY = override.scale;
}
if (override.rotation !== undefined) style.rotation = override.rotation;
if (override.activeColor !== undefined) style.activeColor = override.activeColor;
if (override.dimColor !== undefined) style.dimColor = override.dimColor;
+2
View File
@@ -107,6 +107,8 @@ export function buildCaptionModel(
export function extractTranscript(source: string): TranscriptWord[] {
// Match: (const|let|var) (TRANSCRIPT|script) = [...]
// The array may span multiple lines and contain trailing commas.
// The lazy [\s\S]*? anchors on the first `];` — assumes transcript word
// text never contains a literal `];` string (safe for speech transcripts).
const varPattern = /(?:const|let|var)\s+(?:TRANSCRIPT|script)\s*=\s*(\[[\s\S]*?\]);/;
const match = source.match(varPattern);
+5 -3
View File
@@ -7,6 +7,8 @@ import {
CaptionStyle,
} from "./types";
let nextSplitId = 0;
interface CaptionState {
isEditMode: boolean;
model: CaptionModel | null;
@@ -57,7 +59,7 @@ const initialState = {
sourceFilePath: null,
};
export const useCaptionStore = create<CaptionState>((set, get) => ({
export const useCaptionStore = create<CaptionState>((set) => ({
...initialState,
// Basic
@@ -186,7 +188,7 @@ export const useCaptionStore = create<CaptionState>((set, get) => ({
const firstIds = group.segmentIds.slice(0, splitIndex);
const secondIds = group.segmentIds.slice(splitIndex);
const newGroupId = `group-${Date.now()}-${Math.random().toString(36).slice(2)}`;
const newGroupId = `group-split-${nextSplitId++}`;
const groups = new Map(state.model.groups);
groups.set(groupId, { ...group, segmentIds: firstIds });
groups.set(newGroupId, { ...group, id: newGroupId, segmentIds: secondIds });
@@ -232,7 +234,7 @@ export const useCaptionStore = create<CaptionState>((set, get) => ({
});
// Clear selection if it referenced group2
const selectedGroupId = get().selectedGroupId === groupId2 ? null : get().selectedGroupId;
const selectedGroupId = state.selectedGroupId === groupId2 ? null : state.selectedGroupId;
return { model: { ...state.model, groups, segments, groupOrder }, selectedGroupId };
}),
@@ -376,8 +376,8 @@ export const NLELayout = memo(function NLELayout({
onDrillDown={handleDrillDown}
renderClipContent={renderClipContent}
/>
{timelineFooter}
</div>
{timelineFooter && <div className="flex-shrink-0">{timelineFooter}</div>}
</div>
</>
)}
-1
View File
@@ -20,7 +20,6 @@ When transcribing:
---
Analyze the spoken content to determine caption style. If the user specifies a style, use that. Otherwise, detect tone from the transcript.
## Transcript Source
@@ -39,7 +39,7 @@ The CLI auto-detects and normalizes these formats:
The default model (`small.en`) balances accuracy and speed. For better results, use a larger model:
| Model | Size | Speed | Accuracy | When to use |
| ----------- | ------ | -------- | --------- | ------------------------------------- |
| ---------- | ------ | -------- | --------- | ------------------------------------- |
| `tiny` | 75 MB | Fastest | Low | Quick previews, testing pipeline |
| `base` | 142 MB | Fast | Fair | Short clips, clear audio |
| `small` | 466 MB | Moderate | Good | **Default** — good for most content |