feat(studio): drag keyframes with live beat snapping (#1439)

* feat(studio): drag keyframes with beat snapping

Keyframe diamonds are draggable with live preview and snap to the music
beat grid (requires VITE_STUDIO_ENABLE_KEYFRAMES=1).

Drag model: a tween start point trims the front (end fixed), an end point
resizes (start fixed), an intermediate keyframe moves within the tween
(adjacent segments resize, others untouched; start/end moves remap the
intermediates to preserve their absolute times). The keyframe snaps to the
nearest beat within ~8px, centered exactly on the dot.

Reliability: the commit resolves the dragged element's selection + parsed
animations on demand (awaited) instead of relying on the async DOM-edit
session, picks the tween whose window contains the keyframe's original
time among same-group tweens, and holds the dropped position optimistically
until the cache round-trip lands. Cache clip% precision raised to 0.001%
so the marker lands exactly where dropped.

Pure match/plan logic + unit tests in editor/keyframeMove.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio): harden keyframe drag commit (review follow-ups)

- pickKeyframeTween no longer falls back to ALL animations on a selector
  mismatch — it only picks among the dragged element's own tweens, so a
  class/compound-selector mismatch can't edit a different element. No
  match → no-op.
- computeKeyframeMovePlan bails to a no-op when a keyframe-array tween's
  dragged keyframe can't be located (stale cache / precision drift) instead
  of falling through to an end-point resize that silently rescaled the
  whole tween and re-timed every keyframe.
- usePopulateKeyframeCacheForFile clipPct now uses 0.001% precision
  (matching useGsapAnimationsForElement) so beat-snapped keyframes from the
  file-wide cache also center on the dot and the two caches agree.
- The optimistic drag hold only releases once the cache reflects the
  committed position (a keyframe near the held %), so an unrelated cache
  rebuild no longer flashes the diamond back to its old spot.
- A drag's document listeners are cleaned up on unmount, so an unmount
  mid-drag (clip delete / comp switch / zoom-out) no longer leaks them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* feat(studio): shrink + lower keyframe diamonds under the beat strip

When a clip's track shows the beat-dot strip (the top band), its keyframe
diamonds and connecting lines render at 45% size and centered in the
region below the band, so they don't collide with the dots. Full size and
vertically centered otherwise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio): ignore keyframe re-drag during the optimistic-hold window

After a drop, the diamond is held at its dropped position (via effPct) until the
file round-trip lands, but `pct` passed to handlePointerDown still comes from
props (the pre-drop position). Re-grabbing the same keyframe in that window
would track the drag from a stale origin and commit against the wrong tween (or
no-op via the stale-cache guard). Skip starting a drag while a hold is pending;
it clears on the cache match (≤2s fallback). Click selection is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

---------

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

packages/studio/src/player/components/BeatStrip.tsx

@@ -3,7 +3,7 @@ import { moveBeatCompositionTime, deleteBeatAtCompositionTime } from "../../util
 import { usePlayerStore } from "../store/playerStore";
 import { CLIP_Y } from "./timelineLayout";
 
-const BEAT_BAND_H = 14; // dark band height at top of track
+export const BEAT_BAND_H = 14; // dark band height at top of track
 const BEAT_HIT_W = 12; // grab width per beat (px)
 
 /** Hide both layers when beats are packed tighter than this (px) — too dense to read. */

packages/studio/src/player/components/Timeline.tsx

@@ -19,6 +19,7 @@ import {
   type KeyframeDiamondContextMenuState,
 } from "./KeyframeDiamondContextMenu";
 import { useTimelineClipDrag } from "./useTimelineClipDrag";
+import { snapKeyframePctToBeat } from "./timelineEditing";
 import { ClipContextMenu } from "./ClipContextMenu";
 import {
   GUTTER,
@@ -470,6 +471,16 @@ export const Timeline = memo(function Timeline({
           onDragKeyframe={(el, oldPct, newPct) => {
             onMoveKeyframe?.(el, oldPct, newPct);
           }}
+          onSnapKeyframePct={(el, pct) =>
+            snapKeyframePctToBeat(el, pct, adjustedBeatAnalysis?.beatTimes, pps)
+          }
+          onPickKeyframeElement={(el) => {
+            const elKey = el.key ?? el.id;
+            if (selectedElementId !== elKey) {
+              setSelectedElementId(elKey);
+              onSelectElement?.(el);
+            }
+          }}
           onContextMenuKeyframe={(e, elId, pct) => {
             const el = elements.find((x) => (x.key ?? x.id) === elId);
             if (el) {

packages/studio/src/player/components/TimelineCanvas.tsx

@@ -92,6 +92,10 @@ interface TimelineCanvasProps {
   onClickKeyframe?: (element: TimelineElement, percentage: number) => void;
   onShiftClickKeyframe?: (elementId: string, percentage: number) => void;
   onDragKeyframe?: (element: TimelineElement, oldPct: number, newPct: number) => void;
+  /** Snap a keyframe's clip-relative % to the nearest beat (returns unchanged when none in range). */
+  onSnapKeyframePct?: (element: TimelineElement, pct: number) => number;
+  /** Select the element when a keyframe drag starts (loads its GSAP session). */
+  onPickKeyframeElement?: (element: TimelineElement) => void;
   onContextMenuKeyframe?: (e: React.MouseEvent, elementId: string, percentage: number) => void;
   onContextMenuClip?: (e: React.MouseEvent, element: TimelineElement) => void;
   beatAnalysis?: MusicBeatAnalysis | null;
@@ -140,6 +144,8 @@ export const TimelineCanvas = memo(function TimelineCanvas({
   onClickKeyframe,
   onShiftClickKeyframe,
   onDragKeyframe,
+  onSnapKeyframePct,
+  onPickKeyframeElement,
   onContextMenuKeyframe,
   onContextMenuClip,
   beatAnalysis,
@@ -211,6 +217,14 @@ export const TimelineCanvas = memo(function TimelineCanvas({
         const ts = trackStyles.get(trackNum) ?? getTrackStyle("");
         const isPendingTrack =
           draggedClip?.started === true && !trackOrder.includes(trackNum) && els.length === 0;
+        // The beat-dot strip occupies the top of this track's lane (active track,
+        // or the music track when nothing is selected). When shown, keyframe
+        // diamonds shrink + drop to the bottom half so they don't collide with it.
+        const beatStripOnTrack =
+          (beatAnalysis?.beatTimes?.length ?? 0) >= 2 &&
+          (selectedElementId
+            ? els.some((e) => (e.key ?? e.id) === selectedElementId)
+            : els.some(isMusicTrack));
         return (
           <div
             key={trackNum}
@@ -254,9 +268,7 @@ export const TimelineCanvas = memo(function TimelineCanvas({
               />
               {/* Beat dots on the active track (the one holding the selection),
                   falling back to the music track when nothing is selected. */}
-              {(selectedElementId
-                ? els.some((e) => (e.key ?? e.id) === selectedElementId)
-                : els.some(isMusicTrack)) && (
+              {beatStripOnTrack && (
                 <BeatStrip
                   beatTimes={beatAnalysis?.beatTimes}
                   beatStrengths={beatAnalysis?.beatStrengths}
@@ -422,6 +434,7 @@ export const TimelineCanvas = memo(function TimelineCanvas({
                         keyframesData={keyframeCache.get(elementKey)!}
                         clipWidthPx={Math.max(previewElement.duration * pps, 4)}
                         clipHeightPx={TRACK_H - 2 * CLIP_Y}
+                        beatsActive={beatStripOnTrack}
                         accentColor={clipStyle.accent}
                         isSelected={isSelected}
                         currentPercentage={
@@ -436,6 +449,8 @@ export const TimelineCanvas = memo(function TimelineCanvas({
                         onDragKeyframe={(oldPct, newPct) =>
                           onDragKeyframe?.(previewElement, oldPct, newPct)
                         }
+                        snapPct={(pct) => onSnapKeyframePct?.(previewElement, pct) ?? pct}
+                        onPickForDrag={() => onPickKeyframeElement?.(previewElement)}
                         onContextMenuKeyframe={onContextMenuKeyframe}
                       />
                     )}

packages/studio/src/player/components/TimelineClipDiamonds.tsx

@@ -1,4 +1,5 @@
-import { memo, useRef } from "react";
+import { memo, useEffect, useRef, useState } from "react";
+import { BEAT_BAND_H } from "./BeatStrip";
 
 interface KeyframeEntry {
   percentage: number;
@@ -17,6 +18,9 @@ interface TimelineClipDiamondsProps {
   keyframesData: KeyframeCacheEntry;
   clipWidthPx: number;
   clipHeightPx: number;
+  /** Beat-dot strip is shown on this track → shrink diamonds + drop them into
+   *  the bottom half so they clear the strip at the top. */
+  beatsActive?: boolean;
   accentColor: string;
   isSelected: boolean;
   currentPercentage: number;
@@ -26,6 +30,13 @@ interface TimelineClipDiamondsProps {
   onShiftClickKeyframe?: (elementId: string, percentage: number) => void;
   onDragKeyframe?: (percentage: number, newPercentage: number) => void;
   onContextMenuKeyframe?: (e: React.MouseEvent, elementId: string, percentage: number) => void;
+  /** Snap a clip-relative percentage to the nearest beat (returns it unchanged
+   *  when no beat is within range). Drives live beat-snapping while dragging. */
+  snapPct?: (percentage: number) => number;
+  /** Select this element when a keyframe drag begins, so its GSAP session is
+   *  loaded by the time the move commits (diamonds render on unselected clips
+   *  too, and a drag suppresses the selecting click). */
+  onPickForDrag?: () => void;
 }
 
 const DIAMOND_RATIO = 0.8;
@@ -34,6 +45,7 @@ export const TimelineClipDiamonds = memo(function TimelineClipDiamonds({
   keyframesData,
   clipWidthPx,
   clipHeightPx,
+  beatsActive,
   accentColor,
   isSelected,
   currentPercentage,
@@ -43,13 +55,59 @@ export const TimelineClipDiamonds = memo(function TimelineClipDiamonds({
   onShiftClickKeyframe,
   onDragKeyframe,
   onContextMenuKeyframe,
+  snapPct,
+  onPickForDrag,
 }: TimelineClipDiamondsProps) {
-  const dragRef = useRef<{ startX: number; startPct: number } | null>(null);
+  // Live drag: which keyframe (by original %) is being dragged and its current
+  // (beat-snapped) %, so the diamond + its connecting lines follow the cursor.
+  const dragRef = useRef<{ origPct: number; pct: number; moved: boolean } | null>(null);
+  const [drag, setDrag] = useState<{ origPct: number; pct: number } | null>(null);
+  // Commit through the latest callback, not the one captured at pointer-down:
+  // selecting the element on drag-start loads its GSAP session asynchronously,
+  // and the commit must use the closure that sees the loaded session.
+  const onDragKeyframeRef = useRef(onDragKeyframe);
+  onDragKeyframeRef.current = onDragKeyframe;
+  // Optimistic hold: after a commit, keep the diamond at the dropped position
+  // until the cache reflects the change (the file round-trip rewrites
+  // keyframesData), so it doesn't flash back to the old spot in between.
+  const pendingRef = useRef(false);
+  const pendingHeldPctRef = useRef<number | null>(null);
+  const pendingTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
+  // Cleanup for an in-flight drag's document listeners, so an unmount mid-drag
+  // (clip deleted, comp switch, zoom-out → early return) doesn't leak them.
+  const dragCleanupRef = useRef<(() => void) | null>(null);
+
+  useEffect(() => {
+    if (!pendingRef.current) return;
+    // Only release the optimistic hold once the cache actually reflects the
+    // committed position (a keyframe near the held %). An unrelated cache
+    // rebuild (e.g. elementCount change) rebuilds keyframesData with the SAME
+    // percentages — releasing then would flash the diamond back to the old spot.
+    const held = pendingHeldPctRef.current;
+    if (held != null && !keyframesData.keyframes.some((k) => Math.abs(k.percentage - held) < 0.3)) {
+      return;
+    }
+    pendingRef.current = false;
+    pendingHeldPctRef.current = null;
+    if (pendingTimerRef.current) clearTimeout(pendingTimerRef.current);
+    setDrag(null);
+  }, [keyframesData]);
+
+  useEffect(
+    () => () => {
+      clearTimeout(pendingTimerRef.current ?? undefined);
+      dragCleanupRef.current?.();
+    },
+    [],
+  );
 
   if (clipWidthPx < 20) return null;
 
-  const diamondSize = Math.round(clipHeightPx * DIAMOND_RATIO);
+  // When the beat strip occupies the top band, shrink the diamonds and center
+  // them in the remaining bottom region so they don't collide with it.
+  const diamondSize = Math.round(clipHeightPx * (beatsActive ? 0.45 : DIAMOND_RATIO));
   const half = diamondSize / 2;
+  const centerY = beatsActive ? BEAT_BAND_H + (clipHeightPx - BEAT_BAND_H) / 2 : clipHeightPx / 2;
   const sorted = keyframesData.keyframes.slice().sort((a, b) => a.percentage - b.percentage);
   const baseColor = isSelected ? accentColor : "#a3a3a3";
   const baseOpacity = isSelected ? 0.4 : 0.25;
@@ -66,47 +124,81 @@ export const TimelineClipDiamonds = memo(function TimelineClipDiamonds({
   const handlePointerDown = (e: React.PointerEvent, pct: number) => {
     if (e.button !== 0) return;
     e.stopPropagation();
+    // Ignore a new drag while a prior drop is still settling: `pct` comes from
+    // props (the pre-drop position) but the diamond is held at its dropped spot
+    // via effPct(), so a re-grab would track from a stale origin and commit
+    // against the wrong tween. The hold clears on the cache round-trip (≤2s).
+    if (pendingRef.current) return;
+    // Select the element up front so its GSAP session loads during the drag and
+    // the commit (which resolves the animation from the selection) isn't a no-op.
+    onPickForDrag?.();
     const startX = e.clientX;
+    dragRef.current = { origPct: pct, pct, moved: false };
 
     const handleMove = (me: PointerEvent) => {
+      const d = dragRef.current;
+      if (!d) return;
       const dx = me.clientX - startX;
-      if (Math.abs(dx) > 4) {
-        dragRef.current = { startX, startPct: pct };
+      // 4px dead zone so a click doesn't register as a drag.
+      if (!d.moved && Math.abs(dx) <= 4) return;
+      d.moved = true;
+      const rawPct = Math.max(0, Math.min(100, pct + (dx / clipWidthPx) * 100));
+      const snapped = snapPct ? snapPct(rawPct) : rawPct;
+      d.pct = snapped;
+      setDrag({ origPct: pct, pct: snapped });
+    };
+
+    const handleUp = () => {
+      document.removeEventListener("pointermove", handleMove);
+      document.removeEventListener("pointerup", handleUp);
+      dragCleanupRef.current = null;
+      const d = dragRef.current;
+      dragRef.current = null;
+      const willCommit = !!(d && d.moved && Math.abs(d.pct - d.origPct) > 0.5);
+      if (willCommit && d) {
+        // Hold the dropped position optimistically; the effect clears it once the
+        // cache round-trip lands (fallback timeout in case it never does).
+        pendingRef.current = true;
+        pendingHeldPctRef.current = d.pct;
+        setDrag({ origPct: d.origPct, pct: d.pct });
+        if (pendingTimerRef.current) clearTimeout(pendingTimerRef.current);
+        pendingTimerRef.current = setTimeout(() => {
+          pendingRef.current = false;
+          pendingHeldPctRef.current = null;
+          setDrag(null);
+        }, 2000);
+        onDragKeyframeRef.current?.(d.origPct, d.pct);
+      } else {
+        setDrag(null);
       }
     };
 
-    const handleUp = (ue: PointerEvent) => {
+    dragCleanupRef.current = () => {
       document.removeEventListener("pointermove", handleMove);
       document.removeEventListener("pointerup", handleUp);
-      const start = dragRef.current;
-      dragRef.current = null;
-      if (!start) return;
-      const dx = ue.clientX - start.startX;
-      const dPct = (dx / clipWidthPx) * 100;
-      const newPct = Math.max(0, Math.min(100, Math.round(start.startPct + dPct)));
-      if (Math.abs(newPct - start.startPct) > 0.5) {
-        onDragKeyframe?.(start.startPct, newPct);
-      }
     };
 
     document.addEventListener("pointermove", handleMove);
     document.addEventListener("pointerup", handleUp);
   };
 
+  // Effective % for rendering: the dragged keyframe follows the (snapped) cursor.
+  const effPct = (p: number): number => (drag && drag.origPct === p ? drag.pct : p);
+
   return (
     <div className="absolute inset-0" style={{ zIndex: 3, pointerEvents: "none" }}>
       {sorted.map((kf, i) => {
         if (i === 0) return null;
         const prev = sorted[i - 1]!;
-        const x1 = (prev.percentage / 100) * clipWidthPx;
-        const x2 = (kf.percentage / 100) * clipWidthPx;
+        const x1 = (effPct(prev.percentage) / 100) * clipWidthPx;
+        const x2 = (effPct(kf.percentage) / 100) * clipWidthPx;
         return (
           <div
             key={`line-${i}-${prev.percentage}-${kf.percentage}`}
             className="absolute"
             style={{
               left: x1,
-              top: "50%",
+              top: centerY,
               width: x2 - x1,
               height: 2,
               transform: "translateY(-1px)",
@@ -119,7 +211,7 @@ export const TimelineClipDiamonds = memo(function TimelineClipDiamonds({
       })}
 
       {sorted.map((kf, i) => {
-        const leftPx = (kf.percentage / 100) * clipWidthPx - half;
+        const leftPx = (effPct(kf.percentage) / 100) * clipWidthPx - half;
         const kfKey = `${elementId}:${kf.percentage}`;
         const isKfSelected = selectedKeyframes.has(kfKey);
         const atPlayhead = isSelected && Math.abs(kf.percentage - currentPercentage) < 0.5;
@@ -132,7 +224,7 @@ export const TimelineClipDiamonds = memo(function TimelineClipDiamonds({
             className="absolute"
             style={{
               left: leftPx,
-              top: "50%",
+              top: centerY,
               transform: "translateY(-50%)",
               width: diamondSize,
               height: diamondSize,

packages/studio/src/player/components/timelineEditing.ts

@@ -111,6 +111,34 @@ export function resolveTimelineMove(
   };
 }
 
+/**
+ * Snap a keyframe's clip-relative percentage to the nearest beat within ~8px,
+ * mapping through composition time (pct → time → nearest beat → pct). Returns
+ * the percentage unchanged when no beat is in range, so dragging stays free
+ * between beats.
+ */
+export function snapKeyframePctToBeat(
+  el: { start: number; duration: number },
+  pct: number,
+  beatTimes: number[] | undefined,
+  pixelsPerSecond: number,
+): number {
+  if (!beatTimes || beatTimes.length === 0 || el.duration <= 0) return pct;
+  const t = el.start + (pct / 100) * el.duration;
+  const snapSecs = 8 / Math.max(pixelsPerSecond, 1);
+  let best = t;
+  let bestDist = snapSecs;
+  for (const bt of beatTimes) {
+    const d = Math.abs(bt - t);
+    if (d < bestDist) {
+      bestDist = d;
+      best = bt;
+    }
+  }
+  if (best === t) return pct;
+  return Math.max(0, Math.min(100, ((best - el.start) / el.duration) * 100));
+}
+
 export function resolveTimelineResize(
   input: TimelineResizeInput,
   edge: "start" | "end",