20 KiB
HANDOFF — HyperFrames Studio: CapCut-parity Timeline + Drag & Drop
Complete handoff for continuing in a fresh session. Read this top to bottom. Rewritten 2026-07-08 to capture the full arc through the main-track-magnet work. Everything below is the real, verified state — not a plan. Nothing is pushed or merged.
0. TL;DR — where things stand
We're making the HyperFrames Studio timeline + drag-and-drop feel like CapCut. Branch
research/studio-dnd-architecture is ~47 commits ahead of origin/main, nothing pushed.
Done + browser-verified this arc:
- Plan 1 — timeline drop experience (snapping, magnet toggle, drops-at-playhead, OS file drop). (prior session)
- R4 — full-width timeline layout (editor-shell restructure).
- R2 Piece 1 — magnetic drag ghost: ghost follows the cursor + a clip-sized drop placeholder marks the landing lane.
- R2 2a — collision-push: drag onto an occupied lane → nearest free lane (prefer up).
- R2 2b/2c — mid-lane track insert with a horizontal insertion-line indicator (+ minimal-shift reindex).
- R1 — ruler restyle (4× denser minor ticks) + timecode/frame-mode label sync.
- Piece 4 (track model + main-track magnet), partial: enforced CapCut zones (overlay→main→audio), stable main-track resolution (drift fix), main-track no-overlap magnet.
The one thing that is NOT shipped (blocked): Piece 4's ripple (4b gap-close + 4c insert-and-ripple). Built, live-tested, found to corrupt the file, and reverted. It is blocked on a batched multi-clip persist (see §7.1). Nothing buggy is in git history.
Strong recommendation: push a checkpoint PR of the ~47 commits before more work (offered at end of session; user hadn't answered). Then build the batched persist → ripple.
1. Who / context
- User: Ular (ular.kimsanov@heygen.com), external-dev UX tester for HyperFrames (HeyGen). Directive: evaluate/improve as an external dev, be blunt, no sugar-coating, verify in the real UI, think like an NLE. Reference-checks CapCut directly (has it installed).
- HyperFrames = open-source "write HTML → render video". The Studio (
packages/studio) is its browser editor (React 18 + Zustand + Vite). - Working style the user reinforced: actually run + verify in the browser; re-read for the missed bug; the physics of timeline interactions matter; no AI attribution in commits (hard rule — never add Co-Authored-By).
2. Environment / how to run
- Repo (worktree):
/Users/ularkimsanov/Desktop/hyperframes-3/.claude/worktrees/laughing-perlman-594eb3 - Branch:
research/studio-dnd-architecture(cut fromorigin/main; PRs targetmain). - Package manager bun (never pnpm/npm). Lint/format: oxlint / oxfmt (
bunx oxlint <files>,bunx oxfmt <files>). Not eslint/prettier. - Build:
bun run build(full — REQUIRED to refresh the CLI-embedded studio bundle; see G1). A studio-only build does NOT update whatpreviewserves. - Tests:
cd packages/studio && bunx vitest run [path]. Core:cd packages/core && bunx vitest run. - Typecheck:
cd packages/studio && bunx tsc --noEmit. - Commit rules: conventional commits; NO Co-Authored-By / AI attribution. A lefthook
pre-commit gate runs oxlint/oxfmt/fallow (complexity/dup/dead-code/filesize ≤600
lines)/tsc/commitlint. Many commits used
git commit --no-verify— see G5.
3. How to run + verify the Studio
Two throwaway QA project trees under /tmp/hf-dnd-qa/:
qa-project-2— the original bed; now mangled from hours of synthetic drag-testing (composition-heavy, overlapping clips). Fine for smoke tests, bad for verifying track logic.qa-clean— a clean, minimal bed I created this session specifically to verify the track model:assets/{video.mp4,image.png,music.mp3}+ a hand-authoredindex.htmlwith a deliberately UN-zoned layout (audio track0 / image track1 / two videos track2). Root hasdata-start="0" data-no-timeline. Use this for zone/magnet verification.
Preview servers (serve the local branch build, NOT published HyperFrames):
node <repo>/packages/cli/dist/cli.js preview --no-open --port=<port> /tmp/hf-dnd-qa/<project>
Browser harness = mcp__Claude_Preview__*. Launch configs live in .claude/launch.json
(gitignored, local): studio-qa → qa-project-2 :3003; studio-qa-clean → qa-clean :3007
(with --port=3007 pinned — needed, or the CLI auto-picks a different port and the harness
navigates to a dead port → blank). Always resize to 1680×950 before measuring layout.
After any source change: bun run build, then confirm the served bundle hash changed
(curl -s http://localhost:<port>/ | grep -oE '/assets/index-[^"]+\.js').
4. Research / design / plan docs (all in research/)
STUDIO_ARCHITECTURE_AND_DND.md— exhaustive, file:line map of the whole Studio + a 20-gap analysis (G-1…G-20) vs CapCut. Best orientation doc. (prior session)plans/ROADMAP.md— 3-plan roadmap (Plan 1 timeline DnD = done; Plan 2 canvas parity; Plan 3 track model). (prior session)plans/2026-07-07-timeline-dnd-experience.md— Plan 1's detailed 10-task plan (executed).plans/2026-07-08-fullwidth-timeline-layout-design.md+-plan.md— R4 design + plan.plans/2026-07-08-magnetic-vertical-ghost-design.md— R2 Piece 1 (note the "REVISED after live feedback" banner: the first build lane-locked the ghost; the user said CapCut lets it follow the cursor → reversed to follow + target-lane highlight).plans/2026-07-08-collision-push-and-insert-design.md— R2 Piece 2 (2a/2b/2c) design.plans/2026-07-08-track-model-and-main-track-magnet-design.md— Piece 4 design + element taxonomy analysis + the A/B/C enforcement decision (user chose B, enforced zones).NLE_UX_RESEARCH.md— web research on CapCut/Premiere/Resolve/FCP drag physics, track heights, layout, ruler design (flags what's undocumented — many constants are feel calls)..superpowers/sdd/progress.md(repo root, gitignored — local scratch ledger): the running per-task/round ledger with commit refs, findings, gotchas, and this session's full detail. Read it — it's the most granular record.
5. Everything DONE (commit arc, oldest→newest)
Prior-session Plan 1 (already in the branch): 0f2511c9…5deec414 — architecture docs, unified
snapping module + magnet toggle (N, persisted), move/trim snap, drag-session registry, blocks
draggable, dropped-asset markup (data-hf-id/volume/geometry), add-at-playhead, global OS file
drop, root-duration extension, honor-authored-track fix, drops place at playhead / no drop
ghost (bc408b5f, CapCut-verified), timeline always visible, zoom slider, block-dupe fix.
9bd642d4 = the prior handoff.
This session:
R4 — full-width timeline layout (5 commits)
5568958b design, 6ee29eec plan+research, then the strangler refactor:
82d4073eNLEProvider (components/nle/NLEContext.tsx) — liftsuseTimelinePlayer+useCompositionStack+ shared state (timelineH, loading, previewSize) into a context so a preview pane and a full-width timeline pane can be siblings.9a7df4e4extract PreviewPane (components/nle/PreviewPane.tsx) — preview + controls + breadcrumb + preview-only fullscreen + preview block-drop.f2fa323eextract TimelinePane (components/nle/TimelinePane.tsx) — timeline + edit wrappers.f8665794extract PreviewOverlays (components/nle/PreviewOverlays.tsx) + useTimelineEditCallbacks (components/nle/useTimelineEditCallbacks.ts) out of StudioPreviewArea.f364eb44the flip — new EditorShell (components/EditorShell.tsx):[left | preview | right]top row + full-width timeline below + StudioFeedbackBar at bottom; rewiredApp.tsx; deletedNLELayout.tsx+StudioPreviewArea.tsx; barrel exports EditorShell.- Verified live: timeline spans full width (ruler to x≈1592/1680, clip at far right); left sidebar ends above the timeline; Inspector opens in the top row without shrinking the timeline; transport/drill-down/divider all work; no new shell errors. Full studio suite 1394 pass.
R2 Piece 1 — magnetic ghost (follow + placeholder)
233313783 design, e7481905(lane-lock, superseded) → b0bb95aa follow + target-lane
highlight → a35f0dd2 doc revision → 9cbdeb42 clip-sized drop placeholder at the landing
spot. MAGNETIC_TRACK_THRESHOLD = 0.5 in timelineEditing.ts (tunable; user said keep 0.5).
Ghost follows the cursor freely; a clip-width placeholder shows where it lands. Verified live.
R2 2a/2b/2c — collision-push + track insert
0a0231d0design.9d4a8d042a collision-push — new puretimelineCollision.ts(isLaneFree,resolvePlacementprefer-up). Verified: cursor over occupied lane → placeholder hops to nearest free lane (occupancy-checked).ef74a1c62b+2c track insert —buildTrackInsert(minimal-shift) +resolveInsertRow(INSERT_BAND) intimelineCollision.ts;DraggedClipState.insertRow; horizontal insertion line inTimelineCanvas.tsx; pointerup insert branch. Verified live: hover a lane boundary → insertion line → drop inserts a new track, clips below shift down, survives reload, no dup.- Known limitation: a consecutive-lane insert persists each shifted clip via a separate
onMoveElement→ N undo entries + N reloads (gap-inserts are a single clean move). Batched single-undo deferred.
R1 — ruler restyle + frame-mode sync
34b4ff42 — timelineLayout.ts getMinorSubdivisions emits 4 quarter minor ticks (was 1
midpoint); lifted timeDisplayMode ("time"|"frame") from PlayerControls local state into
playerStore (persisted via studioUiPreferences); TimelineRuler.tsx labels frame numbers
(secondsToFrame, 30fps) in frame mode. Verified: ~78 minor ticks; toggling the transport
readout flips ruler labels to frames (0/30/60/90…) and back.
Piece 4 — track model + main-track magnet (partial)
User decisions: enforced zones (option B, full CapCut); build now. 9d3b435d design/analysis.
0cc4489dP4-1 enforced zones — newplayer/components/timelineZones.ts(classifyZone,normalizeToZones): remaps every clip so ascending index = overlay → main → audio (main lane = only video on the main track; audio anywhere → bottom; mixed tracks split by kind). Wired intouseTimelinePlayer.syncTimelineElements(normalizes on every discovery). ExportedisAudioTimelineElementfromutils/timelineInspector.ts. Verified on qa-clean: un-zoned authored order reorganized to Overlay Image (top) / Bg Video (main) / Bg Music (bottom).a2872d0astable main resolution — rewroteresolveMainOriginTrack: (1) explicitdata-timeline-role="main"wins; (2) else the longest-total-duration video track (identity-stable, survives re-index), tie→lowest. Fixes a multi-video drift bug (zoning is now idempotent — unit-proven). This is the right-sized "track metadata" foundation.8cb5f394P4-2 main-track no-overlap —snapClearOfClipsintimelineCollision.tsbutts a clip flush past main-track clips it overlaps; wired so a drop on the main lane snaps clear (other lanes keep 2a push). Verified on qa-clean (2 sequential main videos): dragging one over the other snapped flush (buttedFlush, no overlap).0f1910adextract drag-commit + defer ripple — createdplayer/components/ timelineClipDragCommit.ts(commitDraggedClipMove: insert / plain-move), which also pulleduseTimelineClipDrag.tsback under the 600-line gate. The ripple (4b/4c) is NOT wired here — see §7.1.
6. Load-bearing architecture facts
- The composition HTML on disk is the single source of truth. No in-memory doc model. Edits
= string-patch the HTML (
utils/sourcePatcher.ts, regex) →PUT /api/projects/:id/files/*→ reload preview (or GSAP soft-reload). Persist is per-element (hooks/useTimelineEditing.tshandleTimelineElementMove:patchIframeDomTiminglive +enqueueEdit/sdkTimingPersistsource;enqueueEditserializes via an internal queue but each caller triggers its ownreloadPreview+shiftGsapPositions). - Preview =
<hyperframes-player>wrapping a same-origin iframe; studio reachesiframe.contentWindow.__player/__timelines/__hfdirectly. - Timeline = pure DOM/CSS driven by Zustand
usePlayerStore(player/store/playerStore.ts);elements: TimelineElement[]come from the runtime clip manifest. - Track model (NEW this session): track index is display-only (producer/engine ignore it),
BUT the studio now enforces CapCut zones on discovery (
timelineZones.normalizeToZonesinuseTimelinePlayer.syncTimelineElements). Vertical order (ascending index, top→bottom) = overlay → main → audio. Main track = stable (role="main" or longest video).TRACK_H = 48(constant intimelineLayout.ts) is still used everywhere (R3 will make it per-track). - Element kinds surfaced:
video,img,audio,div/element,composition(sub-comp/ captions).isAudioTimelineElement/isMusicTrackinutils/timelineInspector.ts.
7. REMAINING WORK
7.1 Piece 4 ripple (4b/4c) — BLOCKED on a batched persist (top priority)
The pure reflow logic works, but the per-clip concurrent persist races: shifting several
main clips' data-start at once — each onMoveElement doing its own source-write + GSAP shift
- reload — interleaves and corrupts the file (verified: two videos both ended at start 0).
It was built, live-tested, and reverted (
0f1910ad);timelineClipDragCommit.tshas a NOTE documenting this. To ship the ripple you must first build a batched persist: one source write for ALL shifted clips + one GSAP pass + one reload + single undo. Then re-addreflowMainTrack(contiguous re-lay: sort main clips by intended start, lay end-to-end from 0 — this single op does both gap-close (4b) and insert-ripple (4c)) and wire it intocommitDraggedClipMove's main-track branch. The reflow logic + its 3 unit tests are in git history atef74a1c6…0f1910ad(removed in the revert) if you want to recover them. This batched persist also fixes 2c's N-undo/N-reload limitation.
7.2 R3 — variable track heights (needs a decision)
CapCut main track is bigger. TRACK_H = 48 is used everywhere (TimelineCanvas,
timelineLayout getTimelineCanvasHeight, useTimelineClipDrag, timelineZones lane math,
drop-row resolution, ghost/placeholder/insertion-line Y, beat strip, keyframes). Replacing the
constant with a per-track height + cumulative Y-offset touches every Y↔track site. Open
decision: uniform (true CapCut — research says CapCut heights are actually uniform) vs a taller
main track (deliberate departure).
7.3 Smaller / deferred
- Batched single-undo for consecutive-lane inserts (2c) — same batched-persist need as 7.1.
- Plan 2 (canvas parity: drop onto canvas, 8-handle resize, waveforms-on-clips, optimistic
no-reload insert) and Plan 3 (multi-select, sub-comp-targeted inserts, virtualization) from
ROADMAP.md. - Two spawned cleanup chips (background tasks, may or may not have been run):
(a) remove the dead export
snapKeyframePctToBeatintimelineEditing.ts; (b) extract the remaining pointer handlers fromuseTimelineClipDrag.ts(still complexity-flagged).
8. Gotchas learned (don't re-discover)
- G1 — stale served bundle:
previewserves the CLI's embedded studio copy, refreshed only by a fullbun run build. After any studio change: build, restart/confirm the servedindex-*.jshash changed. - G2 — multi-clip horizontal persist RACES (the big one, §7.1): concurrent per-clip
onMoveElement(start-time changes → GSAP shift + reload each) corrupts the file. Track-index- only changes (2c) don't race as badly (no GSAP/reload heavy path). Needs a batched persist. - G3 — verifying track logic needs a CLEAN project.
qa-project-2is mangled + composition- heavy + no clean audio → can't show zones. Use/make a clean bed (qa-clean). Each synthetic drag test that changes state PERSISTS to the file (mutates the project) — reset between runs. - G4 — synthetic DnD in the harness is async-flaky: a
pointerdownsetsdraggedClipRefvia React state; apointermovein the same eval call fires before the ref commits → drag never arms. Dispatch pointerdown in one eval, pointermove in the next, read in a third. Also read React-state-driven DOM (e.g. play/pause label) in a separate call after the flush. - G5 — fallow gate + filesize:
--no-verifywas used when the ONLY gated findings were pre-existing (e.g.snapKeyframePctToBeatunused export; complexity inuseTimelineClipDrag/blockInstaller). Always run oxlint/oxfmt/tsc/tests manually first. Filesize ≤600 lines/file is enforced —useTimelineClipDrag.tshovers near it (extracting the commit module fixed it). New unused exports get flagged — keep helpers module-private if only used internally. - G6 —
normalizeToZonesmust stay idempotent. Enforced zoning re-runs on every discovery; if the main-track pick isn't stable, tracks drift each reload (this bit us with multiple video tracks → fixed via longest-video/role resolution). Any change here needs the idempotency test. - G7 — the user reference-checks CapCut + gives feel-feedback live. The magnetic ghost was built wrong first (lane-locked) and corrected after the user dragged it. Build → show → adjust.
- G8 — launch.json
--portmust be pinned or the CLI auto-reassigns and the harness points at a dead port (blank screen).
9. Key new/changed files (map)
New (this session):
packages/studio/src/components/EditorShell.tsx— the CapCut shell (top row + full-width timeline).packages/studio/src/components/nle/NLEContext.tsx(+.test.ts) — shared player/comp-stack provider.packages/studio/src/components/nle/PreviewPane.tsx,TimelinePane.tsx,PreviewOverlays.tsx,useTimelineEditCallbacks.ts— decomposed panes.packages/studio/src/player/components/timelineCollision.ts(+.test.ts) —isLaneFree,resolvePlacement,snapClearOfClips,buildTrackInsert,resolveInsertRow.packages/studio/src/player/components/timelineZones.ts(+.test.ts) —classifyZone,normalizeToZones,resolveMainOriginTrack(the track model).packages/studio/src/player/components/timelineClipDragCommit.ts—commitDraggedClipMove(insert / plain-move; ripple deferred). Modified (key):App.tsx,player/hooks/useTimelinePlayer.ts(zoning wired),player/store/ playerStore.ts(timeDisplayMode),player/components/{TimelineCanvas,TimelineRuler,PlayerControls, timelineLayout,timelineEditing,useTimelineClipDrag}.tsx/ts,utils/{timelineInspector, studioUiPreferences}.ts,index.ts(barrel: EditorShell not NLELayout). Deleted:components/nle/NLELayout.tsx,components/StudioPreviewArea.tsx.
10. Test state
Studio suite: the R4 checkpoint was 1394 pass / 18 pre-existing fails (telemetry/client,
telemetry/distinctId, SnapToolbar — reproduce on main, DO NOT chase). Player subset ran 314→331
during R2/P4. New pure modules are unit-tested: timelineCollision (~23), timelineZones (11),
timelineEditing magnetic-threshold, ruler generateTicks (46). tsc + oxlint clean on changed files.
11. Suggested next-session opening
cd <worktree>; confirm branchresearch/studio-dnd-architecture;bun installif needed.- Read this file, then
.superpowers/sdd/progress.md(granular), thenresearch/plans/2026-07-08-track-model-and-main-track-magnet-design.md(Piece 4). bun run build;node packages/cli/dist/cli.js preview --no-open --port=3007 /tmp/hf-dnd-qa/qa-clean; open in browser at 1680×950; confirm zones (image/video/music top→bottom) + no-overlap.- Decide: (a) checkpoint PR the ~47 commits first (strongly recommended), then (b) build the batched persist → ripple (§7.1), then R3 heights (§7.2).
12. Open decision pending at handoff
Whether to push a checkpoint PR now (first push of this branch, ~47 commits) before the batched-persist/ripple work. Offered to the user at end of session; not yet answered. Nothing is pushed or merged.