mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-08 10:46:06 +00:00
* feat(studio): mirror canvas z-order actions into timeline lanes, badge z overrides
Track order = default paint order; authored z = advanced override.
- timelineZMirror.ts: pure resolver mapping a successful z-menu action to a
timeline lane move — closest track in the action's direction that is free
over the clip's whole span, else a new lane adjacent to the crossed
neighbor; temporal-overlap scope (default pending product sign-off, see
module doc); visual zone only; same-file reference scoping; persistTrack
via the shared authored-space rules. null for non-clips (menu stays
z-only) and at-extreme/no-overlap cases.
- useCanvasZOrderTimelineMirror.ts: after the z commit resolves, the mirror
persists the lane move through the same machinery as a timeline lane drag
(optimistic store update, authoredTrack refresh, rollback); inserts reuse
commitTrackInsert's renumber via a shared buildTrackInsertEdits core. Both
writes share one coalesce key (zReorderCoalesceKey) and fold into ONE undo
entry (test proves it over the real history reducer). The mirror never
triggers the lane->z stacking sync, so it cannot fight the z values the
action just set.
- timelineZOverride.ts + TimelineClip badge: clips whose paint order
contradicts lane order among temporally-overlapping same-context visual
neighbors (laneIsAbove XOR paintsAbove, the stacking-sync predicates) show
a 'z' badge — authored z overrides are surfaced instead of silently
disagreeing with the timeline.
- Timeline.tsx track derivations extracted to useTimelineTrackDerivations
(600-line cap).
* fix(studio): fold mirrored z-order gestures into one undo entry across slow persists
Live verification caught the z write and the mirrored lane write splitting
into two undo entries: the mirror runs after the z persist's server round
trip, which exceeds editHistory's default 300ms coalesce window under real
latency (the unit test's deterministic clock sat inside it).
zReorderCoalesceKey now mints a per-gesture-unique key (monotonic seq, the
laneChangeGestureSeq precedent) and both records carry coalesceMs Infinity —
distinct gestures can never merge, and one gesture always folds regardless
of write latency. coalesceMs threaded through the persist chain alongside
coalesceKey. Also hardens the existing lane-drag move->z fold, which had the
same latent split. Fold test now simulates a 400ms gap (failed before the
fix, passes after); a two-separate-gestures test asserts two entries.
* feat(studio): flashless lane mirror, z-order menu icons, close-gap track menu
- Track-only batch moves (the z-mirror's lane hop and the insert renumber)
skip the GSAP fallback round-trip and the preview reload entirely — the
renderer never reads data-track-index, and the live DOM patch + optimistic
store update cover the UI. Mixed batches keep current behavior. Kills the
canvas blink on mirrored Bring/Send actions (live-verified: an
iframe-scoped marker survives the whole gesture).
- The four z-order menu items get 16px stroke icons (single layer diamond +
directional arrow for Forward/Backward; pierced two-layer stack for
Front/Back); labels unchanged — they are the industry-standard names.
- New track context menu on empty lane space: 'Close gap' (shifts the next
clip and every clip after it on that lane left by the clicked gap's width;
leading gaps count, so a single clip with empty space before it compacts
to 0) and 'Close all gaps' (whole lane contiguous from 0). Pure gap math
in timelineGaps.ts; persists through the drag path's atomic batch move
(one undo per action); refuses when a clip that must shift is locked;
items disable when there is nothing to close.
* fix(studio): rebind-only preview sync for unmutated timing edits, classical z-menu order
Timing edits that rewrote NO GSAP positions (gap closes and moves of
selector-addressed caption clips, zero-delta batches, comps without a
rewritable script) full-reloaded the preview — and the rerun-current-scripts
attempt was wrong for real compositions: re-executing init-style scripts
(three.js scenes, caption engines) is exactly the unsafe case, verified live
by doubled init warnings and a fallback reload anyway.
The correct observation: when mutated === false the existing __timelines are
still valid — only the runtime's clip visibility windows are stale, and the
live DOM timing attributes were already patched. So the no-mutation path now
runs applySoftReloadFinalization only (seek + __hfForceTimelineRebind +
manual-edits reapply), extracted from the soft-reload machinery — zero
script execution. This also un-blinks comps with no GSAP script at all,
which previously always remounted. Rewritten-script soft reloads,
cannot-soft-reload, otherFileChanged, and mutation failures keep their
existing behavior. gsapSoftReload's undo/redo restore section moved verbatim
to gsapUndoRestore.ts for the 600-line cap.
Also: z-order menu items reordered to the classical arrangement (Bring to
Front, Bring Forward, Send Backward, Send to Back).
Live-verified on a three.js-heavy composition: Close-all-gaps shifted 4
caption clips with correct cumulative amounts, the preview iframe was never
remounted (marker survived), and one undo reverted everything.
* fix(studio): bound forward/backward mirror to a one-element step
User-specified semantic: Bring Forward / Send Backward move the clip past
EXACTLY ONE element. The mirror's lane target is now bounded by the next
temporally-overlapping element beyond the crossed neighbor: a free lane
strictly between the two is taken (closest to the neighbor), and when they
are back-to-back a new track is inserted immediately beyond the crossed
element — never past the second one. Previously the resolver took the
closest free lane anywhere beyond the neighbor, which could carry the track
past a second element while the z action only stepped past one — a
track/paint contradiction our own zOverride badge would flag. Front/back
keep whole-set semantics (past everything; back stays above the audio
zone). End-to-end test pins the 3-stacked case through commitZMirrorLaneMove
to the persisted renumbered tracks.
* feat(studio): permanent gap-menu rows with hover and click-select gap highlights
- TrackGapContextMenu always renders both rows; an inapplicable action dims
with a tooltip ("No gap here" / lock reason / "No gaps on this track")
instead of vanishing into a one-item menu. Width badge only when a gap
exists under the pointer.
- Hovering an ACTIONABLE row highlights the strip(s) it would close in the
timeline: the single gap for Close gap, every current gap (leading included)
for Close all gaps. New resolveAllGapIntervals in timelineGaps.ts reports
present-state intervals (epsilon-tolerant, overlap-safe), distinct from
resolveAllTrackGaps' post-compaction starts.
- Click-selecting a single clip paints a quieter tint over its lane's gaps
(suppressed for marquee multi-selection and during drags; the gap-menu hover
wins on its own lane). Derivation lives in useTimelineGapHighlights with the
pure buildTimelineGapStrips exported and unit-tested.
- Strips render in TimelineCanvas with the drop-placeholder geometry (row top
+ clip inset), dashed accent for hover, faint tint for selection.
- Timeline.tsx stayed under the 600-line cap by extracting the scroll-viewport
plumbing (ResizeObserver width + shortcut-hint sync) into
useTimelineScrollViewport, behavior unchanged.
* feat(studio): stronger capcut-style timeline zoom steps
One button press / pinch gesture now moves the zoom meaningfully: step
factors 1.25x/0.8x -> 1.5x/(2/3) (kept reciprocal so in+out round-trips) and
pinch sensitivity 0.0035 -> 0.007. Addresses "zooming several times to get
anywhere" feedback; cursor anchoring unchanged.
* feat(studio): three-way z sync — layers drags mirror timeline lanes, panel tracks live z edits
Completes the layers/canvas/timeline sync triangle: the Layers panel was the
one surface whose reorders never reached the timeline, and the one that went
stale when the other two wrote z flashlessly.
- Layers drag -> minimal z + equal-jump lane mirror. handleReorder now uses
the canvas menu's realization core via resolveZOrderReposition (one
between-z write when a strict gap exists, band-safe scoped renumber
otherwise) instead of computeReorderZValues' all-sibling stamp — that
helper is deleted, completing the #2347 unification follow-up. The drop
then mirrors into a timeline lane move through the same machinery as the
canvas menu (new resolveRepositionLaneMove: the clip lands on a free lane
strictly between its NEW paint neighbors' lanes — nearest clip siblings in
the desired render order, decorations skipped — else a track insert at
that boundary; audio zone never crossed). Both writes share one
per-gesture zReorderCoalesceKey with an unbounded fold window, so a drag
is exactly ONE undo entry; useCanvasZOrderTimelineMirror's plumbing is
factored into useMirrorLaneMoveCommit and reused by the new
useLayerReorderTimelineMirror. A same-slot drop is a hard no-op (new
order-equality guard in resolveZOrderReposition).
- Panel staleness fix: flashless z commits (skipReload) reload nothing and
bump no refreshKey, so the panel's z-sorted order went stale while paused.
handleDomZIndexReorderCommit now bumps a store zEditVersion on apply AND
rollback; the panel re-collects on it. Verified live: the panel re-sorts
the instant a drag commits and again on undo.
- Layer click reveal (useLayerRevealOverride): clicking a layer that stays
hidden at the current frame (animation-parked opacity, non-clip
display/visibility hides, hidden ancestors) temporarily forces the chain
visible with live inline styles — exact priors restored on deselect, on
another reveal, on play, and on unmount; never persisted (file diff == 0
verified live). Clips keep the existing seek-into-window behavior; the
override applies on a short defer so a seek-revealed clip needs none.
- layerOrdering's unused hasExplicitZIndex probe (zero callers) removed.
Live-verified on a bed copy: a 2-position layers drag wrote exactly one
element (z 6->23 + data-track-index 15->2), the timeline lane moved without
a reload, and a single Cmd+Z restored the file byte-identically.
* feat(studio): full-track selection highlight, borderless gap hover strips
- Click-selecting a clip now lights the WHOLE lane minus its clips — leading
gap, inter-clip gaps, and the open space after the last clip to the rendered
end (new resolveLaneEmptyIntervals; displayDuration threaded into the strip
derivation). Still click-only: any drag/resize suppresses the strips, and a
marquee multi-select never shows them.
- The gap-menu hover strips drop the dashed border (user feedback) — fill only,
nudged to 0.18 alpha to keep the same visual weight.
* feat(studio): selected layer paints on top via a reader-transparent z lift
Clicking a layer in the Layers tab now shows the element as if it were at the
very top of the stack while selected — whatever its authored z or panel
position — extending the reveal override (which already forced hidden chains
visible) with a temporary inline z lift:
- liftElementToTop parks the TRUE effective z in data-hf-reveal-prior-z and
writes a far-top inline z; a static element gets a layout-preserving
position:relative with its prior parked in data-hf-reveal-prior-pos. Only
the RENDERER sees the lift: all three studio z readers
(readTimelineElementZIndex, getElementZIndex, readEffectiveZIndex) return
the parked prior while the attribute is present, so the canvas z-menu, the
zOverride badge, the lane mirror, the stacking sync, and the panel sort
keep reasoning on the element's real z.
- Strictly ephemeral: exact priors restored on deselect / another reveal /
play / unmount, each property only while it still holds the value the
override wrote (a later real edit is never clobbered). File diff == 0
verified live across a full lift/restore cycle.
- A z-reorder commit CONSUMES an active lift (handleDomZIndexReorderCommit
reads the parked position for its persist-position:relative static check,
then drops the attributes) — the committed z becomes the truth and the
later restore is a guarded no-op.
* fix(studio): flashless undo/redo — three full-reload causes in the soft-restore path
Cmd+Z blinked the canvas on essentially every undo. Three independent causes
in applyUndoRestoreToPreview, each sufficient on its own:
1. Master-view path gate: activeCompPath is NULL at the master view, so the
'paths[0] === activeCompPath' eligibility check could never match the
index.html restore and every default-view undo full-reloaded at the first
gate. Normalized to the codebase-wide 'activeCompPath ?? "index.html"'.
2. Nested identity innerHTML check: the diff compared each identified
element's innerHTML, but the composition root wraps every clip — any child
change re-detected at the root rejected the restore. Change detection now
compares only each element's OWN attribute surface; structure/text
integrity is still guaranteed by the normalize-residual whole-doc pass
(text nodes, added/removed elements, and un-identified attrs all remain
after normalization and force the full reload).
3. id-only identity: elements addressed by data-hf-id / selector (no DOM id)
fell outside the diff entirely. Identity is now id OR data-hf-id, with the
live sync resolving either.
Also stop re-running an UNCHANGED GSAP script: attribute-only restores (z,
lane, timing, style — the overwhelmingly common undo) now use the rebind-only
finalization (seek + __hfForceTimelineRebind + manual reapply, zero script
execution — the same path as flashless timing edits), instead of tearing down
and rebuilding live timelines or full-reloading when the script can't be
scoped. A restore whose script text genuinely changed still re-runs it via
applySoftReload, and structural restores (split/delete) still full-reload.
Live-verified on the bed (iframe marker): gap-close undo AND redo both keep
the iframe mounted, live DOM lands on the restored values, disk restored
byte-identically.
* feat(studio): left breathing pad before t=0, double zoom sensitivity again
TRACKS_LEFT_PAD (48px) — the horizontal sibling of TRACKS_TOP_PAD: empty lane
surface between the sticky gutter and the ruler's 00:00 / the first clips,
scrolling WITH the content.
- The lanes and the ruler realize it as a plain flow spacer between the
sticky gutter cell and the time-mapped content div, so every
content-relative computation (clip left = t*pps, beat lines, lane-menu
time, clip drag deltas) is untouched by construction.
- Canvas-space overlays shift by the pad: playhead (getTimelinePlayheadLeft),
gap strips, drop placeholder, snap guide, range highlight, marquee clip
rects, beat SVG; the insert line spans the pad.
- Every pointer->time inverse subtracts it symmetrically: seekFromX, razor,
range/marquee anchors, asset drops, and the zoom-anchor gutter basis; fit
pps and the display width account for the consumed viewport width.
- Live-verified: t=0 clip edge, the 00:00 tick, and the playhead line center
all sit at GUTTER + TRACKS_LEFT_PAD, and a ruler click lands the playhead
center exactly under the pointer.
Also doubles the timeline zoom sensitivity again (user feedback after
feel-testing the first bump): button steps 1.5x/(2/3) -> 2x/0.5, pinch
0.007 -> 0.014.
* fix(studio): left pad renders as true empty space, not lane surface
The pad before t=0 inherited each row's background and bottom border from the
row wrapper, so it read as track lanes. Lane visuals now live on the cells:
the sticky gutter keeps its own separator (header column stays delineated),
the time-mapped content div carries the row background + separator, and the
pad spacer stays transparent — bare shell background, no lines. The
new-track insertion line also starts at the pad's end instead of crossing it.
* fix(studio): no vertical line in the ruler band before 00:00
The ruler corner's right border drew the header-boundary line through the
ruler strip, so the band didn't read as starting at 00:00. Dropped it — the
boundary line belongs to the track rows below; the ruler stays completely
clean from the panel edge to the first tick, matching the empty left pad.
* refactor(studio): remove the timeline z-override badge
User decision: the "z" chip on clips never earned its place — dropped
entirely (timelineZOverride.ts + test deleted, TimelineClip badge rendering
and the zOverrideKeys derivation/threading removed). This also eliminates the
review's D2 finding at the root: the badge's cross-document comparison
(stackingContextId ?? null collides across source files in the expanded view)
produced false positives, and there is no longer a detector to mis-fire.
overlapsInTime/paintsAbove lose their export (the badge was their only
external consumer); the paint-order predicate itself is unchanged.
* fix(studio): collision-free expanded child lanes and host-window gap floors
Review findings D1 (blocker) and 4.
- D1: buildChildElements assigned expanded children synthetic display rows as
`host.track + index` — integers that can EQUAL a real clip's lane in another
file (host on 0 with two children puts child #2 on 1). Lane grouping merges
purely by track number, so the collision fused clips from different source
files into one display lane, and lane-scoped actions (the gap menu) then
batch-persisted a foreign file's clip. Children now take FRACTIONS strictly
between the host's lane and the next integer — structurally unable to
collide with any normalized lane, while still rendering as ordered rows
under the host. Regression test pins the reviewer's exact two-file scenario.
- Finding 4: gap math compacted toward absolute 0, but an expanded child's
display time is host-anchored — close/compact could drag it before its host
window and persist a wrong (even negative) local time. All gap functions
now take a lane FLOOR (laneGapFloor: 0 for ordinary lanes, the children's
expandedParentStart for child lanes — single-origin per lane post-D1),
threaded through the menu model, hover highlights, selected-lane strips,
and both commits. Close-gap shifts clamp at the gap's own left edge.
* fix(studio): scope mirror references, insert writes, and crossed-neighbor identity
Review findings 1, 2, and 3.
- Finding 1: buildTrackInsertEdits normalized the FULL display set and
persisted every shifted clip — writing host-lane numbers into OTHER
composition files when expanded children were showing. The renumber write
set is now the edited element's own source file (the sanctioned multi-write
converges one FILE to lane space, never neighbors' files); foreign clips
keep their authored tracks and re-derive display lanes. The locked-clip
refusal scopes the same way. Expanded-origin elements refuse the insert
outright (a new lane is a host-space renumber, meaningless in the child's
file), and the mirrors restrict an expanded child's lane candidates to its
own siblings' lanes — a sub-comp child still mirrors WITHIN its sub-comp
(persisting the sibling's authored track) but can never land on a host lane
with no same-file occupant. authoredTrackForLane's offset fallback rounds:
fractional synthetic rows must never leak fractions into data-track-index.
- Finding 2: the mirror comparison sets required only sameSourceFile, but a
file can contain several CSS stacking contexts and leaf z is only
comparable within one. Both resolvers now scope by samePaintScope — same
source file AND same stackingContextId (the file check also stops null root
contexts of different files from comparing equal in the expanded view).
- Finding 3: the crossed-neighbor key was derived without selectorIndex, so
duplicate class selectors (.sub) resolved to occurrence 0 — a different
clip. The key now carries getSelectorIndex, matching how z-reorder entries
derive theirs.
* fix(studio): z-to-lane gestures are one serialized transaction gated on durable persists
Review findings 5 and 7.
- Finding 5: commitDomEditPatchBatches resolved successfully even when the
server matched NO patch target — the z write never reached disk (the
preview reloads to reconverge) yet the lane mirror still ran, desyncing
track order from what actually paints. The commit now resolves a durability
report ({allMatched, changed}; the save queue and commit types are generic
over the result), and the mirror phase is skipped on allMatched === false.
- Finding 7: the z persist rides the DOM-edit save queue while the lane move
rides the timeline/SDK path — two queues, so a second rapid gesture's z
write could land BETWEEN the first gesture's z and lane phases. Every
z-to-lane gesture (canvas z-order menu AND Layers-panel drag) now runs
through runZLaneGesture: a single module-level tail that serializes the
COMPLETE two-phase transaction, with unit tests for ordering, the
durability gate, and queue resilience to failed gestures. The timeline
lane-drag's inverse (move-then-z-sync) shares its phases' await ordering
already; cross-gesture serialization for that path is noted as follow-up.
- LayersPanel's pure sort helpers moved to layersPanelSort.ts (600-line cap).
* fix(studio): multi-clip GSAP batch mutations roll back on late failure
Review finding 6. finishGroupTimingGsapFallback mutates files sequentially
per clip; a late per-clip failure left the earlier rewrites on disk with no
aggregate history entry — unreachable by undo. foldGsapMutationIntoHistory
already snapshots every touched path before mutating; on a mutation failure
it now restores each path whose disk content changed (all-or-nothing batch),
reports restore errors without masking the original failure, and rethrows.
Regression test drives a two-clip batch whose second rewrite fails and
asserts the first clip's write is restored byte-identically.
* fix(studio): scope mirror inserts to their lane zone
* fix(studio): unify source-scoped clip identity
* fix(studio): isolate track insert topology
* fix(studio): harden timeline paint synchronization
---------
Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
461 lines
19 KiB
TypeScript
461 lines
19 KiB
TypeScript
/**
|
|
* React callbacks for synchronising the player store from iframe runtime data.
|
|
*
|
|
* Covers four related concerns:
|
|
* - processTimelineMessage — turn a clip-manifest postMessage into TimelineElements
|
|
* - enrichMissingCompositions — fill gaps the manifest misses (element-ref starts)
|
|
* - initializeAdapter — called after iframe load: seek, set duration, read elements
|
|
* - onIframeLoad — orchestrates initializeAdapter with a message-based fallback
|
|
*/
|
|
|
|
import { useCallback } from "react";
|
|
import { liveTime, usePlayerStore } from "../store/playerStore";
|
|
import type { TimelineElement, DomClipChild } from "../store/playerStore";
|
|
import { resolveCssStackingContextId } from "@hyperframes/core/runtime/stacking-context";
|
|
import type { PlaybackAdapter, ClipManifestClip, IframeWindow } from "../lib/playbackTypes";
|
|
import {
|
|
parseTimelineFromDOM,
|
|
createTimelineElementFromManifestClip,
|
|
findTimelineDomNodeForClip,
|
|
createImplicitTimelineLayersFromDOM,
|
|
buildStandaloneRootTimelineElement,
|
|
getTimelineElementSelector,
|
|
readTimelineDurationFromDocument,
|
|
} from "../lib/timelineDOM";
|
|
import {
|
|
normalizePreviewViewport,
|
|
autoHealMissingCompositionIds,
|
|
buildMissingCompositionElements,
|
|
} from "../lib/timelineIframeHelpers";
|
|
import { acceptedRuntimeMessageFps, inspectStudioRuntimeMessage } from "../lib/runtimeProtocol";
|
|
|
|
interface UseTimelineSyncCallbacksParams {
|
|
iframeRef: React.RefObject<HTMLIFrameElement | null>;
|
|
probeIntervalRef: React.MutableRefObject<ReturnType<typeof setInterval> | undefined>;
|
|
pendingSeekRef: React.MutableRefObject<number | null>;
|
|
isRefreshingRef: React.MutableRefObject<boolean>;
|
|
getAdapter: () => PlaybackAdapter | null;
|
|
syncTimelineElements: (elements: TimelineElement[], nextDuration?: number) => void;
|
|
setDuration: (v: number) => void;
|
|
setCurrentTime: (v: number) => void;
|
|
setTimelineReady: (v: boolean) => void;
|
|
setIsPlaying: (v: boolean) => void;
|
|
attachIframeShortcutListeners: () => void;
|
|
applyPreviewAudioState: () => void;
|
|
}
|
|
|
|
/**
|
|
* Where should the player seek when the preview (re)loads?
|
|
* Priority: explicit pending seek (saved by refreshPlayer right before a
|
|
* reload) → store-level seek request (deep-link `?t=` hydration) → the store's
|
|
* last known playhead. The last fallback makes the playhead RELOAD-INVARIANT:
|
|
* edits persist + reload the preview, sometimes more than once (App's
|
|
* refreshPreviewDocumentVersion staggers extra bumps at 80/300ms), and the
|
|
* consume-once pendingSeekRef meant any reload after the first found the slot
|
|
* empty and reset the playhead to 0 — the "dropped a file and the playhead
|
|
* jumped to 0" bug. Falling back to the store's playhead means every reload
|
|
* restores position; a fresh project load still starts at 0 because the store
|
|
* resets currentTime on project switch. Invariant: an edit NEVER moves the
|
|
* playhead (the clamp below is the one sanctioned move — content shrank past it).
|
|
*/
|
|
/**
|
|
* Undo the `visibility: hidden` that refreshPlayer sets across a full reload.
|
|
* Safe to call when the iframe was never hidden (idempotent no-op). Every reload
|
|
* completion + failure path funnels through here so the preview can never get
|
|
* stuck invisible.
|
|
*/
|
|
export function revealIframe(iframe: HTMLIFrameElement | null): void {
|
|
if (iframe && iframe.style.visibility === "hidden") {
|
|
iframe.style.visibility = "";
|
|
}
|
|
}
|
|
|
|
export function resolveReloadSeekTime(input: {
|
|
pendingSeek: number | null;
|
|
requestedSeek: number | null;
|
|
storeCurrentTime: number;
|
|
duration: number;
|
|
}): number {
|
|
const target = input.pendingSeek ?? input.requestedSeek ?? input.storeCurrentTime;
|
|
if (!Number.isFinite(target) || target <= 0) return 0;
|
|
// Only clamp to duration when it's a usable positive number. A non-finite or
|
|
// non-positive duration (e.g. the adapter reports NaN mid-reload) would turn
|
|
// Math.min(target, NaN) into NaN and seek(NaN); return the guarded target
|
|
// unclamped instead so the playhead lands at the intended position.
|
|
if (!Number.isFinite(input.duration) || input.duration <= 0) return target;
|
|
return Math.min(target, input.duration);
|
|
}
|
|
|
|
/** Reject non-finite, non-positive, and absurdly large (loop-inflated) values. */
|
|
function sanitizeDurationSeconds(value: number): number {
|
|
return Number.isFinite(value) && value > 0 && value < 7200 ? value : 0;
|
|
}
|
|
|
|
/**
|
|
* The transport TOTAL a clip-manifest message should write to the store.
|
|
*
|
|
* The manifest's `durationInFrames` measures the runtime timeline; some runtimes
|
|
* report only the furthest clip end and ignore the root composition's authored
|
|
* `data-duration`. When that manifest total is SHORTER than the authored root
|
|
* duration, writing it makes the readout stale (playback still runs the full
|
|
* authored window — the user saw "0:44/0:40" on a root authored at 44.5s whose
|
|
* last clip ends at 40s). The authored root duration is the floor for the total,
|
|
* so the readout can never sit below what the file declares. A manifest total
|
|
* that is LONGER (clips extend past the root) still wins — content can only grow
|
|
* the timeline, never shrink it below the authored window.
|
|
*/
|
|
export function resolveTimelineTotalDuration(input: {
|
|
manifestDurationSeconds: number;
|
|
authoredRootDurationSeconds: number;
|
|
}): number {
|
|
return Math.max(
|
|
sanitizeDurationSeconds(input.manifestDurationSeconds),
|
|
sanitizeDurationSeconds(input.authoredRootDurationSeconds),
|
|
);
|
|
}
|
|
|
|
export function useTimelineSyncCallbacks({
|
|
iframeRef,
|
|
probeIntervalRef,
|
|
pendingSeekRef,
|
|
isRefreshingRef,
|
|
getAdapter,
|
|
syncTimelineElements,
|
|
setDuration,
|
|
setCurrentTime,
|
|
setTimelineReady,
|
|
setIsPlaying,
|
|
attachIframeShortcutListeners,
|
|
applyPreviewAudioState,
|
|
}: UseTimelineSyncCallbacksParams) {
|
|
// Convert a runtime timeline message (from iframe postMessage) into TimelineElements
|
|
const processTimelineMessage = useCallback(
|
|
(data: {
|
|
clips: ClipManifestClip[];
|
|
durationInFrames: number;
|
|
scenes?: Array<{ id: string; label: string; start: number; duration: number }>;
|
|
protocolVersion?: unknown;
|
|
capabilities?: unknown;
|
|
fps?: unknown;
|
|
}) => {
|
|
if (!data.clips || data.clips.length === 0) {
|
|
return;
|
|
}
|
|
|
|
usePlayerStore.getState().setClipManifest(data.clips);
|
|
|
|
// Show root-level clips: no parentCompositionId, OR parent is a "phantom wrapper"
|
|
const clipCompositionIds = new Set(data.clips.map((c) => c.compositionId).filter(Boolean));
|
|
const filtered = data.clips.filter(
|
|
(clip) => !clip.parentCompositionId || !clipCompositionIds.has(clip.parentCompositionId),
|
|
);
|
|
let iframeDoc: Document | null = null;
|
|
try {
|
|
iframeDoc = iframeRef.current?.contentDocument ?? null;
|
|
} catch {
|
|
iframeDoc = null;
|
|
}
|
|
|
|
try {
|
|
const iframeWin = iframeRef.current?.contentWindow as
|
|
| (Window & { __clipTree?: import("@hyperframes/core/runtime/clipTree").ClipTree })
|
|
| null;
|
|
const clipTree = iframeWin?.__clipTree;
|
|
const parentMap = new Map<string, string>();
|
|
if (clipTree) {
|
|
const walk = (nodes: typeof clipTree.roots) => {
|
|
for (const node of nodes) {
|
|
if (node.id && node.parentId) parentMap.set(node.id, node.parentId);
|
|
if (node.children.length > 0) walk(node.children);
|
|
}
|
|
};
|
|
walk(clipTree.roots);
|
|
}
|
|
|
|
// Descend into each sub-composition host: its internal elements (group
|
|
// wrappers + their children) carry no `data-start`, so the clip
|
|
// tree/manifest never enumerate them. Surface them studio-side as DOM
|
|
// children + parent links so the timeline can expand a sub-comp/group
|
|
// row to show them. Manifest stays lean (timed clips only).
|
|
const domClipChildren: DomClipChild[] = [];
|
|
if (iframeDoc) {
|
|
for (const clip of data.clips) {
|
|
if (clip.kind !== "composition" || !clip.id) continue;
|
|
const hostEl = iframeDoc.getElementById(clip.id);
|
|
if (!hostEl) continue;
|
|
const hostId = clip.id;
|
|
const innerRoot = hostEl.querySelector("[data-hf-inner-root]") ?? hostEl;
|
|
// Collect the sub-comp's id'd descendants (grouped OR ungrouped) so they
|
|
// expand into timeline rows. Descends through id-less structural wrappers
|
|
// (the inlined sub-comp body), and one level into groups for drill-in.
|
|
const collect = (parentEl: Element, parentId: string) => {
|
|
for (const child of Array.from(parentEl.children)) {
|
|
if (!child.id) {
|
|
collect(child, parentId); // unwrap id-less structural containers
|
|
continue;
|
|
}
|
|
const isGroup = child.hasAttribute("data-hf-group");
|
|
domClipChildren.push({
|
|
id: child.id,
|
|
parentId,
|
|
hostId,
|
|
label: isGroup ? child.getAttribute("data-hf-group") || child.id : child.id,
|
|
stackingContextId: resolveCssStackingContextId(child),
|
|
});
|
|
parentMap.set(child.id, parentId);
|
|
if (isGroup) collect(child, child.id);
|
|
}
|
|
};
|
|
collect(innerRoot, hostId);
|
|
}
|
|
}
|
|
usePlayerStore.getState().setClipParentMap(parentMap);
|
|
usePlayerStore.getState().setDomClipChildren(domClipChildren);
|
|
} catch {
|
|
// cross-origin or __clipTree not available — maps stay empty
|
|
}
|
|
|
|
const usedHostEls = new Set<Element>();
|
|
const els: TimelineElement[] = filtered.map((clip, index) => {
|
|
const hostEl = iframeDoc
|
|
? findTimelineDomNodeForClip(iframeDoc, clip, index, usedHostEls)
|
|
: null;
|
|
if (hostEl) usedHostEls.add(hostEl);
|
|
return createTimelineElementFromManifestClip({
|
|
clip,
|
|
fallbackIndex: index,
|
|
doc: iframeDoc,
|
|
hostEl,
|
|
});
|
|
});
|
|
const rawDuration = data.durationInFrames / acceptedRuntimeMessageFps(data);
|
|
// Clamp non-finite or absurdly large durations — the runtime can emit
|
|
// Infinity when it detects a loop-inflated GSAP timeline without an
|
|
// explicit data-duration on the root composition. Floor the manifest total
|
|
// at the authored root `data-duration` so a runtime that measures only the
|
|
// furthest clip end (shorter than the authored window) can't leave a stale,
|
|
// too-short total in the transport (the "0:44/0:40" bug).
|
|
const newDuration = resolveTimelineTotalDuration({
|
|
manifestDurationSeconds: rawDuration,
|
|
authoredRootDurationSeconds: readTimelineDurationFromDocument(iframeDoc),
|
|
});
|
|
const effectiveDuration = newDuration > 0 ? newDuration : usePlayerStore.getState().duration;
|
|
const clampedEls =
|
|
effectiveDuration > 0
|
|
? els
|
|
.filter((element) => element.start < effectiveDuration)
|
|
.map((element) => ({
|
|
...element,
|
|
duration: Math.min(element.duration, effectiveDuration - element.start),
|
|
}))
|
|
.filter((element) => element.duration > 0)
|
|
: els;
|
|
const timelineEls =
|
|
iframeDoc && effectiveDuration > 0
|
|
? [
|
|
...clampedEls,
|
|
...createImplicitTimelineLayersFromDOM(iframeDoc, effectiveDuration, clampedEls),
|
|
]
|
|
: clampedEls;
|
|
if (timelineEls.length > 0) {
|
|
syncTimelineElements(timelineEls, newDuration > 0 ? newDuration : undefined);
|
|
}
|
|
},
|
|
[iframeRef, syncTimelineElements],
|
|
);
|
|
|
|
const enrichMissingCompositions = useCallback(() => {
|
|
try {
|
|
const iframe = iframeRef.current;
|
|
const doc = iframe?.contentDocument;
|
|
const iframeWin = iframe?.contentWindow as IframeWindow | null;
|
|
if (!doc || !iframeWin) return;
|
|
|
|
const currentEls = usePlayerStore.getState().elements;
|
|
const rootDuration = usePlayerStore.getState().duration;
|
|
const { missing, updatedEls, patched } = buildMissingCompositionElements(
|
|
doc,
|
|
iframeWin,
|
|
currentEls,
|
|
rootDuration,
|
|
);
|
|
|
|
if (missing.length > 0 || patched) {
|
|
// Dedup: ensure no missing element duplicates an existing one
|
|
const finalIds = new Set(updatedEls.map((e) => e.id));
|
|
const dedupedMissing = missing.filter((m) => !finalIds.has(m.id));
|
|
syncTimelineElements([...updatedEls, ...dedupedMissing]);
|
|
}
|
|
} catch {}
|
|
}, [iframeRef, syncTimelineElements]);
|
|
|
|
const initializeAdapter = useCallback(() => {
|
|
const adapter = getAdapter();
|
|
if (!adapter || adapter.getDuration() <= 0) return false;
|
|
|
|
adapter.pause();
|
|
// Honor a seek requested before the adapter was ready. It may sit in either
|
|
// place: `pendingSeekRef` if the store subscription was mounted when requestSeek
|
|
// fired, or only in the store's `requestedSeekTime` if it fired earlier still
|
|
// (deep-link hydration runs before the player subscription mounts, so the request
|
|
// never reaches pendingSeekRef). Reconciling with the store here is what makes a
|
|
// deep-linked `?t=` land instead of starting at 0.
|
|
const storeSeek = usePlayerStore.getState().requestedSeekTime;
|
|
const startTime = resolveReloadSeekTime({
|
|
pendingSeek: pendingSeekRef.current,
|
|
requestedSeek: storeSeek,
|
|
storeCurrentTime: usePlayerStore.getState().currentTime,
|
|
duration: adapter.getDuration(),
|
|
});
|
|
pendingSeekRef.current = null;
|
|
if (storeSeek != null) usePlayerStore.getState().clearSeekRequest();
|
|
|
|
// Force a REAL render at startTime, not a no-op. After a post-edit reload the
|
|
// freshly rebuilt GSAP timeline can already report being at `startTime`
|
|
// internally (the reload restores the same playhead), so a single
|
|
// `adapter.seek(startTime)` is a GSAP no-op — `tl.seek(t)` at the current time
|
|
// doesn't re-evaluate. That's why a just-dropped clip stayed invisible until
|
|
// the user nudged the playhead: its element's state was never applied at the
|
|
// restore position. Seeking to a DIFFERENT guard value first (a hair off, or 0
|
|
// when startTime is already ~0) guarantees the follow-up seek to `startTime`
|
|
// crosses a time boundary and re-renders every clip — including the new one.
|
|
const guardTime = startTime > 0.001 ? Math.max(0, startTime - 0.001) : 0.001;
|
|
adapter.seek(guardTime);
|
|
adapter.seek(startTime);
|
|
// The correct frame is now rendered — reveal the iframe that refreshPlayer hid
|
|
// for the reload, so the user sees the restored frame directly (never the raw
|
|
// all-clips DOM). Cleared unconditionally: any later failure path must not leave
|
|
// the preview stuck invisible.
|
|
revealIframe(iframeRef.current);
|
|
// Keep non-React listeners such as the capture link and time display in sync
|
|
// with the initial adapter seek on iframe load.
|
|
liveTime.notify(startTime);
|
|
const adapterDur = adapter.getDuration();
|
|
if (
|
|
Number.isFinite(adapterDur) &&
|
|
adapterDur > 0 &&
|
|
adapterDur < 7200 &&
|
|
adapterDur !== usePlayerStore.getState().duration
|
|
) {
|
|
setDuration(adapterDur);
|
|
}
|
|
setCurrentTime(startTime);
|
|
if (!isRefreshingRef.current) {
|
|
setTimelineReady(true);
|
|
}
|
|
isRefreshingRef.current = false;
|
|
setIsPlaying(false);
|
|
|
|
try {
|
|
const iframe = iframeRef.current;
|
|
const doc = iframe?.contentDocument;
|
|
const iframeWin = iframe?.contentWindow as IframeWindow | null;
|
|
if (doc && iframeWin) {
|
|
normalizePreviewViewport(doc, iframeWin);
|
|
autoHealMissingCompositionIds(doc);
|
|
attachIframeShortcutListeners();
|
|
}
|
|
|
|
const manifest = iframeWin?.__clipManifest;
|
|
if (manifest && manifest.clips.length > 0) {
|
|
processTimelineMessage(manifest);
|
|
}
|
|
enrichMissingCompositions();
|
|
applyPreviewAudioState();
|
|
|
|
if (usePlayerStore.getState().elements.length === 0 && doc) {
|
|
const els = parseTimelineFromDOM(doc, adapter.getDuration());
|
|
if (els.length > 0) syncTimelineElements(els);
|
|
}
|
|
if (usePlayerStore.getState().elements.length === 0 && doc) {
|
|
const rootComp = doc.querySelector("[data-composition-id]");
|
|
const rootDuration = adapter.getDuration();
|
|
if (rootComp && rootDuration > 0) {
|
|
const fallbackElement = buildStandaloneRootTimelineElement({
|
|
compositionId: rootComp.getAttribute("data-composition-id") || "composition",
|
|
tagName: (rootComp as HTMLElement).tagName || "div",
|
|
rootDuration,
|
|
iframeSrc: iframe?.src || "",
|
|
selector: getTimelineElementSelector(rootComp),
|
|
});
|
|
if (fallbackElement) syncTimelineElements([fallbackElement]);
|
|
}
|
|
}
|
|
} catch {}
|
|
return true;
|
|
}, [
|
|
getAdapter,
|
|
setDuration,
|
|
setCurrentTime,
|
|
setTimelineReady,
|
|
setIsPlaying,
|
|
processTimelineMessage,
|
|
enrichMissingCompositions,
|
|
syncTimelineElements,
|
|
attachIframeShortcutListeners,
|
|
applyPreviewAudioState,
|
|
iframeRef,
|
|
isRefreshingRef,
|
|
pendingSeekRef,
|
|
]);
|
|
|
|
const onIframeLoad = useCallback(() => {
|
|
applyPreviewAudioState();
|
|
if (probeIntervalRef.current) clearInterval(probeIntervalRef.current);
|
|
|
|
// Fast path: adapter already available (in-place reloads, cached compositions)
|
|
if (initializeAdapter()) return;
|
|
|
|
// The runtime posts "state" or "timeline" messages once ready.
|
|
// Listen for those instead of polling.
|
|
const iframe = iframeRef.current;
|
|
let settled = false;
|
|
|
|
const trySettle = () => {
|
|
if (settled) return;
|
|
if (initializeAdapter()) {
|
|
settled = true;
|
|
window.removeEventListener("message", onMessage);
|
|
if (probeIntervalRef.current) clearInterval(probeIntervalRef.current);
|
|
}
|
|
};
|
|
|
|
const onMessage = (e: MessageEvent) => {
|
|
if (e.source && iframe && e.source !== iframe.contentWindow) return;
|
|
const data = e.data;
|
|
if (data?.source === "hf-preview" && (data?.type === "state" || data?.type === "timeline")) {
|
|
// The main message handler owns protocol-error diagnostics. This readiness-only
|
|
// listener mirrors its acceptance gate without dispatching a duplicate event:
|
|
// an unsupported runtime must not make the iframe appear successfully settled.
|
|
if (inspectStudioRuntimeMessage(data).status === "unsupported") return;
|
|
trySettle();
|
|
}
|
|
};
|
|
window.addEventListener("message", onMessage);
|
|
|
|
// Safety net: if no message arrives within 5s, try one last time then give up.
|
|
probeIntervalRef.current = setTimeout(() => {
|
|
if (!settled) {
|
|
trySettle();
|
|
}
|
|
window.removeEventListener("message", onMessage);
|
|
// Never leave the preview stuck invisible if the runtime never settled
|
|
// (initializeAdapter reveals on success; this covers the give-up case).
|
|
revealIframe(iframeRef.current);
|
|
}, 5000) as unknown as ReturnType<typeof setInterval>;
|
|
}, [initializeAdapter, iframeRef, probeIntervalRef, applyPreviewAudioState]);
|
|
|
|
// Stable refs so mount-effect closures always call the latest version
|
|
const processTimelineMessageRef = { current: processTimelineMessage };
|
|
const enrichMissingCompositionsRef = { current: enrichMissingCompositions };
|
|
|
|
return {
|
|
processTimelineMessage,
|
|
processTimelineMessageRef,
|
|
enrichMissingCompositions,
|
|
enrichMissingCompositionsRef,
|
|
initializeAdapter,
|
|
onIframeLoad,
|
|
};
|
|
}
|