Files
hyperframes/packages/core/src/mediaLink.ts
T
Vance IngallsandClaude Opus 5.5 45b8e171e2 feat(studio): sync origin, out-of-sync repair, linked selection toggle and grouped clip menu (7/8) (#4820)
* feat(studio): order the clip menu into time, sound, picture and delete groups

Split and Freeze frame; Normalize, Voice, link items and Duck; Look and Crop;
then Delete beside Delete this clip only. Each item file renders its pieces
separately, empty groups collapse, and the viewport clamp uses the measured
menu height instead of a row-count estimate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio-server): mint freeze-frame right-half links with the shared lk-N helper

Freeze frame now calls core relinkSplitHalves, so split halves and freeze
halves share one link-id convention instead of L -> L-2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): apply speed edits to every data-link member in one undo step

The inspector speed slider, rate lane edits and ramp presets on a linked clip
now commit once, writing the selection and, via expandToLinkedMembers, each
partner from its own current value. Unlinked clips keep their usual path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(studio): drop the audio parent tick in favour of the data-link chain badge

A linked clip now shows one link indicator: the chain badge driven by
data-link. The waveform no longer draws its 1px parent tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): follow late clip-menu rows and target link partners like S2

The menu height follows a ResizeObserver so rows that read the live preview
after mount still clamp inside the viewport. Speed fan-out builds partner
targets with buildPatchTarget, matching the link edit plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): show the sound strip and audio menu items without studio's shell

The audible-video waveform strip drops AudioWaveform's 16px label band, which
left a 0px canvas on short lanes. Normalize loudness and Duck under voice fall
back to the timeline session's project, the live preview iframe store and a new
onNotice edit callback when no StudioShellProvider is mounted.
usePreviewIframeStore is exported so a host can publish its preview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): show duck under voice only when a voice overlaps

The item now appears only when an overlapping voice exists or the bed is
already ducked. Its check mark moves to the right edge so unchecked rows
align with siblings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): name the partner in the clip menu link and unlink items

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): clip menu stops offering unlink once the clips are unlinked

The timeline skipped store updates when only data-link changed,
so the menu kept reading the stale link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(core): sync origin, offset math and move/slip into sync for linked pairs

data-sync-origin survives Unlink; split gives right halves a fresh origin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio-server): split re-mints the sync origin of unlinked pairs too

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): detach, link and keep-sound cutout write a sync origin that unlink keeps

TimelineElement.syncOrigin is read on both parse paths; merge back drops it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(studio): expect sync-origin on keep-sound cutout write

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): linked selection toggle in the timeline toolbar, on by default

Off: a click selects one clip; drag, trim, razor, speed and delete stay on it
(delete unlinks the survivor). Alt-click still selects one half when on.
Right-click holds the out-of-sync indicator preference. Both persist in UI prefs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(studio): link groups share a label colour; out-of-sync badges with move and slip

Red badge on both halves of a sync-origin pair that drifted (frames, s:ff past
a second); hidden when rates differ or the preference is off. The whole badge
opens Move / Slip into Sync; each edits that clip alone in one undo step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): take link label colours from theme tokens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(sdk): syncOffset, moveIntoSync and slipIntoSync for sync-origin pairs

Core gains findSyncPartner, the DOM pairing shared by the SDK and the CLI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): timeline rows report syncOffsetFrames and print out-of-sync

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(cli): the timeline row fixture carries syncOffsetFrames

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): data-sync-origin in hyperframes-core

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(core,sdk): keep a one-sided cut's sync origin and slip the in-point playback reads

relinkSplitHalves reminted the right half's sync origin even when only one
member of an unlinked source pair was cut, so the uncut partner kept the old
origin and the new half lost its badge. The origin is now reminted only when
both the video and the audio were cut.

Slip into Sync wrote data-media-start whenever it existed, but playback
reads data-playback-start first, so a clip carrying both never moved. The
link timing reader now uses playback's precedence and the slip writes the
attribute that wins; lint compares the same field. A sync partner is also
looked up only in the clip's own composition (core) or source file (Studio).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): flip the out-of-sync menu above the pointer near the bottom edge

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: trim comments the sync-origin layer added

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(core): split the scoped candidate lookup out of findSyncPartner

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(core,studio): find sync partners inside the link scope

findSyncPartner and Studio's syncPartnerOf now scope by the same rule as
link groups: the nearest data-composition-id or data-composition-file
ancestor, plus the row's compositionScope in Studio. An origin reused in a
file host or an inline composition no longer pairs across it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 06:08:37 +00:00

208 lines
7.6 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { readMediaOffsetSeconds } from "@hyperframes/parsers/media-duration";
/**
* Linked clips: members sharing a `data-link` id are edited as one (a detached
* video + its audio). An editing contract only; playback and render ignore it.
*/
export const MEDIA_LINK_ATTR = "data-link";
interface AttributeReader {
getAttribute(name: string): string | null;
}
export interface LinkTiming {
start: number;
duration: number;
mediaStart: number;
playbackRate: number;
}
const LINK_TIMING_FIELDS: ReadonlyArray<[keyof LinkTiming, string]> = [
["start", "start"],
["duration", "duration"],
["mediaStart", "media-start"],
["playbackRate", "playback-rate"],
];
const SYNC_TOLERANCE_S = 1e-3;
function readNumber(el: AttributeReader, names: string[], fallback: number): number {
for (const name of names) {
const raw = el.getAttribute(name);
if (raw === null || raw.trim() === "") continue;
const value = Number(raw);
if (Number.isFinite(value)) return value;
}
return fallback;
}
export function readLinkTiming(el: AttributeReader): LinkTiming {
return {
start: readNumber(el, ["data-start"], 0),
duration: readNumber(el, ["data-duration"], 0),
mediaStart: readMediaOffsetSeconds((name) => el.getAttribute(name)),
playbackRate: readNumber(el, ["data-playback-rate"], 1),
};
}
/** The timing fields (`start`, `duration`, `media-start`, `playback-rate`) on which members disagree. */
export function linkTimingMismatches(members: readonly AttributeReader[]): string[] {
const timings = members.map(readLinkTiming);
const first = timings[0];
if (!first) return [];
return LINK_TIMING_FIELDS.filter(([key]) =>
timings.some((timing) => Math.abs(timing[key] - first[key]) > SYNC_TOLERANCE_S),
).map(([, field]) => field);
}
const LINK_SCOPE_SELECTOR = "[data-composition-id], [data-composition-file]";
/** The composition a link group stays inside: the nearest inline composition or file host. */
export function linkScopeOf(el: Element): Element | null {
return el.parentElement?.closest(LINK_SCOPE_SELECTOR) ?? null;
}
export function mintLinkId(taken: Iterable<string>): string {
const used = new Set(taken);
let n = 1;
while (used.has(`lk-${n}`)) n += 1;
return `lk-${n}`;
}
function takenLinkIds(doc: Document): string[] {
return Array.from(
doc.querySelectorAll(`[id], [${MEDIA_LINK_ATTR}], [${SYNC_ORIGIN_ATTR}]`),
).flatMap((el) => [
el.id,
el.getAttribute(MEDIA_LINK_ATTR) ?? "",
el.getAttribute(SYNC_ORIGIN_ATTR) ?? "",
]);
}
function renameShared(
doc: Document,
ids: readonly string[],
attr: string,
taken: Set<string>,
): void {
const renamed = new Map<string, string>();
for (const id of ids) {
const el = doc.getElementById(id);
const old = el?.getAttribute(attr);
if (!el || !old) continue;
let fresh = renamed.get(old);
if (!fresh) {
fresh = mintLinkId(taken);
taken.add(fresh);
renamed.set(old, fresh);
}
el.setAttribute(attr, fresh);
}
}
/** Split right halves get a fresh link id; a sync origin is reminted only when video and audio were both cut. */
export function relinkSplitHalves(doc: Document, rightHalfIds: readonly string[]): void {
const taken = new Set(takenLinkIds(doc));
renameShared(doc, rightHalfIds, MEDIA_LINK_ATTR, taken);
renameShared(doc, idsWithBothSidesCut(doc, rightHalfIds), SYNC_ORIGIN_ATTR, taken);
}
function idsWithBothSidesCut(doc: Document, rightHalfIds: readonly string[]): string[] {
const tagsByOrigin = new Map<string, Set<string>>();
const originOf = (id: string) => doc.getElementById(id)?.getAttribute(SYNC_ORIGIN_ATTR) ?? null;
for (const id of rightHalfIds) {
const origin = originOf(id);
const tag = doc.getElementById(id)?.tagName.toLowerCase();
if (!origin || !tag) continue;
tagsByOrigin.set(origin, (tagsByOrigin.get(origin) ?? new Set()).add(tag));
}
return rightHalfIds.filter((id) => {
const tags = tagsByOrigin.get(originOf(id) ?? "");
return tags?.has("video") === true && tags.has("audio");
});
}
/** Shared by a detached or linked video + audio of one file; it survives Unlink so a drifted pair can resync. */
export const SYNC_ORIGIN_ATTR = "data-sync-origin";
export type SyncTiming = Pick<LinkTiming, "start" | "mediaStart" | "playbackRate">;
const rateOf = (timing: SyncTiming) => (timing.playbackRate > 0 ? timing.playbackRate : 1);
/** Where source time zero lands on the timeline: `start − mediaStart / rate`. */
export function sourceZeroTime(timing: SyncTiming): number {
return timing.start - timing.mediaStart / rateOf(timing);
}
/** Frames `clip` sits from `partner`, positive when it plays late; null when their rates differ. */
export function syncOffsetFrames(
clip: SyncTiming,
partner: SyncTiming,
fps: number,
): number | null {
if (Math.abs(rateOf(clip) - rateOf(partner)) > 1e-6) return null;
const frames = Math.round((sourceZeroTime(clip) - sourceZeroTime(partner)) * fps);
return Object.is(frames, -0) ? 0 : frames;
}
/** `+10`, `-1:21` (seconds:frames once past one second). */
export function formatSyncOffset(frames: number, fps: number): string {
const sign = frames < 0 ? "-" : "+";
const whole = Math.abs(frames);
const perSecond = Math.max(1, Math.round(fps));
if (whole < perSecond) return `${sign}${whole}`;
const seconds = Math.floor(whole / perSecond);
const rest = String(whole % perSecond).padStart(2, "0");
return `${sign}${seconds}:${rest}`;
}
/** Move into Sync: the start that puts `clip` back on `partner`, or null before zero. */
export function moveIntoSyncStart(clip: SyncTiming, partner: SyncTiming): number | null {
const start = clip.start - (sourceZeroTime(clip) - sourceZeroTime(partner));
return start >= -SYNC_TOLERANCE_S ? Math.max(0, start) : null;
}
/** Slip into Sync: the media start that syncs `clip` in place, or null before the file's start. */
export function slipIntoSyncMediaStart(clip: SyncTiming, partner: SyncTiming): number | null {
const mediaStart = (clip.start - sourceZeroTime(partner)) * rateOf(clip);
return mediaStart >= -SYNC_TOLERANCE_S ? Math.max(0, mediaStart) : null;
}
interface PairableElement extends AttributeReader {
tagName: string;
closest(selectors: string): Element | null;
ownerDocument: Document;
}
function sharedSeconds(a: LinkTiming, b: LinkTiming): number {
return Math.min(a.start + a.duration, b.start + b.duration) - Math.max(a.start, b.start);
}
/** `el`'s source partner (same sync origin, other tag): most shared timeline, then nearest start. */
function sameOriginInScope(el: PairableElement, tag: string, origin: string): Element[] {
const scopeOf = (node: PairableElement) => node.closest(LINK_SCOPE_SELECTOR) ?? el.ownerDocument;
const scope = scopeOf(el);
return Array.from(scope.querySelectorAll(tag)).filter(
(candidate) =>
candidate.getAttribute(SYNC_ORIGIN_ATTR) === origin && scopeOf(candidate) === scope,
);
}
export function findSyncPartner(el: PairableElement): Element | null {
const origin = el.getAttribute(SYNC_ORIGIN_ATTR);
const tag = el.tagName.toLowerCase();
if (!origin || (tag !== "video" && tag !== "audio")) return null;
const partnerTag = tag === "video" ? "audio" : "video";
const own = readLinkTiming(el);
let best: { el: Element; shared: number; distance: number } | null = null;
for (const candidate of sameOriginInScope(el, partnerTag, origin)) {
const timing = readLinkTiming(candidate);
const shared = sharedSeconds(own, timing);
const distance = Math.abs(timing.start - own.start);
if (!best || shared > best.shared || (shared === best.shared && distance < best.distance)) {
best = { el: candidate, shared, distance };
}
}
return best?.el ?? null;
}