feat(studio,lint): carve targets voiceover groups — always, when plural

Plural voiceover carve now targets a group instead of naming each clip:
`resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its
current members at analysis time, so a clip added to the group later is
covered without touching `sources`. The picker (`useFxCarve.ts`) offers a
grouped voice as one option instead of one row per member, tests overlap
as a union of member spans (a group overlaps the bed if ANY member does),
and prefers a qualifying group over its individual members in
`autoSourceIds`.

Picking two or more ungrouped voice clips in the carve flow now mints a
group behind them (`mintGroupId`, de-duped against every id in the
document) and writes `data-audio-group` on each picked clip atomically,
one undo entry — `createAudioGroupAndAssignMembers` in
`timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target
write shape. The DSP is untouched: `mixCarveSources` already sums
multiple sources correctly (verified in the design doc's own
investigation) — this only fixes the picker.

New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/
media.ts`, alongside `audio_volume_double_automation`) warns when a
`data-fx-carve`'s `sources` names two or more plain clip ids instead of a
group — the shape that silently rots when a clip is added. `/hyperframes-
audio` states the same rule as an invariant, not a tip, with the grouped-
narration HTML example from the design doc.

The group-matching and auto-group logic (`withAutoGroupedSources`,
`collectCarveCandidates`) is split into `useFxCarveGrouping.ts` —
`useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings`
is deliberately NOT an `async function`: wrapping it in one would force a
microtask on every call, including the synchronous branch — the exact bug
`withAutoGroupedSources`'s own sync-when-possible contract exists to avoid,
and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures)
before fixing it back to a plain function the caller conditionally awaits.

Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and
`useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600
lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker
IIFE to clear fallow's complexity gate.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
2026-08-20 16:39:29 -07:00
co-authored by Claude Sonnet 5
parent 5dc93a25ad
commit 351b219d2b
22 changed files with 914 additions and 68 deletions
@@ -2,7 +2,11 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { usePlayerStore, type TimelineElement } from "../player";
import { toggleTimelineElementHidden, toggleTimelineTrackHidden } from "./timelineTrackVisibility";
import {
createAudioGroupAndAssignMembers,
toggleTimelineElementHidden,
toggleTimelineTrackHidden,
} from "./timelineTrackVisibility";
afterEach(() => {
document.body.innerHTML = "";
@@ -395,3 +399,139 @@ describe("toggleTimelineElementHidden", () => {
expect(recordEdit.mock.calls[0]?.[0]?.label).toBe("Hide 2 elements");
});
});
describe("createAudioGroupAndAssignMembers", () => {
it("writes data-audio-group on every member in ONE atomic edit and updates the player store", async () => {
const iframe = document.createElement("iframe");
document.body.append(iframe);
if (iframe.contentDocument) {
iframe.contentDocument.body.innerHTML = `
<audio id="narration"></audio>
<audio id="interview-guest"></audio>
`;
}
const files = new Map([
[
"index.html",
`<audio id="narration" data-start="0" data-duration="5"></audio>
<audio id="interview-guest" data-start="10" data-duration="5"></audio>
<audio id="sfx-boom" data-start="0" data-duration="1"></audio>`,
],
]);
stubProjectFiles(files);
const narration = element({
id: "narration",
key: "index.html:#narration",
domId: "narration",
track: 0,
});
const guest = element({
id: "interview-guest",
key: "index.html:#interview-guest",
domId: "interview-guest",
track: 1,
});
usePlayerStore.getState().setElements([narration, guest]);
const writes = new Map<string, string>();
const recordEdit = vi.fn();
const changedPaths = await createAudioGroupAndAssignMembers({
projectId: "project-1",
activeCompPath: "index.html",
elements: [narration, guest],
groupId: "voiceover",
previewIframe: iframe,
writeProjectFile: async (path, content) => {
writes.set(path, content);
},
recordEdit,
domEditSaveTimestampRef: { current: 0 },
pendingTimelineEditPathRef: { current: new Set() },
});
expect(changedPaths).toEqual(["index.html"]);
expect(
iframe.contentDocument?.getElementById("narration")?.getAttribute("data-audio-group"),
).toBe("voiceover");
expect(
iframe.contentDocument?.getElementById("interview-guest")?.getAttribute("data-audio-group"),
).toBe("voiceover");
// One write carrying BOTH members — per-element writes would clobber each
// other (each starts from the original file content).
expect(writes.get("index.html")).toContain(
'id="narration" data-start="0" data-duration="5" data-audio-group="voiceover"',
);
expect(writes.get("index.html")).toContain(
'id="interview-guest" data-start="10" data-duration="5" data-audio-group="voiceover"',
);
expect(writes.get("index.html")).toContain(
'id="sfx-boom" data-start="0" data-duration="1"></audio>',
);
expect(recordEdit).toHaveBeenCalledTimes(1);
expect(recordEdit.mock.calls[0]?.[0]?.label).toBe("Group 2 voice clips");
expect(
usePlayerStore.getState().elements.find((el) => el.key === "index.html:#narration")
?.audioGroup,
).toBe("voiceover");
expect(
usePlayerStore.getState().elements.find((el) => el.key === "index.html:#interview-guest")
?.audioGroup,
).toBe("voiceover");
});
it("does nothing for fewer than two elements — grouping is a plural concept", async () => {
const recordEdit = vi.fn();
const changedPaths = await createAudioGroupAndAssignMembers({
projectId: "project-1",
activeCompPath: "index.html",
elements: [element({ id: "narration", domId: "narration" })],
groupId: "voiceover",
previewIframe: null,
writeProjectFile: async () => {},
recordEdit,
domEditSaveTimestampRef: { current: 0 },
pendingTimelineEditPathRef: { current: new Set() },
});
expect(changedPaths).toEqual([]);
expect(recordEdit).not.toHaveBeenCalled();
});
it("reverts the optimistic live patch when the save fails", async () => {
const iframe = document.createElement("iframe");
document.body.append(iframe);
if (iframe.contentDocument) {
iframe.contentDocument.body.innerHTML = `
<audio id="narration"></audio>
<audio id="interview-guest"></audio>
`;
}
// No stubbed fetch: readFileContent's request will fail, forcing the
// catch path.
const narration = element({ id: "narration", domId: "narration" });
const guest = element({ id: "interview-guest", domId: "interview-guest" });
await expect(
createAudioGroupAndAssignMembers({
projectId: "project-1",
activeCompPath: "index.html",
elements: [narration, guest],
groupId: "voiceover",
previewIframe: iframe,
writeProjectFile: async () => {},
recordEdit: vi.fn(),
domEditSaveTimestampRef: { current: 0 },
pendingTimelineEditPathRef: { current: new Set() },
}),
).rejects.toThrow();
expect(
iframe.contentDocument?.getElementById("narration")?.hasAttribute("data-audio-group"),
).toBe(false);
expect(
iframe.contentDocument?.getElementById("interview-guest")?.hasAttribute("data-audio-group"),
).toBe(false);
});
});
@@ -8,6 +8,7 @@ import {
} from "../player/components/timelineTrackDisplay";
import { saveProjectFilesWithHistory } from "../utils/studioFileHistory";
import { isAudioTimelineElement } from "../utils/timelineInspector";
import { HF_AUDIO_GROUP_ATTR } from "@hyperframes/core/audio-groups";
import { readTagSnippetByTarget, type PatchOperation } from "../utils/sourcePatcher";
import {
applyPatchByTarget,
@@ -277,6 +278,114 @@ export async function toggleTimelineElementHidden({
});
}
function patchLiveAudioGroupState(
iframe: HTMLIFrameElement | null,
elements: readonly TimelineElement[],
groupId: string | null,
activeCompPath: string | null,
): void {
for (const element of elements) {
const target = findTimelineElementInIframe(iframe, element, activeCompPath);
if (!target) continue;
if (groupId) target.setAttribute(HF_AUDIO_GROUP_ATTR, groupId);
else target.removeAttribute(HF_AUDIO_GROUP_ATTR);
}
}
interface CreateAudioGroupAndAssignMembersInput {
projectId: string;
activeCompPath: string | null;
elements: readonly TimelineElement[];
groupId: string;
previewIframe: HTMLIFrameElement | null;
writeProjectFile: (path: string, content: string) => Promise<void>;
recordEdit: (input: RecordEditInput) => Promise<void>;
domEditSaveTimestampRef: MutableRef<number>;
pendingTimelineEditPathRef: MutableRef<Set<string>>;
}
/**
* Group two or more voice clips: write `data-audio-group="<groupId>"` on
* every one of them, atomically, one undo entry — the same multi-target shape
* `setElementsHidden` uses for mute. The group needs no `<hf-audio-group>`
* element of its own to exist: `resolveAudioGroups` already degrades
* gracefully to label = id when one is absent, and a naming dialog is out of
* scope here — the id itself is the default name.
*/
// fallow-ignore-next-line complexity
export async function createAudioGroupAndAssignMembers({
projectId,
activeCompPath,
elements,
groupId,
previewIframe,
writeProjectFile,
recordEdit,
domEditSaveTimestampRef,
pendingTimelineEditPathRef,
}: CreateAudioGroupAndAssignMembersInput): Promise<string[]> {
if (elements.length < 2) return [];
patchLiveAudioGroupState(previewIframe, elements, groupId, activeCompPath);
reseekPreviewRuntime(previewIframe);
const groupOperation: PatchOperation = {
type: "attribute",
property: HF_AUDIO_GROUP_ATTR,
value: groupId,
};
const originalByPath = new Map<string, string>();
const files: Record<string, string> = {};
try {
for (const [targetPath, fileElements] of groupElementsByTargetPath(elements, activeCompPath)) {
let patchedContent = await readFileContent(projectId, targetPath);
originalByPath.set(targetPath, patchedContent);
for (const element of fileElements) {
const patchTarget = buildPatchTarget(element);
if (!patchTarget) {
throw new Error(`Timeline element ${element.id} is missing a patchable target`);
}
if (readTagSnippetByTarget(patchedContent, patchTarget) === undefined) {
throw new Error(`Unable to patch timeline element ${element.id} in ${targetPath}`);
}
patchedContent = applyPatchByTarget(patchedContent, patchTarget, groupOperation);
}
files[targetPath] = patchedContent;
pendingTimelineEditPathRef.current.add(targetPath);
}
domEditSaveTimestampRef.current = Date.now();
const changedPaths = await saveProjectFilesWithHistory({
projectId,
label: `Group ${elements.length} voice clips`,
kind: "timeline",
files,
readFile: async (path) => {
const original = originalByPath.get(path);
if (original !== undefined) return original;
return readFileContent(projectId, path);
},
writeFile: writeProjectFile,
recordEdit,
});
domEditSaveTimestampRef.current = Date.now();
for (const element of elements) {
usePlayerStore.getState().updateElement(element.key ?? element.id, { audioGroup: groupId });
}
return changedPaths;
} catch (error) {
// Mirrors setElementsHidden's failure path: the optimistic live patch
// already ran, so a save failure has to be unwound or the preview shows a
// grouping that never made it to disk.
patchLiveAudioGroupState(previewIframe, elements, null, activeCompPath);
reseekPreviewRuntime(previewIframe);
throw error;
}
}
export function useTimelineTrackVisibilityEditing({
projectIdRef,
activeCompPath,
@@ -407,3 +516,67 @@ export function useTimelineElementVisibilityEditing({
],
);
}
/**
* The write behind B6's auto-group: pick two or more voice clips in the carve
* picker and they land in a group instead of naming each other by id. Same
* expanded-rows resolution as element-visibility, for the same reason — a
* nested sub-composition child has no entry in the raw store list.
*/
export function useAudioGroupCarveAssignment({
projectIdRef,
activeCompPath,
showToast,
writeProjectFile,
recordEdit,
domEditSaveTimestampRef,
previewIframeRef,
pendingTimelineEditPathRef,
isRecordingRef,
}: UseTimelineElementVisibilityEditingInput): (
clipIds: readonly string[],
groupId: string,
) => Promise<void> {
const expandedElements = useExpandedTimelineElements();
return useCallback(
async (clipIds: readonly string[], groupId: string) => {
if (isRecordingRef?.current) {
showToast("Cannot edit timeline while recording", "error");
return;
}
const pid = projectIdRef.current;
if (!pid) return;
const keys = new Set(clipIds);
const elements = expandedElements.filter((item) => keys.has(item.key ?? item.id));
try {
await createAudioGroupAndAssignMembers({
projectId: pid,
activeCompPath,
elements,
groupId,
previewIframe: previewIframeRef.current,
writeProjectFile,
recordEdit,
domEditSaveTimestampRef,
pendingTimelineEditPathRef,
});
} catch (error) {
console.error("[Timeline] Failed to group voice clips", error);
const message = error instanceof Error ? error.message : "Failed to group voice clips";
showToast(message);
}
},
[
activeCompPath,
expandedElements,
previewIframeRef,
writeProjectFile,
recordEdit,
domEditSaveTimestampRef,
pendingTimelineEditPathRef,
isRecordingRef,
showToast,
projectIdRef,
],
);
}
@@ -0,0 +1,20 @@
import { useMemo } from "react";
import type { TimelineElement } from "../player/store/timelineElement";
/**
* The stored `duration` lags a moment behind an edit that pushes an element
* past it (drag, trim, paste) — this is the actual end of the timeline, the
* later of the stored duration and the furthest element's end.
*/
export function useEffectiveTimelineDuration(
timelineDuration: number,
timelineElements: readonly TimelineElement[],
): number {
return useMemo(() => {
const maxEnd =
timelineElements.length > 0
? Math.max(...timelineElements.map((el) => el.start + el.duration))
: 0;
return Math.max(timelineDuration, maxEnd);
}, [timelineDuration, timelineElements]);
}
@@ -0,0 +1,70 @@
import { useCallback, useEffect, useRef } from "react";
import { waitForMediaJob } from "../components/studioMediaJobs";
import type { BackgroundRemovalProgress } from "../components/editor/propertyPanelTypes";
interface RemoveBackgroundOptions {
createBackgroundPlate?: boolean;
quality?: "fast" | "balanced" | "best";
onProgress?: (progress: BackgroundRemovalProgress) => void;
}
/**
* One removal in flight at a time: starting a second one aborts whichever job
* is still running, so a stale progress callback can't overwrite a newer
* result. Unmounting aborts too, or the job would keep running against a
* panel that is no longer there to show its progress.
*/
export function useRemoveBackground(
projectId: string,
refreshFileTree: () => Promise<void>,
showToast: (message: string, kind?: "info" | "error") => void,
) {
const abortRef = useRef<AbortController | null>(null);
useEffect(
() => () => {
abortRef.current?.abort();
},
[],
);
return useCallback(
// fallow-ignore-next-line complexity
async (inputPath: string, options: RemoveBackgroundOptions) => {
const response = await fetch(
`/api/projects/${encodeURIComponent(projectId)}/media/remove-background`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
inputPath,
createBackgroundPlate: options.createBackgroundPlate === true,
quality: options.quality ?? "balanced",
}),
},
);
const data = (await response.json().catch(() => ({}))) as {
jobId?: string;
error?: string;
};
if (!response.ok || !data.jobId) {
throw new Error(data.error || `Background removal failed (${response.status})`);
}
showToast("Removing background...", "info");
abortRef.current?.abort();
const controller = new AbortController();
abortRef.current = controller;
try {
const result = await waitForMediaJob(data.jobId, options.onProgress, controller.signal);
await refreshFileTree();
showToast(`Created transparent asset: ${result.outputPath.split("/").pop()}`, "info");
return result;
} finally {
if (abortRef.current === controller) {
abortRef.current = null;
}
}
},
[projectId, refreshFileTree, showToast],
);
}
@@ -28,6 +28,7 @@ import {
import type { PersistTimelineEditInput } from "./timelineEditingHelpers";
import type { TimelineStackingReorderIntent } from "../player/components/timelineEditing";
import {
useAudioGroupCarveAssignment,
useTimelineElementVisibilityEditing,
useTimelineTrackVisibilityEditing,
} from "./timelineTrackVisibility";
@@ -388,6 +389,18 @@ export function useTimelineEditing({
forceReloadSdkSession,
});
const handleAutoGroupCarveSources = useAudioGroupCarveAssignment({
projectIdRef,
activeCompPath,
showToast,
writeProjectFile,
recordEdit,
domEditSaveTimestampRef,
previewIframeRef,
pendingTimelineEditPathRef,
isRecordingRef,
});
// fallow-ignore-next-line complexity
const handleTimelineElementsDelete = useCallback(
// fallow-ignore-next-line complexity
@@ -558,6 +571,7 @@ export function useTimelineEditing({
handleTimelineElementResize,
handleToggleTrackHidden,
handleToggleElementHidden,
handleAutoGroupCarveSources,
handleTimelineElementDelete,
handleTimelineElementsDelete,
handleTimelineElementSplit: handleRazorSplit,