mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
feat(studio,lint): carve targets voiceover groups — always, when plural
Plural voiceover carve now targets a group instead of naming each clip: `resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its current members at analysis time, so a clip added to the group later is covered without touching `sources`. The picker (`useFxCarve.ts`) offers a grouped voice as one option instead of one row per member, tests overlap as a union of member spans (a group overlaps the bed if ANY member does), and prefers a qualifying group over its individual members in `autoSourceIds`. Picking two or more ungrouped voice clips in the carve flow now mints a group behind them (`mintGroupId`, de-duped against every id in the document) and writes `data-audio-group` on each picked clip atomically, one undo entry — `createAudioGroupAndAssignMembers` in `timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target write shape. The DSP is untouched: `mixCarveSources` already sums multiple sources correctly (verified in the design doc's own investigation) — this only fixes the picker. New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/ media.ts`, alongside `audio_volume_double_automation`) warns when a `data-fx-carve`'s `sources` names two or more plain clip ids instead of a group — the shape that silently rots when a clip is added. `/hyperframes- audio` states the same rule as an invariant, not a tip, with the grouped- narration HTML example from the design doc. The group-matching and auto-group logic (`withAutoGroupedSources`, `collectCarveCandidates`) is split into `useFxCarveGrouping.ts` — `useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings` is deliberately NOT an `async function`: wrapping it in one would force a microtask on every call, including the synchronous branch — the exact bug `withAutoGroupedSources`'s own sync-when-possible contract exists to avoid, and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures) before fixing it back to a plain function the caller conditionally awaits. Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and `useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600 lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker IIFE to clear fallow's complexity gate. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
8d48a6f52f
commit
1abad17650
@@ -1,5 +1,10 @@
|
||||
import { beforeEach, describe, expect, it } from "vitest";
|
||||
import { audioGroupOf, HF_AUDIO_GROUP_ATTR, resolveAudioGroups } from "./audioGroups.js";
|
||||
import {
|
||||
audioGroupOf,
|
||||
HF_AUDIO_GROUP_ATTR,
|
||||
resolveAudioGroups,
|
||||
resolveCarveSourceIds,
|
||||
} from "./audioGroups.js";
|
||||
|
||||
beforeEach(() => {
|
||||
document.body.innerHTML = "";
|
||||
@@ -64,6 +69,47 @@ describe("audioGroupOf", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveCarveSourceIds", () => {
|
||||
it("expands a group id to its current members", () => {
|
||||
document.body.innerHTML = `
|
||||
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
||||
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
||||
`;
|
||||
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2"]);
|
||||
});
|
||||
|
||||
it("picks up a member added to the group after the carve was set (analysis-time, not frozen)", () => {
|
||||
document.body.innerHTML = `
|
||||
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
||||
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
||||
`;
|
||||
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2"]);
|
||||
document.body.insertAdjacentHTML(
|
||||
"beforeend",
|
||||
`<audio id="vo-3" data-audio-group="voiceover"></audio>`,
|
||||
);
|
||||
expect(resolveCarveSourceIds(document, ["voiceover"])).toEqual(["vo-1", "vo-2", "vo-3"]);
|
||||
});
|
||||
|
||||
it("passes through a plain clip id that still exists", () => {
|
||||
document.body.innerHTML = `<audio id="vo-1"></audio>`;
|
||||
expect(resolveCarveSourceIds(document, ["vo-1"])).toEqual(["vo-1"]);
|
||||
});
|
||||
|
||||
it("drops an id that resolves to nothing — a deleted clip, an empty or vanished group", () => {
|
||||
document.body.innerHTML = `<audio id="vo-1"></audio>`;
|
||||
expect(resolveCarveSourceIds(document, ["vo-1", "deleted", "no-such-group"])).toEqual(["vo-1"]);
|
||||
});
|
||||
|
||||
it("dedupes and preserves first-seen order across a mix of group and plain ids", () => {
|
||||
document.body.innerHTML = `
|
||||
<audio id="vo-1" data-audio-group="voiceover"></audio>
|
||||
<audio id="vo-2" data-audio-group="voiceover"></audio>
|
||||
`;
|
||||
expect(resolveCarveSourceIds(document, ["voiceover", "vo-1"])).toEqual(["vo-1", "vo-2"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe(HF_AUDIO_GROUP_ATTR, () => {
|
||||
it("is the attribute name membership is keyed on", () => {
|
||||
expect(HF_AUDIO_GROUP_ATTR).toBe("data-audio-group");
|
||||
|
||||
@@ -51,6 +51,37 @@ export function resolveAudioGroups(root: ParentNode): HfAudioGroup[] {
|
||||
return groups;
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand a list of source ids for a carve: a plain id passes through if it
|
||||
* still exists, a group id expands to its CURRENT members. Resolved fresh
|
||||
* every time — group membership is never frozen into the carve's own
|
||||
* attribute, so adding a fourth voice to a group already named in a carve's
|
||||
* `sources` picks it up on the next analysis without editing that carve.
|
||||
*
|
||||
* Dedupes and preserves first-seen order; an id that resolves to nothing
|
||||
* (a deleted clip, an empty or vanished group) is dropped rather than kept
|
||||
* as a dangling reference the analysis would only fail to find anyway.
|
||||
*/
|
||||
export function resolveCarveSourceIds(doc: Document, ids: readonly string[]): string[] {
|
||||
const groupsById = new Map(resolveAudioGroups(doc).map((group) => [group.id, group] as const));
|
||||
const seen = new Set<string>();
|
||||
const out: string[] = [];
|
||||
const add = (id: string): void => {
|
||||
if (seen.has(id)) return;
|
||||
seen.add(id);
|
||||
out.push(id);
|
||||
};
|
||||
for (const id of ids) {
|
||||
const group = groupsById.get(id);
|
||||
if (group) {
|
||||
group.memberIds.forEach(add);
|
||||
} else if (doc.getElementById(id)) {
|
||||
add(id);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The group a member belongs to, or null. Groups do not nest — this ignores
|
||||
* `data-audio-group` on an `<hf-audio-group>` element itself.
|
||||
*
|
||||
|
||||
@@ -555,3 +555,51 @@ describe("audio_volume_double_automation", () => {
|
||||
expect(res.findings.some((f) => f.code === "audio_volume_double_automation")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("audio_carve_ungrouped_sources", () => {
|
||||
const withCarve = (carveJson: string, extra = "") => `<!DOCTYPE html><html><body>
|
||||
<div id="root" data-composition-id="main" data-start="0" data-width="1920" data-height="1080" data-duration="10">
|
||||
<audio id="bed" src="bed.wav" data-start="0" data-duration="10" data-fx-carve='${carveJson}'></audio>
|
||||
${extra}
|
||||
</div>
|
||||
</body></html>`;
|
||||
|
||||
it("warns when sources names two or more plain clip ids", async () => {
|
||||
const res = await lintHyperframeHtml(
|
||||
withCarve(`{"enabled":true,"sources":["vo-1","vo-2"],"strength":0.35}`),
|
||||
);
|
||||
const finding = res.findings.find((f) => f.code === "audio_carve_ungrouped_sources");
|
||||
expect(finding?.severity).toBe("warning");
|
||||
expect(finding?.elementId).toBe("bed");
|
||||
});
|
||||
|
||||
it("stays quiet when sources names a group", async () => {
|
||||
const res = await lintHyperframeHtml(
|
||||
withCarve(
|
||||
`{"enabled":true,"sources":["voiceover"],"strength":0.35}`,
|
||||
`<hf-audio-group id="voiceover" data-label="Voiceover"></hf-audio-group>`,
|
||||
),
|
||||
);
|
||||
expect(res.findings.some((f) => f.code === "audio_carve_ungrouped_sources")).toBe(false);
|
||||
});
|
||||
|
||||
it("stays quiet for a single-clip sources list", async () => {
|
||||
const res = await lintHyperframeHtml(
|
||||
withCarve(`{"enabled":true,"sources":["narration"],"strength":0.35}`),
|
||||
);
|
||||
expect(res.findings.some((f) => f.code === "audio_carve_ungrouped_sources")).toBe(false);
|
||||
});
|
||||
|
||||
it("still warns when one entry is a group and the rest are plain clip ids", async () => {
|
||||
// Mixing a group with two more bare clip ids is still an ungrouped-source
|
||||
// rot risk for those two clips — only fully-grouped sources are silent.
|
||||
const res = await lintHyperframeHtml(
|
||||
withCarve(
|
||||
`{"enabled":true,"sources":["voiceover","vo-3","vo-4"],"strength":0.35}`,
|
||||
`<hf-audio-group id="voiceover" data-label="Voiceover"></hf-audio-group>`,
|
||||
),
|
||||
);
|
||||
const finding = res.findings.find((f) => f.code === "audio_carve_ungrouped_sources");
|
||||
expect(finding?.severity).toBe("warning");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -632,6 +632,8 @@ export const mediaRules: Array<(ctx: LintContext) => HyperframeLintFinding[]> =
|
||||
|
||||
// audio_volume_tween_overrides_gain
|
||||
findVolumeTweenOverridesGainFindings,
|
||||
// audio_carve_ungrouped_sources
|
||||
findCarveUngroupedSourcesFindings,
|
||||
];
|
||||
|
||||
/**
|
||||
@@ -722,3 +724,48 @@ function findVolumeDoubleAutomationFindings(ctx: LintContext): HyperframeLintFin
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
/**
|
||||
* A carve's `sources` naming two or more plain clip ids is the normative
|
||||
* mistake groups exist to prevent (groups doc §1.6): the list silently rots
|
||||
* when a voice clip is added or removed, since nothing re-derives it. Naming
|
||||
* a group instead means membership resolves at analysis time. Silent when
|
||||
* `sources` already names a group, or names at most one clip.
|
||||
*/
|
||||
function findCarveUngroupedSourcesFindings(ctx: LintContext): HyperframeLintFinding[] {
|
||||
const groupIds = new Set(
|
||||
ctx.tags.filter((tag) => tag.name === "hf-audio-group").map((tag) => readAttr(tag.raw, "id")),
|
||||
);
|
||||
|
||||
const findings: HyperframeLintFinding[] = [];
|
||||
for (const tag of ctx.tags) {
|
||||
const raw = readDecodedAttr(tag.raw, "data-fx-carve");
|
||||
if (raw === null) continue;
|
||||
const trimmed = raw.trim();
|
||||
if (!trimmed.startsWith("{")) continue;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(trimmed);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const sources = (parsed as { sources?: unknown }).sources;
|
||||
if (!Array.isArray(sources)) continue;
|
||||
const clipIds = sources.filter(
|
||||
(id): id is string => typeof id === "string" && !groupIds.has(id),
|
||||
);
|
||||
if (clipIds.length < 2) continue;
|
||||
|
||||
const elementId = readAttr(tag.raw, "id") || undefined;
|
||||
findings.push({
|
||||
code: "audio_carve_ungrouped_sources",
|
||||
severity: "warning",
|
||||
message: `${elementId ? `#${elementId}'s` : "This"} carve names ${clipIds.length} voice clips directly (${clipIds.join(", ")}) instead of a group.`,
|
||||
elementId,
|
||||
fixHint:
|
||||
"Group the voice clips and carve against the group — a hand-rolled clip list silently rots when a clip is added.",
|
||||
snippet: truncateSnippet(tag.raw),
|
||||
});
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
|
||||
@@ -37,6 +37,7 @@ import { useCompositionDimensions } from "./hooks/useCompositionDimensions";
|
||||
import { useToast } from "./hooks/useToast";
|
||||
import { useCompositionContentLoader } from "./hooks/useCompositionContentLoader";
|
||||
import { useStudioUrlState } from "./hooks/useStudioUrlState";
|
||||
import { useEffectiveTimelineDuration } from "./hooks/useEffectiveTimelineDuration";
|
||||
import {
|
||||
buildStudioContextValue,
|
||||
useGlobalFileDrop,
|
||||
@@ -95,13 +96,10 @@ export function StudioApp() {
|
||||
const setTimelineSelectionSet = usePlayerStore((s) => s.setSelectedElementIds);
|
||||
const timelineDuration = usePlayerStore((s) => s.duration);
|
||||
const isPlaying = usePlayerStore((s) => s.isPlaying);
|
||||
const effectiveTimelineDuration = useMemo(() => {
|
||||
const maxEnd =
|
||||
timelineElements.length > 0
|
||||
? Math.max(...timelineElements.map((el) => el.start + el.duration))
|
||||
: 0;
|
||||
return Math.max(timelineDuration, maxEnd);
|
||||
}, [timelineDuration, timelineElements]);
|
||||
const effectiveTimelineDuration = useEffectiveTimelineDuration(
|
||||
timelineDuration,
|
||||
timelineElements,
|
||||
);
|
||||
const { toasts, showToast, dismissToast } = useToast();
|
||||
const panelLayout = usePanelLayout({
|
||||
rightCollapsed: initialUrlStateRef.current.rightCollapsed,
|
||||
@@ -532,6 +530,7 @@ export function StudioApp() {
|
||||
domEditSaveTimestampRef={domEditSaveTimestampRef}
|
||||
recordEdit={editHistory.recordEdit}
|
||||
onToggleElementHidden={timelineEditing.handleToggleElementHidden}
|
||||
onAutoGroupCarveSources={timelineEditing.handleAutoGroupCarveSources}
|
||||
onAddMediaOverlay={handleAddMediaOverlay}
|
||||
/>
|
||||
)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useCallback, useEffect, useRef } from "react";
|
||||
import { useCallback } from "react";
|
||||
import type { StudioRightPanelProps } from "./StudioRightPanel.types";
|
||||
|
||||
export type { StudioRightPanelProps };
|
||||
@@ -20,15 +20,14 @@ import { usePanelLayoutContext } from "../contexts/PanelLayoutContext";
|
||||
import { useFileManagerContext } from "../contexts/FileManagerContext";
|
||||
import { useDomEditContext } from "../contexts/DomEditContext";
|
||||
import { usePlayerStore } from "../player";
|
||||
import { waitForMediaJob } from "./studioMediaJobs";
|
||||
import {
|
||||
applyColorGradingScopeUpdate,
|
||||
EMPTY_COLOR_GRADING_SCOPE_RESULT,
|
||||
type ColorGradingScope,
|
||||
} from "./studioColorGradingScope";
|
||||
import type { BackgroundRemovalProgress } from "./editor/propertyPanelTypes";
|
||||
import { timelineKeysForSelections } from "../utils/studioHelpers";
|
||||
import { useInspectorSplitResize } from "../hooks/useInspectorSplitResize";
|
||||
import { useRemoveBackground } from "../hooks/useRemoveBackground";
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
export function StudioRightPanel({
|
||||
@@ -45,6 +44,7 @@ export function StudioRightPanel({
|
||||
domEditSaveTimestampRef,
|
||||
recordEdit,
|
||||
onToggleElementHidden,
|
||||
onAutoGroupCarveSources,
|
||||
onAddMediaOverlay,
|
||||
}: StudioRightPanelProps) {
|
||||
const {
|
||||
@@ -163,14 +163,6 @@ export function StudioRightPanel({
|
||||
handleInspectorSplitResizeMove,
|
||||
handleInspectorSplitResizeEnd,
|
||||
} = useInspectorSplitResize();
|
||||
const backgroundRemovalAbortRef = useRef<AbortController | null>(null);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
backgroundRemovalAbortRef.current?.abort();
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
const renderJobs = renderQueue.jobs as RenderJob[];
|
||||
const inspectorTabActive = rightPanelTab === "design" || rightPanelTab === "layers";
|
||||
@@ -237,52 +229,7 @@ export function StudioRightPanel({
|
||||
],
|
||||
);
|
||||
|
||||
const handleRemoveBackground = useCallback(
|
||||
// fallow-ignore-next-line complexity
|
||||
async (
|
||||
inputPath: string,
|
||||
options: {
|
||||
createBackgroundPlate?: boolean;
|
||||
quality?: "fast" | "balanced" | "best";
|
||||
onProgress?: (progress: BackgroundRemovalProgress) => void;
|
||||
},
|
||||
) => {
|
||||
const response = await fetch(
|
||||
`/api/projects/${encodeURIComponent(projectId)}/media/remove-background`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
inputPath,
|
||||
createBackgroundPlate: options.createBackgroundPlate === true,
|
||||
quality: options.quality ?? "balanced",
|
||||
}),
|
||||
},
|
||||
);
|
||||
const data = (await response.json().catch(() => ({}))) as {
|
||||
jobId?: string;
|
||||
error?: string;
|
||||
};
|
||||
if (!response.ok || !data.jobId) {
|
||||
throw new Error(data.error || `Background removal failed (${response.status})`);
|
||||
}
|
||||
showToast("Removing background...", "info");
|
||||
backgroundRemovalAbortRef.current?.abort();
|
||||
const controller = new AbortController();
|
||||
backgroundRemovalAbortRef.current = controller;
|
||||
try {
|
||||
const result = await waitForMediaJob(data.jobId, options.onProgress, controller.signal);
|
||||
await refreshFileTree();
|
||||
showToast(`Created transparent asset: ${result.outputPath.split("/").pop()}`, "info");
|
||||
return result;
|
||||
} finally {
|
||||
if (backgroundRemovalAbortRef.current === controller) {
|
||||
backgroundRemovalAbortRef.current = null;
|
||||
}
|
||||
}
|
||||
},
|
||||
[projectId, refreshFileTree, showToast],
|
||||
);
|
||||
const handleRemoveBackground = useRemoveBackground(projectId, refreshFileTree, showToast);
|
||||
|
||||
/**
|
||||
* A dial being dragged writes to the preview and stops there.
|
||||
@@ -328,6 +275,7 @@ export function StudioRightPanel({
|
||||
copiedAgentPrompt={copiedAgentPrompt}
|
||||
onClearSelection={clearDomSelection}
|
||||
onToggleElementHidden={onToggleElementHidden}
|
||||
onAutoGroupCarveSources={onAutoGroupCarveSources}
|
||||
onUngroup={handleUngroupSelection}
|
||||
onSetStyle={handleDomStyleCommit}
|
||||
onSetAttribute={handleDomAttributeCommit}
|
||||
|
||||
@@ -51,5 +51,6 @@ export interface StudioRightPanelProps extends StudioEditPersistenceProps {
|
||||
files: Record<string, { before: string; after: string }>;
|
||||
}) => Promise<void>;
|
||||
onToggleElementHidden?: ToggleHiddenHandler;
|
||||
onAutoGroupCarveSources?: (clipIds: readonly string[], groupId: string) => Promise<void>;
|
||||
onAddMediaOverlay?: AddMediaOverlayHandler;
|
||||
}
|
||||
|
||||
@@ -79,6 +79,7 @@ export function PropertyPanelFlat({
|
||||
onRemoveTextField,
|
||||
onAskAgent,
|
||||
onToggleElementHidden,
|
||||
onAutoGroupCarveSources,
|
||||
onImportAssets,
|
||||
onAddMediaOverlay,
|
||||
onImportFonts,
|
||||
@@ -440,6 +441,7 @@ export function PropertyPanelFlat({
|
||||
element={element}
|
||||
onSetAttributeQuiet={onSetAttributeQuiet ?? onSetAttributeLive}
|
||||
onSetAttributeLive={onSetAttributeLive}
|
||||
onAutoGroupCarveSources={onAutoGroupCarveSources}
|
||||
/>
|
||||
),
|
||||
});
|
||||
|
||||
@@ -1507,6 +1507,142 @@ describe("AudioFxGroup carve source list", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("AudioFxGroup carve targets groups (B6)", () => {
|
||||
// These tests are the only ones in this file whose auto-carve effect makes a
|
||||
// real cross-element write call (`onAutoGroupCarveSources`), which can be a
|
||||
// genuine Promise. None of this file's other `mount*` helpers ever unmount
|
||||
// their React root — harmless everywhere else because their effects only
|
||||
// ever touch a plain `onSetAttributeQuiet` mock, so an orphaned root left
|
||||
// over from an earlier test does nothing observable if it ever re-renders.
|
||||
// Here it can re-fire the auto-group effect against WHATEVER a later test's
|
||||
// fixture put in the (file-global) `document`, calling a long-dead test's
|
||||
// mock — unmounting is what prevents that.
|
||||
const roots: ReturnType<typeof createRoot>[] = [];
|
||||
afterEach(() => {
|
||||
for (const root of roots.splice(0)) act(() => root.unmount());
|
||||
});
|
||||
|
||||
/** A bed, plus tracks that may carry `data-audio-group`, plus an optional auto-group handler. */
|
||||
const mountWith = (
|
||||
tracks: { id: string; group?: string; start?: string; duration?: string }[],
|
||||
onAutoGroupCarveSources?: (clipIds: readonly string[], groupId: string) => Promise<void>,
|
||||
) => {
|
||||
const bed = document.createElement("audio");
|
||||
bed.id = "bed";
|
||||
document.body.append(bed);
|
||||
for (const t of tracks) {
|
||||
const el = document.createElement("audio");
|
||||
el.id = t.id;
|
||||
if (t.group) el.setAttribute("data-audio-group", t.group);
|
||||
if (t.start !== undefined) el.setAttribute("data-start", t.start);
|
||||
if (t.duration !== undefined) el.setAttribute("data-duration", t.duration);
|
||||
document.body.append(el);
|
||||
}
|
||||
const onSetAttributeQuiet = vi.fn();
|
||||
const host = document.createElement("div");
|
||||
document.body.append(host);
|
||||
const root = createRoot(host);
|
||||
roots.push(root);
|
||||
act(() => {
|
||||
root.render(
|
||||
<AudioFxGroup
|
||||
element={{ dataAttributes: {}, id: "bed", element: bed } as unknown as DomEditSelection}
|
||||
onSetAttributeQuiet={onSetAttributeQuiet}
|
||||
onSetAttributeLive={vi.fn()}
|
||||
onAutoGroupCarveSources={onAutoGroupCarveSources}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
const offered = Array.from(host.querySelectorAll<HTMLElement>("[data-carve-source]"));
|
||||
const options = offered.map((el) => el.dataset["carveSource"] ?? "");
|
||||
const boxes = offered.filter((el): el is HTMLInputElement => el instanceof HTMLInputElement);
|
||||
return { host, options, boxes, onSetAttributeQuiet };
|
||||
};
|
||||
|
||||
it("offers one entry for a group and hides its members individually", () => {
|
||||
const { options } = mountWith([
|
||||
{ id: "vo-1", group: "voiceover" },
|
||||
{ id: "vo-2", group: "voiceover" },
|
||||
]);
|
||||
expect(options).toEqual(["voiceover"]);
|
||||
});
|
||||
|
||||
it("offers the group when only ONE member overlaps the bed (union, not per-clip)", () => {
|
||||
// The bed spans the whole default window (no start/duration on the
|
||||
// selection itself in this harness resolves to [0, Infinity)), so give the
|
||||
// members explicit, non-overlapping-with-each-other spans and confirm the
|
||||
// group still appears as long as at least one of them is in range.
|
||||
const { options } = mountWith([
|
||||
{ id: "vo-1", group: "voiceover", start: "0", duration: "5" },
|
||||
{ id: "vo-2", group: "voiceover", start: "1000", duration: "5" },
|
||||
]);
|
||||
expect(options).toEqual(["voiceover"]);
|
||||
});
|
||||
|
||||
it("auto-selects the group over its individual members", () => {
|
||||
const { onSetAttributeQuiet } = mountWith([
|
||||
{ id: "vo-1", group: "voiceover" },
|
||||
{ id: "vo-2", group: "voiceover" },
|
||||
]);
|
||||
const write = writeTo(onSetAttributeQuiet.mock.calls, "data-fx-carve");
|
||||
expect(JSON.parse(String(write![1])).sources).toEqual(["voiceover"]);
|
||||
});
|
||||
|
||||
it("auto-groups a multi-voice carve into one named group, atomically", async () => {
|
||||
// Two ungrouped, both voice-classified: the existing "every candidate
|
||||
// carves itself" mount effect names both by id, which is exactly the
|
||||
// plural-ungrouped case B6 intercepts — the carve should land on the
|
||||
// minted group, not on the two ids directly.
|
||||
const onAutoGroupCarveSources = vi.fn().mockResolvedValue(undefined);
|
||||
const { onSetAttributeQuiet } = mountWith(
|
||||
[
|
||||
{ id: "narration", start: "0", duration: "5" },
|
||||
{ id: "interview-guest", start: "10", duration: "5" },
|
||||
],
|
||||
onAutoGroupCarveSources,
|
||||
);
|
||||
expect(onAutoGroupCarveSources).toHaveBeenCalledWith(
|
||||
["narration", "interview-guest"],
|
||||
"voiceover",
|
||||
);
|
||||
// Explicitly await the exact promise `assignGroup` returned — a bare
|
||||
// `await Promise.resolve()`/`setTimeout` flush is guessing how deep the
|
||||
// chain behind it goes (its `.then(...)`, the write, and the re-analysis
|
||||
// `setCarve` awaits afterward). Left genuinely unresolved when this test
|
||||
// returns, that chain settles during a LATER test instead, after this
|
||||
// one's mocks and DOM are gone.
|
||||
await act(async () => {
|
||||
await onAutoGroupCarveSources.mock.results[0]?.value;
|
||||
// Two more turns of the microtask queue: one for the `.then(...)` that
|
||||
// builds the grouped settings, one for `setCarve`'s own trailing
|
||||
// `await analyse(next)` (a no-op here — the fixture's tracks have no
|
||||
// `src`, so `resolveCarveVoices` returns empty and `analyse` exits
|
||||
// immediately, but it still has to actually run to completion).
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
});
|
||||
const write = writeTo(onSetAttributeQuiet.mock.calls, "data-fx-carve");
|
||||
expect(write).toBeTruthy();
|
||||
expect(JSON.parse(String(write![1])).sources).toEqual(["voiceover"]);
|
||||
});
|
||||
|
||||
it("leaves sources alone when one of them already names a group", () => {
|
||||
const onAutoGroupCarveSources = vi.fn();
|
||||
const { onSetAttributeQuiet } = mountWith(
|
||||
[
|
||||
{ id: "vo-1", group: "voiceover" },
|
||||
{ id: "vo-2", group: "voiceover" },
|
||||
],
|
||||
onAutoGroupCarveSources,
|
||||
);
|
||||
// The default-carve effect already named the group (previous test above),
|
||||
// so no auto-group call should ever fire for an all-group source list.
|
||||
expect(onAutoGroupCarveSources).not.toHaveBeenCalled();
|
||||
const write = writeTo(onSetAttributeQuiet.mock.calls, "data-fx-carve");
|
||||
expect(JSON.parse(String(write![1])).sources).toEqual(["voiceover"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("AudioFxGroup carve source range", () => {
|
||||
const spanned = (tracks: { id: string; start?: string; duration?: string }[]): string[] => {
|
||||
const bed = document.createElement("audio");
|
||||
|
||||
@@ -55,6 +55,7 @@ export function AudioFxGroup({
|
||||
element,
|
||||
onSetAttributeQuiet: onSetAttributeQuietRaw,
|
||||
onSetAttributeLive,
|
||||
onAutoGroupCarveSources,
|
||||
}: {
|
||||
element: DomEditSelection;
|
||||
/**
|
||||
@@ -70,6 +71,8 @@ export function AudioFxGroup({
|
||||
onSetAttributeQuiet: (attr: string, value: string | null) => void | Promise<void>;
|
||||
/** Continuous, non-persisting write for a dial being dragged. */
|
||||
onSetAttributeLive: (attr: string, value: string | null) => void | Promise<void>;
|
||||
/** Write `data-audio-group` on every named clip, atomically, one undo entry. */
|
||||
onAutoGroupCarveSources?: (clipIds: readonly string[], groupId: string) => Promise<void>;
|
||||
}) {
|
||||
const chain = ((): HfAudioFxChain => {
|
||||
const raw = element.dataAttributes?.[HF_AUDIO_FX_DATA_KEY];
|
||||
@@ -231,6 +234,7 @@ export function AudioFxGroup({
|
||||
onSetAttributeQuiet,
|
||||
writeAutomation,
|
||||
setAnalysing,
|
||||
onAutoGroupCarveSources,
|
||||
);
|
||||
|
||||
const { runLeveller, auditionTransport, auditioningLevel, auditionLevel, removeLeveller } =
|
||||
|
||||
@@ -27,6 +27,7 @@ export type PropertyPanelFlatProps = Pick<
|
||||
| "onRemoveTextField"
|
||||
| "onAskAgent"
|
||||
| "onToggleElementHidden"
|
||||
| "onAutoGroupCarveSources"
|
||||
| "onImportAssets"
|
||||
| "onAddMediaOverlay"
|
||||
| "onImportFonts"
|
||||
|
||||
@@ -92,6 +92,8 @@ export interface PropertyPanelProps {
|
||||
onRemoveTextField: (fieldKey: string) => void;
|
||||
onAskAgent: () => void;
|
||||
onToggleElementHidden?: (elementKey: string, hidden: boolean) => void | Promise<void>;
|
||||
/** B6: group two or more picked voice clips, atomically, one undo entry. */
|
||||
onAutoGroupCarveSources?: (clipIds: readonly string[], groupId: string) => Promise<void>;
|
||||
onImportAssets?: (files: FileList, dir?: string) => Promise<string[]>;
|
||||
onAddMediaOverlay?: AddMediaOverlayHandler;
|
||||
fontAssets?: ImportedFontAsset[];
|
||||
|
||||
Binary file not shown.
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* B6's normative rule: plural voiceover carve always targets a group.
|
||||
*
|
||||
* Split out of `useFxCarve.ts`, which owned all of this before the file grew
|
||||
* past a size where "carve targets a group" was still one thing to read —
|
||||
* same reason `useFxCarve.ts` itself was split out of
|
||||
* `propertyPanelAudioFxGroup.tsx`.
|
||||
*/
|
||||
|
||||
import { classifyAudioName, type HfCarveSettings } from "@hyperframes/core/audio-carve";
|
||||
import { resolveAudioGroups } from "@hyperframes/core/audio-groups";
|
||||
|
||||
/**
|
||||
* An id for a new voiceover group, de-duped against every id already in the
|
||||
* document — group ids and plain element ids share one namespace, so both
|
||||
* have to be checked.
|
||||
*/
|
||||
function mintGroupId(doc: Document): string {
|
||||
const taken = new Set([
|
||||
...resolveAudioGroups(doc).map((g) => g.id),
|
||||
...Array.from(doc.querySelectorAll("[id]")).map((el) => el.id),
|
||||
]);
|
||||
if (!taken.has("voiceover")) return "voiceover";
|
||||
let n = 2;
|
||||
while (taken.has(`voiceover-${n}`)) n += 1;
|
||||
return `voiceover-${n}`;
|
||||
}
|
||||
|
||||
export function isPromiseLike<T>(value: T | Promise<T>): value is Promise<T> {
|
||||
return typeof (value as { then?: unknown })?.then === "function";
|
||||
}
|
||||
|
||||
/**
|
||||
* Plural voiceover carve, always against a group — normative, not a
|
||||
* suggestion (groups doc §1.6). Picking a second ungrouped voice clip mints a
|
||||
* group behind the two of them and points the carve at it instead, the same
|
||||
* way a hand-authored composition is expected to work; a source list already
|
||||
* naming a group is left alone; this only fires on a run of plain clip ids.
|
||||
*
|
||||
* Returns the settings unchanged, synchronously, when there is nothing to do —
|
||||
* NOT wrapped in a promise even though the caller may `await` the result. An
|
||||
* `async` function always yields a microtask, which would push every
|
||||
* `setCarve` call (grouped or not) one tick later than before this existed;
|
||||
* several tests assert on `onSetAttributeQuiet` synchronously after mount and
|
||||
* would miss that write.
|
||||
*/
|
||||
function withAutoGroupedSources(
|
||||
doc: Document,
|
||||
next: HfCarveSettings,
|
||||
assignGroup: ((clipIds: readonly string[], groupId: string) => Promise<void>) | undefined,
|
||||
): HfCarveSettings | Promise<HfCarveSettings> {
|
||||
if (!assignGroup || next.sources.length < 2) return next;
|
||||
const groupIds = new Set(resolveAudioGroups(doc).map((g) => g.id));
|
||||
if (next.sources.some((id) => groupIds.has(id))) return next;
|
||||
const groupId = mintGroupId(doc);
|
||||
return assignGroup(next.sources, groupId).then(() => ({ ...next, sources: [groupId] }));
|
||||
}
|
||||
|
||||
/**
|
||||
* `setCarve`'s first step: apply the auto-group rule, if a document and a
|
||||
* write-back are both available.
|
||||
*
|
||||
* Deliberately NOT an `async function`: wrapping this in one would make every
|
||||
* call return a promise-wrapped value, forcing the caller's `await` to yield
|
||||
* a microtask even on the synchronous branch — the exact bug
|
||||
* `withAutoGroupedSources`'s own sync-when-possible contract exists to avoid.
|
||||
* The caller does the `isPromiseLike` check (re-exported from here) and only
|
||||
* awaits the branch that is genuinely async.
|
||||
*/
|
||||
export function resolveNextCarveSettings(
|
||||
nextRaw: HfCarveSettings | null,
|
||||
doc: Document | undefined,
|
||||
assignGroup: ((clipIds: readonly string[], groupId: string) => Promise<void>) | undefined,
|
||||
): HfCarveSettings | Promise<HfCarveSettings> | null {
|
||||
return nextRaw && doc ? withAutoGroupedSources(doc, nextRaw, assignGroup) : nextRaw;
|
||||
}
|
||||
|
||||
export interface CarveCandidate {
|
||||
id: string;
|
||||
label: string;
|
||||
kind: ReturnType<typeof classifyAudioName>;
|
||||
}
|
||||
|
||||
/**
|
||||
* One row per ungrouped clip that overlaps the bed, one row per group that has
|
||||
* ANY overlapping member (union, not per-clip — a narration group spanning
|
||||
* the whole timeline is relevant even if each of its segments only overlaps
|
||||
* part of the bed). Grouped members never appear individually.
|
||||
*/
|
||||
export function collectCarveCandidates(
|
||||
doc: Document,
|
||||
others: readonly HTMLAudioElement[],
|
||||
overlapsBed: (a: Element) => boolean,
|
||||
): CarveCandidate[] {
|
||||
const groupByMember = new Map(
|
||||
resolveAudioGroups(doc).flatMap((group) => group.memberIds.map((id) => [id, group] as const)),
|
||||
);
|
||||
const offeredGroupIds = new Set<string>();
|
||||
const described: CarveCandidate[] = [];
|
||||
for (const a of others) {
|
||||
const group = groupByMember.get(a.id);
|
||||
if (!group) {
|
||||
if (overlapsBed(a)) {
|
||||
described.push({
|
||||
id: a.id,
|
||||
label: a.id,
|
||||
kind: classifyAudioName(a.id, a.getAttribute("src")),
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (offeredGroupIds.has(group.id)) continue;
|
||||
const members = group.memberIds
|
||||
.map((id) => doc.getElementById(id))
|
||||
// Not `instanceof HTMLAudioElement`: these belong to the composition's
|
||||
// iframe document, so the constructor is a different realm's and the
|
||||
// instanceof is false for every one (mirrors resolveCarveVoices in
|
||||
// useFxCarve.ts).
|
||||
.filter((el): el is HTMLElement => el?.tagName === "AUDIO");
|
||||
if (!members.some(overlapsBed)) continue;
|
||||
offeredGroupIds.add(group.id);
|
||||
described.push({
|
||||
id: group.id,
|
||||
label: `${group.label} (${members.length})`,
|
||||
kind: classifyAudioName(
|
||||
group.label,
|
||||
...members.flatMap((m) => [m.id, m.getAttribute("src")]),
|
||||
),
|
||||
});
|
||||
}
|
||||
return described;
|
||||
}
|
||||
@@ -2,7 +2,11 @@
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { usePlayerStore, type TimelineElement } from "../player";
|
||||
import { toggleTimelineElementHidden, toggleTimelineTrackHidden } from "./timelineTrackVisibility";
|
||||
import {
|
||||
createAudioGroupAndAssignMembers,
|
||||
toggleTimelineElementHidden,
|
||||
toggleTimelineTrackHidden,
|
||||
} from "./timelineTrackVisibility";
|
||||
|
||||
afterEach(() => {
|
||||
document.body.innerHTML = "";
|
||||
@@ -395,3 +399,139 @@ describe("toggleTimelineElementHidden", () => {
|
||||
expect(recordEdit.mock.calls[0]?.[0]?.label).toBe("Hide 2 elements");
|
||||
});
|
||||
});
|
||||
|
||||
describe("createAudioGroupAndAssignMembers", () => {
|
||||
it("writes data-audio-group on every member in ONE atomic edit and updates the player store", async () => {
|
||||
const iframe = document.createElement("iframe");
|
||||
document.body.append(iframe);
|
||||
if (iframe.contentDocument) {
|
||||
iframe.contentDocument.body.innerHTML = `
|
||||
<audio id="narration"></audio>
|
||||
<audio id="interview-guest"></audio>
|
||||
`;
|
||||
}
|
||||
|
||||
const files = new Map([
|
||||
[
|
||||
"index.html",
|
||||
`<audio id="narration" data-start="0" data-duration="5"></audio>
|
||||
<audio id="interview-guest" data-start="10" data-duration="5"></audio>
|
||||
<audio id="sfx-boom" data-start="0" data-duration="1"></audio>`,
|
||||
],
|
||||
]);
|
||||
stubProjectFiles(files);
|
||||
|
||||
const narration = element({
|
||||
id: "narration",
|
||||
key: "index.html:#narration",
|
||||
domId: "narration",
|
||||
track: 0,
|
||||
});
|
||||
const guest = element({
|
||||
id: "interview-guest",
|
||||
key: "index.html:#interview-guest",
|
||||
domId: "interview-guest",
|
||||
track: 1,
|
||||
});
|
||||
usePlayerStore.getState().setElements([narration, guest]);
|
||||
|
||||
const writes = new Map<string, string>();
|
||||
const recordEdit = vi.fn();
|
||||
|
||||
const changedPaths = await createAudioGroupAndAssignMembers({
|
||||
projectId: "project-1",
|
||||
activeCompPath: "index.html",
|
||||
elements: [narration, guest],
|
||||
groupId: "voiceover",
|
||||
previewIframe: iframe,
|
||||
writeProjectFile: async (path, content) => {
|
||||
writes.set(path, content);
|
||||
},
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef: { current: 0 },
|
||||
pendingTimelineEditPathRef: { current: new Set() },
|
||||
});
|
||||
|
||||
expect(changedPaths).toEqual(["index.html"]);
|
||||
expect(
|
||||
iframe.contentDocument?.getElementById("narration")?.getAttribute("data-audio-group"),
|
||||
).toBe("voiceover");
|
||||
expect(
|
||||
iframe.contentDocument?.getElementById("interview-guest")?.getAttribute("data-audio-group"),
|
||||
).toBe("voiceover");
|
||||
// One write carrying BOTH members — per-element writes would clobber each
|
||||
// other (each starts from the original file content).
|
||||
expect(writes.get("index.html")).toContain(
|
||||
'id="narration" data-start="0" data-duration="5" data-audio-group="voiceover"',
|
||||
);
|
||||
expect(writes.get("index.html")).toContain(
|
||||
'id="interview-guest" data-start="10" data-duration="5" data-audio-group="voiceover"',
|
||||
);
|
||||
expect(writes.get("index.html")).toContain(
|
||||
'id="sfx-boom" data-start="0" data-duration="1"></audio>',
|
||||
);
|
||||
expect(recordEdit).toHaveBeenCalledTimes(1);
|
||||
expect(recordEdit.mock.calls[0]?.[0]?.label).toBe("Group 2 voice clips");
|
||||
expect(
|
||||
usePlayerStore.getState().elements.find((el) => el.key === "index.html:#narration")
|
||||
?.audioGroup,
|
||||
).toBe("voiceover");
|
||||
expect(
|
||||
usePlayerStore.getState().elements.find((el) => el.key === "index.html:#interview-guest")
|
||||
?.audioGroup,
|
||||
).toBe("voiceover");
|
||||
});
|
||||
|
||||
it("does nothing for fewer than two elements — grouping is a plural concept", async () => {
|
||||
const recordEdit = vi.fn();
|
||||
const changedPaths = await createAudioGroupAndAssignMembers({
|
||||
projectId: "project-1",
|
||||
activeCompPath: "index.html",
|
||||
elements: [element({ id: "narration", domId: "narration" })],
|
||||
groupId: "voiceover",
|
||||
previewIframe: null,
|
||||
writeProjectFile: async () => {},
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef: { current: 0 },
|
||||
pendingTimelineEditPathRef: { current: new Set() },
|
||||
});
|
||||
expect(changedPaths).toEqual([]);
|
||||
expect(recordEdit).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reverts the optimistic live patch when the save fails", async () => {
|
||||
const iframe = document.createElement("iframe");
|
||||
document.body.append(iframe);
|
||||
if (iframe.contentDocument) {
|
||||
iframe.contentDocument.body.innerHTML = `
|
||||
<audio id="narration"></audio>
|
||||
<audio id="interview-guest"></audio>
|
||||
`;
|
||||
}
|
||||
// No stubbed fetch: readFileContent's request will fail, forcing the
|
||||
// catch path.
|
||||
const narration = element({ id: "narration", domId: "narration" });
|
||||
const guest = element({ id: "interview-guest", domId: "interview-guest" });
|
||||
|
||||
await expect(
|
||||
createAudioGroupAndAssignMembers({
|
||||
projectId: "project-1",
|
||||
activeCompPath: "index.html",
|
||||
elements: [narration, guest],
|
||||
groupId: "voiceover",
|
||||
previewIframe: iframe,
|
||||
writeProjectFile: async () => {},
|
||||
recordEdit: vi.fn(),
|
||||
domEditSaveTimestampRef: { current: 0 },
|
||||
pendingTimelineEditPathRef: { current: new Set() },
|
||||
}),
|
||||
).rejects.toThrow();
|
||||
|
||||
expect(
|
||||
iframe.contentDocument?.getElementById("narration")?.hasAttribute("data-audio-group"),
|
||||
).toBe(false);
|
||||
expect(
|
||||
iframe.contentDocument?.getElementById("interview-guest")?.hasAttribute("data-audio-group"),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
} from "../player/components/timelineTrackDisplay";
|
||||
import { saveProjectFilesWithHistory } from "../utils/studioFileHistory";
|
||||
import { isAudioTimelineElement } from "../utils/timelineInspector";
|
||||
import { HF_AUDIO_GROUP_ATTR } from "@hyperframes/core/audio-groups";
|
||||
import { readTagSnippetByTarget, type PatchOperation } from "../utils/sourcePatcher";
|
||||
import {
|
||||
applyPatchByTarget,
|
||||
@@ -277,6 +278,114 @@ export async function toggleTimelineElementHidden({
|
||||
});
|
||||
}
|
||||
|
||||
function patchLiveAudioGroupState(
|
||||
iframe: HTMLIFrameElement | null,
|
||||
elements: readonly TimelineElement[],
|
||||
groupId: string | null,
|
||||
activeCompPath: string | null,
|
||||
): void {
|
||||
for (const element of elements) {
|
||||
const target = findTimelineElementInIframe(iframe, element, activeCompPath);
|
||||
if (!target) continue;
|
||||
if (groupId) target.setAttribute(HF_AUDIO_GROUP_ATTR, groupId);
|
||||
else target.removeAttribute(HF_AUDIO_GROUP_ATTR);
|
||||
}
|
||||
}
|
||||
|
||||
interface CreateAudioGroupAndAssignMembersInput {
|
||||
projectId: string;
|
||||
activeCompPath: string | null;
|
||||
elements: readonly TimelineElement[];
|
||||
groupId: string;
|
||||
previewIframe: HTMLIFrameElement | null;
|
||||
writeProjectFile: (path: string, content: string) => Promise<void>;
|
||||
recordEdit: (input: RecordEditInput) => Promise<void>;
|
||||
domEditSaveTimestampRef: MutableRef<number>;
|
||||
pendingTimelineEditPathRef: MutableRef<Set<string>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Group two or more voice clips: write `data-audio-group="<groupId>"` on
|
||||
* every one of them, atomically, one undo entry — the same multi-target shape
|
||||
* `setElementsHidden` uses for mute. The group needs no `<hf-audio-group>`
|
||||
* element of its own to exist: `resolveAudioGroups` already degrades
|
||||
* gracefully to label = id when one is absent, and a naming dialog is out of
|
||||
* scope here — the id itself is the default name.
|
||||
*/
|
||||
// fallow-ignore-next-line complexity
|
||||
export async function createAudioGroupAndAssignMembers({
|
||||
projectId,
|
||||
activeCompPath,
|
||||
elements,
|
||||
groupId,
|
||||
previewIframe,
|
||||
writeProjectFile,
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef,
|
||||
pendingTimelineEditPathRef,
|
||||
}: CreateAudioGroupAndAssignMembersInput): Promise<string[]> {
|
||||
if (elements.length < 2) return [];
|
||||
|
||||
patchLiveAudioGroupState(previewIframe, elements, groupId, activeCompPath);
|
||||
reseekPreviewRuntime(previewIframe);
|
||||
|
||||
const groupOperation: PatchOperation = {
|
||||
type: "attribute",
|
||||
property: HF_AUDIO_GROUP_ATTR,
|
||||
value: groupId,
|
||||
};
|
||||
const originalByPath = new Map<string, string>();
|
||||
const files: Record<string, string> = {};
|
||||
|
||||
try {
|
||||
for (const [targetPath, fileElements] of groupElementsByTargetPath(elements, activeCompPath)) {
|
||||
let patchedContent = await readFileContent(projectId, targetPath);
|
||||
originalByPath.set(targetPath, patchedContent);
|
||||
|
||||
for (const element of fileElements) {
|
||||
const patchTarget = buildPatchTarget(element);
|
||||
if (!patchTarget) {
|
||||
throw new Error(`Timeline element ${element.id} is missing a patchable target`);
|
||||
}
|
||||
if (readTagSnippetByTarget(patchedContent, patchTarget) === undefined) {
|
||||
throw new Error(`Unable to patch timeline element ${element.id} in ${targetPath}`);
|
||||
}
|
||||
patchedContent = applyPatchByTarget(patchedContent, patchTarget, groupOperation);
|
||||
}
|
||||
|
||||
files[targetPath] = patchedContent;
|
||||
pendingTimelineEditPathRef.current.add(targetPath);
|
||||
}
|
||||
|
||||
domEditSaveTimestampRef.current = Date.now();
|
||||
const changedPaths = await saveProjectFilesWithHistory({
|
||||
projectId,
|
||||
label: `Group ${elements.length} voice clips`,
|
||||
kind: "timeline",
|
||||
files,
|
||||
readFile: async (path) => {
|
||||
const original = originalByPath.get(path);
|
||||
if (original !== undefined) return original;
|
||||
return readFileContent(projectId, path);
|
||||
},
|
||||
writeFile: writeProjectFile,
|
||||
recordEdit,
|
||||
});
|
||||
domEditSaveTimestampRef.current = Date.now();
|
||||
for (const element of elements) {
|
||||
usePlayerStore.getState().updateElement(element.key ?? element.id, { audioGroup: groupId });
|
||||
}
|
||||
return changedPaths;
|
||||
} catch (error) {
|
||||
// Mirrors setElementsHidden's failure path: the optimistic live patch
|
||||
// already ran, so a save failure has to be unwound or the preview shows a
|
||||
// grouping that never made it to disk.
|
||||
patchLiveAudioGroupState(previewIframe, elements, null, activeCompPath);
|
||||
reseekPreviewRuntime(previewIframe);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function useTimelineTrackVisibilityEditing({
|
||||
projectIdRef,
|
||||
activeCompPath,
|
||||
@@ -407,3 +516,67 @@ export function useTimelineElementVisibilityEditing({
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The write behind B6's auto-group: pick two or more voice clips in the carve
|
||||
* picker and they land in a group instead of naming each other by id. Same
|
||||
* expanded-rows resolution as element-visibility, for the same reason — a
|
||||
* nested sub-composition child has no entry in the raw store list.
|
||||
*/
|
||||
export function useAudioGroupCarveAssignment({
|
||||
projectIdRef,
|
||||
activeCompPath,
|
||||
showToast,
|
||||
writeProjectFile,
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef,
|
||||
previewIframeRef,
|
||||
pendingTimelineEditPathRef,
|
||||
isRecordingRef,
|
||||
}: UseTimelineElementVisibilityEditingInput): (
|
||||
clipIds: readonly string[],
|
||||
groupId: string,
|
||||
) => Promise<void> {
|
||||
const expandedElements = useExpandedTimelineElements();
|
||||
return useCallback(
|
||||
async (clipIds: readonly string[], groupId: string) => {
|
||||
if (isRecordingRef?.current) {
|
||||
showToast("Cannot edit timeline while recording", "error");
|
||||
return;
|
||||
}
|
||||
const pid = projectIdRef.current;
|
||||
if (!pid) return;
|
||||
const keys = new Set(clipIds);
|
||||
const elements = expandedElements.filter((item) => keys.has(item.key ?? item.id));
|
||||
try {
|
||||
await createAudioGroupAndAssignMembers({
|
||||
projectId: pid,
|
||||
activeCompPath,
|
||||
elements,
|
||||
groupId,
|
||||
previewIframe: previewIframeRef.current,
|
||||
writeProjectFile,
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef,
|
||||
pendingTimelineEditPathRef,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("[Timeline] Failed to group voice clips", error);
|
||||
const message = error instanceof Error ? error.message : "Failed to group voice clips";
|
||||
showToast(message);
|
||||
}
|
||||
},
|
||||
[
|
||||
activeCompPath,
|
||||
expandedElements,
|
||||
previewIframeRef,
|
||||
writeProjectFile,
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef,
|
||||
pendingTimelineEditPathRef,
|
||||
isRecordingRef,
|
||||
showToast,
|
||||
projectIdRef,
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
import { useMemo } from "react";
|
||||
import type { TimelineElement } from "../player/store/timelineElement";
|
||||
|
||||
/**
|
||||
* The stored `duration` lags a moment behind an edit that pushes an element
|
||||
* past it (drag, trim, paste) — this is the actual end of the timeline, the
|
||||
* later of the stored duration and the furthest element's end.
|
||||
*/
|
||||
export function useEffectiveTimelineDuration(
|
||||
timelineDuration: number,
|
||||
timelineElements: readonly TimelineElement[],
|
||||
): number {
|
||||
return useMemo(() => {
|
||||
const maxEnd =
|
||||
timelineElements.length > 0
|
||||
? Math.max(...timelineElements.map((el) => el.start + el.duration))
|
||||
: 0;
|
||||
return Math.max(timelineDuration, maxEnd);
|
||||
}, [timelineDuration, timelineElements]);
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
import { useCallback, useEffect, useRef } from "react";
|
||||
import { waitForMediaJob } from "../components/studioMediaJobs";
|
||||
import type { BackgroundRemovalProgress } from "../components/editor/propertyPanelTypes";
|
||||
|
||||
interface RemoveBackgroundOptions {
|
||||
createBackgroundPlate?: boolean;
|
||||
quality?: "fast" | "balanced" | "best";
|
||||
onProgress?: (progress: BackgroundRemovalProgress) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* One removal in flight at a time: starting a second one aborts whichever job
|
||||
* is still running, so a stale progress callback can't overwrite a newer
|
||||
* result. Unmounting aborts too, or the job would keep running against a
|
||||
* panel that is no longer there to show its progress.
|
||||
*/
|
||||
export function useRemoveBackground(
|
||||
projectId: string,
|
||||
refreshFileTree: () => Promise<void>,
|
||||
showToast: (message: string, kind?: "info" | "error") => void,
|
||||
) {
|
||||
const abortRef = useRef<AbortController | null>(null);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
abortRef.current?.abort();
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
return useCallback(
|
||||
// fallow-ignore-next-line complexity
|
||||
async (inputPath: string, options: RemoveBackgroundOptions) => {
|
||||
const response = await fetch(
|
||||
`/api/projects/${encodeURIComponent(projectId)}/media/remove-background`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
inputPath,
|
||||
createBackgroundPlate: options.createBackgroundPlate === true,
|
||||
quality: options.quality ?? "balanced",
|
||||
}),
|
||||
},
|
||||
);
|
||||
const data = (await response.json().catch(() => ({}))) as {
|
||||
jobId?: string;
|
||||
error?: string;
|
||||
};
|
||||
if (!response.ok || !data.jobId) {
|
||||
throw new Error(data.error || `Background removal failed (${response.status})`);
|
||||
}
|
||||
showToast("Removing background...", "info");
|
||||
abortRef.current?.abort();
|
||||
const controller = new AbortController();
|
||||
abortRef.current = controller;
|
||||
try {
|
||||
const result = await waitForMediaJob(data.jobId, options.onProgress, controller.signal);
|
||||
await refreshFileTree();
|
||||
showToast(`Created transparent asset: ${result.outputPath.split("/").pop()}`, "info");
|
||||
return result;
|
||||
} finally {
|
||||
if (abortRef.current === controller) {
|
||||
abortRef.current = null;
|
||||
}
|
||||
}
|
||||
},
|
||||
[projectId, refreshFileTree, showToast],
|
||||
);
|
||||
}
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
import type { PersistTimelineEditInput } from "./timelineEditingHelpers";
|
||||
import type { TimelineStackingReorderIntent } from "../player/components/timelineEditing";
|
||||
import {
|
||||
useAudioGroupCarveAssignment,
|
||||
useTimelineElementVisibilityEditing,
|
||||
useTimelineTrackVisibilityEditing,
|
||||
} from "./timelineTrackVisibility";
|
||||
@@ -388,6 +389,18 @@ export function useTimelineEditing({
|
||||
forceReloadSdkSession,
|
||||
});
|
||||
|
||||
const handleAutoGroupCarveSources = useAudioGroupCarveAssignment({
|
||||
projectIdRef,
|
||||
activeCompPath,
|
||||
showToast,
|
||||
writeProjectFile,
|
||||
recordEdit,
|
||||
domEditSaveTimestampRef,
|
||||
previewIframeRef,
|
||||
pendingTimelineEditPathRef,
|
||||
isRecordingRef,
|
||||
});
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
const handleTimelineElementsDelete = useCallback(
|
||||
// fallow-ignore-next-line complexity
|
||||
@@ -558,6 +571,7 @@ export function useTimelineEditing({
|
||||
handleTimelineElementResize,
|
||||
handleToggleTrackHidden,
|
||||
handleToggleElementHidden,
|
||||
handleAutoGroupCarveSources,
|
||||
handleTimelineElementDelete,
|
||||
handleTimelineElementsDelete,
|
||||
handleTimelineElementSplit: handleRazorSplit,
|
||||
|
||||
@@ -146,7 +146,14 @@ interface PlayerState extends KeyframeSlice, AutomationSelectionSlice, Thumbnail
|
||||
updates: Partial<
|
||||
Pick<
|
||||
TimelineElement,
|
||||
"start" | "duration" | "track" | "zIndex" | "hasExplicitZIndex" | "playbackStart" | "hidden"
|
||||
| "start"
|
||||
| "duration"
|
||||
| "track"
|
||||
| "zIndex"
|
||||
| "hasExplicitZIndex"
|
||||
| "playbackStart"
|
||||
| "hidden"
|
||||
| "audioGroup"
|
||||
>
|
||||
>,
|
||||
) => void;
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
"files": 121
|
||||
},
|
||||
"hyperframes-audio": {
|
||||
"hash": "94aba963d262d71d",
|
||||
"hash": "f224ea9481998c08",
|
||||
"files": 6
|
||||
},
|
||||
"hyperframes-cli": {
|
||||
|
||||
@@ -236,6 +236,31 @@ so one analysis covers all of them: the bands come from all the speech there is,
|
||||
the envelopes rise wherever any of it is happening. Voices that never play while the
|
||||
bed does are left out; they cannot mask it.
|
||||
|
||||
**A carve against more than one clip id is wrong. Group the clips and carve
|
||||
against the group.** This is an invariant, not a tip. Naming clips one by one has
|
||||
to be exhaustively right and stays right only until the next edit — a fourth
|
||||
narration clip added later plays outside the carve's awareness, and the bed
|
||||
fails to duck under it silently. Naming the group instead resolves membership at
|
||||
analysis time, so a clip added to the group later is covered without touching
|
||||
`sources` at all:
|
||||
|
||||
```html
|
||||
<!-- group the narration, then carve the bed against the group -->
|
||||
<audio id="vo-intro" data-audio-group="voiceover" …></audio>
|
||||
<audio id="vo-middle" data-audio-group="voiceover" …></audio>
|
||||
<audio id="vo-outro" data-audio-group="voiceover" …></audio>
|
||||
|
||||
<audio
|
||||
id="music"
|
||||
data-fx-carve='{"enabled":true,"sources":["voiceover"],"strength":0.25}'
|
||||
…
|
||||
></audio>
|
||||
```
|
||||
|
||||
A `sources` list naming two or more plain clip ids instead of a group is caught
|
||||
by the `audio_carve_ungrouped_sources` lint rule — it still works, but it is the
|
||||
version that silently rots when a clip is added.
|
||||
|
||||
**One knob.** `strength` is 0..1 and derives everything: how deep to cut, how
|
||||
many bands, how wide, how far to favour intelligibility over raw voice energy,
|
||||
how far the level may drop, how far under the voice to aim. Those six move
|
||||
|
||||
Reference in New Issue
Block a user