mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
feat(studio): typed audio values, clip fades, audible video drops (#4250)
* feat(studio): typed audio values, clip fades, audible video drops Audio rows in the media inspector (Volume, Speed, Media start) take a typed value: click the readout, type "-6 dB", "1.5x" or "2.5s", Enter. Speed presets are hidden on <audio> clips. Clip-edge fades: data-fade-in / data-fade-out in seconds, honoured by the preview runtime (per-tick gain), the render mixer (afade after volume) and Studio (Fade in / Fade out rows, and drag dots at the top corners of audio clips and audible videos). Linear on both sides so preview and render agree; fades that outrun the clip scale to meet inside it. Dropped videos with an audio stream now land with data-has-audio="true" instead of muted, so the mixer keeps their sound. The media metadata endpoint reports hasAudio from the ffprobe call it already makes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(audio): one owner for fades, dB, and afade-after-volume Illegal fade values become no-fade at fadeGain; typed inspector input still returns null so the readout stays red. Master and group mixes share one filter chain with volume then afade. Fade-out stays anchored to the trimmed clip end. * fix(studio): share audio helpers and bound typed media values * fix(studio): reserve sibling fade duration when editing --------- Co-authored-by: Jake Moran <jakemoran2011@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
This commit is contained in:
co-authored by
Jake Moran
Claude Fable 5.1
Miguel Angel Simon Sierra
parent
3091dcb787
commit
839ae426dd
@@ -126,6 +126,7 @@ Audio has no visual lifecycle.
|
||||
| `data-media-start` / `data-playback-start` | Video, audio, nested composition | Offset into the source file, used by trim and split. Two groups of readers disagree, so the right name depends on the element. **Read only `data-media-start`:** the timing compiler, the HTML parser, `hyperframes validate` (which only inspects `<audio>`), and the engine's audio mixer (which feeds ffmpeg `-ss`). **Read `data-playback-start` first, falling back to `data-media-start`:** the runtime player, Studio (which also writes it), and `hyperframes snapshot`. Because the audio mixer reads only `data-media-start`, a `<video>` authored with just `data-playback-start` renders a trimmed picture over untrimmed audio. Set the name by kind: **`<video>` / `<audio>` → `data-media-start`**; **nested composition → `data-playback-start`** — composition hosts are inspected only by the playback-start-first readers (the media-start-only ones are all `<video>`/`<audio>`-scoped). `data-media-start` still works there as a fallback, but `data-playback-start` is the canonical name Studio writes and normalises to for new composition hosts, so the other name works until an edit rewrites it (it is the [child-timeline offset](/concepts/compositions)). |
|
||||
| `data-playback-rate` | Video, audio, nested composition | Playback multiplier from `0.1` to `10`. A `rate` lane in `data-automation` overrides it; see [Speed ramps](/reference/speed-ramps) |
|
||||
| `data-volume` | Video and audio | Static gain. `1` is 0 dB, `0` is silence, and values above `1` boost up to `3.98` (+12 dB) |
|
||||
| `data-fade-in` / `data-fade-out` | Video and audio | Clip-edge fades in seconds: gain rises 0 → 1 over the first `data-fade-in` seconds and falls 1 → 0 over the last `data-fade-out` seconds of the clip. Linear, anchored to the clip's own edges (a fade-out follows a later trim), and multiplied on top of `data-volume` and any volume lane. Fades that together exceed the clip are scaled to meet inside it. Studio drags them from the dots at an audio clip's top corners or types them in the inspector's Fade rows |
|
||||
| `data-has-audio="true"` | Video | Declares that the video contributes audio |
|
||||
|
||||
Video and audio may omit `data-duration` when their intrinsic duration is known
|
||||
|
||||
@@ -11,7 +11,8 @@ Core attributes for controlling element timing and behavior.
|
||||
## Media
|
||||
|
||||
- `data-media-start="2"` — Media playback offset / trim point (seconds)
|
||||
- `data-volume="0.8"` — Audio/video gain. `1` is 0 dB, `0` is silence, and values above `1` boost up to `3.98` (+12 dB). For fades and ducking use the `data-automation` volume lane (`hyperframes-core` skill, `creator-editing-recipes.md`)
|
||||
- `data-volume="0.8"` — Audio/video gain. `1` is 0 dB, `0` is silence, and values above `1` boost up to `3.98` (+12 dB). For ducking and shaped envelopes use the `data-automation` volume lane (`hyperframes-core` skill, `creator-editing-recipes.md`)
|
||||
- `data-fade-in="0.5"` / `data-fade-out="1"` — Clip-edge fades (seconds): linear gain ramps anchored to the clip's start and end, multiplied on top of `data-volume` and any volume lane
|
||||
- `data-has-audio="true"` — Indicates video has an audio track
|
||||
|
||||
## Composition
|
||||
|
||||
@@ -283,6 +283,7 @@ Background music or sound effects. Audio clips are invisible.
|
||||
```
|
||||
|
||||
- `data-media-start` — Playback begins at this time in the source audio file (seconds). Default: `0`.
|
||||
- `data-fade-in` / `data-fade-out` — (optional) Clip-edge fades in seconds. Gain ramps linearly from silence over the first `data-fade-in` seconds and back to silence over the last `data-fade-out` seconds of the clip, on top of `data-volume` and any `data-automation` volume lane. Anchored to the clip's edges, so a fade-out follows a later trim. Also honoured on `<video>` audio.
|
||||
- `data-duration` — (optional) How long the clip occupies on the timeline, in seconds. Playback runs from `data-media-start` for up to `data-duration` seconds. If the source media runs out before `data-duration` elapses, playback naturally stops (the clip remains mounted but silent). If omitted, defaults to the remaining duration of the source file from `data-media-start`.
|
||||
|
||||
## Two Layers: Primitives and Scripts
|
||||
|
||||
@@ -188,6 +188,12 @@
|
||||
"types": "./dist/audioGain.d.ts",
|
||||
"environments": ["browser", "bun", "node"]
|
||||
},
|
||||
"./audio-fade": {
|
||||
"source": "./src/audioFade.ts",
|
||||
"runtime": "./dist/audioFade.js",
|
||||
"types": "./dist/audioFade.d.ts",
|
||||
"environments": ["browser", "bun", "node"]
|
||||
},
|
||||
"./speed-ramp": {
|
||||
"source": "./src/speedRamp.ts",
|
||||
"runtime": "./dist/speedRamp.js",
|
||||
|
||||
@@ -202,6 +202,12 @@
|
||||
"import": "./src/audioGain.ts",
|
||||
"types": "./src/audioGain.ts"
|
||||
},
|
||||
"./audio-fade": {
|
||||
"bun": "./src/audioFade.ts",
|
||||
"node": "./dist/audioFade.js",
|
||||
"import": "./src/audioFade.ts",
|
||||
"types": "./src/audioFade.ts"
|
||||
},
|
||||
"./speed-ramp": {
|
||||
"bun": "./src/speedRamp.ts",
|
||||
"node": "./dist/speedRamp.js",
|
||||
@@ -546,6 +552,10 @@
|
||||
"import": "./dist/audioGain.js",
|
||||
"types": "./dist/audioGain.d.ts"
|
||||
},
|
||||
"./audio-fade": {
|
||||
"import": "./dist/audioFade.js",
|
||||
"types": "./dist/audioFade.d.ts"
|
||||
},
|
||||
"./speed-ramp": {
|
||||
"import": "./dist/speedRamp.js",
|
||||
"types": "./dist/speedRamp.d.ts"
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
clampFadesToDuration,
|
||||
fadeGain,
|
||||
formatFadeSeconds,
|
||||
readElementFades,
|
||||
readFadeSeconds,
|
||||
} from "./audioFade.js";
|
||||
|
||||
describe("readFadeSeconds", () => {
|
||||
it("reads a positive number and treats everything else as no fade", () => {
|
||||
expect(readFadeSeconds("0.5")).toBe(0.5);
|
||||
expect(readFadeSeconds("2")).toBe(2);
|
||||
expect(readFadeSeconds(null)).toBe(0);
|
||||
expect(readFadeSeconds("")).toBe(0);
|
||||
expect(readFadeSeconds("-1")).toBe(0);
|
||||
expect(readFadeSeconds("NaN")).toBe(0);
|
||||
expect(readFadeSeconds("abc")).toBe(0);
|
||||
});
|
||||
|
||||
it("reads both attributes off an element", () => {
|
||||
const attrs: Record<string, string> = { "data-fade-in": "0.25", "data-fade-out": "1.5" };
|
||||
expect(readElementFades({ getAttribute: (n) => attrs[n] ?? null })).toEqual({
|
||||
fadeIn: 0.25,
|
||||
fadeOut: 1.5,
|
||||
});
|
||||
expect(readElementFades({ getAttribute: () => null })).toEqual({ fadeIn: 0, fadeOut: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe("clampFadesToDuration", () => {
|
||||
it("leaves fades that fit alone", () => {
|
||||
expect(clampFadesToDuration({ fadeIn: 1, fadeOut: 2 }, 10)).toEqual({ fadeIn: 1, fadeOut: 2 });
|
||||
});
|
||||
it("scales fades that outrun the clip so they meet inside it", () => {
|
||||
expect(clampFadesToDuration({ fadeIn: 6, fadeOut: 6 }, 6)).toEqual({ fadeIn: 3, fadeOut: 3 });
|
||||
expect(clampFadesToDuration({ fadeIn: 3, fadeOut: 1 }, 2)).toEqual({
|
||||
fadeIn: 1.5,
|
||||
fadeOut: 0.5,
|
||||
});
|
||||
});
|
||||
it("keeps authored fades on a clip of unknown duration", () => {
|
||||
expect(clampFadesToDuration({ fadeIn: 4, fadeOut: 4 }, Number.POSITIVE_INFINITY)).toEqual({
|
||||
fadeIn: 4,
|
||||
fadeOut: 4,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("fadeGain", () => {
|
||||
const fades = { fadeIn: 2, fadeOut: 1 };
|
||||
it("is unity with no fades", () => {
|
||||
expect(fadeGain(0, 10, { fadeIn: 0, fadeOut: 0 })).toBe(1);
|
||||
expect(fadeGain(10, 10, { fadeIn: 0, fadeOut: 0 })).toBe(1);
|
||||
});
|
||||
it("ramps linearly from silence over the fade-in", () => {
|
||||
expect(fadeGain(0, 10, fades)).toBe(0);
|
||||
expect(fadeGain(1, 10, fades)).toBe(0.5);
|
||||
expect(fadeGain(2, 10, fades)).toBe(1);
|
||||
expect(fadeGain(5, 10, fades)).toBe(1);
|
||||
});
|
||||
it("ramps linearly to silence over the fade-out, anchored at the clip end", () => {
|
||||
expect(fadeGain(9, 10, fades)).toBe(1);
|
||||
expect(fadeGain(9.5, 10, fades)).toBe(0.5);
|
||||
expect(fadeGain(10, 10, fades)).toBe(0);
|
||||
expect(fadeGain(11, 10, fades)).toBe(0);
|
||||
});
|
||||
it("multiplies where the two fades overlap on a short clip", () => {
|
||||
// 2 s clip, both fades scaled to 1 s each: midpoint is the seam.
|
||||
expect(fadeGain(1, 2, { fadeIn: 2, fadeOut: 2 })).toBe(1);
|
||||
expect(fadeGain(0.5, 2, { fadeIn: 2, fadeOut: 2 })).toBe(0.5);
|
||||
});
|
||||
it("never applies a fade-out to a clip of unknown duration", () => {
|
||||
expect(fadeGain(100, Number.POSITIVE_INFINITY, fades)).toBe(1);
|
||||
expect(fadeGain(1, Number.POSITIVE_INFINITY, fades)).toBe(0.5);
|
||||
});
|
||||
it("treats 0, negative, and NaN fades as no fade", () => {
|
||||
expect(fadeGain(0, 10, { fadeIn: 0, fadeOut: 0 })).toBe(1);
|
||||
expect(fadeGain(0, 10, { fadeIn: -1, fadeOut: -4 })).toBe(1);
|
||||
expect(fadeGain(0, 10, { fadeIn: Number.NaN, fadeOut: Number.NaN })).toBe(1);
|
||||
});
|
||||
it("returns a finite 0..1 gain for NaN elapsed and for fades longer than the clip", () => {
|
||||
expect(fadeGain(Number.NaN, 10, fades)).toBe(1);
|
||||
const overlapping = fadeGain(0.5, 2, { fadeIn: 10, fadeOut: 10 });
|
||||
expect(overlapping).toBeGreaterThanOrEqual(0);
|
||||
expect(overlapping).toBeLessThanOrEqual(1);
|
||||
expect(Number.isFinite(overlapping)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatFadeSeconds", () => {
|
||||
it("writes short attribute text", () => {
|
||||
expect(formatFadeSeconds(0)).toBe("0");
|
||||
expect(formatFadeSeconds(1)).toBe("1");
|
||||
expect(formatFadeSeconds(0.5)).toBe("0.5");
|
||||
expect(formatFadeSeconds(0.333)).toBe("0.33");
|
||||
expect(formatFadeSeconds(-2)).toBe("0");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,65 @@
|
||||
/** Clip-edge fades: `data-fade-in` / `data-fade-out` in seconds, linear, multiplied on the resolved level. */
|
||||
|
||||
export const HF_AUDIO_FADE_IN_ATTR = "data-fade-in";
|
||||
export const HF_AUDIO_FADE_OUT_ATTR = "data-fade-out";
|
||||
|
||||
export const HF_AUDIO_FADE_IN_DATA_KEY = HF_AUDIO_FADE_IN_ATTR.slice("data-".length);
|
||||
export const HF_AUDIO_FADE_OUT_DATA_KEY = HF_AUDIO_FADE_OUT_ATTR.slice("data-".length);
|
||||
|
||||
export interface AudioFades {
|
||||
/** Seconds from the clip start over which gain rises 0 → 1. */
|
||||
fadeIn: number;
|
||||
/** Seconds before the clip end over which gain falls 1 → 0. */
|
||||
fadeOut: number;
|
||||
}
|
||||
|
||||
export const NO_FADES: Readonly<AudioFades> = Object.freeze({ fadeIn: 0, fadeOut: 0 });
|
||||
|
||||
function finiteFadeSeconds(value: number): number {
|
||||
return Number.isFinite(value) && value > 0 ? value : 0;
|
||||
}
|
||||
|
||||
/** Attribute text as seconds: finite and positive, else 0 (no fade). */
|
||||
export function readFadeSeconds(raw: string | null | undefined): number {
|
||||
if (raw == null || raw === "") return 0;
|
||||
return finiteFadeSeconds(Number.parseFloat(raw));
|
||||
}
|
||||
|
||||
export function readElementFades(el: { getAttribute(name: string): string | null }): AudioFades {
|
||||
return {
|
||||
fadeIn: readFadeSeconds(el.getAttribute(HF_AUDIO_FADE_IN_ATTR)),
|
||||
fadeOut: readFadeSeconds(el.getAttribute(HF_AUDIO_FADE_OUT_ATTR)),
|
||||
};
|
||||
}
|
||||
|
||||
/** Scale fades that outrun the clip so they meet inside it. Infinite duration keeps them as authored. */
|
||||
export function clampFadesToDuration(fades: AudioFades, duration: number): AudioFades {
|
||||
const fadeIn = finiteFadeSeconds(fades.fadeIn);
|
||||
const fadeOut = finiteFadeSeconds(fades.fadeOut);
|
||||
if (!Number.isFinite(duration) || duration <= 0) return { fadeIn, fadeOut };
|
||||
const total = fadeIn + fadeOut;
|
||||
if (total <= duration) return { fadeIn, fadeOut };
|
||||
const scale = duration / total;
|
||||
return { fadeIn: fadeIn * scale, fadeOut: fadeOut * scale };
|
||||
}
|
||||
|
||||
/** Gain at `elapsed` into a clip of `duration`. 1 where no fade runs; the two multiply where they meet. */
|
||||
export function fadeGain(elapsed: number, duration: number, fades: AudioFades): number {
|
||||
if (!Number.isFinite(elapsed)) return 1;
|
||||
const { fadeIn, fadeOut } = clampFadesToDuration(fades, duration);
|
||||
let gain = 1;
|
||||
if (fadeIn > 0) gain *= clamp01(elapsed / fadeIn);
|
||||
if (fadeOut > 0 && Number.isFinite(duration)) gain *= clamp01((duration - elapsed) / fadeOut);
|
||||
return gain;
|
||||
}
|
||||
|
||||
/** Attribute text Studio writes: up to 2 decimals, no trailing zeros. */
|
||||
export function formatFadeSeconds(seconds: number): string {
|
||||
const rounded = Math.round(finiteFadeSeconds(seconds) * 100) / 100;
|
||||
return Number.isInteger(rounded) ? `${rounded}` : rounded.toFixed(2).replace(/0+$/, "");
|
||||
}
|
||||
|
||||
function clamp01(value: number): number {
|
||||
if (!Number.isFinite(value) || value < 0) return 0;
|
||||
return value > 1 ? 1 : value;
|
||||
}
|
||||
@@ -4,12 +4,22 @@ import {
|
||||
formatAudioGain,
|
||||
AUDIO_GAIN_FADER_MIN,
|
||||
MAX_AUDIO_GAIN,
|
||||
audioDbToGain,
|
||||
audioGainToDb,
|
||||
audioGainToFaderPosition,
|
||||
audioGainToText,
|
||||
audioFaderPositionToGain,
|
||||
} from "./audioGain";
|
||||
|
||||
describe("audio gain fader", () => {
|
||||
it("converts decibels to linear gain", () => {
|
||||
expect(audioDbToGain(0)).toBe(1);
|
||||
expect(audioDbToGain(-6)).toBeCloseTo(10 ** (-6 / 20), 10);
|
||||
expect(MAX_AUDIO_GAIN).toBe(audioDbToGain(12));
|
||||
expect(audioGainToDb(1)).toBeCloseTo(0, 10);
|
||||
expect(audioGainToDb(audioDbToGain(-6))).toBeCloseTo(-6, 10);
|
||||
});
|
||||
|
||||
it("puts unity gain at the physical midpoint", () => {
|
||||
expect(audioGainToFaderPosition(1)).toBe(0);
|
||||
expect(audioFaderPositionToGain(0)).toBe(1);
|
||||
|
||||
@@ -6,7 +6,15 @@
|
||||
* shared ceiling here so Studio, preview, and render cannot drift.
|
||||
*/
|
||||
export const MAX_AUDIO_GAIN_DB = 12;
|
||||
export const MAX_AUDIO_GAIN = 10 ** (MAX_AUDIO_GAIN_DB / 20);
|
||||
/** Linear gain from a decibel value. Owner of the dB → gain formula. */
|
||||
export function audioDbToGain(db: number): number {
|
||||
return 10 ** (db / 20);
|
||||
}
|
||||
/** Inverse of `audioDbToGain`. Caller must pass a positive finite gain. */
|
||||
export function audioGainToDb(gain: number): number {
|
||||
return 20 * Math.log10(gain);
|
||||
}
|
||||
export const MAX_AUDIO_GAIN = audioDbToGain(MAX_AUDIO_GAIN_DB);
|
||||
|
||||
/** Studio fader coordinates. Unity is deliberately the physical midpoint. */
|
||||
export const AUDIO_GAIN_FADER_MIN = -100;
|
||||
@@ -87,13 +95,13 @@ export function audioFaderPositionToGain(position: number): number {
|
||||
safe < 0
|
||||
? (safe / Math.abs(AUDIO_GAIN_FADER_MIN)) * Math.abs(MIN_AUDIO_GAIN_DB)
|
||||
: (safe / AUDIO_GAIN_FADER_MAX) * MAX_AUDIO_GAIN_DB;
|
||||
return 10 ** (db / 20);
|
||||
return audioDbToGain(db);
|
||||
}
|
||||
|
||||
export function audioGainToFaderPosition(gain: number): number {
|
||||
const safe = clampAudioGain(gain);
|
||||
if (safe === 0) return AUDIO_GAIN_FADER_MIN;
|
||||
const db = 20 * Math.log10(safe);
|
||||
const db = audioGainToDb(safe);
|
||||
const position =
|
||||
db < 0
|
||||
? (db / Math.abs(MIN_AUDIO_GAIN_DB)) * Math.abs(AUDIO_GAIN_FADER_MIN)
|
||||
@@ -104,7 +112,7 @@ export function audioGainToFaderPosition(gain: number): number {
|
||||
export function audioGainToText(gain: number): string {
|
||||
const safe = clampAudioGain(gain);
|
||||
if (safe === 0) return "-∞ dB";
|
||||
const db = 20 * Math.log10(safe);
|
||||
const db = audioGainToDb(safe);
|
||||
const rounded = Math.abs(db) < 0.05 ? 0 : db;
|
||||
return (rounded > 0 ? "+" : "") + rounded.toFixed(1) + " dB";
|
||||
}
|
||||
|
||||
@@ -465,6 +465,56 @@ describe("syncRuntimeMedia", () => {
|
||||
expect(only).toBeCloseTo(0.55, 5);
|
||||
});
|
||||
|
||||
it("applies data-fade-in / data-fade-out on top of data-volume, anchored to the clip edges", () => {
|
||||
const at = (t: number) => {
|
||||
const clip = createMockClip({ start: 2, end: 12, duration: 10, volume: 0.8 });
|
||||
Object.defineProperty(clip.el, "readyState", { value: 4, writable: true });
|
||||
clip.el.setAttribute("data-fade-in", "2");
|
||||
clip.el.setAttribute("data-fade-out", "1");
|
||||
clip.fades = { fadeIn: 2, fadeOut: 1 };
|
||||
let authored = -1;
|
||||
syncRuntimeMedia({
|
||||
clips: [clip],
|
||||
timeSeconds: t,
|
||||
playing: true,
|
||||
playbackRate: 1,
|
||||
onElementVolume: (_el, _effective, authorVolume) => {
|
||||
authored = authorVolume;
|
||||
},
|
||||
});
|
||||
return authored;
|
||||
};
|
||||
expect(at(2.5)).toBeCloseTo(0.2, 5); // a quarter into the 2 s fade-in
|
||||
expect(at(3)).toBeCloseTo(0.4, 5); // halfway through the fade-in
|
||||
expect(at(6)).toBeCloseTo(0.8, 5); // body of the clip: data-volume alone
|
||||
expect(at(11.5)).toBeCloseTo(0.4, 5); // halfway through the 1 s fade-out
|
||||
expect(at(11.9)).toBeCloseTo(0.08, 5); // almost at the clip's end
|
||||
});
|
||||
|
||||
it("never writes NaN or a negative volume for 0, negative, NaN, or longer-than-clip fades", () => {
|
||||
const cases: Array<{ fadeIn: number; fadeOut: number }> = [
|
||||
{ fadeIn: 0, fadeOut: 0 },
|
||||
{ fadeIn: -2, fadeOut: -1 },
|
||||
{ fadeIn: Number.NaN, fadeOut: Number.NaN },
|
||||
{ fadeIn: 40, fadeOut: 40 },
|
||||
];
|
||||
for (const fades of cases) {
|
||||
const clip = createMockClip({ start: 0, end: 10, duration: 10, volume: 0.5, fades });
|
||||
Object.defineProperty(clip.el, "readyState", { value: 4, writable: true });
|
||||
for (const t of [0, 5, 10, 12]) {
|
||||
syncRuntimeMedia({
|
||||
clips: [clip],
|
||||
timeSeconds: t,
|
||||
playing: true,
|
||||
playbackRate: 1,
|
||||
});
|
||||
expect(Number.isFinite(clip.el.volume)).toBe(true);
|
||||
expect(clip.el.volume).toBeGreaterThanOrEqual(0);
|
||||
expect(clip.el.volume).toBeLessThanOrEqual(1);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("sends boosted author gain to Web Audio while keeping the native element legal", () => {
|
||||
const clip = createMockClip({ start: 0, end: 10, volume: 3.98 });
|
||||
Object.defineProperty(clip.el, "readyState", { value: 4, writable: true });
|
||||
|
||||
@@ -2,6 +2,7 @@ import { swallow } from "./diagnostics";
|
||||
import { isClipVisibleAt, isInClipWindow } from "./clipWindow";
|
||||
import { interpolateVolumeGain, type VolumeKeyframe } from "./mediaVolumeEnvelope.js";
|
||||
import { elementVolumeLaneGain } from "./audioAutomationVolume.js";
|
||||
import { fadeGain, NO_FADES, readElementFades, type AudioFades } from "../audioFade.js";
|
||||
import { readElementPlaybackRate, readElementRateSpec, readMediaStart } from "./playbackRate.js";
|
||||
import { rateAt, sourceTimeAt, timeAtSourceTime, type RateSpec } from "../speedRamp.js";
|
||||
import { clampAudioGain } from "../audioGain.js";
|
||||
@@ -61,6 +62,8 @@ export type RuntimeMediaClip = {
|
||||
* race between the 60 Hz transport tick and GSAP's own seek.
|
||||
*/
|
||||
volumeKeyframes?: VolumeKeyframe[];
|
||||
/** Clip-edge fades from `data-fade-in` / `data-fade-out`; see audioFade.ts. */
|
||||
fades?: AudioFades;
|
||||
};
|
||||
|
||||
export function refreshRuntimeMediaCache(params?: {
|
||||
@@ -126,6 +129,7 @@ export function refreshRuntimeMediaCache(params?: {
|
||||
rate,
|
||||
loop,
|
||||
sourceDuration,
|
||||
fades: readElementFades(el),
|
||||
};
|
||||
mediaClips.push(clip);
|
||||
if (el.tagName === "VIDEO") videoClips.push(clip);
|
||||
@@ -362,6 +366,12 @@ export function syncRuntimeMedia(params: {
|
||||
authorVolume = fallbackAuthorVolume;
|
||||
}
|
||||
|
||||
// Clip-local fade on top of the resolved level, matching render's afade-after-volume.
|
||||
const fades = clip.fades ?? NO_FADES;
|
||||
if (fades.fadeIn > 0 || fades.fadeOut > 0) {
|
||||
authorVolume *= fadeGain(params.timeSeconds - clip.start, clip.duration, fades);
|
||||
}
|
||||
|
||||
// A data-hidden ancestor is silent in the export (audioMixer.ts drops
|
||||
// it), so preview matches. Folded into the per-tick volume, not
|
||||
// el.muted (RULES trap: el.muted is the transport's ownership flag).
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { buildFadeFilters, buildTrackInputFilter, parseAudioElements } from "./audioMixer.js";
|
||||
|
||||
describe("parseAudioElements — clip-edge fades", () => {
|
||||
it("reads data-fade-in / data-fade-out in seconds and omits them when absent", () => {
|
||||
const [faded, plain] = parseAudioElements(
|
||||
`<div data-composition-id="root" data-start="0" data-duration="10">
|
||||
<audio id="bed" src="bed.wav" data-start="0" data-duration="10" data-fade-in="0.5" data-fade-out="2"></audio>
|
||||
<audio id="hit" src="hit.wav" data-start="1" data-duration="1"></audio>
|
||||
</div>`,
|
||||
);
|
||||
expect(faded).toMatchObject({ id: "bed", fadeIn: 0.5, fadeOut: 2 });
|
||||
expect(plain).not.toHaveProperty("fadeIn");
|
||||
expect(plain).not.toHaveProperty("fadeOut");
|
||||
});
|
||||
|
||||
it("treats a negative or unreadable fade as none", () => {
|
||||
const [el] = parseAudioElements(
|
||||
`<div data-composition-id="root" data-start="0" data-duration="10">
|
||||
<audio id="bed" src="bed.wav" data-start="0" data-duration="10" data-fade-in="-1" data-fade-out="soon"></audio>
|
||||
</div>`,
|
||||
);
|
||||
expect(el).not.toHaveProperty("fadeIn");
|
||||
expect(el).not.toHaveProperty("fadeOut");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildFadeFilters", () => {
|
||||
it("is empty for a track without fades", () => {
|
||||
expect(buildFadeFilters({ start: 0, end: 10 })).toBe("");
|
||||
});
|
||||
|
||||
it("emits afade in and out in the track's own stream time, ready to append after volume", () => {
|
||||
expect(buildFadeFilters({ start: 3, end: 13, fadeIn: 0.5, fadeOut: 2 })).toBe(
|
||||
",afade=t=in:st=0:d=0.5,afade=t=out:st=8:d=2",
|
||||
);
|
||||
expect(buildFadeFilters({ start: 0, end: 10, fadeOut: 1 })).toBe(",afade=t=out:st=9:d=1");
|
||||
expect(buildFadeFilters({ start: 0, end: 10, fadeIn: 1 })).toBe(",afade=t=in:st=0:d=1");
|
||||
});
|
||||
|
||||
it("scales fades that outrun the clip so they meet inside it, matching the preview", () => {
|
||||
// 4 s clip asked for 4 s in + 4 s out: each becomes 2 s and the fade-out starts at 2.
|
||||
expect(buildFadeFilters({ start: 0, end: 4, fadeIn: 4, fadeOut: 4 })).toBe(
|
||||
",afade=t=in:st=0:d=2,afade=t=out:st=2:d=2",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps fade-out anchored to the clip end after a trim", () => {
|
||||
expect(buildFadeFilters({ start: 2, end: 12, fadeOut: 3 })).toBe(",afade=t=out:st=7:d=3");
|
||||
expect(buildFadeFilters({ start: 2, end: 8, fadeOut: 3 })).toBe(",afade=t=out:st=3:d=3");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildTrackInputFilter", () => {
|
||||
it("appends afade after the volume filter on the chain master and group mixes share", () => {
|
||||
const filter = buildTrackInputFilter(
|
||||
{ start: 0, end: 10, fadeIn: 1, fadeOut: 2 },
|
||||
0,
|
||||
"volume=0.5",
|
||||
20,
|
||||
);
|
||||
expect(filter.indexOf("volume=0.5")).toBeGreaterThanOrEqual(0);
|
||||
expect(filter.indexOf("volume=0.5")).toBeLessThan(filter.indexOf("afade="));
|
||||
expect(filter).toContain("volume=0.5,afade=t=in:st=0:d=1,afade=t=out:st=8:d=2");
|
||||
});
|
||||
});
|
||||
@@ -11,6 +11,7 @@ import { parseHTML } from "linkedom";
|
||||
import { extractAudioMetadata } from "../utils/ffprobe.js";
|
||||
import { isNotMediaPayload } from "../utils/notMediaPayload.js";
|
||||
import { clampAudioGain } from "@hyperframes/core/audio-gain";
|
||||
import { clampFadesToDuration, readElementFades } from "@hyperframes/core/audio-fade";
|
||||
import {
|
||||
downloadToTemp,
|
||||
isHttpUrl,
|
||||
@@ -349,6 +350,37 @@ function buildVolumeExpression(track: AudioTrack, ignoreKeyframes = false): stri
|
||||
return `volume=${escapeExpressionCommas(expression)}:eval=frame`;
|
||||
}
|
||||
|
||||
/** afade stages after the volume filter (leading comma), or "" when none. Stream time 0 is the clip start. */
|
||||
export function buildFadeFilters(
|
||||
track: Pick<AudioTrack, "start" | "end" | "fadeIn" | "fadeOut">,
|
||||
): string {
|
||||
const duration = track.end - track.start;
|
||||
const { fadeIn, fadeOut } = clampFadesToDuration(
|
||||
{ fadeIn: track.fadeIn ?? 0, fadeOut: track.fadeOut ?? 0 },
|
||||
duration,
|
||||
);
|
||||
const stages: string[] = [];
|
||||
if (fadeIn > 0) stages.push(`afade=t=in:st=0:d=${formatFilterNumber(fadeIn)}`);
|
||||
if (fadeOut > 0 && duration > 0) {
|
||||
stages.push(
|
||||
`afade=t=out:st=${formatFilterNumber(Math.max(0, duration - fadeOut))}:d=${formatFilterNumber(fadeOut)}`,
|
||||
);
|
||||
}
|
||||
return stages.length ? `,${stages.join(",")}` : "";
|
||||
}
|
||||
|
||||
/** One track: trim, volume, afade, delay/pad. asetpts after apad so FFmpeg 5-8 delayed branches do not land at t=0. */
|
||||
export function buildTrackInputFilter(
|
||||
track: Pick<AudioTrack, "start" | "end" | "fadeIn" | "fadeOut" | "tailSeconds">,
|
||||
index: number,
|
||||
volumeFilter: string,
|
||||
totalDuration: number,
|
||||
): string {
|
||||
const delayMs = Math.round(track.start * 1000);
|
||||
const trimDuration = track.end - track.start + (track.tailSeconds ?? 0);
|
||||
return `[${index}:a]atrim=0:${formatFilterNumber(trimDuration)},${volumeFilter}${buildFadeFilters(track)},adelay=${delayMs}|${delayMs},apad,asetpts=N/SR/TB,atrim=0:${formatFilterNumber(totalDuration)}[a${index}]`;
|
||||
}
|
||||
|
||||
interface ExtractResult {
|
||||
success: boolean;
|
||||
outputPath: string;
|
||||
@@ -585,6 +617,7 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
): AudioElement => {
|
||||
const layerAttr = el.getAttribute("data-layer");
|
||||
const volumeAttr = el.getAttribute("data-volume");
|
||||
const fades = readElementFades(el);
|
||||
const fxChain = el.getAttribute(HF_AUDIO_FX_ATTR);
|
||||
const automation = el.getAttribute(HF_AUDIO_AUTOMATION_ATTR);
|
||||
// Audio only in v1 (matches resolveAudioGroups, which only scans
|
||||
@@ -600,6 +633,8 @@ export function parseAudioElements(html: string): AudioElement[] {
|
||||
playbackRate: readElementRateSpec(el),
|
||||
layer: layerAttr ? parseInt(layerAttr) : 0,
|
||||
volume: volumeAttr ? parseFloat(volumeAttr) : 1.0,
|
||||
...(fades.fadeIn > 0 ? { fadeIn: fades.fadeIn } : {}),
|
||||
...(fades.fadeOut > 0 ? { fadeOut: fades.fadeOut } : {}),
|
||||
...(fxChain ? { fxChain } : {}),
|
||||
...(automation ? { automation } : {}),
|
||||
...(group
|
||||
@@ -835,27 +870,8 @@ async function mixAudioTracks(
|
||||
const buildFilterComplex = (ignoreAutomation: boolean): string => {
|
||||
const filterParts: string[] = [];
|
||||
tracks.forEach((track, i) => {
|
||||
const delayMs = Math.round(track.start * 1000);
|
||||
// A clip's own audio ends at `end`, but an FX tail is still decaying past
|
||||
// it. Trimming at the boundary is what cut every reverb short; the final
|
||||
// atrim below still holds the mix to the composition's length, so a tail
|
||||
// can run over what follows but never past the end of the video.
|
||||
const trimDuration = track.end - track.start + (track.tailSeconds ?? 0);
|
||||
const volumeFilter = buildVolumeExpression(track, ignoreAutomation);
|
||||
// `apad` then `atrim` is the portable pad-to-length shape: PR #2769 moved
|
||||
// off `apad=whole_dur=` because some FFmpeg builds reject that option
|
||||
// outright ("Error applying option 'whole_dur': Option not found").
|
||||
// But on FFmpeg 5.x through 8.0.x the samples `apad` appends carry
|
||||
// timestamps the following `atrim` misreads, so a delayed branch lands at
|
||||
// t=0 and, once four or more branches are mixed, the last one disappears
|
||||
// entirely. `asetpts=N/SR/TB` renumbers the padded stream from the sample
|
||||
// count before the trim reads it, which fixes the misplacement while
|
||||
// keeping the filter set every build supports. Verified correct on 4.2.7,
|
||||
// 7.0.2, an 8.x nightly and 8.1.1; the un-reset form is wrong on the
|
||||
// middle two.
|
||||
filterParts.push(
|
||||
`[${i}:a]atrim=0:${formatFilterNumber(trimDuration)},${volumeFilter},adelay=${delayMs}|${delayMs},apad,asetpts=N/SR/TB,atrim=0:${formatFilterNumber(totalDuration)}[a${i}]`,
|
||||
);
|
||||
filterParts.push(buildTrackInputFilter(track, i, volumeFilter, totalDuration));
|
||||
});
|
||||
|
||||
const mixInputs = tracks.map((_, i) => `[a${i}]`).join("");
|
||||
@@ -1023,14 +1039,8 @@ async function mixGroupMembers(
|
||||
|
||||
const buildInputFilters = (ignoreKeyframes: boolean) =>
|
||||
memberTracks.map((track, i) => {
|
||||
const delayMs = Math.round(track.start * 1000);
|
||||
const trimDuration = track.end - track.start + (track.tailSeconds ?? 0);
|
||||
const volumeFilter = buildVolumeExpression(track, ignoreKeyframes);
|
||||
// Same `asetpts=N/SR/TB` as the master mix above, for the same reason and
|
||||
// on the same builds: these are delayed branches padded to length and then
|
||||
// amix'd, so without the renumbering a delayed member lands at t=0 and a
|
||||
// group of four or more loses its last one.
|
||||
return `[${i}:a]atrim=0:${formatFilterNumber(trimDuration)},${volumeFilter},adelay=${delayMs}|${delayMs},apad,asetpts=N/SR/TB,atrim=0:${formatFilterNumber(totalDuration)}[a${i}]`;
|
||||
return buildTrackInputFilter(track, i, volumeFilter, totalDuration);
|
||||
});
|
||||
const mixInputs = memberTracks.map((_, i) => `[a${i}]`).join("");
|
||||
|
||||
@@ -1391,6 +1401,8 @@ export async function processCompositionAudio(
|
||||
// Gain is already in the samples when baked, so mix at unity.
|
||||
volume: bakedEnvelope ? 1.0 : (element.volume ?? 1.0),
|
||||
volumeKeyframes: bakedEnvelope ? undefined : (envelopeKeyframes ?? undefined),
|
||||
...(element.fadeIn ? { fadeIn: element.fadeIn } : {}),
|
||||
...(element.fadeOut ? { fadeOut: element.fadeOut } : {}),
|
||||
...(tailSeconds > 0 ? { tailSeconds } : {}),
|
||||
};
|
||||
|
||||
|
||||
@@ -16,6 +16,9 @@ export interface AudioElement {
|
||||
layer: number;
|
||||
volume?: number;
|
||||
volumeKeyframes?: AudioVolumeKeyframe[];
|
||||
/** Clip-edge fades from `data-fade-in` / `data-fade-out`, seconds; absent means none. */
|
||||
fadeIn?: number;
|
||||
fadeOut?: number;
|
||||
/** Serialised FX chain JSON from `data-fx-chain`, when set. */
|
||||
fxChain?: string;
|
||||
/** Serialised automation JSON from `data-automation`, when set. */
|
||||
@@ -44,6 +47,9 @@ export interface AudioTrack {
|
||||
duration: number;
|
||||
volume: number;
|
||||
volumeKeyframes?: AudioVolumeKeyframe[];
|
||||
/** Clip-edge fades in seconds, applied by `afade` after the volume filter. Fade-out ends at `end`. */
|
||||
fadeIn?: number;
|
||||
fadeOut?: number;
|
||||
/**
|
||||
* Seconds of FX tail past `end` that the mix should let through — a reverb or
|
||||
* delay still decaying when the clip's own audio stops. Absent means cut at
|
||||
|
||||
@@ -82,6 +82,35 @@ describe("probeMediaMetadata", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("reports whether a video carries an audio stream", async () => {
|
||||
const withAudio = await probeMediaMetadata("/tmp/clip.mp4", () => ({
|
||||
status: 0,
|
||||
stdout: JSON.stringify({
|
||||
streams: [
|
||||
{ codec_type: "video", codec_name: "h264" },
|
||||
{ codec_type: "audio", codec_name: "aac" },
|
||||
],
|
||||
}),
|
||||
stderr: "",
|
||||
}));
|
||||
expect(withAudio).toMatchObject({ kind: "video", hasAudio: true });
|
||||
|
||||
const silent = await probeMediaMetadata("/tmp/silent.mp4", () => ({
|
||||
status: 0,
|
||||
stdout: JSON.stringify({ streams: [{ codec_type: "video", codec_name: "h264" }] }),
|
||||
stderr: "",
|
||||
}));
|
||||
expect(silent).toMatchObject({ kind: "video", hasAudio: false });
|
||||
|
||||
// An audio file is not a video; the flag is a video question only.
|
||||
const audioFile = await probeMediaMetadata("/tmp/music.wav", () => ({
|
||||
status: 0,
|
||||
stdout: "{}",
|
||||
stderr: "",
|
||||
}));
|
||||
expect(audioFile).not.toHaveProperty("hasAudio");
|
||||
});
|
||||
|
||||
it("ignores attached cover art and reads the real video stream", async () => {
|
||||
const metadata = await probeMediaMetadata("/tmp/clip.mp4", () => ({
|
||||
status: 0,
|
||||
|
||||
@@ -66,6 +66,8 @@ export interface MediaColorMetadata {
|
||||
export interface MediaMetadata {
|
||||
kind: "video" | "image" | "audio" | "unknown";
|
||||
color: MediaColorMetadata;
|
||||
/** Video audio stream, when probed. Absent means the drop path stays muted. */
|
||||
hasAudio?: boolean;
|
||||
probeError?: string;
|
||||
}
|
||||
|
||||
@@ -221,7 +223,11 @@ export async function probeMediaMetadata(
|
||||
if (kind === "image") return item.codec_type === "video";
|
||||
return item.codec_type === kind && item.disposition?.attached_pic !== 1;
|
||||
});
|
||||
return { kind, color: classifyMediaColor(stream) };
|
||||
const metadata: MediaMetadata = { kind, color: classifyMediaColor(stream) };
|
||||
if (kind === "video") {
|
||||
metadata.hasAudio = (parsed.streams ?? []).some((item) => item.codec_type === "audio");
|
||||
}
|
||||
return metadata;
|
||||
} catch {
|
||||
return { kind, color: classifyMediaColor(null), probeError: "ffprobe returned invalid json" };
|
||||
}
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { MAX_AUDIO_GAIN } from "@hyperframes/core/audio-gain";
|
||||
import { RATE_RANGE } from "@hyperframes/core/audio-automation";
|
||||
import { parseGainInput, parseRateInput, parseSecondsInput } from "./audioInspectorInput";
|
||||
|
||||
describe("parseGainInput", () => {
|
||||
it("reads decibels as the readout shows them", () => {
|
||||
expect(parseGainInput("0")).toBeCloseTo(1, 6);
|
||||
expect(parseGainInput("+0.5 dB")).toBeCloseTo(10 ** (0.5 / 20), 6);
|
||||
expect(parseGainInput("-6")).toBeCloseTo(10 ** (-6 / 20), 6);
|
||||
expect(parseGainInput("-6dB")).toBeCloseTo(10 ** (-6 / 20), 6);
|
||||
expect(parseGainInput("-inf")).toBe(0);
|
||||
});
|
||||
it("also takes a linear multiplier or a percentage", () => {
|
||||
expect(parseGainInput("x0.8")).toBeCloseTo(0.8, 6);
|
||||
expect(parseGainInput("0.8x")).toBeCloseTo(0.8, 6);
|
||||
expect(parseGainInput("80%")).toBeCloseTo(0.8, 6);
|
||||
});
|
||||
it("clamps to the fader's ceiling and refuses nonsense", () => {
|
||||
expect(parseGainInput("+40 dB")).toBeCloseTo(MAX_AUDIO_GAIN, 6);
|
||||
expect(parseGainInput("loud")).toBeNull();
|
||||
expect(parseGainInput("")).toBeNull();
|
||||
expect(parseGainInput("x-1")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseRateInput", () => {
|
||||
it("reads a multiplier with or without the x, and a percentage", () => {
|
||||
expect(parseRateInput("1.5x")).toBe(1.5);
|
||||
expect(parseRateInput("1.5")).toBe(1.5);
|
||||
expect(parseRateInput("x2")).toBe(2);
|
||||
expect(parseRateInput("150%")).toBe(1.5);
|
||||
});
|
||||
it("clamps to the rate range and refuses zero or text", () => {
|
||||
expect(parseRateInput("100x")).toBe(RATE_RANGE.max);
|
||||
expect(parseRateInput("0.001")).toBe(RATE_RANGE.min);
|
||||
expect(parseRateInput("0")).toBeNull();
|
||||
expect(parseRateInput("fast")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseSecondsInput", () => {
|
||||
it("reads seconds, milliseconds and minutes:seconds", () => {
|
||||
expect(parseSecondsInput("2.5")).toBe(2.5);
|
||||
expect(parseSecondsInput("2.5s")).toBe(2.5);
|
||||
expect(parseSecondsInput("500ms")).toBe(0.5);
|
||||
expect(parseSecondsInput("1:02.5")).toBe(62.5);
|
||||
expect(parseSecondsInput("0")).toBe(0);
|
||||
});
|
||||
it("refuses negatives and text", () => {
|
||||
expect(parseSecondsInput("-1")).toBeNull();
|
||||
expect(parseSecondsInput("soon")).toBeNull();
|
||||
expect(parseSecondsInput("")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
import { audioDbToGain, clampAudioGain } from "@hyperframes/core/audio-gain";
|
||||
import { RATE_RANGE } from "@hyperframes/core/audio-automation";
|
||||
|
||||
const NUMBER = String.raw`[-+]?(?:\d+\.?\d*|\.\d+)`;
|
||||
|
||||
/** Gain as the readout shows it (dB, "x0.8", "80%"). Returns linear gain, or null to refuse. */
|
||||
export function parseGainInput(text: string): number | null {
|
||||
const t = text.trim().toLowerCase().replace(/\s+/g, "");
|
||||
if (!t) return null;
|
||||
if (t === "-inf" || t === "-infinity" || t === "-∞" || t === "silence" || t === "mute") return 0;
|
||||
let m = new RegExp(`^(${NUMBER})%$`).exec(t);
|
||||
if (m) return finiteGain(Number(m[1]) / 100);
|
||||
m = new RegExp(`^(?:x(${NUMBER})|(${NUMBER})x)$`).exec(t);
|
||||
if (m) return finiteGain(Number(m[1] ?? m[2]));
|
||||
m = new RegExp(`^(${NUMBER})(?:db)?$`).exec(t);
|
||||
if (m) return finiteGain(audioDbToGain(Number(m[1])));
|
||||
return null;
|
||||
}
|
||||
|
||||
function finiteGain(gain: number): number | null {
|
||||
return Number.isFinite(gain) && gain >= 0 ? clampAudioGain(gain) : null;
|
||||
}
|
||||
|
||||
/** Playback rate: "1.5x", "1.5", "150%". Clamped to the rate lane's range. */
|
||||
export function parseRateInput(text: string): number | null {
|
||||
const t = text.trim().toLowerCase().replace(/\s+/g, "");
|
||||
if (!t) return null;
|
||||
let value: number;
|
||||
let m = new RegExp(`^(${NUMBER})%$`).exec(t);
|
||||
if (m) value = Number(m[1]) / 100;
|
||||
else {
|
||||
m = new RegExp(`^(?:x(${NUMBER})|(${NUMBER})x?)$`).exec(t);
|
||||
if (!m) return null;
|
||||
value = Number(m[1] ?? m[2]);
|
||||
}
|
||||
if (!Number.isFinite(value) || value <= 0) return null;
|
||||
return Math.min(RATE_RANGE.max, Math.max(RATE_RANGE.min, value));
|
||||
}
|
||||
|
||||
/** Non-negative duration or offset: "2.5", "2.5s", "500ms", "1:02.5". Returns seconds, or null to refuse. */
|
||||
export function parseSecondsInput(text: string): number | null {
|
||||
const t = text.trim().toLowerCase().replace(/\s+/g, "");
|
||||
if (!t) return null;
|
||||
let m = new RegExp(`^(\\d+):(\\d{1,2}(?:\\.\\d*)?)$`).exec(t);
|
||||
if (m) return finiteSeconds(Number(m[1]) * 60 + Number(m[2]));
|
||||
m = new RegExp(`^(${NUMBER})ms$`).exec(t);
|
||||
if (m) return finiteSeconds(Number(m[1]) / 1000);
|
||||
m = new RegExp(`^(${NUMBER})(?:s|sec|secs)?$`).exec(t);
|
||||
if (m) return finiteSeconds(Number(m[1]));
|
||||
return null;
|
||||
}
|
||||
|
||||
function finiteSeconds(seconds: number): number | null {
|
||||
return Number.isFinite(seconds) && seconds >= 0 ? seconds : null;
|
||||
}
|
||||
@@ -474,3 +474,175 @@ describe("FlatMediaSection — fit/position", () => {
|
||||
act(() => root.unmount());
|
||||
});
|
||||
});
|
||||
|
||||
function makeAudioElement(dataAttributes: Record<string, string> = {}): DomEditSelection {
|
||||
const el = document.createElement("audio");
|
||||
el.setAttribute("src", "assets/music.wav");
|
||||
return makeVideoElement({
|
||||
element: el,
|
||||
id: "music",
|
||||
selector: "#music",
|
||||
label: "Music",
|
||||
tagName: "audio",
|
||||
dataAttributes: { duration: "10", ...dataAttributes },
|
||||
});
|
||||
}
|
||||
|
||||
function renderWithRate(element: DomEditSelection, onSetAttribute = vi.fn()) {
|
||||
const host = document.createElement("div");
|
||||
document.body.append(host);
|
||||
const root = createRoot(host);
|
||||
act(() => {
|
||||
root.render(
|
||||
<FlatMediaSection
|
||||
projectDir={null}
|
||||
element={element}
|
||||
styles={{}}
|
||||
onSetStyle={vi.fn()}
|
||||
onSetAttribute={onSetAttribute}
|
||||
onSetHtmlAttribute={vi.fn()}
|
||||
rate={{
|
||||
automated: false,
|
||||
automatedValue: undefined,
|
||||
onAutomate: vi.fn(),
|
||||
onRemoveAutomation: vi.fn(),
|
||||
onCommitAt: vi.fn(),
|
||||
canApplyPreset: true,
|
||||
onApplyPreset: vi.fn(),
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
return { host, root, onSetAttribute };
|
||||
}
|
||||
|
||||
function labelsOf(host: HTMLElement): string[] {
|
||||
return [...host.querySelectorAll('[data-flat-slider-track="true"]')].map(
|
||||
(track) => track.getAttribute("aria-label") ?? "",
|
||||
);
|
||||
}
|
||||
|
||||
describe("FlatMediaSection — audio clips", () => {
|
||||
it("offers speed presets on video but not on audio", () => {
|
||||
const video = renderWithRate(makeVideoElement({ dataAttributes: { duration: "10" } }));
|
||||
expect(video.host.textContent).toContain("Speed preset");
|
||||
act(() => video.root.unmount());
|
||||
|
||||
const audio = renderWithRate(makeAudioElement());
|
||||
expect(audio.host.textContent).not.toContain("Speed preset");
|
||||
act(() => audio.root.unmount());
|
||||
});
|
||||
|
||||
it("shows Fade in / Fade out rows for audio, and for video only when it carries audio", () => {
|
||||
const audio = renderWithRate(makeAudioElement());
|
||||
expect(labelsOf(audio.host)).toEqual(
|
||||
expect.arrayContaining(["Volume", "Speed", "Media start", "Fade in", "Fade out"]),
|
||||
);
|
||||
act(() => audio.root.unmount());
|
||||
|
||||
const silentVideo = renderWithRate(makeVideoElement({ dataAttributes: { duration: "10" } }));
|
||||
expect(labelsOf(silentVideo.host)).not.toContain("Fade in");
|
||||
act(() => silentVideo.root.unmount());
|
||||
|
||||
const audibleVideo = renderWithRate(
|
||||
makeVideoElement({ dataAttributes: { duration: "10", "has-audio": "true" } }),
|
||||
);
|
||||
expect(labelsOf(audibleVideo.host)).toContain("Fade out");
|
||||
act(() => audibleVideo.root.unmount());
|
||||
});
|
||||
|
||||
it("reads the authored fades and writes data-fade-in from the slider, clearing it at zero", () => {
|
||||
const { host, root, onSetAttribute } = renderWithRate(
|
||||
makeAudioElement({ "fade-in": "0.5", "fade-out": "2" }),
|
||||
);
|
||||
const readouts = [...host.querySelectorAll('[data-flat-slider-value="true"]')].map(
|
||||
(node) => node.textContent,
|
||||
);
|
||||
expect(readouts).toEqual(expect.arrayContaining(["0.50s", "2.00s"]));
|
||||
|
||||
const fadeInTrack = host.querySelector<HTMLElement>(
|
||||
'[data-flat-slider-track="true"][aria-label="Fade in"]',
|
||||
);
|
||||
if (!fadeInTrack) throw new Error("expected a Fade in slider");
|
||||
Object.defineProperty(fadeInTrack, "getBoundingClientRect", {
|
||||
value: () => ({ left: 0, width: 100, top: 0, height: 2, right: 100, bottom: 2 }),
|
||||
});
|
||||
// The other fade reserves 2 s: a quarter of the remaining 8 s is 2 s.
|
||||
act(() => {
|
||||
fadeInTrack.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true, clientX: 25 }));
|
||||
fadeInTrack.dispatchEvent(new MouseEvent("pointerup", { bubbles: true, clientX: 25 }));
|
||||
});
|
||||
expect(onSetAttribute).toHaveBeenCalledWith("fade-in", "2");
|
||||
act(() => {
|
||||
fadeInTrack.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true, clientX: 0 }));
|
||||
fadeInTrack.dispatchEvent(new MouseEvent("pointerup", { bubbles: true, clientX: 0 }));
|
||||
});
|
||||
// An empty write removes the attribute rather than leaving data-fade-in="0" behind.
|
||||
expect(onSetAttribute).toHaveBeenCalledWith("fade-in", "");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it.each([
|
||||
["Media start", "media-start", "45.00", {}, "9999"],
|
||||
["Fade in", "fade-in", "10", {}, "9999"],
|
||||
["Fade out", "fade-out", "10", {}, "9999"],
|
||||
["Fade in", "fade-in", "5", { "fade-out": "5" }, "8"],
|
||||
["Fade out", "fade-out", "5", { "fade-in": "5" }, "8"],
|
||||
] as const)("bounds typed %s to the slider limit", (label, attribute, expected, fades, typed) => {
|
||||
const { host, root, onSetAttribute } = renderWithRate(
|
||||
makeAudioElement({ "source-duration": "45", ...fades }),
|
||||
);
|
||||
const row = host.querySelector<HTMLElement>(
|
||||
`[data-flat-slider-track="true"][aria-label="${label}"]`,
|
||||
)?.parentElement;
|
||||
const readout = row?.querySelector<HTMLElement>('[data-flat-slider-value="true"]');
|
||||
if (!readout) throw new Error(`expected ${label} readout`);
|
||||
act(() => readout.dispatchEvent(new MouseEvent("click", { bubbles: true })));
|
||||
const input = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
if (!input) throw new Error(`expected ${label} input`);
|
||||
act(() => {
|
||||
typeInto(input, typed);
|
||||
input.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
|
||||
});
|
||||
expect(onSetAttribute.mock.calls).toEqual([[attribute, expected]]);
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("commits a typed volume in dB and a typed fade in seconds", () => {
|
||||
const { host, root, onSetAttribute } = renderWithRate(makeAudioElement());
|
||||
const readouts = host.querySelectorAll<HTMLElement>('[data-flat-slider-value="true"]');
|
||||
// Volume is the first slider row.
|
||||
act(() => readouts[0]?.dispatchEvent(new MouseEvent("click", { bubbles: true })));
|
||||
const volumeInput = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
if (!volumeInput) throw new Error("expected the volume readout to open for typing");
|
||||
act(() => {
|
||||
typeInto(volumeInput, "-6 dB");
|
||||
volumeInput.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
|
||||
});
|
||||
const [attr, value] = onSetAttribute.mock.calls.at(-1) ?? [];
|
||||
expect(attr).toBe("volume");
|
||||
expect(Number(value)).toBeCloseTo(10 ** (-6 / 20), 2);
|
||||
|
||||
const fadeOutRow = host.querySelector<HTMLElement>(
|
||||
'[data-flat-slider-track="true"][aria-label="Fade out"]',
|
||||
)?.parentElement;
|
||||
const fadeOutReadout = fadeOutRow?.querySelector<HTMLElement>(
|
||||
'[data-flat-slider-value="true"]',
|
||||
);
|
||||
act(() => fadeOutReadout?.dispatchEvent(new MouseEvent("click", { bubbles: true })));
|
||||
const fadeInput = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
if (!fadeInput) throw new Error("expected the fade readout to open for typing");
|
||||
act(() => {
|
||||
typeInto(fadeInput, "1.25s");
|
||||
fadeInput.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
|
||||
});
|
||||
expect(onSetAttribute).toHaveBeenLastCalledWith("fade-out", "1.25");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
});
|
||||
|
||||
function typeInto(input: HTMLInputElement, value: string) {
|
||||
const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set;
|
||||
setter?.call(input, value);
|
||||
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
}
|
||||
|
||||
@@ -25,6 +25,14 @@ import {
|
||||
audioGainToFaderPosition,
|
||||
audioGainToText,
|
||||
} from "@hyperframes/core/audio-gain";
|
||||
import {
|
||||
HF_AUDIO_FADE_IN_DATA_KEY,
|
||||
HF_AUDIO_FADE_OUT_DATA_KEY,
|
||||
clampFadesToDuration,
|
||||
formatFadeSeconds,
|
||||
readFadeSeconds,
|
||||
} from "@hyperframes/core/audio-fade";
|
||||
import { parseGainInput, parseRateInput, parseSecondsInput } from "./audioInspectorInput";
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
export function FlatMediaSection({
|
||||
@@ -92,6 +100,10 @@ export function FlatMediaSection({
|
||||
(el as HTMLMediaElement).duration ||
|
||||
0;
|
||||
const mediaStartMax = Math.max(30, Math.ceil(sourceDuration || mediaStart + 10));
|
||||
const fadeIn = readFadeSeconds(element.dataAttributes[HF_AUDIO_FADE_IN_DATA_KEY]);
|
||||
const fadeOut = readFadeSeconds(element.dataAttributes[HF_AUDIO_FADE_OUT_DATA_KEY]);
|
||||
const clipDuration = Number.parseFloat(element.dataAttributes.duration ?? "") || 0;
|
||||
const fadeMax = clipDuration > 0 ? clipDuration : 10;
|
||||
const hasLoop = el.hasAttribute("loop");
|
||||
const hasMuted = el.hasAttribute("muted");
|
||||
const hasAudio = element.dataAttributes["has-audio"] === "true";
|
||||
@@ -256,6 +268,13 @@ export function FlatMediaSection({
|
||||
void onSetAttribute("volume", formatAudioGain(gain));
|
||||
}
|
||||
}}
|
||||
onCommitText={(text) => {
|
||||
const gain = parseGainInput(text);
|
||||
if (gain === null) return false;
|
||||
if (volumeAutomated) onCommitVolumeAt?.(gain);
|
||||
else void onSetAttribute("volume", formatAudioGain(gain));
|
||||
return true;
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
<AutomationToggle
|
||||
@@ -285,6 +304,13 @@ export function FlatMediaSection({
|
||||
void onSetAttribute("playback-rate", formatNumericValue(speed));
|
||||
}
|
||||
}}
|
||||
onCommitText={(text) => {
|
||||
const speed = parseRateInput(text);
|
||||
if (speed === null) return false;
|
||||
if (rate?.automated) rate.onCommitAt(speed);
|
||||
else void onSetAttribute("playback-rate", formatNumericValue(speed));
|
||||
return true;
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
<AutomationToggle
|
||||
@@ -295,7 +321,10 @@ export function FlatMediaSection({
|
||||
onRemoveAutomation={rate ? () => rate.onRemoveAutomation() : undefined}
|
||||
/>
|
||||
</div>
|
||||
{rate?.canApplyPreset && (
|
||||
{/* Speed presets are picture-driven ramps (slow-mo reveals, whip
|
||||
speed-ups); on a bare audio clip they only warp pitch, so the
|
||||
row is video-only. */}
|
||||
{rate?.canApplyPreset && !isAudio && (
|
||||
<FlatSelectRow
|
||||
label="Speed preset"
|
||||
value=""
|
||||
@@ -312,7 +341,21 @@ export function FlatMediaSection({
|
||||
tier={mediaStart === 0 ? "default" : "explicitCustom"}
|
||||
displayValue={formatTimingValue(mediaStart)}
|
||||
onCommit={(next) => void onSetAttribute("media-start", (next / 100).toFixed(2))}
|
||||
onCommitText={(text) => {
|
||||
const seconds = parseSecondsInput(text);
|
||||
if (seconds === null) return false;
|
||||
void onSetAttribute("media-start", Math.min(seconds, mediaStartMax).toFixed(2));
|
||||
return true;
|
||||
}}
|
||||
/>
|
||||
{(isAudio || hasAudio) && (
|
||||
<MediaFadeSliders
|
||||
fadeIn={fadeIn}
|
||||
fadeOut={fadeOut}
|
||||
fadeMax={fadeMax}
|
||||
onSetAttribute={onSetAttribute}
|
||||
/>
|
||||
)}
|
||||
<FlatToggle
|
||||
label="Loop"
|
||||
checked={hasLoop}
|
||||
@@ -371,3 +414,46 @@ export function FlatMediaSection({
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function MediaFadeSliders({
|
||||
fadeIn,
|
||||
fadeOut,
|
||||
fadeMax,
|
||||
onSetAttribute,
|
||||
}: {
|
||||
fadeIn: number;
|
||||
fadeOut: number;
|
||||
fadeMax: number;
|
||||
onSetAttribute: (attr: string, value: string) => void | Promise<void>;
|
||||
}) {
|
||||
const fadeText = (seconds: number) => (seconds > 0 ? formatFadeSeconds(seconds) : "");
|
||||
return (["in", "out"] as const).map((edge) => {
|
||||
const seconds = edge === "in" ? fadeIn : fadeOut;
|
||||
const dataKey = edge === "in" ? HF_AUDIO_FADE_IN_DATA_KEY : HF_AUDIO_FADE_OUT_DATA_KEY;
|
||||
const edgeMax = Math.max(0, fadeMax - (edge === "in" ? fadeOut : fadeIn));
|
||||
return (
|
||||
<FlatSlider
|
||||
key={edge}
|
||||
label={`Fade ${edge}`}
|
||||
value={Math.round(seconds * 100)}
|
||||
min={0}
|
||||
max={Math.round(edgeMax * 100)}
|
||||
tier={seconds === 0 ? "default" : "explicitCustom"}
|
||||
displayValue={formatTimingValue(seconds)}
|
||||
onCommit={(next) => void onSetAttribute(dataKey, fadeText(next / 100))}
|
||||
onCommitText={(text) => {
|
||||
const parsed = parseSecondsInput(text);
|
||||
if (parsed === null) return false;
|
||||
const next = Math.min(parsed, edgeMax);
|
||||
const bounded = clampFadesToDuration(
|
||||
{ fadeIn: edge === "in" ? next : fadeIn, fadeOut: edge === "out" ? next : fadeOut },
|
||||
fadeMax,
|
||||
);
|
||||
void onSetAttribute(dataKey, fadeText(edge === "in" ? bounded.fadeIn : bounded.fadeOut));
|
||||
return true;
|
||||
}}
|
||||
onReset={seconds > 0 ? () => void onSetAttribute(dataKey, "") : undefined}
|
||||
/>
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1285,3 +1285,95 @@ describe("FlatSelectRow — label/value options", () => {
|
||||
act(() => root.unmount());
|
||||
});
|
||||
});
|
||||
|
||||
describe("FlatSlider — typed value", () => {
|
||||
it("keeps the readout inert without onCommitText", () => {
|
||||
const { host, root } = renderInto(
|
||||
<FlatSlider
|
||||
label="Opacity"
|
||||
value={10}
|
||||
min={0}
|
||||
max={100}
|
||||
tier="default"
|
||||
displayValue="10%"
|
||||
onCommit={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
const readout = host.querySelector<HTMLElement>('[data-flat-slider-value="true"]');
|
||||
expect(readout?.getAttribute("role")).toBeNull();
|
||||
act(() => readout?.dispatchEvent(new MouseEvent("click", { bubbles: true })));
|
||||
expect(host.querySelector('[data-flat-slider-input="true"]')).toBeNull();
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("turns the readout into an input on click and commits the typed text on Enter", () => {
|
||||
const onCommitText = vi.fn().mockReturnValue(true);
|
||||
const { host, root } = renderInto(
|
||||
<FlatSlider
|
||||
label="Volume"
|
||||
value={0}
|
||||
min={-100}
|
||||
max={100}
|
||||
tier="default"
|
||||
displayValue="0 dB"
|
||||
onCommit={vi.fn()}
|
||||
onCommitText={onCommitText}
|
||||
/>,
|
||||
);
|
||||
const readout = host.querySelector<HTMLElement>('[data-flat-slider-value="true"]');
|
||||
expect(readout?.getAttribute("role")).toBe("button");
|
||||
act(() => readout?.dispatchEvent(new MouseEvent("click", { bubbles: true })));
|
||||
const input = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
if (!input) throw new Error("expected the readout to become an input");
|
||||
expect(input.value).toBe("0 dB");
|
||||
act(() => {
|
||||
setReactInputValue(input, "-6 dB");
|
||||
input.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
|
||||
});
|
||||
expect(onCommitText).toHaveBeenCalledWith("-6 dB");
|
||||
expect(host.querySelector('[data-flat-slider-input="true"]')).toBeNull();
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("stays open and marks itself invalid when the text is refused, and closes on Escape", () => {
|
||||
const onCommitText = vi.fn().mockReturnValue(false);
|
||||
const { host, root } = renderInto(
|
||||
<FlatSlider
|
||||
label="Speed"
|
||||
value={500}
|
||||
min={0}
|
||||
max={1000}
|
||||
tier="default"
|
||||
displayValue="1x"
|
||||
onCommit={vi.fn()}
|
||||
onCommitText={onCommitText}
|
||||
/>,
|
||||
);
|
||||
act(() =>
|
||||
host
|
||||
.querySelector<HTMLElement>('[data-flat-slider-value="true"]')
|
||||
?.dispatchEvent(new MouseEvent("click", { bubbles: true })),
|
||||
);
|
||||
const input = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
if (!input) throw new Error("expected an input");
|
||||
act(() => {
|
||||
setReactInputValue(input, "fast");
|
||||
input.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
|
||||
});
|
||||
expect(onCommitText).toHaveBeenCalledWith("fast");
|
||||
const still = host.querySelector<HTMLInputElement>('[data-flat-slider-input="true"]');
|
||||
expect(still).not.toBeNull();
|
||||
expect(still?.getAttribute("aria-invalid")).toBe("true");
|
||||
act(() => still?.dispatchEvent(new KeyboardEvent("keydown", { key: "Escape", bubbles: true })));
|
||||
expect(host.querySelector('[data-flat-slider-input="true"]')).toBeNull();
|
||||
expect(host.querySelector('[data-flat-slider-value="true"]')?.textContent).toBe("1x");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
});
|
||||
|
||||
/** Drives React's controlled input the way a keystroke would. */
|
||||
function setReactInputValue(input: HTMLInputElement, value: string) {
|
||||
const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")?.set;
|
||||
setter?.call(input, value);
|
||||
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
}
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { useEffect, useRef, useState, type ReactNode } from "react";
|
||||
import { type ReactNode, useEffect, useRef, useState } from "react";
|
||||
import { useTrackDesignInput } from "../../contexts/DesignPanelInputContext";
|
||||
import { RotateCcw } from "../../icons/SystemIcons";
|
||||
import { CommitField } from "./propertyPanelPrimitives";
|
||||
import { FlatSliderReadout } from "./propertyPanelFlatSliderReadout";
|
||||
import {
|
||||
VALUE_TIER_LABEL_CLASS,
|
||||
VALUE_TIER_VALUE_CLASS,
|
||||
@@ -278,6 +279,7 @@ export function FlatSlider({
|
||||
centerTick,
|
||||
onReset,
|
||||
onCommit,
|
||||
onCommitText,
|
||||
}: {
|
||||
label: string;
|
||||
value: number;
|
||||
@@ -290,6 +292,8 @@ export function FlatSlider({
|
||||
centerTick?: boolean;
|
||||
onReset?: () => void;
|
||||
onCommit: (nextValue: number) => void;
|
||||
/** Typed readout: return false to refuse the text, keep the field open, and mark it invalid. */
|
||||
onCommitText?: (text: string) => boolean | void;
|
||||
}) {
|
||||
const track = useTrackDesignInput();
|
||||
// `draft` gives the knob instant, drag-local visual feedback. `onCommit` is
|
||||
@@ -538,14 +542,14 @@ export function FlatSlider({
|
||||
style={{ left: `${clampedPct}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span
|
||||
data-flat-slider-value="true"
|
||||
className={`w-11 shrink-0 text-right font-mono text-[10px] ${
|
||||
tier === "explicitCustom" ? "text-panel-text-0" : "text-panel-text-3"
|
||||
}`}
|
||||
>
|
||||
{displayValue}
|
||||
</span>
|
||||
<FlatSliderReadout
|
||||
label={label}
|
||||
displayValue={displayValue}
|
||||
tier={tier}
|
||||
disabled={disabled}
|
||||
onCommitText={onCommitText}
|
||||
onCommitted={() => track("input", label)}
|
||||
/>
|
||||
{(centerTick || onReset) && (
|
||||
<span data-flat-slider-reset-slot="true" className="w-3.5 shrink-0">
|
||||
{tier === "explicitCustom" && onReset && (
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
|
||||
/** Click-to-type readout. Enter/blur commits; a refused value stays open and turns red. */
|
||||
export function FlatSliderReadout({
|
||||
label,
|
||||
displayValue,
|
||||
tier,
|
||||
disabled,
|
||||
onCommitText,
|
||||
onCommitted,
|
||||
}: {
|
||||
label: string;
|
||||
displayValue: string;
|
||||
tier: "default" | "explicitCustom";
|
||||
disabled?: boolean;
|
||||
/** Receives the raw text; return false to refuse it and keep the field open. */
|
||||
onCommitText?: (text: string) => boolean | void;
|
||||
/** Fires after an accepted commit, for telemetry. */
|
||||
onCommitted?: () => void;
|
||||
}) {
|
||||
const [editing, setEditing] = useState(false);
|
||||
const [text, setText] = useState("");
|
||||
const [invalid, setInvalid] = useState(false);
|
||||
const inputRef = useRef<HTMLInputElement | null>(null);
|
||||
const editable = Boolean(onCommitText) && !disabled;
|
||||
|
||||
useEffect(() => {
|
||||
if (!editing) return;
|
||||
inputRef.current?.focus();
|
||||
inputRef.current?.select();
|
||||
}, [editing]);
|
||||
|
||||
const begin = () => {
|
||||
if (!editable) return;
|
||||
setText(displayValue);
|
||||
setInvalid(false);
|
||||
setEditing(true);
|
||||
};
|
||||
// Enter and blur both commit, so a value typed and then clicked away from is
|
||||
// not silently dropped. A refused value stays open on Enter (the author is
|
||||
// still there to fix it) and is discarded on blur (they have moved on).
|
||||
const commit = (keepOpenIfRefused: boolean) => {
|
||||
const accepted = onCommitText?.(text.trim());
|
||||
if (accepted === false && keepOpenIfRefused) {
|
||||
setInvalid(true);
|
||||
inputRef.current?.select();
|
||||
return;
|
||||
}
|
||||
setEditing(false);
|
||||
if (accepted !== false) onCommitted?.();
|
||||
};
|
||||
|
||||
if (editing) {
|
||||
return (
|
||||
<input
|
||||
ref={inputRef}
|
||||
data-flat-slider-input="true"
|
||||
aria-label={`${label} value`}
|
||||
aria-invalid={invalid || undefined}
|
||||
value={text}
|
||||
spellCheck={false}
|
||||
onChange={(e) => {
|
||||
setText(e.target.value);
|
||||
setInvalid(false);
|
||||
}}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === "Enter") {
|
||||
e.preventDefault();
|
||||
commit(true);
|
||||
} else if (e.key === "Escape") {
|
||||
e.preventDefault();
|
||||
setEditing(false);
|
||||
}
|
||||
// Keep the timeline's own shortcuts from firing on every keystroke.
|
||||
e.stopPropagation();
|
||||
}}
|
||||
onBlur={() => commit(false)}
|
||||
className={`w-11 shrink-0 rounded-[3px] border bg-panel-surface px-1 text-right font-mono text-[10px] text-panel-text-0 outline-none ${
|
||||
invalid ? "border-red-400" : "border-panel-accent"
|
||||
}`}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<span
|
||||
data-flat-slider-value="true"
|
||||
role={editable ? "button" : undefined}
|
||||
tabIndex={editable ? 0 : undefined}
|
||||
title={editable ? "Click to type a value" : undefined}
|
||||
onClick={begin}
|
||||
onKeyDown={(e) => {
|
||||
if (!editable) return;
|
||||
if (e.key === "Enter" || e.key === " ") {
|
||||
e.preventDefault();
|
||||
begin();
|
||||
}
|
||||
}}
|
||||
className={`w-11 shrink-0 text-right font-mono text-[10px] ${
|
||||
tier === "explicitCustom" ? "text-panel-text-0" : "text-panel-text-3"
|
||||
} ${editable ? "cursor-text rounded-[3px] hover:bg-panel-hover hover:text-panel-text-0" : ""}`}
|
||||
>
|
||||
{displayValue}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { mediaMetadataUrl } from "../../utils/studioHelpers";
|
||||
import { useLivePreviewIframe } from "../../player/store/previewIframeStore";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState, type RefObject } from "react";
|
||||
import {
|
||||
@@ -288,12 +289,7 @@ export function useColorGradingController({
|
||||
return;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
fetch(
|
||||
`/api/projects/${encodeURIComponent(projectId)}/media/metadata?path=${encodeURIComponent(
|
||||
selectedAssetPath,
|
||||
)}`,
|
||||
{ signal: controller.signal },
|
||||
)
|
||||
fetch(mediaMetadataUrl(projectId, selectedAssetPath), { signal: controller.signal })
|
||||
.then(async (response) => {
|
||||
if (!response.ok) return { ok: false as const };
|
||||
const data: MediaMetadataResponse | null = await response.json();
|
||||
|
||||
@@ -18,7 +18,11 @@ import {
|
||||
} from "../utils/timelineAssetDrop";
|
||||
import { generateId } from "../utils/generateId";
|
||||
import { saveProjectFilesWithHistory, type RecordEditInput } from "../utils/studioFileHistory";
|
||||
import { collectHtmlIds, resolveDroppedAssetDuration } from "../utils/studioHelpers";
|
||||
import {
|
||||
collectHtmlIds,
|
||||
resolveDroppedAssetDuration,
|
||||
resolveDroppedAssetHasAudio,
|
||||
} from "../utils/studioHelpers";
|
||||
import { formatTimelineAttributeNumber } from "./timelineEditingHelpers";
|
||||
import { readFileContent } from "./timelineTimingSync";
|
||||
import { commitTimelineCompositionInsertion } from "../utils/timelineCompositionInsert";
|
||||
@@ -113,6 +117,9 @@ export function useTimelineAssetDropOps({
|
||||
? durationOverride
|
||||
: await resolveDroppedAssetDuration(pid, assetPath, kind);
|
||||
const normalizedDuration = Number(formatTimelineAttributeNumber(duration));
|
||||
// A video with an audio stream lands audible; the mixer only hears a
|
||||
// <video> marked data-has-audio, and a muted drop was losing the sound.
|
||||
const hasAudio = await resolveDroppedAssetHasAudio(pid, assetPath, kind);
|
||||
const newId = buildTimelineAssetId(assetPath, collectHtmlIds(originalContent));
|
||||
const resolvedAssetSrc = resolveTimelineAssetSrc(targetPath, assetPath);
|
||||
|
||||
@@ -146,6 +153,7 @@ export function useTimelineAssetDropOps({
|
||||
duration: normalizedDuration,
|
||||
track,
|
||||
zIndex: newElementZIndex,
|
||||
hasAudio,
|
||||
geometry: fitTimelineAssetGeometry(
|
||||
null,
|
||||
resolveTimelineAssetCompositionSize(originalContent),
|
||||
|
||||
@@ -4,6 +4,7 @@ import { defaultTimelineTheme, getClipHandleOpacity, type TimelineTheme } from "
|
||||
import type { TimelineEditCapabilities } from "./timelineEditing";
|
||||
import { isAudioTimelineElement } from "../../utils/timelineInspector";
|
||||
import { timelineClipFocusId } from "./timelineNavigationIdentity";
|
||||
import { TimelineClipFades } from "./TimelineClipFades";
|
||||
|
||||
interface TimelineClipProps {
|
||||
el: TimelineElement;
|
||||
@@ -74,6 +75,7 @@ export const TimelineClip = memo(function TimelineClip({
|
||||
"--clip-border-active": theme.clipBorderActive,
|
||||
"--clip-handle": theme.handleColor,
|
||||
} as CSSProperties;
|
||||
const isAudioClip = isAudioTimelineElement(el);
|
||||
const clipClassName = [
|
||||
"timeline-clip",
|
||||
"absolute",
|
||||
@@ -81,7 +83,7 @@ export const TimelineClip = memo(function TimelineClip({
|
||||
isSelected ? "is-selected" : "",
|
||||
isHovered ? "is-hovered" : "",
|
||||
isDragging ? "is-dragging" : "",
|
||||
isAudioTimelineElement(el) ? "is-audio" : "",
|
||||
isAudioClip ? "is-audio" : "",
|
||||
]
|
||||
.filter((className) => className.length > 0)
|
||||
.join(" ");
|
||||
@@ -198,6 +200,17 @@ export const TimelineClip = memo(function TimelineClip({
|
||||
</span>
|
||||
)}
|
||||
{children}
|
||||
{/* Fade handles + ramps for anything the mixer hears — audio clips and
|
||||
videos marked data-has-audio. They write data-fade-in/out on the clip
|
||||
and are the timeline half of the inspector's Fade rows. */}
|
||||
{(isAudioClip || el.hasAudio) && !isGestureActor && (
|
||||
<TimelineClipFades
|
||||
el={el}
|
||||
pps={pps}
|
||||
widthPx={widthPx}
|
||||
showHandles={(isHovered || isSelected) && !isDragging}
|
||||
/>
|
||||
)}
|
||||
</button>
|
||||
);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
// @vitest-environment happy-dom
|
||||
|
||||
import React, { act } from "react";
|
||||
import { createRoot } from "react-dom/client";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import type { TimelineElement } from "../store/playerStore";
|
||||
import { TimelineClipFades } from "./TimelineClipFades";
|
||||
import { TimelineEditProvider } from "../../contexts/TimelineEditContext";
|
||||
|
||||
(globalThis as unknown as { IS_REACT_ACT_ENVIRONMENT: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
afterEach(() => {
|
||||
document.body.innerHTML = "";
|
||||
});
|
||||
|
||||
const clip: TimelineElement = {
|
||||
id: "music",
|
||||
tag: "audio",
|
||||
src: "assets/music.wav",
|
||||
start: 2,
|
||||
duration: 10,
|
||||
track: 1,
|
||||
fadeIn: 1,
|
||||
fadeOut: 2,
|
||||
};
|
||||
|
||||
function render(
|
||||
el: TimelineElement,
|
||||
options: { showHandles?: boolean; provide?: boolean; onClipPointerDown?: () => void } = {},
|
||||
) {
|
||||
const host = document.createElement("div");
|
||||
document.body.append(host);
|
||||
const root = createRoot(host);
|
||||
const onSetElementAttributeLive = vi.fn();
|
||||
const onSetElementAttributeQuiet = vi.fn().mockResolvedValue(undefined);
|
||||
// Stands in for the TimelineClip button the handles live inside.
|
||||
const node = (
|
||||
<div data-testid="clip" onPointerDown={options.onClipPointerDown}>
|
||||
<TimelineClipFades
|
||||
el={el}
|
||||
pps={100}
|
||||
widthPx={el.duration * 100}
|
||||
showHandles={options.showHandles ?? true}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
act(() => {
|
||||
root.render(
|
||||
options.provide === false ? (
|
||||
node
|
||||
) : (
|
||||
<TimelineEditProvider value={{ onSetElementAttributeLive, onSetElementAttributeQuiet }}>
|
||||
{node}
|
||||
</TimelineEditProvider>
|
||||
),
|
||||
);
|
||||
});
|
||||
return { host, root, onSetElementAttributeLive, onSetElementAttributeQuiet };
|
||||
}
|
||||
|
||||
function pointer(type: string, clientX: number, pointerId = 1) {
|
||||
const event = new MouseEvent(type, { bubbles: true, clientX, button: 0 });
|
||||
Object.defineProperty(event, "pointerId", { value: pointerId });
|
||||
return event;
|
||||
}
|
||||
|
||||
function armCapture(handle: HTMLElement) {
|
||||
let captured = false;
|
||||
Object.defineProperty(handle, "setPointerCapture", { value: () => (captured = true) });
|
||||
Object.defineProperty(handle, "releasePointerCapture", { value: () => (captured = false) });
|
||||
Object.defineProperty(handle, "hasPointerCapture", { value: () => captured });
|
||||
}
|
||||
|
||||
describe("TimelineClipFades", () => {
|
||||
it("shades the fade-in and fade-out as ramps sized by pps", () => {
|
||||
const { host, root } = render(clip, { showHandles: false, provide: false });
|
||||
const fadeIn = host.querySelector('[data-testid="clip-fade-in"]');
|
||||
const fadeOut = host.querySelector('[data-testid="clip-fade-out"]');
|
||||
// 1 s in at 100 px/s: the wedge spans 0..100; 2 s out on a 1000 px clip: 800..1000.
|
||||
expect(fadeIn?.getAttribute("points")).toBe("0,0 100,0 0,100");
|
||||
expect(fadeOut?.getAttribute("points")).toBe("800,0 1000,0 1000,100");
|
||||
// Outside a provider there is nothing to write to, so no handles either.
|
||||
expect(host.querySelector('[data-testid="clip-fade-handle-in"]')).toBeNull();
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("renders nothing for a clip without fades when the handles are hidden", () => {
|
||||
const { host, root } = render(
|
||||
{ ...clip, fadeIn: undefined, fadeOut: undefined },
|
||||
{
|
||||
showHandles: false,
|
||||
},
|
||||
);
|
||||
expect(host.querySelector('[data-testid="clip"]')?.innerHTML).toBe("");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("drags the fade-in dot to the right, previewing live and committing once on release", () => {
|
||||
const { host, root, onSetElementAttributeLive, onSetElementAttributeQuiet } = render(clip);
|
||||
const handle = host.querySelector<HTMLElement>('[data-testid="clip-fade-handle-in"]');
|
||||
if (!handle) throw new Error("expected a fade-in handle");
|
||||
armCapture(handle);
|
||||
// The handle sits at the 1 s mark (x=100). Drag 150 px right → 2.5 s.
|
||||
act(() => handle.dispatchEvent(pointer("pointerdown", 100)));
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", 200)));
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", 250)));
|
||||
expect(onSetElementAttributeLive).toHaveBeenLastCalledWith(clip, "data-fade-in", "2.5");
|
||||
expect(onSetElementAttributeQuiet).not.toHaveBeenCalled();
|
||||
// The ramp follows the pointer before anything is persisted.
|
||||
expect(host.querySelector('[data-testid="clip-fade-in"]')?.getAttribute("points")).toBe(
|
||||
"0,0 250,0 0,100",
|
||||
);
|
||||
act(() => handle.dispatchEvent(pointer("pointerup", 250)));
|
||||
expect(onSetElementAttributeQuiet).toHaveBeenCalledTimes(1);
|
||||
expect(onSetElementAttributeQuiet).toHaveBeenCalledWith(clip, "data-fade-in", "2.5", "Fade in");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("drags the fade-out dot to the left, growing the fade, and never past the fade-in", () => {
|
||||
const { host, root, onSetElementAttributeLive, onSetElementAttributeQuiet } = render(clip);
|
||||
const handle = host.querySelector<HTMLElement>('[data-testid="clip-fade-handle-out"]');
|
||||
if (!handle) throw new Error("expected a fade-out handle");
|
||||
armCapture(handle);
|
||||
// Handle at x=800 (2 s before the end). 300 px left → 5 s.
|
||||
act(() => handle.dispatchEvent(pointer("pointerdown", 800)));
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", 500)));
|
||||
expect(onSetElementAttributeLive).toHaveBeenLastCalledWith(clip, "data-fade-out", "5");
|
||||
// Way past the start: clamps to duration − fadeIn = 9 s.
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", -500)));
|
||||
expect(onSetElementAttributeLive).toHaveBeenLastCalledWith(clip, "data-fade-out", "9");
|
||||
act(() => handle.dispatchEvent(pointer("pointerup", -500)));
|
||||
expect(onSetElementAttributeQuiet).toHaveBeenCalledWith(clip, "data-fade-out", "9", "Fade out");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("removes the attribute when dragged back to zero", () => {
|
||||
const { host, root, onSetElementAttributeQuiet } = render(clip);
|
||||
const handle = host.querySelector<HTMLElement>('[data-testid="clip-fade-handle-in"]');
|
||||
if (!handle) throw new Error("expected a fade-in handle");
|
||||
armCapture(handle);
|
||||
act(() => handle.dispatchEvent(pointer("pointerdown", 100)));
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", -40)));
|
||||
act(() => handle.dispatchEvent(pointer("pointerup", -40)));
|
||||
expect(onSetElementAttributeQuiet).toHaveBeenCalledWith(clip, "data-fade-in", null, "Fade in");
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("puts the live value back and writes nothing when the gesture is cancelled", () => {
|
||||
const { host, root, onSetElementAttributeLive, onSetElementAttributeQuiet } = render(clip);
|
||||
const handle = host.querySelector<HTMLElement>('[data-testid="clip-fade-handle-in"]');
|
||||
if (!handle) throw new Error("expected a fade-in handle");
|
||||
armCapture(handle);
|
||||
act(() => handle.dispatchEvent(pointer("pointerdown", 100)));
|
||||
act(() => handle.dispatchEvent(pointer("pointermove", 300)));
|
||||
act(() => handle.dispatchEvent(pointer("pointercancel", 300)));
|
||||
expect(onSetElementAttributeLive).toHaveBeenLastCalledWith(clip, "data-fade-in", "1");
|
||||
expect(onSetElementAttributeQuiet).not.toHaveBeenCalled();
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
||||
it("does not let a press on the dot start the clip's own move gesture", () => {
|
||||
const outer = vi.fn();
|
||||
const { host, root } = render(clip, { onClipPointerDown: outer });
|
||||
const handle = host.querySelector<HTMLElement>('[data-testid="clip-fade-handle-in"]');
|
||||
if (!handle) throw new Error("expected a fade-in handle");
|
||||
armCapture(handle);
|
||||
act(() => handle.dispatchEvent(pointer("pointerdown", 100)));
|
||||
expect(outer).not.toHaveBeenCalled();
|
||||
act(() => root.unmount());
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,302 @@
|
||||
import {
|
||||
useEffect,
|
||||
useRef,
|
||||
useState,
|
||||
type CSSProperties,
|
||||
type KeyboardEvent,
|
||||
type PointerEvent,
|
||||
} from "react";
|
||||
import {
|
||||
HF_AUDIO_FADE_IN_ATTR,
|
||||
HF_AUDIO_FADE_OUT_ATTR,
|
||||
clampFadesToDuration,
|
||||
formatFadeSeconds,
|
||||
} from "@hyperframes/core/audio-fade";
|
||||
import type { TimelineElement } from "../store/playerStore";
|
||||
import { useTimelineEditContextOptional } from "../../contexts/TimelineEditContext";
|
||||
|
||||
type FadeEdge = "in" | "out";
|
||||
|
||||
const HANDLE_SIZE = 10;
|
||||
const HANDLE_HIT = 16;
|
||||
/** Pixels of pointer travel before a press on the handle counts as a drag. */
|
||||
const DRAG_THRESHOLD_PX = 2;
|
||||
|
||||
interface TimelineClipFadesProps {
|
||||
el: TimelineElement;
|
||||
pps: number;
|
||||
widthPx: number;
|
||||
/** Handles show on hover/selection; the shaded ramps show whenever a fade is set. */
|
||||
showHandles: boolean;
|
||||
}
|
||||
|
||||
/** Corner dots that drag `data-fade-in` / `data-fade-out`; the wedge is the faded region. */
|
||||
export function TimelineClipFades({ el, pps, widthPx, showHandles }: TimelineClipFadesProps) {
|
||||
const { onSetElementAttributeLive, onSetElementAttributeQuiet } =
|
||||
useTimelineEditContextOptional();
|
||||
const canEdit = Boolean(onSetElementAttributeLive && onSetElementAttributeQuiet);
|
||||
|
||||
// Draft holds the value under the pointer during a drag; it also bridges
|
||||
// the gap between release and the store re-reading the file, so the ramp
|
||||
// does not snap back for a frame.
|
||||
const [draft, setDraft] = useState<{ edge: FadeEdge; seconds: number } | null>(null);
|
||||
const [dragging, setDragging] = useState<FadeEdge | null>(null);
|
||||
const authoredIn = el.fadeIn ?? 0;
|
||||
const authoredOut = el.fadeOut ?? 0;
|
||||
useEffect(() => setDraft(null), [authoredIn, authoredOut]);
|
||||
|
||||
const fades = clampFadesToDuration(
|
||||
{
|
||||
fadeIn: draft?.edge === "in" ? draft.seconds : authoredIn,
|
||||
fadeOut: draft?.edge === "out" ? draft.seconds : authoredOut,
|
||||
},
|
||||
el.duration,
|
||||
);
|
||||
const inPx = Math.min(widthPx, fades.fadeIn * pps);
|
||||
const outPx = Math.min(widthPx, fades.fadeOut * pps);
|
||||
|
||||
const gesture = useRef<{
|
||||
edge: FadeEdge;
|
||||
pointerId: number;
|
||||
originClientX: number;
|
||||
originSeconds: number;
|
||||
otherSeconds: number;
|
||||
moved: boolean;
|
||||
last: number;
|
||||
} | null>(null);
|
||||
|
||||
const attrFor = (edge: FadeEdge) =>
|
||||
edge === "in" ? HF_AUDIO_FADE_IN_ATTR : HF_AUDIO_FADE_OUT_ATTR;
|
||||
const attrText = (seconds: number) => (seconds > 0 ? formatFadeSeconds(seconds) : null);
|
||||
|
||||
const secondsAt = (clientX: number): number => {
|
||||
const g = gesture.current;
|
||||
if (!g) return 0;
|
||||
const deltaSeconds = (clientX - g.originClientX) / Math.max(pps, 1e-6);
|
||||
// Fade-in grows to the right, fade-out grows to the left.
|
||||
const raw = g.edge === "in" ? g.originSeconds + deltaSeconds : g.originSeconds - deltaSeconds;
|
||||
const limit = Math.max(0, el.duration - g.otherSeconds);
|
||||
const clamped = Math.min(limit, Math.max(0, raw));
|
||||
return Math.round(clamped * 100) / 100;
|
||||
};
|
||||
|
||||
const onHandlePointerDown = (edge: FadeEdge) => (e: PointerEvent<HTMLDivElement>) => {
|
||||
if (e.button !== 0 || !canEdit) return;
|
||||
// Ours, not the clip's: a press here must not start a move or trim.
|
||||
e.stopPropagation();
|
||||
e.preventDefault();
|
||||
e.currentTarget.setPointerCapture(e.pointerId);
|
||||
gesture.current = {
|
||||
edge,
|
||||
pointerId: e.pointerId,
|
||||
originClientX: e.clientX,
|
||||
originSeconds: edge === "in" ? authoredIn : authoredOut,
|
||||
otherSeconds: edge === "in" ? authoredOut : authoredIn,
|
||||
moved: false,
|
||||
last: edge === "in" ? authoredIn : authoredOut,
|
||||
};
|
||||
setDragging(edge);
|
||||
};
|
||||
|
||||
const onHandlePointerMove = (e: PointerEvent<HTMLDivElement>) => {
|
||||
const g = gesture.current;
|
||||
if (!g || g.pointerId !== e.pointerId) return;
|
||||
if (!g.moved && Math.abs(e.clientX - g.originClientX) < DRAG_THRESHOLD_PX) return;
|
||||
g.moved = true;
|
||||
const seconds = secondsAt(e.clientX);
|
||||
if (seconds === g.last) return;
|
||||
g.last = seconds;
|
||||
setDraft({ edge: g.edge, seconds });
|
||||
onSetElementAttributeLive?.(el, attrFor(g.edge), attrText(seconds));
|
||||
};
|
||||
|
||||
type Gesture = NonNullable<typeof gesture.current>;
|
||||
|
||||
/** Ends the pointer gesture and returns it, or null when the event is not ours. */
|
||||
const endGesture = (e: PointerEvent<HTMLDivElement>): Gesture | null => {
|
||||
const g = gesture.current;
|
||||
if (!g || g.pointerId !== e.pointerId) return null;
|
||||
gesture.current = null;
|
||||
setDragging(null);
|
||||
if (e.currentTarget.hasPointerCapture(e.pointerId)) {
|
||||
e.currentTarget.releasePointerCapture(e.pointerId);
|
||||
}
|
||||
return g;
|
||||
};
|
||||
|
||||
/** Puts the live document back where the file has it and drops the draft. */
|
||||
const revertGesture = (g: Gesture) => {
|
||||
if (g.moved) onSetElementAttributeLive?.(el, attrFor(g.edge), attrText(g.originSeconds));
|
||||
setDraft(null);
|
||||
};
|
||||
|
||||
const finish = (e: PointerEvent<HTMLDivElement>, cancelled: boolean) => {
|
||||
const g = endGesture(e);
|
||||
if (!g) return;
|
||||
if (cancelled || !g.moved) return revertGesture(g);
|
||||
setDraft({ edge: g.edge, seconds: g.last });
|
||||
void onSetElementAttributeQuiet?.(
|
||||
el,
|
||||
attrFor(g.edge),
|
||||
attrText(g.last),
|
||||
g.edge === "in" ? "Fade in" : "Fade out",
|
||||
);
|
||||
};
|
||||
|
||||
const onHandleKeyDown = (e: KeyboardEvent<HTMLDivElement>) => {
|
||||
const g = gesture.current;
|
||||
if (e.key !== "Escape" || !g) return;
|
||||
e.preventDefault();
|
||||
gesture.current = null;
|
||||
setDragging(null);
|
||||
revertGesture(g);
|
||||
};
|
||||
|
||||
const showIn = fades.fadeIn > 0;
|
||||
const showOut = fades.fadeOut > 0;
|
||||
const handlesVisible = showHandles || dragging !== null;
|
||||
if (!showIn && !showOut && !handlesVisible) return null;
|
||||
|
||||
const handleStyle = (leftPx: number): CSSProperties => ({
|
||||
position: "absolute",
|
||||
top: -(HANDLE_HIT - HANDLE_SIZE) / 2 + 1,
|
||||
left: leftPx - HANDLE_HIT / 2,
|
||||
width: HANDLE_HIT,
|
||||
height: HANDLE_HIT,
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
justifyContent: "center",
|
||||
cursor: "ew-resize",
|
||||
zIndex: 6,
|
||||
opacity: handlesVisible ? 1 : 0,
|
||||
pointerEvents: handlesVisible && canEdit ? "auto" : "none",
|
||||
touchAction: "none",
|
||||
outline: "none",
|
||||
});
|
||||
|
||||
return (
|
||||
<>
|
||||
{(showIn || showOut) && (
|
||||
<svg
|
||||
aria-hidden="true"
|
||||
data-testid="clip-fade-ramps"
|
||||
style={{
|
||||
position: "absolute",
|
||||
inset: 0,
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
pointerEvents: "none",
|
||||
zIndex: 5,
|
||||
overflow: "visible",
|
||||
}}
|
||||
viewBox={`0 0 ${Math.max(widthPx, 1)} 100`}
|
||||
preserveAspectRatio="none"
|
||||
>
|
||||
{/* The shaded wedge is the gain that is NOT there: above the ramp. */}
|
||||
{showIn && (
|
||||
<>
|
||||
<polygon
|
||||
data-testid="clip-fade-in"
|
||||
points={`0,0 ${inPx},0 0,100`}
|
||||
fill="var(--timeline-fade-shade)"
|
||||
/>
|
||||
<line
|
||||
x1={0}
|
||||
y1={100}
|
||||
x2={inPx}
|
||||
y2={0}
|
||||
stroke="var(--clip-handle)"
|
||||
strokeOpacity={0.9}
|
||||
strokeWidth={1.25}
|
||||
vectorEffect="non-scaling-stroke"
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
{showOut && (
|
||||
<>
|
||||
<polygon
|
||||
data-testid="clip-fade-out"
|
||||
points={`${widthPx - outPx},0 ${widthPx},0 ${widthPx},100`}
|
||||
fill="var(--timeline-fade-shade)"
|
||||
/>
|
||||
<line
|
||||
x1={widthPx - outPx}
|
||||
y1={0}
|
||||
x2={widthPx}
|
||||
y2={100}
|
||||
stroke="var(--clip-handle)"
|
||||
strokeOpacity={0.9}
|
||||
strokeWidth={1.25}
|
||||
vectorEffect="non-scaling-stroke"
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</svg>
|
||||
)}
|
||||
{canEdit && (
|
||||
<>
|
||||
<div
|
||||
role="slider"
|
||||
tabIndex={-1}
|
||||
aria-label="Fade in"
|
||||
aria-valuemin={0}
|
||||
aria-valuemax={Math.max(0, el.duration - fades.fadeOut)}
|
||||
aria-valuenow={fades.fadeIn}
|
||||
aria-valuetext={`${formatFadeSeconds(fades.fadeIn)}s`}
|
||||
data-testid="clip-fade-handle-in"
|
||||
title={`Fade in: ${formatFadeSeconds(fades.fadeIn)}s — drag to change`}
|
||||
style={handleStyle(inPx)}
|
||||
onPointerDown={onHandlePointerDown("in")}
|
||||
onPointerMove={onHandlePointerMove}
|
||||
onPointerUp={(e) => finish(e, false)}
|
||||
onPointerCancel={(e) => finish(e, true)}
|
||||
onKeyDown={onHandleKeyDown}
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
onDoubleClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<FadeDot active={dragging === "in"} />
|
||||
</div>
|
||||
<div
|
||||
role="slider"
|
||||
tabIndex={-1}
|
||||
aria-label="Fade out"
|
||||
aria-valuemin={0}
|
||||
aria-valuemax={Math.max(0, el.duration - fades.fadeIn)}
|
||||
aria-valuenow={fades.fadeOut}
|
||||
aria-valuetext={`${formatFadeSeconds(fades.fadeOut)}s`}
|
||||
data-testid="clip-fade-handle-out"
|
||||
title={`Fade out: ${formatFadeSeconds(fades.fadeOut)}s — drag to change`}
|
||||
style={handleStyle(widthPx - outPx)}
|
||||
onPointerDown={onHandlePointerDown("out")}
|
||||
onPointerMove={onHandlePointerMove}
|
||||
onPointerUp={(e) => finish(e, false)}
|
||||
onPointerCancel={(e) => finish(e, true)}
|
||||
onKeyDown={onHandleKeyDown}
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
onDoubleClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<FadeDot active={dragging === "out"} />
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function FadeDot({ active }: { active: boolean }) {
|
||||
return (
|
||||
<span
|
||||
aria-hidden="true"
|
||||
style={{
|
||||
display: "block",
|
||||
width: HANDLE_SIZE,
|
||||
height: HANDLE_SIZE,
|
||||
borderRadius: "50%",
|
||||
background: active ? "var(--timeline-fade-dot-active)" : "var(--clip-handle)",
|
||||
boxShadow: "0 0 0 1.5px var(--timeline-fade-dot-ring)",
|
||||
transform: active ? "scale(1.15)" : undefined,
|
||||
transition: "transform 80ms ease-out",
|
||||
}}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import type { ClipManifestClip } from "./playbackTypes";
|
||||
import { isFinitePositive } from "./playbackAdapter";
|
||||
import { getSourceScopedSelectorIndex } from "../../utils/sourceScopedSelectorIndex";
|
||||
import { HF_AUDIO_GROUP_TAG } from "@hyperframes/core/audio-groups";
|
||||
import { readElementFades } from "@hyperframes/core/audio-fade";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Layer-reveal lift transparency
|
||||
@@ -173,8 +174,27 @@ function applyPlaybackMetadataFromElement(entry: TimelineElement, el: Element):
|
||||
}
|
||||
}
|
||||
|
||||
/** Sets or clears an optional field, so a re-parse after the attribute is removed drops it. */
|
||||
function setOptional<K extends keyof TimelineElement>(
|
||||
entry: TimelineElement,
|
||||
key: K,
|
||||
value: TimelineElement[K] | undefined,
|
||||
): void {
|
||||
if (value === undefined) delete entry[key];
|
||||
else entry[key] = value;
|
||||
}
|
||||
|
||||
/** `data-has-audio` and the clip-edge fades: what the mixer hears and how it enters and leaves. */
|
||||
function applyAudioMetadataFromElement(entry: TimelineElement, el: Element): void {
|
||||
setOptional(entry, "hasAudio", el.getAttribute("data-has-audio") === "true" ? true : undefined);
|
||||
const fades = readElementFades(el);
|
||||
setOptional(entry, "fadeIn", fades.fadeIn > 0 ? fades.fadeIn : undefined);
|
||||
setOptional(entry, "fadeOut", fades.fadeOut > 0 ? fades.fadeOut : undefined);
|
||||
}
|
||||
|
||||
export function applyMediaMetadataFromElement(entry: TimelineElement, el: Element): void {
|
||||
applyPlaybackMetadataFromElement(entry, el);
|
||||
applyAudioMetadataFromElement(entry, el);
|
||||
|
||||
const mediaEl = resolveMediaElement(el);
|
||||
if (!mediaEl) return;
|
||||
|
||||
@@ -54,6 +54,11 @@ export interface TimelineElement {
|
||||
playbackRate?: number;
|
||||
sourceDuration?: number;
|
||||
volume?: number;
|
||||
/** `data-has-audio="true"` on a video: its sound is mixed, so it takes audio affordances (fades). */
|
||||
hasAudio?: boolean;
|
||||
/** Clip-edge fades from `data-fade-in` / `data-fade-out`, seconds; absent means none. */
|
||||
fadeIn?: number;
|
||||
fadeOut?: number;
|
||||
/** Verbatim `data-fx-chain` / `data-automation`; see automationLaneData. */
|
||||
fxChain?: string;
|
||||
automation?: string;
|
||||
|
||||
@@ -69,6 +69,10 @@
|
||||
--timeline-clip-border-hover: rgba(60, 230, 172, 0.75);
|
||||
--timeline-clip-border-active: rgba(60, 230, 172, 0.55);
|
||||
--timeline-handle: rgba(255, 255, 255, 0.55);
|
||||
/* Audio clip fades: the shaded wedge over the faded gain, and the corner dots. */
|
||||
--timeline-fade-shade: rgba(0, 0, 0, 0.38);
|
||||
--timeline-fade-dot-active: rgba(255, 255, 255, 1);
|
||||
--timeline-fade-dot-ring: rgba(0, 0, 0, 0.55);
|
||||
--timeline-resize-seam: rgba(255, 255, 255, 0.12);
|
||||
--timeline-resize-active: rgba(255, 255, 255, 0.24);
|
||||
--timeline-track-clip-fill: rgba(255, 255, 255, 0.055);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { useMemo } from "react";
|
||||
import { audioDbToGain, audioGainToDb } from "@hyperframes/core/audio-gain";
|
||||
import { usePlayerStore, type TimelineElement } from "../player";
|
||||
import { isAudioTimelineElement } from "./timelineInspector";
|
||||
|
||||
@@ -12,7 +13,7 @@ const HOLD_MS = 1200;
|
||||
|
||||
/** Linear peak (0..1) to a 0..1 bar fraction along the piecewise-linear dB scale. */
|
||||
export function levelToFraction(linear: number): number {
|
||||
const db = linear > 0 ? 20 * Math.log10(linear) : FLOOR_DB;
|
||||
const db = linear > 0 ? audioGainToDb(linear) : FLOOR_DB;
|
||||
if (db >= 0) return 1;
|
||||
if (db <= FLOOR_DB) return 0;
|
||||
const j = STOPS.findIndex((stop) => db >= stop);
|
||||
@@ -23,7 +24,7 @@ export function levelToFraction(linear: number): number {
|
||||
|
||||
/** Fraction of the bar height at which a dB mark sits, 0 at the bottom. */
|
||||
export function markFraction(db: number): number {
|
||||
return levelToFraction(10 ** (db / 20));
|
||||
return levelToFraction(audioDbToGain(db));
|
||||
}
|
||||
|
||||
export interface MeterChannel {
|
||||
|
||||
@@ -377,6 +377,27 @@ export async function resolveDroppedAssetDuration(
|
||||
return duration;
|
||||
}
|
||||
|
||||
export function mediaMetadataUrl(projectId: string, assetPath: string): string {
|
||||
return `/api/projects/${encodeURIComponent(projectId)}/media/metadata?path=${encodeURIComponent(assetPath)}`;
|
||||
}
|
||||
|
||||
/** Dropped video audio stream from the metadata endpoint. Failure answers false so the drop still lands muted. */
|
||||
export async function resolveDroppedAssetHasAudio(
|
||||
projectId: string,
|
||||
assetPath: string,
|
||||
kind: TimelineAssetKind,
|
||||
): Promise<boolean> {
|
||||
if (kind !== "video") return false;
|
||||
try {
|
||||
const response = await fetch(mediaMetadataUrl(projectId, assetPath));
|
||||
if (!response.ok) return false;
|
||||
const data = (await response.json()) as { metadata?: { hasAudio?: boolean } } | null;
|
||||
return data?.metadata?.hasAudio === true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveDroppedAssetDimensions(
|
||||
projectId: string,
|
||||
assetPath: string,
|
||||
|
||||
@@ -198,6 +198,38 @@ describe("insertTimelineAssetIntoSource", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildTimelineAssetInsertHtml — video audio", () => {
|
||||
const base = {
|
||||
id: "clip_asset",
|
||||
hfId: "hf-vid-1",
|
||||
assetPath: "assets/clip.mp4",
|
||||
kind: "video" as const,
|
||||
start: 0,
|
||||
duration: 8,
|
||||
track: 1,
|
||||
zIndex: 2,
|
||||
};
|
||||
|
||||
it("inserts a video muted when nothing says it carries audio", () => {
|
||||
const html = buildTimelineAssetInsertHtml(base);
|
||||
expect(html).toContain(" muted ");
|
||||
expect(html).not.toContain("data-has-audio");
|
||||
});
|
||||
|
||||
it("inserts a video with an audio stream audible: data-has-audio and no muted", () => {
|
||||
const html = buildTimelineAssetInsertHtml({ ...base, hasAudio: true });
|
||||
expect(html).toContain('data-has-audio="true"');
|
||||
expect(html).not.toContain("muted");
|
||||
expect(html).toContain("playsinline");
|
||||
});
|
||||
|
||||
it("keeps a video without an audio stream muted", () => {
|
||||
const html = buildTimelineAssetInsertHtml({ ...base, hasAudio: false });
|
||||
expect(html).toContain(" muted ");
|
||||
expect(html).not.toContain("data-has-audio");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildTimelineAssetInsertHtml markup quality", () => {
|
||||
const base = {
|
||||
id: "clip_1",
|
||||
|
||||
@@ -116,6 +116,8 @@ export function buildTimelineAssetInsertHtml(input: {
|
||||
track: number;
|
||||
zIndex: number;
|
||||
geometry?: { left: number; top: number; width: number; height: number };
|
||||
/** Video only: true inserts `data-has-audio="true"` with no `muted`. Unknown or false stays muted. */
|
||||
hasAudio?: boolean;
|
||||
}): string {
|
||||
const sharedAttrs = `id="${input.id}" data-hf-id="${input.hfId}" class="clip" src="${input.assetPath}" data-start="${input.start}" data-duration="${input.duration}" data-track-index="${input.track}"`;
|
||||
const geometry = input.geometry ?? { left: 0, top: 0, width: 640, height: 360 };
|
||||
@@ -126,7 +128,10 @@ export function buildTimelineAssetInsertHtml(input: {
|
||||
}
|
||||
|
||||
if (input.kind === "video") {
|
||||
return `<video ${sharedAttrs} muted playsinline style="${visualStyles}"></video>`;
|
||||
// `muted` and `data-has-audio="true"` are mutually exclusive by the lint
|
||||
// contract (video_has_audio_but_muted): an audible drop takes the latter.
|
||||
const audio = input.hasAudio ? 'data-has-audio="true"' : "muted";
|
||||
return `<video ${sharedAttrs} ${audio} playsinline style="${visualStyles}"></video>`;
|
||||
}
|
||||
|
||||
return `<audio ${sharedAttrs} data-volume="1" style="z-index: ${input.zIndex}"></audio>`;
|
||||
|
||||
Reference in New Issue
Block a user