feat(studio): redesign Asset tab + fix beat analysis auto-trigger (#1684)

* feat(media-use): core infrastructure — manifest, cache, adopt, probe

Foundation for media-use — the media resolution layer for HyperFrames.

- manifest.mjs: JSONL read/write/find for .media/manifest.jsonl
- index-gen.mjs: regenerate agent-readable index.md from manifest
- cache.mjs: content-addressed global cache at ~/.media/ (SHA-256, sentinel)
- freeze.mjs: download URL or copy local file to .media/
- probe.mjs: extract duration/dimensions via ffprobe
- adopt.mjs: scan assets/ directory, register existing files with metadata
- 19 passing tests (manifest round-trip, cache, promote, index generation)

* fix(media-use): oxfmt formatting + cap freeze download size

Format adopt/cache/probe/manifest.test (CI oxfmt --check gate).
Cap freezeUrl downloads at 256MB so a hostile/runaway URL can't fill
the disk (addresses CodeQL #670: network data written to file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(media-use): resolve engine + all providers + brand from frame.md

- resolve.mjs: cheapest-first cascade
- BGM/SFX via heygen --headers, Image/Icon via heygen asset search
- Brand tokens from frame.md / design.md (local, no API)
- SKILL.md: full agent docs + hyperframes.dev/design redirect
- Router skill + workflow skill references

* fix(media-use): oxfmt formatting for resolve + providers

Format brand/heygen-search/providers/sfx providers + resolve.mjs
(CI oxfmt --check gate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(media-use): align providers with the real heygen CLI surface (v0.1.6)

Verified live against the official Go `heygen` CLI v0.1.6 with a valid key:

- Caller attribution: pass `--headers 'X-HeyGen-Client-Source: media-use'`
  (the allowlisted flag the CLI added for media-use in v0.1.6). The old
  `--x-source media-use` was never a real flag and broke every call.
- Command is `asset search` (the `list` leaf was dropped in v0.1.6), not
  `asset search list`.
- `--min-score` is sent server-side: honored by `audio sounds list`, but the
  `asset search` backend rejects it and returns no score field, so only the
  audio providers pass it (image/icon don't).
- Drop hardcoded `ext` so resolve.mjs derives it from the URL: catalog icons
  are .png (not .svg), some BGM is .wav (not .mp3).

Also: surface CLI/auth failures on stderr instead of swallowing them as
'no results', carry icon width/height through, and document the heygen CLI
install + >= v0.1.6 requirement.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(studio): redesign Asset tab + fix beat analysis auto-trigger

Asset tab: categorized sections, filter chips, text search, audio
spectrum visualizer, "in use" badge, manifest metadata, panel tokens.

Beat fix: only run analysis when a beats file exists on disk.

* fix(studio): oxfmt formatting for AssetsTab

CI oxfmt --check gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Miguel Ángel
2026-06-24 20:29:22 -04:00
committed by GitHub
co-authored by Claude Opus 4.8
parent b2dc353725
commit 92befea305
4 changed files with 726 additions and 286 deletions
@@ -92,33 +92,48 @@ export function useMusicBeatAnalysis(): void {
return;
}
let cancelled = false;
let promise = analysisCache.get(musicSrc);
if (!promise) {
promise = analyzeMusicFromUrl(musicSrc);
cacheAnalysis(musicSrc, promise);
}
const beatPath = beatFilePathForSrc(musicSrc);
promise
.then(async (analysis) => {
const io = ioRef.current;
// Only run expensive audio decode + beat analysis when the user has an
// explicit beats file saved. Without one, skip entirely — no surprise
// green lines on the timeline after dragging unrelated assets.
(async () => {
if (!beatPath || !io) return;
let hasSavedBeats = false;
try {
const content = await io.readOptionalProjectFile(beatPath);
const parsed = content ? parseBeats(content) : null;
hasSavedBeats = !!(parsed && parsed.times.length > 0);
} catch {
/* no file */
}
if (cancelled) return;
if (!hasSavedBeats) {
setBeatAnalysis(null);
return;
}
let promise = analysisCache.get(musicSrc);
if (!promise) {
promise = analyzeMusicFromUrl(musicSrc);
cacheAnalysis(musicSrc, promise);
}
try {
const analysis = await promise;
const detected = { times: analysis.beatTimes, strengths: analysis.beatStrengths };
const io = ioRef.current;
if (!io) return;
const { times, strengths, hasFile } = await resolveBeats(beatPath, detected, io);
const { times, strengths } = await resolveBeats(beatPath, detected, io);
if (cancelled) return;
setBeatEdits(null);
resetBeatHistory();
setBeatAnalysis({ ...analysis, beatTimes: times, beatStrengths: strengths });
// Seed a missing file through the SAME debounced writer the edits use, so
// the initial write can't race a near-simultaneous edit's persist.
if (beatPath && !hasFile && times.length > 0) usePlayerStore.getState().beatPersist?.();
})
.catch(() => {
if (cancelled) return;
setBeatAnalysis(null);
analysisCache.delete(musicSrc);
});
} catch {
if (!cancelled) {
setBeatAnalysis(null);
analysisCache.delete(musicSrc);
}
}
})();
return () => {
cancelled = true;