feat(core): swap undecodable video to its proxy at runtime (#2592)

* feat(studio-server): serve H.264 proxies from the preview route

Wires the codec manifest and the transcoder into the preview surface: the route
negotiates a proxy via a query param and serves it through the existing range
and ETag machinery, composition HTML carries a codec map for the runtime, and
hostile assets pre-warm so a first play does not wait on a cold transcode.
Exposes the three subpath exports the CLI surfaces consume upstack.

Drops the TEMP fallow entry added with the transcoder: it has real importers now.

* fix(studio-server): publish media proxy exports

* fix(parsers): scan HTML comments linearly

* feat(cli): let projects opt out of automatic proxying

Adds media.autoProxy to hyperframes.json plus --proxy/--no-proxy flags, and
forwards the resolved value into the studio and preview servers and the vite
adapter. Lands before the runtime slice that turns auto-proxying on, so the
switch exists before there is any behavior to switch off.

* fix(cli): align media config schema

* feat(core): swap undecodable video to its proxy at runtime

Adds the browser-side half: before first load the runtime consults the injected
codec map and swaps a hostile source to its proxy, and if a video still reports
zero decodable width it rescues it reactively. An HEVC file carrying AAC fires
no error event, so zero videoWidth, not the error event, is the reliable signal.
Audio elements and alpha sources are never proxied, render mode never proxies,
and each swap evicts the element's stale sync state and reports once.

This completes the loop: auto-proxying is live for preview and studio from here.
The opt-out (media.autoProxy, --no-proxy) shipped in the previous slice.
This commit is contained in:
Miguel Ángel
2026-07-16 23:01:39 -04:00
committed by GitHub
parent 6458807066
commit 39b588cbd0
4 changed files with 956 additions and 4 deletions
+31
View File
@@ -24,6 +24,7 @@ import {
resolveRuntimeMediaClipDuration,
syncRuntimeMedia,
} from "./media";
import { handleErrorForProxy, handleMetadataForProxy, maybeProxyProactively } from "./mediaProxy";
import { probeAndCacheElementVolume, type VolumeKeyframe } from "./mediaVolumeEnvelope.js";
import { createPickerModule } from "./picker";
import { createRuntimePlayer, type RuntimePlayerTransport } from "./player";
@@ -1644,10 +1645,29 @@ export function initSandboxRuntimeModular(): void {
}, METADATA_REBIND_DEBOUNCE_MS);
};
// Reactive/tertiary undecodable-media triggers (see mediaProxy.ts). Wrapped
// as event listeners here — rather than exported directly — because
// `addEventListener` hands the listener an `Event`, not the element;
// `event.currentTarget` recovers it. Bound/unbound alongside the metadata
// listeners below, reusing `metadataBoundMedia` as the once-per-element
// dedupe (no separate tracking set needed).
const onMediaLoadedMetadataForProxy = (event: Event) => {
if (event.currentTarget instanceof HTMLMediaElement) {
handleMetadataForProxy(event.currentTarget);
}
};
const onMediaErrorForProxy = (event: Event) => {
if (event.currentTarget instanceof HTMLMediaElement) {
handleErrorForProxy(event.currentTarget);
}
};
const unbindMediaMetadataListeners = () => {
for (const mediaEl of metadataBoundMedia) {
mediaEl.removeEventListener("loadedmetadata", scheduleMetadataDurationHydration);
mediaEl.removeEventListener("durationchange", scheduleMetadataDurationHydration);
mediaEl.removeEventListener("loadedmetadata", onMediaLoadedMetadataForProxy);
mediaEl.removeEventListener("error", onMediaErrorForProxy);
}
metadataBoundMedia.clear();
};
@@ -1664,6 +1684,17 @@ export function initSandboxRuntimeModular(): void {
}
mediaEl.addEventListener("loadedmetadata", scheduleMetadataDurationHydration);
mediaEl.addEventListener("durationchange", scheduleMetadataDurationHydration);
// Reactive (zero-videoWidth) + tertiary (error event) proxy-fallback
// triggers. Inert in render mode / when the codec map is absent /
// for <audio> — all guarded inside mediaProxy.ts itself.
mediaEl.addEventListener("loadedmetadata", onMediaLoadedMetadataForProxy);
mediaEl.addEventListener("error", onMediaErrorForProxy);
// Proactive proxy-fallback trigger: consult the codec map and swap
// BEFORE the eager load() below, so a known-hostile asset never even
// attempts to load (and error-flash) the original. No-op in render
// mode, for <audio>, or when the codec map is absent.
maybeProxyProactively(mediaEl);
// Eagerly preload media data so audio/video is buffered before the user
// clicks play. Without this, the first play() call fires on un-fetched