Commit Graph
6 Commits
Author SHA1 Message Date
Miguel Ángel 4c6fa9a790 fix(studio): the preview stays paused, survives typing, and opens without a re-encode (#3910)
* chore(studio): remove the write-only self-write timestamp ref

`domEditSaveTimestampRef` was assigned at 26 sites across 54 files and read
nowhere. It used to feed a 2 s "suppress the watcher reload after our own
write" window; that mechanism was replaced by content-hash identity in
sdkSelfWriteRegistry, whose header still says why (a clock cannot tell an
SDK self-write echo from an undo landing in the same window). The reader
went with that change, the writers did not, and three comments kept
describing the timestamp as protection that no longer existed.

Pure deletion: the ref, every prop and parameter that threaded it, every
assignment, the five comments citing it, and the orphaned test helpers.
No behaviour change; the studio suite, typecheck, lint and format are
green.

Not removed: the __hfSuppressSceneMutations wrapper in gsapSoftReload.ts.
It looked undefined from inside the studio package, but shader-transitions
installs it on the preview window (hyper-shader.ts) so soft reloads do not
invalidate cached transitions. It is live.

* docs(studio): plan for loading the preview during the shell's first layout

* fix(runtime): a paused preview stays still after any seek

Sub-compositions kept animating while the transport was paused and the
Studio button showed play. A render-seek unpauses every sibling timeline so
GSAP propagates the root's totalTime into them, and nothing paused them
again; parented to the global ticker, they free-ran at 1x while the master
timeline and the clock stayed stopped. The Studio reaches that seek path
through its seek-driven fallback adapter right after a preview reload,
which is why editing text was what set it off. Captured live in the user's
preview: seven child timelines advancing 0.210s per 200ms sample, all
gsapPaused=false, transport not playing.

Three fixes under one invariant: while the clock is paused nothing runs,
and the button never disagrees with the runtime.

- The sibling rearm is a lease, not a resting state: seekTimelineAndAdapters
  now returns every timeline it unpaused to paused in a finally. The second
  rearm after the child re-seek only ever changed the leaked state and is
  gone, with the comment that justified the leak by describing one caller.
- The paused side of the transport tick policed nothing; it now stops any
  timed media element found running. A parked transport runs no ticks, so a
  capture-phase play listener wakes it.
- Studio's canvas click resumed by setting the store flag alone; it now
  requests playback through the player so adapter, rAF loop and flag agree.

Regression tests fail without each of the three changes.

* perf(studio-server): stop re-encoding videos the browser already plays on open

Opening a project kicked off a background transcode for every asset whose
codec some browser might not decode. VP9 is on that list for Safari's sake,
so a Chrome user opening a project with a VP9 avatar paid a ~10s, ~60
CPU-second re-encode across six cores on every open, concurrently with the
browser's first layout, for an output nothing ever requested: across four
recorded sessions the browser asked for the proxy zero times.

One predicate was doing two jobs. "Could some browser fail on this" is a
property of the asset and decides what gets injected into the page;
"will this client request the substitute" decides whether to spend CPU
before being asked. The codec table now carries an explicit prewarm flag:
HEVC and ProRes (no browser decodes them) still warm; VP9 and AV1 are
injected but transcoded lazily by the existing ?hf-proxy= request, which
already reports failure as a 502.

Pre-warms requested and proxies served are now counted on the existing
structured-stderr telemetry channel, because a speculative job with a 0%
hit rate emits no error and had been invisible. A single lookup helper
also closes a prototype-key gap where one path used Object.hasOwn and the
other a bare index.

Regression test: a VP9 and an HEVC asset in one composition yield exactly
one pre-warm, for the HEVC. Fails on the previous gate.

* perf(cli): serve the studio bundle compressed and cache hashed assets

The 4.1MB studio bundle was served uncompressed with Cache-Control:
no-store, so every open of the studio re-downloaded and re-parsed it. Vite
names built assets with an 8-character content hash, so their bytes can
never change: those now ship gzip-compressed (1.25MB on the wire,
byte-identical after inflation) with a one-year immutable cache policy.
Unhashed files under public/ keep no-store, and the HTML shell, which
previously sent no Cache-Control at all, now sends no-store explicitly:
it is the only thing that names the current hashed bundle, and it must
keep revalidating for the immutable policy to be safe.

The hash test only accepts the segment after the last hyphen and requires
a digit, underscore or capital, so an ordinary hyphenated name like
user-Guide-v2.js is not mistaken for a hash and served forever.

* perf(cli): warm the preview route before the browser opens

The browser's first request for a project's preview paid the server's cold
compiler import and first bundle in the request path, after the studio
shell's own first-layout stall, so both costs landed on the user's
time-to-first-frame in series. The CLI now issues one fire-and-forget
request for the preview route from openStudioBrowser, the single funnel
for every launch path, before the --no-open early return so pasted URLs
benefit too. Measured on the demo project: first client request 231ms
before, 101ms after; the ETag 304 path afterwards is under 1ms.

The fetch carries a 10s abort so an unsettled connection cannot keep an
otherwise-finished CLI process alive, matching the package's existing
convention. The shader query params the player appends are not part of
the route's cache key, so the plain route URL warms the same entry.

* perf(studio): request the player chunk before the shell's first layout

The dynamic import of @hyperframes/player ran inside the preview's mount
effect, which React schedules after the shell's first layout. On a cold
open that layout stalls the main thread for seconds, so the chunk request
waited behind it for no reason. The import is now kicked at module scope,
behind a typeof window guard that preserves the documented SSR contract
(the module registers a custom element at load), and the mount effect
awaits the already-in-flight promise. Verified in the built bundle: the
preload call sits at module top level, so the request goes out on bundle
evaluation.

* fix(studio): editing a text property no longer reloads the preview

Every keystroke in the Design panel writes the composition file, the file
watcher announces the change to the studio, and the studio decides whether
the change was its own. Over the CLI's event stream that decision has
never worked: the browser hands the handler a MessageEvent whose data is
a JSON string, and three of the four payload readers (version, write
token, content) only understood an already-parsed object, so they read
every field as absent. An absent token means "someone else edited the
file", and the studio hard-reloaded the preview iframe on its own edit,
blanking it for seconds. The path reader alone knew how to unwrap the
string, which is why the event was recognised well enough to reload and
never well enough to suppress.

The envelope is now decoded once, at the boundary, by one function that
all three transports feed; the readers share one field accessor so they
cannot diverge again. The production event-stream rung is extracted into
an exported channel so a test can drive a real MessageEvent through the
listener it registers, which was impossible before because vitest defines
import.meta.hot and the selection never reached that rung under test.

A second, smaller cause: the server attached the write receipt to the
first subscriber only and deleted it on read, so any other listener saw
an unlabelled change. Reads are now non-destructive with the TTL as the
only eviction, scanning newest-first because identical bytes written
twice inside the TTL (undo, retyping a value) share a version and the
older token was already spent. The file version now ships with every
event, receipt or not, so duplicate deliveries of one change dedupe
instead of reloading once each.

shouldReloadSdkSession had no production callers and a signature that
invited an undecoded delivery straight back into this bug; it is removed.
consumeFileWriteReceipt stays as a deprecated alias for one release.

Regression tests: a Studio write delivered as a real SSE MessageEvent is
suppressed; two subscribers of one watcher event reload once; a genuinely
external write still reloads; a repeat of earlier bytes gets the newest
token; a receipt past the TTL is not recognised. Each fails on the code
before it.

* fix(runtime): paused-time media playback is borrowed, not banned

The paused-side enforcement added in the previous commit had no notion of
provenance, so it stopped two features that legitimately play media while the
transport clock is paused. Both were deterministic, not racy: the capture-phase
`play` listener means the very play() that starts them wakes the transport that
stops them.

- The colour-grading preview (colorGrading.ts startPreviewPlayback) plays a
  video while paused to render grading previews. It went dark on the first
  paused tick.
- The Studio's scrub audition (timelineIframeHelpers.ts applyScrub) plays the
  music track for ~140 ms while paused so a playhead drag is audible. Same path
  killed it.

A runtime-owned lease fixes both without weakening the enforcement. One owner: a
WeakSet in the runtime closure, with lease/release published on the existing
window.__hf surface for the Studio, which reaches the element across the iframe
boundary and cannot call into the closure. The grading runtime is constructed
with the pair directly. Both the cheap probe and the sync path skip leased
elements while the clock is paused; during playback the transport owns everything
again. Anything that plays while paused without a lease is, by definition, the
defect the enforcement exists for, and is still stopped.

Two corrections to the previous commit's reasoning:

The old leak broke the render path too, deterministically, not only the preview.
packages/producer/src/services/fileServer.ts:375-379 seekToTime flushes the
virtualized rAF queue, and GSAP's global ticker with it, after renderSeek and
before the frame screenshot (fileServer.ts:657-664, hf.seek). A sibling left
unpaused therefore advanced by the full inter-frame delta into the captured
frame. The finally added in the previous commit fixes that as well.

Deleting the second activateSiblingTimelines is safe because nothing between
frames reads a sibling's paused(), verified by grep over the deterministic
adapters, syncTimedElementVisibility, the hf-timelines-built handler and
__hfReseekGpu. Not merely because seekStandaloneRegisteredTimelines pauses each
child. The code comment now gives that reason.

Tests, each proven non-vacuous by reverting the piece it guards:
- a leased element survives repeated paused ticks and is stopped once released
- the colour-grading preview survives, through the real init wiring
- the scrub borrows the element and gives it back on stop
- the existing test that an unleased element is still stopped keeps passing

* test(runtime): a lease ends when the transport plays or the borrower stops

Two assertions the review found not load-bearing. Dropping the isPlaying
branch so leased media stayed exempt during playback left every test
green; a leased out-of-window clip is now asserted stopped the moment the
transport plays. Deleting the release in the grading stop closure also
left the suite green, because the pause on the next line satisfied the
assertion; a restart after stop is now asserted stopped, which only the
release makes true.

* test(studio): the event-stream channel must open /api/events

* fix(studio-server): stop printing a proxy diagnostic line per clip on every render

Review of the pre-warm commit found the diagnostic louder than the thing it
diagnoses, and the two counters measuring different things under one name.

The per-asset `prewarm_requested` line is now behind
HYPERFRAMES_DEBUG_MEDIA_PROXY, matching isGpuProbeDebugEnabled in
packages/engine/src/utils/gpuEncoder.ts. A composition with fifty hostile
clips printed fifty JSON lines into a clack-formatted terminal on every
re-render. One summary line is written at process exit instead, using the
same process.on("exit") shutdown hook as packages/cli/src/cli.ts.

The counters now share a unit. prewarmsRequested counts per asset per render;
proxyRequests counted every HTTP request, including 304s. It now increments
once per resolveProxy call, after the ETag shortcut, so a revalidated repeat
no longer reads as fresh demand. An unconditional Range refill still counts,
and the docstring says so rather than claiming otherwise.

The HEVC justification was false on macOS Chrome, which answers canPlayType
for hvc1 with "probably" and keeps the source, so the pre-warm is redeemed
there only through the reactive zero-videoWidth path. prewarm stays true
because Chrome on Windows/Linux and Firefox do need the substitute; the
comment and test names now say "no cross-platform decode" instead of
"browsers never decode it".

Two test gaps closed. Nulling vp9's representativeMime left the suite green
while making the client skip canPlayType and proxy on every browser, which
would reinstate exactly the transcodes this work removed; the mimes are now
pinned. The Object.prototype test passed with the hasOwn guard deleted, so it
now goes through probeAssetCodec, the input that actually misbehaves without
it. Both fail when the change is reverted.

* perf(cli): stop gzipping the studio bundle on loopback

Compression made the cold open slower on the only transport this server has.
It binds 127.0.0.1 with no --host, and measured there the six bundle assets
took 69.8 ms with gzip against 7.9 ms raw; the 4.1 MB chunk alone was 49.4 ms
against 3.1 ms. hono/compress is removed, which also retires the Vary header
question it raised. If remote serving ever matters, compress at build time
rather than per request.

The cache policy is now decided by route instead of by filename. Reading a
content hash out of a name cannot work: rollup's alphabet is base64url and
includes a hyphen, so roughly one hashed file in ten was misread as unhashed,
and the immutable header also leaked onto hand-authored public/ files served
by the same handler. packages/studio/vite.config.ts sets neither
build.assetsDir nor publicDir, so dist/assets holds only rollup's hashed
emits and every public/ file lands at the dist root. /assets/* is therefore
immutable and /icons/* and /favicon.svg keep revalidating, with no heuristic
in between.

The shell is no-cache rather than no-store. It carries no ETag, so both force
the same full refetch, but no-store puts the document on Chrome's bfcache
blocklist: leaving Studio and pressing Back would cold-boot the app instead
of restoring it.

* docs(runtime): say what the paused-media probe actually filters

* revert(cli): drop the preview prewarm that held the CLI event loop

The fire-and-forget warm added in c9b4263c7 keeps the CLI alive until it
settles. cli.ts drains the event loop instead of calling process.exit, and
two launch paths return straight into exit, so the warm delays them: 2660 ms
with the warm suppressed, 8779 ms against a server answering in 6 s, 13293 ms
against one that hangs. The 10 s abort bounds the hold, it does not remove it.
This is the hazard already documented at preview.ts:1605-1609.

Moving the warm into the server process was the obvious fix, but measurement
says there is nothing there to warm. tsup bundles @hyperframes/core/compiler
into cli.js -- bundleToSingleHtml is a plain inline function in the artifact --
so the await import() at studioServer.ts:397 resolves an already-evaluated
namespace. A probe at listen time measures that import at 1 ms, and a paired
run shows no gain: first request 256 ms without a compiler kick, 308 ms with
one. The cold cost the plan attributed to the module load is somewhere else.

Two claims in the reverted comment were also wrong: routes/preview.ts:332 is
an ETag, not a bundle cache, and the player appends no shader params -- the
only query param is variables, which IS part of the ETag.

Reverts packages/cli to origin/main exactly.

* fix(studio): let a remount retry the player chunk after a failed load

1037456aa hoisted the dynamic import to module scope and shared one promise
across every mount. On rejection the .then never ran again: retryPreviewRef
and previewError stayed null, compositionLoading stayed true, so the preview
sat on an infinite spinner with the retry button at Player.tsx:462 unreachable,
and remounting <Player key={activeKey}> -- the recovery path that used to work
-- awaited the same rejected promise.

Memoize through a getter that clears the memo on rejection instead. The
module-scope kick still fires the chunk request before the shell's first
layout, and a remount performs a fresh import as it did before the hoist.

Discloses what the earlier commit did not: the player barrel re-exports
Player, so every studio module importing that barrel now evaluates this one
and eagerly loads the real player bundle. That is 27 DOM-env test files;
the full studio suite passes, 439 files and 4838 tests.

* docs(studio-server): trim the proxy-counter comment to the repo's four-line cap

* fix(studio): stop one failed player import poisoning every later mount

The previous commit claimed a remount could retry the player chunk after a
failed load. In a browser it cannot: the module map caches a failed fetch
as an errored entry for the document's lifetime, so a repeated import of
the same specifier rejects from cache without touching the network
(measured in headless Chrome: three attempts, one request). Clearing the
memo on rejection is still right, because it stops every later mount
awaiting the same poisoned promise, but recovery is a page reload, and the
comment now says so.

The tests said what vitest does, not what the browser does. Two assertions
survived their own mutation: removing the memoisation left the attempt
counter unchanged because vitest caches a resolved mock, and deleting the
module-scope kick, the optimisation this branch exists for, left the suite
green. Both are now asserted directly: the kick has run once at import
time before any mount, and every mount receives the identical promise.

* chore(ci): allow the deletion of the dead sdk-session reload test

* fix(runtime): the timeline resolver no longer unpauses children it cannot drive

Exercising the branch in a real browser still showed sub-compositions
free-running while paused, on a path no seek follows: the timeline
resolver, which runs on every rebind (after an edit, on the periodic bind),
unpaused every registered child BEFORE trying to nest it into the root.
A child the root actually holds is driven by the paused root and is
harmless; a child the root never takes stays on GSAP's global ticker, and
unpaused there means running. The seek fix in this branch could not
reach it, because a rebind is not a seek.

The resolver now reads back which children the root holds and unpauses
only those; the same rule applies to the composite fallback timelines.
Standalone registry children stay paused, as the transport's per-child
seek already requires.

Regression test: a hosted registry child whose root cannot nest it stays
paused across a forced rebind. Fails on the previous resolver.

* test(runtime): pin that the resolver unpauses a child only once the root holds it

The resolver's positive half had no fixture: nothing anywhere nested a
candidate for real, so computing the held set before the add loop instead
of after it left the suite green. A root that reports children only once
added now distinguishes the two orders, and the unpause is asserted at
the moment it happens, because the transport's first seek pauses every
hosted child again straight after resolution. Also drops a cast that
re-declared getChildren, which RuntimeTimelineLike already has.

* fix(runtime): the paused side stops every clip the transport drives

The paused-side enforcement scanned only media carrying its own data-start.
A clip inside a composition inherits its timing from the host and has no
data-start of its own, yet the transport plays it, so it could start while
paused and run unopposed, exactly the hole the invariant was written to
close. Reviewer probe: a hosted video with data-duration only kept
running while paused; the same element started under __player.play().

"Media the transport drives" now has one definition, shared by the media
cache and the paused-side probe, so neither can be narrower than the
other. Regression test: hosted media with no data-start started while
paused is stopped. Fails against the data-start-only scan.
2026-09-12 12:56:00 -04:00
Miguel Ángel e64a22893a fix(studio): preserve alpha proxy playback (#2625)
* fix(studio): preserve alpha proxy playback

* test(cli): pin VP8 alpha proxy pre-resolution

* fix(studio): emit textarea field-sizing CSS

* test(studio): verify textarea CSS generation
2026-07-17 21:56:27 -04:00
Miguel Ángel e8371a7acc feat(media): alpha-capable authoring proxies (#2598)
* feat(media): alpha-capable authoring proxies

Alpha sources were refused a proxy before the codec map ever asked whether the
browser could decode them, so a ProRes 4444 alpha file (which no browser
previews at all) rendered black forever, while an alpha WebM (which previews
fine) was already covered by the browser-safe check on the next line. The alpha
veto earned nothing and cost the one case that needed help.

Alpha is now a target-codec choice rather than a veto: alpha sources transcode
to VP9 + yuva420p in WebM, everything else keeps the existing H.264/MP4 path
byte for byte. Only files no browser can preview are proxied, which is the rule
the runtime already followed everywhere else.

WebM cannot carry AAC, so the VP9 path uses Opus and drops the MP4-only
faststart flag. PROXY_PARAMS_VERSION moves to v3 so clients stop serving the
previously cached proxies.

Safari does not decode VP9 alpha and still shows black for alpha sources, as it
does today: this is better on Chromium and Firefox and no worse anywhere.

* fix(media): infer proxy variant for rescue

* fix(media): preserve alpha proxy hardening after restack
2026-07-17 03:26:55 -04:00
Miguel Ángel 74b4f1e8c3 feat(cli): serve proxies from play and the static project server (#2593)
* feat(studio-server): serve H.264 proxies from the preview route

Wires the codec manifest and the transcoder into the preview surface: the route
negotiates a proxy via a query param and serves it through the existing range
and ETag machinery, composition HTML carries a codec map for the runtime, and
hostile assets pre-warm so a first play does not wait on a cold transcode.
Exposes the three subpath exports the CLI surfaces consume upstack.

Drops the TEMP fallow entry added with the transcoder: it has real importers now.

* fix(studio-server): publish media proxy exports

* fix(parsers): scan HTML comments linearly

* feat(cli): let projects opt out of automatic proxying

Adds media.autoProxy to hyperframes.json plus --proxy/--no-proxy flags, and
forwards the resolved value into the studio and preview servers and the vite
adapter. Lands before the runtime slice that turns auto-proxying on, so the
switch exists before there is any behavior to switch off.

* fix(cli): align media config schema

* feat(core): swap undecodable video to its proxy at runtime

Adds the browser-side half: before first load the runtime consults the injected
codec map and swaps a hostile source to its proxy, and if a video still reports
zero decodable width it rescues it reactively. An HEVC file carrying AAC fires
no error event, so zero videoWidth, not the error event, is the reliable signal.
Audio elements and alpha sources are never proxied, render mode never proxies,
and each swap evicts the element's stale sync state and reports once.

This completes the loop: auto-proxying is live for preview and studio from here.
The opt-out (media.autoProxy, --no-proxy) shipped in the previous slice.

* feat(cli): serve proxies from play, present, and the static project server

Adds proxy negotiation to the CLI-side servers and gives play byte-range
serving it never had, so a swapped video can seek. The static project server
behind check, snapshot, compare and friends injects the codec map once, so all
of its callers inherit the behavior; snapshot forwards its own proxy flag.

* fix(cli): serve proxies for camera formats
2026-07-16 23:02:01 -04:00
Xuanru LiandClaude Opus 4.8 906c8d04f8 fix(cli): localize remote assets before validate so it matches render (#2001)
validate served the composition over a loopback origin and let headless
Chrome fetch remote <img crossorigin>/@font-face assets cross-origin, while
the render pipeline downloads them to disk first. Buckets whose CORS
allowlist omits the loopback origin then failed the CORS-mode request with a
false net::ERR_FAILED that never occurs in the real render, pushing authors
(and agent pipelines) to delete crossorigin — which disables WebGL
color-grading/shaders for that asset.

Reuse producer's localizeRemote{Media,Image,FontFace}Sources in validate,
downloading into a temp dir served as an extra static-server asset root
(project dir untouched, cleaned up after). validate now matches render.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:20:18 -07:00
Miguel Ángel 466ee08ffa fix(cli): serve project media with HTTP Range so validate reads WAV duration (#1811)
* fix(cli): serve project media with HTTP Range so validate reads WAV duration

The local static server used by validate/snapshot/layout answered every
asset request with a plain 200 and no Accept-Ranges header. Chromium treats
such resources as non-seekable, and for WAV that makes the media element
report `.duration` as Infinity no matter how long it buffers (readyState
reaches HAVE_ENOUGH_DATA but duration never resolves). The duration audit in
validate then emitted a spurious "Could not read the duration of N media
element(s) within the validate timeout" warning for a perfectly valid local
WAV, and a longer --timeout never helped because the value is never going to
arrive. MP3/MP4 carry duration in their container metadata so they were
unaffected.

Serve files with Range support (206 + Content-Range, plus Accept-Ranges on
the full 200) so the element is seekable. WAV duration now resolves, the
false warning is gone, and the genuine "media shorter than its slot" check
works for WAV for the first time.

* perf(cli): stream Range responses instead of buffering the whole file

serveFileWithRange read the entire asset with readFileSync and then sliced
it, so a 1KB Range of a 50MB MP4 still allocated the full 50MB per request.
Switch to statSync for the total size and createReadStream(filePath, { start,
end }) piped to the response, reading only the requested window. Behavior is
unchanged: 206 + Content-Range + Content-Length for a satisfiable range, 416
for an unsatisfiable one, Accept-Ranges advertised on every response, and a
plain 200 full-body stream when there is no Range header. writeHead is
deferred to the stream's open event so a failed open still answers 500, and
the fd closes on end/error. This benefits MP4 seek too, not just WAV duration.

Extend staticProjectServer.test.ts with an 8MB-file case that pulls a 4-byte
slice from deep inside and asserts the streamed bytes, Content-Range, and
Content-Length are correct.
2026-06-30 10:43:20 -07:00