mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-19 11:26:02 +00:00
c2ca2291cab668d8ffca15021e3843494c8bdd9f
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c2ca2291ca |
fix(studio): dropping an asset between or above timeline rows opens a new row (#4160)
* fix(studio): dropping an asset between or above timeline lanes opens a new track there * refactor(studio): reuse the source patcher and shared drop-placement type for track inserts * fix(studio): the dropped clip must carry its peers' sourceFile, not the resolved path planDropTrackInsert stamped the synthetic dropped element with the resolved target path string, but real elements on the main document carry no sourceFile at all. sameSourceFile compared the two directly, so the dropped element always read as belonging to a different file and every peer was skipped as unwritable, silently landing the drop on an existing track instead of opening one. * refactor(studio): trim drop-track comments and share one can-insert-track predicate Drop the TimelinePlacement alias, compute the file-or-asset check once for dragover and drop, and shorten two comments that narrated history. * fix(studio): plan a dropped-in track against the visible rows, not clips the timeline hides The insert row counts the rows the timeline shows, but the plan was built from every same-file clip. A clip the timeline does not show as its own row shifted the boundary and renumbered lanes it should have left alone. * test(studio): assert the dropped clip lands on the opened lane * fix(studio): a row drop measures its boundary on the same row order a clip drag does The drop path indexed the insert row against the rows the timeline shows, which include audio-group anchor rows, but planned it against a plain sorted track list. An audio group above the boundary shifted the new row by one. The placement now carries the displayed row order and the planner uses it, as the clip drag already does. |
||
|
|
42f947f4c3 |
fix(studio): select and reveal a timeline clip after a drop (#4067)
* fix(studio): select and reveal a timeline clip after it's dropped Dropping an asset, block, or composition onto the timeline placed the clip but never selected it, so the user had to go find it. Wire the same select+reveal pair AssetCard/AudioRow already use into all three drop paths via a shared selectAndRevealTimelineElement helper. Root-caused a shared bug along the way: useTimelineSelectionPreviewSync compared the resolved selection count against only the ids already present in timelineElements, not against every id actually wanted. A freshly-selected id not yet in timelineElements (a just-dropped, pasted, or duplicated clip) made that comparison pass on zero resolved out of zero expected, falling through into a loud selection clear instead of the intended quiet retry. Fixed the comparison and covered drop, paste, and duplicate-of-a-group. * refactor(studio): drop the dead fallback in the new store-key helper Simplify-pass finding: both new selectAndRevealTimelineElement call sites always pass a DOM id they just minted, so the `?? fallback` after deriveTimelineStoreKey (which only returns undefined for a domId-less, selector-less caller) could never run, and the pattern was duplicated at both call sites. Added a non-optional variant for callers that already hold a guaranteed DOM id, and dropped the now-redundant "old code" narration from a test comment. * fix(studio): cap the selection-sync retry so a dead member cannot stall the group The bail-and-retry added for the drop/paste/duplicate race had no upper bound: a group member that can never resolve (deleted out of band, a stale id) made the whole group's selection retry forever instead of settling. Cap it at a few attempts, then apply whatever did resolve, same as the pre-fix graceful degradation, only bounded now instead of unconditional on every sync. * fix(studio): select and reveal a block dropped onto the preview canvas handlePreviewBlockDrop (a real drag-and-drop onto the preview pane, wired from usePreviewBlockDrop) was the fourth caller of addBlockToProject's hostKey but never selected the new element, unlike the other three drop paths this change wires up. * fix(studio): close a stale retry-budget gap and dedupe the store-key helpers Post-rebase simplify pass and independent review against the new head: deriveTimelineStoreKeyForDomId now calls deriveTimelineStoreKey instead of reimplementing its body, the composition-insert path uses the same helper instead of hand-building the sourceFile#domId format, and the selection-sync retry counter resets on deselect so reselecting an already-exhausted, permanently-unresolvable id gets a fresh grace window instead of degrading on the first attempt. Adds coverage for the asset-drop selection wiring and the deselect/reselect retry-budget reset. * style(studio): shorten a retry-budget docstring past the comment-line cap * fix(studio): quiet the just-dropped-clip toast and reveal the full clip The selection-not-found toast fired on the routine first miss while a freshly dropped or pasted clip was still resolving; it now waits until the retry budget is exhausted. The timeline focus coordinator likewise cleared a reveal request on its first miss before the dropped element had landed in logicalRows, cutting the clip off at the viewport edge; it now tolerates a bounded number of retries like its selection-sync sibling, so AssetCard and AudioRow reveals benefit too. * refactor(studio): share one retry-budget tracker between the two hooks The selection-sync and timeline-focus retry counters were the same shape, incremented and reset independently. Extracted recordRetryAttempt so both hooks call one tracker instead of maintaining their own copy. * fix(studio): stop the post-edit scroll restore from undoing a fresh reveal A dropped clip's reveal could still land cut off at the viewport edge even with the retry fix: Timeline's "restore scroll after an edit re-derives the elements" effect and the focus coordinator's reveal both react to the same element-list change, but the restore effect captured lastScrollLeftRef before the reveal had a chance to update it, so its deferred frame stomped the reveal's scroll position back to the stale pre-drop value. The reveal now keeps lastScrollLeftRef in sync with its own write, and the restore effect reads the ref inside its frame instead of capturing it early, so whichever effect runs last within the commit wins with the freshest value. Confirmed against the original repro (a Playwright drop scripted through the real dev server) that scrollLeft converges to the fully-revealed value and stays there. * docs(studio): note the load-bearing hook order behind the scroll-race fix Flagged by an independent review of the prior commit: nothing in the suite would catch a future refactor that reorders these two hook calls and reintroduces the stale-scroll race. * fix(studio): bound an unresolved focus request by wall time, not just retries Answers two review questions on the scroll-race fix. (1) The retry budget only advances when this effect re-runs, which needs some other dependency to change; a request whose target never appears and which nothing else ever re-renders sat unresolved forever, staying eligible to resolve against an unrelated element that later reused its id. A setTimeout backstop now clears it after 4s regardless of re-renders, independent of the retry-count path. New test proves the leak and the fix (fails without the timeout, passes with it). (2) The "hook order is load-bearing" comment in Timeline.tsx no longer holds once the geometry effect's ref read is deferred into its own frame: a new test co-renders both hooks in both registration orders and both converge, so the order claim is false for the current code. Deleted the comment; the same test fails in geometry-first order when run against the pre-deferral code, confirming order mattered before and does not now. * refactor(studio): drop a nonce check the store already performs Simplify pass on the wall-time backstop commit. The timeout callback guarded clearTimelineFocus behind a nonce match, but the store action already no-ops on a mismatched nonce (a zustand same-reference return short-circuits it) — the guard duplicated a check the setter owns, and was inconsistent with the sibling budget-exhausted branch two lines up that already calls it unguarded. Confirmed no other findings: the two-mechanism clearing (retry count plus wall time) is genuine, not redundant, and the timer has no leak path. |
||
|
|
bf39a72787 |
fix(studio): dropped timeline assets land where you drop them (#4062)
* fix(studio): dropped timeline assets land where you drop them Dropping an asset, an OS file, or a block onto the timeline always placed the new clip at the current playhead time, discarding the actual drop position; only the track (vertical position) tracked the pointer. Composition drops already used the correct pointer position. Unify all drop kinds onto that same, correct placement path, and drop the now-pointless duration clamp so a drop past the end of the timeline extends it, matching what composition drops already did. * fix(studio): grow the timeline when a drop lands past its end Asset/file and composition drops could place a clip past the root composition's duration without growing it, so the editor showed the clip fine but the render truncated it. Every sibling edit path (block install, move, resize) already grows the duration on the same rule; these two didn't. Also restores the pointer-placement decision's provenance in the drop docblock and PR description, and marks the now-production-dead clamp branch in resolveTimelineAssetDrop so the next reader doesn't assume the default is live. * fix(studio): revert the composition-drop duration extension The composition-drop path already grows the root composition's duration server-side (compositionInsertion.ts, before the response is even returned): this was verified wrong via two independent review passes after being merged in by mistake. The client-side extend added on top of that was redundant, and in edge cases could re-apply a slightly different rounding than the server's own. * fix(studio): remove the dead clampStartToDuration option resolveTimelineAssetDrop had one production caller and it always passed clampStartToDuration: false; the clamped branch and the duration field that only fed it were exercised solely by this file's own tests, not by anything shipping. Drop the option, the branch, and duration/durationRef everywhere they were only feeding it (timelineDragDrop.ts, Timeline.tsx, and the touched test fixtures). No test asserted a drop past duration relying on clamping, so this is behavior-neutral for every existing test. |
||
|
|
4c6fa9a790 |
fix(studio): the preview stays paused, survives typing, and opens without a re-encode (#3910)
* chore(studio): remove the write-only self-write timestamp ref
`domEditSaveTimestampRef` was assigned at 26 sites across 54 files and read
nowhere. It used to feed a 2 s "suppress the watcher reload after our own
write" window; that mechanism was replaced by content-hash identity in
sdkSelfWriteRegistry, whose header still says why (a clock cannot tell an
SDK self-write echo from an undo landing in the same window). The reader
went with that change, the writers did not, and three comments kept
describing the timestamp as protection that no longer existed.
Pure deletion: the ref, every prop and parameter that threaded it, every
assignment, the five comments citing it, and the orphaned test helpers.
No behaviour change; the studio suite, typecheck, lint and format are
green.
Not removed: the __hfSuppressSceneMutations wrapper in gsapSoftReload.ts.
It looked undefined from inside the studio package, but shader-transitions
installs it on the preview window (hyper-shader.ts) so soft reloads do not
invalidate cached transitions. It is live.
* docs(studio): plan for loading the preview during the shell's first layout
* fix(runtime): a paused preview stays still after any seek
Sub-compositions kept animating while the transport was paused and the
Studio button showed play. A render-seek unpauses every sibling timeline so
GSAP propagates the root's totalTime into them, and nothing paused them
again; parented to the global ticker, they free-ran at 1x while the master
timeline and the clock stayed stopped. The Studio reaches that seek path
through its seek-driven fallback adapter right after a preview reload,
which is why editing text was what set it off. Captured live in the user's
preview: seven child timelines advancing 0.210s per 200ms sample, all
gsapPaused=false, transport not playing.
Three fixes under one invariant: while the clock is paused nothing runs,
and the button never disagrees with the runtime.
- The sibling rearm is a lease, not a resting state: seekTimelineAndAdapters
now returns every timeline it unpaused to paused in a finally. The second
rearm after the child re-seek only ever changed the leaked state and is
gone, with the comment that justified the leak by describing one caller.
- The paused side of the transport tick policed nothing; it now stops any
timed media element found running. A parked transport runs no ticks, so a
capture-phase play listener wakes it.
- Studio's canvas click resumed by setting the store flag alone; it now
requests playback through the player so adapter, rAF loop and flag agree.
Regression tests fail without each of the three changes.
* perf(studio-server): stop re-encoding videos the browser already plays on open
Opening a project kicked off a background transcode for every asset whose
codec some browser might not decode. VP9 is on that list for Safari's sake,
so a Chrome user opening a project with a VP9 avatar paid a ~10s, ~60
CPU-second re-encode across six cores on every open, concurrently with the
browser's first layout, for an output nothing ever requested: across four
recorded sessions the browser asked for the proxy zero times.
One predicate was doing two jobs. "Could some browser fail on this" is a
property of the asset and decides what gets injected into the page;
"will this client request the substitute" decides whether to spend CPU
before being asked. The codec table now carries an explicit prewarm flag:
HEVC and ProRes (no browser decodes them) still warm; VP9 and AV1 are
injected but transcoded lazily by the existing ?hf-proxy= request, which
already reports failure as a 502.
Pre-warms requested and proxies served are now counted on the existing
structured-stderr telemetry channel, because a speculative job with a 0%
hit rate emits no error and had been invisible. A single lookup helper
also closes a prototype-key gap where one path used Object.hasOwn and the
other a bare index.
Regression test: a VP9 and an HEVC asset in one composition yield exactly
one pre-warm, for the HEVC. Fails on the previous gate.
* perf(cli): serve the studio bundle compressed and cache hashed assets
The 4.1MB studio bundle was served uncompressed with Cache-Control:
no-store, so every open of the studio re-downloaded and re-parsed it. Vite
names built assets with an 8-character content hash, so their bytes can
never change: those now ship gzip-compressed (1.25MB on the wire,
byte-identical after inflation) with a one-year immutable cache policy.
Unhashed files under public/ keep no-store, and the HTML shell, which
previously sent no Cache-Control at all, now sends no-store explicitly:
it is the only thing that names the current hashed bundle, and it must
keep revalidating for the immutable policy to be safe.
The hash test only accepts the segment after the last hyphen and requires
a digit, underscore or capital, so an ordinary hyphenated name like
user-Guide-v2.js is not mistaken for a hash and served forever.
* perf(cli): warm the preview route before the browser opens
The browser's first request for a project's preview paid the server's cold
compiler import and first bundle in the request path, after the studio
shell's own first-layout stall, so both costs landed on the user's
time-to-first-frame in series. The CLI now issues one fire-and-forget
request for the preview route from openStudioBrowser, the single funnel
for every launch path, before the --no-open early return so pasted URLs
benefit too. Measured on the demo project: first client request 231ms
before, 101ms after; the ETag 304 path afterwards is under 1ms.
The fetch carries a 10s abort so an unsettled connection cannot keep an
otherwise-finished CLI process alive, matching the package's existing
convention. The shader query params the player appends are not part of
the route's cache key, so the plain route URL warms the same entry.
* perf(studio): request the player chunk before the shell's first layout
The dynamic import of @hyperframes/player ran inside the preview's mount
effect, which React schedules after the shell's first layout. On a cold
open that layout stalls the main thread for seconds, so the chunk request
waited behind it for no reason. The import is now kicked at module scope,
behind a typeof window guard that preserves the documented SSR contract
(the module registers a custom element at load), and the mount effect
awaits the already-in-flight promise. Verified in the built bundle: the
preload call sits at module top level, so the request goes out on bundle
evaluation.
* fix(studio): editing a text property no longer reloads the preview
Every keystroke in the Design panel writes the composition file, the file
watcher announces the change to the studio, and the studio decides whether
the change was its own. Over the CLI's event stream that decision has
never worked: the browser hands the handler a MessageEvent whose data is
a JSON string, and three of the four payload readers (version, write
token, content) only understood an already-parsed object, so they read
every field as absent. An absent token means "someone else edited the
file", and the studio hard-reloaded the preview iframe on its own edit,
blanking it for seconds. The path reader alone knew how to unwrap the
string, which is why the event was recognised well enough to reload and
never well enough to suppress.
The envelope is now decoded once, at the boundary, by one function that
all three transports feed; the readers share one field accessor so they
cannot diverge again. The production event-stream rung is extracted into
an exported channel so a test can drive a real MessageEvent through the
listener it registers, which was impossible before because vitest defines
import.meta.hot and the selection never reached that rung under test.
A second, smaller cause: the server attached the write receipt to the
first subscriber only and deleted it on read, so any other listener saw
an unlabelled change. Reads are now non-destructive with the TTL as the
only eviction, scanning newest-first because identical bytes written
twice inside the TTL (undo, retyping a value) share a version and the
older token was already spent. The file version now ships with every
event, receipt or not, so duplicate deliveries of one change dedupe
instead of reloading once each.
shouldReloadSdkSession had no production callers and a signature that
invited an undecoded delivery straight back into this bug; it is removed.
consumeFileWriteReceipt stays as a deprecated alias for one release.
Regression tests: a Studio write delivered as a real SSE MessageEvent is
suppressed; two subscribers of one watcher event reload once; a genuinely
external write still reloads; a repeat of earlier bytes gets the newest
token; a receipt past the TTL is not recognised. Each fails on the code
before it.
* fix(runtime): paused-time media playback is borrowed, not banned
The paused-side enforcement added in the previous commit had no notion of
provenance, so it stopped two features that legitimately play media while the
transport clock is paused. Both were deterministic, not racy: the capture-phase
`play` listener means the very play() that starts them wakes the transport that
stops them.
- The colour-grading preview (colorGrading.ts startPreviewPlayback) plays a
video while paused to render grading previews. It went dark on the first
paused tick.
- The Studio's scrub audition (timelineIframeHelpers.ts applyScrub) plays the
music track for ~140 ms while paused so a playhead drag is audible. Same path
killed it.
A runtime-owned lease fixes both without weakening the enforcement. One owner: a
WeakSet in the runtime closure, with lease/release published on the existing
window.__hf surface for the Studio, which reaches the element across the iframe
boundary and cannot call into the closure. The grading runtime is constructed
with the pair directly. Both the cheap probe and the sync path skip leased
elements while the clock is paused; during playback the transport owns everything
again. Anything that plays while paused without a lease is, by definition, the
defect the enforcement exists for, and is still stopped.
Two corrections to the previous commit's reasoning:
The old leak broke the render path too, deterministically, not only the preview.
packages/producer/src/services/fileServer.ts:375-379 seekToTime flushes the
virtualized rAF queue, and GSAP's global ticker with it, after renderSeek and
before the frame screenshot (fileServer.ts:657-664, hf.seek). A sibling left
unpaused therefore advanced by the full inter-frame delta into the captured
frame. The finally added in the previous commit fixes that as well.
Deleting the second activateSiblingTimelines is safe because nothing between
frames reads a sibling's paused(), verified by grep over the deterministic
adapters, syncTimedElementVisibility, the hf-timelines-built handler and
__hfReseekGpu. Not merely because seekStandaloneRegisteredTimelines pauses each
child. The code comment now gives that reason.
Tests, each proven non-vacuous by reverting the piece it guards:
- a leased element survives repeated paused ticks and is stopped once released
- the colour-grading preview survives, through the real init wiring
- the scrub borrows the element and gives it back on stop
- the existing test that an unleased element is still stopped keeps passing
* test(runtime): a lease ends when the transport plays or the borrower stops
Two assertions the review found not load-bearing. Dropping the isPlaying
branch so leased media stayed exempt during playback left every test
green; a leased out-of-window clip is now asserted stopped the moment the
transport plays. Deleting the release in the grading stop closure also
left the suite green, because the pause on the next line satisfied the
assertion; a restart after stop is now asserted stopped, which only the
release makes true.
* test(studio): the event-stream channel must open /api/events
* fix(studio-server): stop printing a proxy diagnostic line per clip on every render
Review of the pre-warm commit found the diagnostic louder than the thing it
diagnoses, and the two counters measuring different things under one name.
The per-asset `prewarm_requested` line is now behind
HYPERFRAMES_DEBUG_MEDIA_PROXY, matching isGpuProbeDebugEnabled in
packages/engine/src/utils/gpuEncoder.ts. A composition with fifty hostile
clips printed fifty JSON lines into a clack-formatted terminal on every
re-render. One summary line is written at process exit instead, using the
same process.on("exit") shutdown hook as packages/cli/src/cli.ts.
The counters now share a unit. prewarmsRequested counts per asset per render;
proxyRequests counted every HTTP request, including 304s. It now increments
once per resolveProxy call, after the ETag shortcut, so a revalidated repeat
no longer reads as fresh demand. An unconditional Range refill still counts,
and the docstring says so rather than claiming otherwise.
The HEVC justification was false on macOS Chrome, which answers canPlayType
for hvc1 with "probably" and keeps the source, so the pre-warm is redeemed
there only through the reactive zero-videoWidth path. prewarm stays true
because Chrome on Windows/Linux and Firefox do need the substitute; the
comment and test names now say "no cross-platform decode" instead of
"browsers never decode it".
Two test gaps closed. Nulling vp9's representativeMime left the suite green
while making the client skip canPlayType and proxy on every browser, which
would reinstate exactly the transcodes this work removed; the mimes are now
pinned. The Object.prototype test passed with the hasOwn guard deleted, so it
now goes through probeAssetCodec, the input that actually misbehaves without
it. Both fail when the change is reverted.
* perf(cli): stop gzipping the studio bundle on loopback
Compression made the cold open slower on the only transport this server has.
It binds 127.0.0.1 with no --host, and measured there the six bundle assets
took 69.8 ms with gzip against 7.9 ms raw; the 4.1 MB chunk alone was 49.4 ms
against 3.1 ms. hono/compress is removed, which also retires the Vary header
question it raised. If remote serving ever matters, compress at build time
rather than per request.
The cache policy is now decided by route instead of by filename. Reading a
content hash out of a name cannot work: rollup's alphabet is base64url and
includes a hyphen, so roughly one hashed file in ten was misread as unhashed,
and the immutable header also leaked onto hand-authored public/ files served
by the same handler. packages/studio/vite.config.ts sets neither
build.assetsDir nor publicDir, so dist/assets holds only rollup's hashed
emits and every public/ file lands at the dist root. /assets/* is therefore
immutable and /icons/* and /favicon.svg keep revalidating, with no heuristic
in between.
The shell is no-cache rather than no-store. It carries no ETag, so both force
the same full refetch, but no-store puts the document on Chrome's bfcache
blocklist: leaving Studio and pressing Back would cold-boot the app instead
of restoring it.
* docs(runtime): say what the paused-media probe actually filters
* revert(cli): drop the preview prewarm that held the CLI event loop
The fire-and-forget warm added in
|
||
|
|
2b65b4efce |
fix(studio): harden composition timeline reliability (#2615)
* fix(studio): preserve composition playback continuity * feat(studio): drag compositions into the timeline * fix(studio): collapse expanded composition move aliases * fix(studio): make timeline cuts atomic * fix(studio): group inspector gesture history * test(studio): cover masked text selection * fix(studio): harden composition timeline reliability * fix(studio): satisfy CI source gates * fix(studio): harden composition mutation requests |
||
|
|
a33b3f35e1 |
fix(studio): address PR #2347 review findings (rounds 1-2)
Review 1 (restore commit): - asset reveal now clears any open preview overlay (stuck-overlay repro: preview on A, click already-added B — A stayed open over the reveal) - duration readout rolls back on failed persist: captureDurationRollback snapshots store + live root data-duration before the optimistic sync and restores both in every move/resize/delete/group catch (golden's previousDuration pattern) - asset preview opened during running playback dismisses immediately (the RAF loop bypasses the store, so the subscription alone never fired) - persistTimelineBatchEdit resolves the target (findTagByTarget) before treating identical output as a no-op — a mistargeted member now throws like the single-element path instead of being silently dropped - a post-mutation history-fold failure no longer suppresses the preview sync: fold errors are surfaced separately and the rewritten script still syncs (previously the preview kept stale GSAP positions with no recovery) - timelineRevealScroll guards degenerate viewports (windowSize <= 0) - CodeQL: encodeURIComponent(projectId) at all timelineTimingSync fetches Review 2 (single-source-of-truth pass): - createTimelineElementFromManifestClip — the one manifest->element boundary — now carries authoredTrack and stackingContextId; expanded sub-comp children preserve both (authoredTrack in their OWN file's space) - authoredTrackForLane scopes occupants to the dragged clip's sourceFile (a foreign file's authored values are a different coordinate space); nearest-same-file-lane offset fallback - optimistic store updates mirror the persisted track into authoredTrack (and roll it back on failure), so consecutive drags before a reload resolve from fresh data - spill sub-lanes: documented decision — dropping onto a spill lane is a legitimate same-track join (occupants share the authored track by construction); false 'never a lane-move target' docstring rewritten - single-element fallback persists vertical-only moves (early return now requires neither start nor track changed; live DOM patch includes data-track-index) - canonical contextKey helper for stacking-context normalization - new pipeline test crosses the REAL factory boundary (sparse authored tracks -> factory -> expansion -> normalize -> drag commit -> persisted attribute), no injected fields |