* feat(studio): expose Studio's live state to an agentic browser
Registers a `studio_look` tool on `document.modelContext`, so an agent in a
browser that supports it can read what Studio knows: the open project and
composition, the playhead, the human's current selection with its
capabilities, and the timeline's elements with a handle for each.
The API is `document.modelContext`, not `navigator.modelContext`. The latter
is a polyfill compatibility shim rather than a spec member, so feature
detecting it is wrong even where a published sample appears to work.
Three decisions worth knowing:
Registration happens ONCE per mount, with the dependencies held in a ref that
every render refreshes. Depending on the handlers instead re-runs on nearly
every interaction, because the DomEdit actions object changes identity with
the selection and the element list. Each re-run aborts the registration signal
and unregisters everything, and the spec warns that a quick unregister-then-
reregister can apply an old call's arguments against the new schema. The test
for this is the important one in the unit; breaking the empty dependency array
fails it and nothing else.
Tools resolve with a tagged result, they never reject. That is forced by the
spec: a rejected `execute` has its reason discarded and the caller sees a bare
UnknownError, so rejecting would guarantee the agent cannot learn why an edit
failed.
Elements are addressed by a minted handle, not by `TimelineElement.id`. That
id is a synthesised identity, so `getElementById` misses most elements; the
handle carries `data-hf-id`, else the DOM id, else a selector plus occurrence.
Mounted from `EditorShell` rather than `App`, because the DomEdit contexts are
only readable below `DomEditProvider` and `App.tsx` is three lines under the
600-line cap.
The undo signal is reported as the shell actually exposes it, `canUndo` and a
label, rather than as a revision counter. The depth lives in component-local
state and is not reachable without plumbing it through the shell context, so
the field says what it is instead of implying precision it does not have.
Writes are not in this change. `canWrite` is optimistic and the comment says
so; the write tools need a real guard against the paused-save and external-
conflict states, which are not on any context this component can reach yet.
* fix(studio): bound WebMCP look filters
* fix(studio): remove premature WebMCP write state
* docs(studio): name WebMCP singleton assumption
* fix(studio): surface WebMCP registration failures
* perf(studio): define timeline viewport budgets and fixtures
* test(studio): gate timeline viewport performance in Chromium
* refactor(studio): isolate clip drag lifecycle
* refactor(studio): extract timeline render contracts
* perf(studio): centralize timeline viewport geometry
* perf(studio): follow playhead across virtualized rows
* perf(studio): add timeline clip-window index primitive
* perf(studio): virtualize timeline clip windows
* perf(studio): stop timeline scroll work when row virtualization is off
The row virtualization stack made the timeline publish a viewport snapshot
on every scroll frame and swap `renderClipContent` across every mounted clip
at gesture start and settle. Both are windowing concessions, and neither was
gated on the flag, so the build users actually run paid for them while
mounting all 1,000 clips anyway. Measured on a 3,000-clip project: median
scroll step 16.6ms to 76.9ms, p95 17.9ms to 189.4ms, 40 long tasks to 247.
Gate both on the row virtualization flag. The scroll path now stops at the
door when the flag is off, so `isScrolling` stays false and resize-driven
and programmatic syncs still publish through the immediate path.
The flag moves into its own module: the scroll-viewport hook needs to read
it, and the virtualization hook already imports the viewport snapshot type
back, which would have closed an import cycle.
Also release the perf fixture lease from the fixture rather than from the
test-hook effect. Loading a fixture writes player state, which changed that
effect's dependency identities and tore it down on the next frame, so the
lease was revoked moments after it was taken and live iframe discovery
overwrote the fixture before the gate could measure it.
The e2e gate gains a flag-off arm (`test:timeline-default`, 1,000 elements)
next to the existing flag-on one. It refuses the 50,000-element combination,
verifies from the mounted DOM that the server under test matches the
requested flag, and skips the DOM-size budgets for the unvirtualized build
rather than relaxing them, so a skipped budget never reads as a passed one.
Verified against a live Studio dev server on the fixture project:
flag off, before: interactionP95 303.1ms, longest task 194ms, 0/5 runs pass
flag off, after: interactionP95 33.6ms, longest task 0ms, 5/5 runs pass
flag on, after: interactionP95 33.2ms, 4/5 runs pass, exit 0
The flag-on arm's fourth run reproducibly reports a 55-58ms long task
against a 50ms budget. That is the residual tail of the window swap itself,
tracked separately and not addressed here.
* ci(studio): run the timeline viewport gate on studio changes
The gate has existed since the row virtualization stack landed but nothing
under `.github/` referenced it, so it only ever ran when someone ran it by
hand. That is how the flag-off scroll regression reached eight merged-ready
PRs without anything noticing.
Adds a `studio-timeline-viewport` job that boots two Studio dev servers, one
per flag state, and runs both arms of the gate against them. Two servers are
needed because row virtualization is read from `import.meta.env` at module
load, so one process cannot serve both builds.
Scoped to a new `studio` paths filter rather than the broad `code` one: the
gate only says anything about `packages/studio`, `packages/core` and
`packages/studio-server`.
Adds a `ci` tier. It applies the constrained budgets without any emulation,
because a hosted runner is already slower and noisier than the machine the
strict numbers were recorded on, while the existing `low-resource` tier would
throttle it a further 4x and measure the throttle rather than the build.
The fixture composition is tracked under `tests/e2e/fixtures` but Studio
resolves projects from the gitignored `data/projects`, so the job copies it
into place instead of a project directory being committed.
Both arms run in about 7 seconds each locally, so the job cost is almost
entirely dependency install and the workspace build it shares with
`studio-load-smoke`.
* fix(ci): preserve both timeline gate evidence arms
* ci(studio): report timeline gate arm statuses
* ci(studio): require timeline gate evidence artifacts
* fix(studio): keep dense keyframes readable
* fix(ci): resolve timeline stack audit findings
What: extends three leaf modules to their final NLE-stack form, tests in the
same change: timelineInspector (isAudioTimelineElement, resolveBeatSourceTrack),
timelineZoom (zoom/pps math incl. computePinnedZoomPercent), and
studioUiPreferences (persisted editor prefs).
Why: leaf dependencies of the NLE timeline stack; landing them first keeps
the later glue PRs to wiring.
How: additive from the consumer side — every export main already uses is
unchanged (typecheck against main's consumers passes untouched); every new
export is exercised by a test in this PR.
Test plan: bunx vitest run on the three test files; tsc --noEmit in
packages/studio; fallow audit --base origin/main clean.
Adds a Blocks tab to the Studio left sidebar with the full 78-item registry
catalog (58 blocks + 20 components). Users can browse by category, search by
title/description, preview CDN-hosted poster thumbnails with video-on-hover,
and install items on-demand with one click or drag-to-timeline.
Core changes:
- BlockCategory type + resolveBlockCategory() for 7 categories (Captions, VFX,
Transitions, Effects, Social, Data, Scenes)
- Registry API routes: GET /api/registry/blocks (catalog) + POST install
- StudioApiAdapter extended with listRegistryCatalog + installRegistryBlock
- Vite adapter reads from disk; CLI adapter fetches from GitHub (24h cache)
- BlockParam interface + params on 6 blocks for future parameter controls
Studio UI:
- 4th sidebar tab "Blocks" with responsive grid, category pills, search bar
- BlockCard: CDN poster thumbnail, video autoplay on hover, duration + WebGL badges
- On-demand install: blocks append as sub-compositions on timeline; components
overlay at start=0 spanning full duration with transparent background patching
- TIMELINE_BLOCK_MIME drag-and-drop to timeline
- BlockParamsPanel (Phase 3 scaffold) auto-opens for parameterized blocks
Registry manifests:
- All 58 blocks backfilled with preview: { video, poster } CDN URLs
- All 20 components normalized to object format + poster URLs added
- 6 blocks annotated with params (Liquid Glass/Background, Portal, Chart,
Logo Outro, Magnetic)
- flowchart-vertical preview generated and uploaded to CDN
* fix(studio): add smooth preview zoom with pinch/Ctrl+scroll
- Scale iframe content from inside (contentDocument.documentElement) instead
of scaling the parent div, avoiding compositor re-rasterization on every
zoom frame — critical for smooth zoom on high-refresh displays (240Hz)
- Document-level capture-phase wheel handler bypasses the DomEditOverlay
- Center-based zoom (no pan drift from pointer-anchored formulas)
- Transient HUD shows zoom % briefly, no persistent UI controls
- Double-click preview area to reset zoom to fit
- Drag-to-pan when zoomed past 100%
- Momentum scroll suppression after pinch gesture (400ms cooldown)
- Delta clamping (MAX_DELTA=10) prevents overshooting on fast gestures
- toDomPrecision rounds transform values to 4 decimals (matches tldraw)
- Zoom state persisted to localStorage with 200ms debounce
- Exposes --preview-zoom CSS custom property for overlay coordinate mapping
- Fix infinite render loop in NLELayout (onIframeRef → refreshPreviewDocumentVersion)
* fix(studio): use CSS zoom instead of transform scale for preview zoom
CSS transform: scale() on a div containing an iframe causes compositor
cross-layer sync issues that produce visible frame tearing on high-refresh
displays (240Hz ProMotion). CSS zoom property changes the actual rendered
size without compositor layer synchronization, eliminating the jumping.
- Replace transform: scale(Z) with zoom: Z on the stage div
- Keep transform: translate() for panning (compositor-friendly, no iframe)
- Overlays work correctly since getBoundingClientRect() includes zoom
- Remove will-change, transition hacks, pointer-events toggles
* fix(ci): use apt-get for ffmpeg in preview-regression workflow
The FedericoCarboni/setup-ffmpeg action downloads from an external URL
that has been persistently unreachable, causing CI failures. Switch to
apt-get install which uses Ubuntu's package repos (same as ci.yml and
player-perf.yml).
* fix(studio): clear zoom timers on NLEPreview unmount
settleTimerRef, hudTimerRef, and retiringTimerRef could fire after
component unmount. Add cleanup effect to prevent stale callbacks.
* feat(studio): persist sidebar, timeline, and playback speed across reloads
Wire up studioUiPreferences for the three remaining UI states requested
in #752: left sidebar collapsed, timeline visibility, and playback rate.
All three now survive page reloads using the same localStorage key as
preview zoom.
Miguel Ángel