* perf(studio): settle the edit accuracy bench on frames and network, not a fixed window
settled() waited for the box and the preview frames to hold still for 500 ms,
7 to 15 times a case. It now resolves once they read the same on 3 animation
frames in a row while the Studio page has no request in flight (the event
stream and server-rendered timeline thumbnails excepted) and the server has
announced every project file written since its last file-change event, so a
save, a preview reload or a reload the event starts still holds it open.
EDIT_ACCURACY_STILL_MS restores the fixed window with no network signal.
Also polls studio_seek every 50 ms instead of 250 ms while Studio opens.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* perf(studio): keep one producer Chrome open across edit accuracy renders
Every render launched and closed its own producer Chrome. The run now holds a
lease on the engine's pooled browser (a capture session with the renders'
launch settings and its page closed), so each render opens a page in that
browser instead. Each render still gets its own page and file server, and
clears the browser cache first. A held browser that failed to launch or whose
Chrome died is dropped and held again, never reused.
Studio's server stays per case: it serves one project folder, stamps it on
start and keeps that folder's undo history open, so switching folders under it
would leak history between cases. Its start is polled every 50 ms instead of
200 ms.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(studio): import render.mjs at the top of the edit accuracy render test
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(studio): restart the settle window when a busy request's frame goes away
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* style(studio): format the edit accuracy settle test
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(studio): seek the edit bench only after Studio binds the timeline
openStudio now polls every 50 ms, so after a reload its first studio_seek
often lands before the runtime binds the timeline (Studio still reports
duration 0, but the playhead moves). The bind then replays the playhead from
near 0 (core init.ts progress(0.0001) then totalTime). A tween that starts on
the playhead was already initialised by the early seek, so GSAP skips its
first keyframe. The preview then holds a box no fresh seek or render shows.
In resize-fromto-px-r0-nested-z100-on that box is 90 px off, and it persists
until the next seek.
Wait for studio_look to report a duration before seeking.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(studio): bench moves pressed right after Add keyframe
* fix(studio): an undo that reloads the preview keeps the selection, so a drag after it moves
* fix(studio): a drag started while a resize saves keeps its animations paused
* fix(parsers): an edit pins no position on a keyframed tween it did not touch
* fix(studio): a move pressed while the preview changes lands on the new preview, at the time pressed
* test(studio): bench waiting presses, a held box resized twice, and reloads at the last playhead
* refactor(studio): tidy the waiting press, bound its wait, and freeze edit stamps after early exits
* test(studio): load the property panel once so its first test stops timing out
* fix(studio): presses queued on a held box keep their release, hover and newest outline
* test(studio): judge bench undo by the history groups Studio claims, not by file writes
* refactor(studio): one waiting-box lookup in the bench and one raf-pause write per release
Behaviour-identical tidy of the waiting-press work: the bench reads the waiting
box through one helper, releaseActiveGesture sets the raf pause once, clearPointerState
writes the false it always computed, and previewGestureStarts calls the opts setters
directly instead of through aliases named like the gesture hook's gated setters.
* fix(studio): a queued press lands on the element it pressed and survives a window switch
* fix(studio): undo cancels a waiting press, a stuck save stops holding presses, scene swaps hold them
* refactor(studio): keep the complexity waiver on clearPointerState itself
* fix(studio): undo of a queued press leaves nothing waiting, and hosts keep their own undo
* fix(studio): a waiting press keeps the box still after a blur, and undo reports what it took back
* style(studio): bring the PR's comments within the comment limits
* fix(studio): bank the four bench cases this fixes and meet the comment, complexity and duplication gates
* test(studio): pin that a waiting press is never a pending edit, and name the new duplication waivers
* test(studio): the edit accuracy bench uses the port its server bound
A busy port made the CLI bind the next free one, and the bench then failed
the case with "studio moved from port N to N+1" instead of using it.
* test(studio): drop a comment the doc line already says
* test(studio): the bench server test removes its temp dir
* test(studio): the bench stops its Studio server on Windows too
Windows has no process groups, so the group kill never reached the server and
the stop waited forever (the new server test timed out on windows-latest).
taskkill /T, the CLI helper for the same job, ends the tree there.
* test(studio): a failed taskkill surfaces, and the server opens no console on Windows
* test(studio): score edits on GSAP-animated size, scale, rotation, keyframes and from/fromTo
* test(studio): settle the edit bench only once no shadow preview is waiting to be swapped in
* test(studio): end a drag at its recorded pointer-up, even when no frame painted while held
* ci(studio): run the keyframed edit accuracy cases in CI across 20 shards
* refactor(studio): split the keyframe render and stray-css checks into small helpers
* test(studio): bank the edit bench's smooth verdict as smooth, judged on the worst frame's work
* style(studio): name the edit gate's smooth count helper
* test(studio): count smooth edit cases only among the accurate ones
* test(studio): only keyframed cases wait for the preview swap, so others keep main's undo timing
* refactor(studio): keep the edit bench's keyframe check apart from the hidden preview scan
* test(studio): check the keyframed cases are in the full grid without pinning its size
* test(studio): quarantine the nudge sequence whose redo waits behind preview videos (#4889)
* test(studio): bank the 255 keyframed edit cases that pass
* test(studio): catch teleports and multi-step edit bugs in the edit accuracy bench
* test(studio): share the edit accuracy case harness between single gestures and sequences
* test(studio): keep edit accuracy helpers private unless shared
* test(studio): score plain-CSS edits on pages that load GSAP for other elements
* test(studio): add an idle GSAP axis value so plain-CSS edits are scored on GSAP pages
* test(studio): report a slow undo save as late with its latency, apart from a lost one
* test(studio): sample the gap between back-to-back drags and allow one frame of lag
* test(studio): read the edge from the element's fractional size, so a crop edge is exact
* test(studio): keep waiting for a late save until an undone file stays put for 3 s
* ci(studio): run the edit accuracy grid in 12 shards to stay near 25 minutes each
* test(studio): read edit bench frames just before paint, so unpainted states never count
* test(studio): scope the edit bench frame sampler so page globals cannot clash
* test(studio): let a lagging box catch up the pointer travel it owes, so a held frame is no jump
* test(studio): record every save a sequence makes, even two that land 10 ms apart
* test(studio): end a drag's teleport scoring at its pointer-up, not at the next element's press
* test(studio): bank for each edit accuracy case a run that agrees with the gate's verdict
* test(studio): fail the edit accuracy gate on a teleport or an unmeasured drag
* test(studio): fail the edit accuracy gate when a text edit is not saved or shown
* test(studio): measure but do not gate the edit accuracy cases a known Studio race flips
* test(studio): bank the edit accuracy cases that newly pass with the teleport metric
* test(studio): quarantine the two nudge sequences whose redo write sits at the 2 s limit
* style(studio): cut three bench comments to one line each
* test(studio): gate edit accuracy against the base branch baseline
Work in progress: the ratchet and its tests; the CI job follows.
* test(studio): run the edit accuracy gate in CI across eight shards
Shares the built CLI from the Build job, re-runs twice each case that passes on the base branch and failed, and posts one sticky comment from the gate job.
* test(studio): mark the ratchet accuracy check as intentionally branchy for fallow
* test(studio): confirm newly passing edit accuracy cases 2 of 3 before banking them
The shards re-ran only regressions, so one lucky pass of a base-failing case failed the gate as unbanked and was never listed as unstable. Every case whose verdict differs from the base branch, either way, is now re-run twice and judged 2 of 3. A render error fails the gate like an unsettled snapshot, a malformed baseline.json fails the job instead of reading as empty, and the header is 3 lines.
* ci(studio): run the edit accuracy gate for every package the bench builds or imports
* ci(studio): also run the edit accuracy gate for lint, sdk and the Studio and CLI build inputs
* ci(studio): run the edit accuracy gate when a bench package's manifest or exports change
* test(studio): bank the 120 edit accuracy cases that sub-pixel dragging now passes
* test(studio): measure render drift against a producer frame in the edit accuracy bench
Finds the target's box in a producer-captured frame by its flat colour, with sub-pixel edges from area moments, and compares it with the reloaded preview. The baseline gains the render column from the same full run on main.
* test(studio): mark the pixel box scan as intentionally complex for fallow
* test(studio): regenerate the edit accuracy baseline with render drift on main
Same full run on main 14e87005d as #4760's baseline, with the render column: render passes 618/660, 70/660 pass, 314/660 pass everything but smoothness.
* test(studio): fail only render drift when the producer cannot render a case
A producer capture failure used to turn the case into a harness error that wiped every accuracy metric. It now sets render to null with renderError, fails render alone, and is counted in the table.
* test(studio): shorten the render drift box comment to two lines
* test(studio): start an edit accuracy benchmark for manual editing (work in progress)
* test(studio): an edit accuracy benchmark for manual editing
Drives real pointer and keyboard gestures in the built CLI Studio on generated one-element projects and scores tracking, drop jump, reload drift, undo fidelity and smoothness in composition px.
* test(studio): move render drift out of the first benchmark PR
Render drift against a producer frame ships in the stacked follow-up so this PR stays near the size budget.
* test(studio): keep the edit accuracy bench clean under fallow
Builds the grid as a product of its axes, drops unused exports and marks the harness functions that fallow scores as complex.
* test(studio): report edit accuracy with the Studio and bench commits it measured
Moves scoring and the report files into report.mjs, stamps both commits at start, counts undo byte failures apart from the worst box, and survives Studio swapping the preview iframe mid-read.
* test(studio): bank the edit accuracy baseline measured on main
660 cases on main ae914334f: 0 pass all six checks, 304 pass everything but smoothness. Five cases that hit a harness error in the full run were rerun and spliced in.
* test(studio): state the edit accuracy build prerequisite in the runner header
* test(studio): keep the edit accuracy baseline one line per case under the formatter
* test(studio): store edit accuracy baseline values rounded up
Rounding to the nearest 0.01 stored seven 0.502 px tracking results as 0.5, which reads as a pass at the 0.5 px limit.
* test(studio): score edit smoothness on dropped frames and main-thread work per frame
* test(studio): measure edit accuracy on a still preview and score press jump
Snapshots wait for the preview frames and the box to hold still for 1 s, so Studio's asynchronous reload after a save is measured, not raced. A preview that never settles fails the metrics it feeds instead of erroring.
Adds press jump (box at press against the box before it, 0.5 px). Smoothness counts only dropped frames beyond a blank-page control driven the same way in the same Chrome; unknown trace data fails smoothness alone.
The runner takes the suite lock per chunk of 8 cases (--lock), reads the port Studio announces, keeps undo history and tmp projects inside the case dir and removes them on a signal, and stamps the CLI tree's commit and bundle hash.
* test(studio): stamp the edit accuracy build with the Studio bundle as well as the CLI
The CLI bundle hashed the same for two different Studio builds; Studio's index.html names its hashed assets, so the stamp now covers it.
* test(studio): regenerate the edit accuracy baseline on main after the settle fix
Full grid on main 14e87005d with the settle fix, press jump and the smoothness control: 70/660 pass, 314/660 pass everything but smoothness, 0 harness errors, 0 unsettled. Scored without render, which #4764 adds.
* test(studio): wait for each edit accuracy undo write before sending redo
Undo and redo each wait up to 15 s for their own write, like the settle; redo is sent only once undo landed, and a write that never lands fails undo with undoTimeout instead of scoring an untested redo. The reload waits for the files to hold still. The headline is now the accuracy count, with smoothness on its own line against the control. The settle compares each read with the window's first read, and a CLI outside a git tree no longer stops the run.
Miguel Ángel