mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-10 22:20:14 +00:00
feat(studio): resolver-parity shadow tripwire (decoupled telemetry) (#1547)
* feat(studio): resolver-parity shadow tripwire (decoupled telemetry) New sdkResolverShadow.ts module: checks whether the SDK session resolves the same element id the server path would address, then verifies value parity after in-memory dispatch. Emits sdk_resolver_shadow telemetry on divergence. Decoupled from STUDIO_SDK_CUTOVER_ENABLED via its own flag STUDIO_SDK_RESOLVER_SHADOW_ENABLED (default false). Headline signal: element_not_found — the resolver divergence class that caused the v0.6.110 regression. Writer-parity suite (#1533) cannot see this class; this tripwire exists specifically to catch it. All 12 acceptance-test-plan items pass (A1-A3, B4-B6, C7-C10, D11, E12). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(studio): default STUDIO_SDK_RESOLVER_SHADOW_ENABLED to true Tripwire should run out of the box — operators opt out, not in. * fix(studio): resolver shadow must not mutate the live session (restore via inverse patches) The shadow runs on the SAME sdkSession the cutover path uses, one line before sdkCutoverPersist. sdkResolverShadowCheck dispatched the edit into that session to read values back but never undid it — so with the shadow enabled the edit was pre-applied, and sdkCutoverPersist then saw before === after and silently fell back to the server path. Enabling the tripwire disabled cutover. Fix: capture the inverse patches of the shadow dispatch (session.on("patch")) and applyPatches them to restore the session before returning, on every path (success, dispatch_error, element_not_found after dispatch). The session ends the check exactly as it started; cutover's before/after diff is unaffected. Tests: B5 now asserts the live session is restored (color back to original, not left on the shadow value) and B5b proves a cutover-style before/dispatch/after diff still fires after a preceding shadow run. The earlier B5 used two separate sessions and so never exercised the shared-session path the bug lived in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(studio): extend resolver shadow to timing/delete/gsap-add chokepoints The shadow only ran on the DOM-edit path (inline-style/text/attribute via onTrySdkPersist) — blind to the rest of the cutover surface, which is where the resolver bugs that motivated it actually live (v0.6.110 was a GSAP property op; CF2 #15/#16 were timing-resolver bugs). On-for-everyone telemetry that only sees style/text/attr edits misses the riskiest paths. Adds a read-only element-resolution tripwire (recordResolverParity) — emits the headline `element_not_found` signal when the SDK can't resolve a target the server path is addressing, with NO dispatch/mutation. Wired before the cutover gate (decoupled) in the element-targeted chokepoints: sdkTimingPersist, sdkDeletePersist, and sdkGsapTweenPersist's add op. To avoid a circular import (sdkResolverShadow imported patchOpsToSdkEditOps from sdkCutover; sdkCutover now imports recordResolverParity from sdkResolverShadow), patchOpsToSdkEditOps moves to a neutral sdkOpMapping.ts that both import from. animationId-resolving GSAP ops (set/remove tween, keyframe ops, deleteAllForSelector) resolve an animation, not an element, so element-resolution parity doesn't apply — left as a follow-up (separate animation-resolution signal). Tests: recordResolverParity emit-on-divergence / parity-no-op / flag-off-no-op / read-only (no mutation). Full studio suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(studio): resolver shadow covers animationId GSAP ops (animation_not_found) Extends the tripwire to the GSAP-edit surface that resolves an animationId rather than an element: setGsapTween/removeGsapTween, addGsapKeyframe, removeGsapKeyframe, removeGsapProperty, removeAllKeyframes, convertToKeyframes. Adds recordAnimationResolverParity — read-only, emits the new `animation_not_found` kind when the SDK can't resolve the animationId the server GSAP path is addressing. The SDK's resolvable animation ids are the located ids attached to elements (buildAnimationIdMap), so a target absent from every element's animationIds is a resolver divergence. No dispatch, no mutation. Wired centrally in dispatchGsapOpAndPersist via an optional resolverTarget arg (runs before its cutover gate); sdkGsapTweenPersist records inline before its own leading gate (set/remove → animation parity, add → element parity). deleteAllForSelector resolves by selector, not an id — left out. Tests: animation_not_found on unresolved id / parity no-op on a real located id / flag-off no-op. Full studio suite green; no circular dep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(studio): resolver shadow review fixes — divergence-only emit + restore in finally Addresses PR #1547 review (Miga, Rames): - #1 (medium): runResolverShadow emitted `sdk_resolver_shadow` on every edit, including parity (mismatchCount 0) — a PostHog event per style/text/attr edit at default-ON. Now emits only on divergence, matching recordResolverParity / recordAnimationResolverParity. Parity is silent across all three paths. - restore() moved into a `finally` in sdkResolverShadowCheck: if checkOpValue throws between dispatch and restore, the patch listener no longer leaks and the shared session is always undone (the cutover-coupling failure mode this module guards against). dispatch errors still return dispatch_error. - Comment on why batch is compatible with per-op inverse capture (a future SDK refactor that coalesces batch must keep emitting inverse patches). Tests: A2 now forces a divergence to emit; A2b pins parity-is-silent; A4 covers null/undefined hfId no-op. 26 shadow tests, full studio suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Vance Ingalls
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
75d92b55e0
commit
933b88ec33
@@ -0,0 +1,313 @@
|
||||
/**
|
||||
* SDK resolver-parity tripwire (telemetry-only).
|
||||
*
|
||||
* Checks whether the SDK session resolves the same element id the server
|
||||
* patch path would target, then optionally verifies value parity after an
|
||||
* in-memory dispatch. Emits `sdk_resolver_shadow` on any divergence.
|
||||
*
|
||||
* Headline signal: `element_not_found` — the resolver divergence class that
|
||||
* caused the v0.6.110 regression. The writer-parity suite (#1533) cannot see
|
||||
* this class; this tripwire exists specifically to catch it.
|
||||
*
|
||||
* Decoupled from `STUDIO_SDK_CUTOVER_ENABLED`. Gated by its own flag
|
||||
* `STUDIO_SDK_RESOLVER_SHADOW_ENABLED` (default ON during the soak — collect
|
||||
* wild telemetry; flip off / remove once resolver parity is proven).
|
||||
* Telemetry-only — never writes to disk, never affects the user-visible edit.
|
||||
*/
|
||||
|
||||
import type { Composition, JsonPatchOp } from "@hyperframes/sdk";
|
||||
import type { PatchOperation } from "./sourcePatcher";
|
||||
import { STUDIO_SDK_RESOLVER_SHADOW_ENABLED } from "../components/editor/manualEditingAvailability";
|
||||
import { patchOpsToSdkEditOps } from "./sdkOpMapping";
|
||||
import { trackStudioEvent } from "./studioTelemetry";
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
export interface SdkResolverMismatch {
|
||||
kind: "element_not_found" | "value_mismatch" | "dispatch_error" | "animation_not_found";
|
||||
hfId?: string;
|
||||
animationId?: string;
|
||||
property?: string;
|
||||
expected?: string | null;
|
||||
actual?: string | null | undefined;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
// ─── Op helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
// Drop studio-internal data-hf-* markers the SDK model doesn't represent.
|
||||
function isShadowableOp(op: PatchOperation): boolean {
|
||||
const name =
|
||||
op.type === "attribute"
|
||||
? op.property.startsWith("data-")
|
||||
? op.property
|
||||
: `data-${op.property}`
|
||||
: op.type === "html-attribute"
|
||||
? op.property
|
||||
: null;
|
||||
return name === null || !name.startsWith("data-hf-");
|
||||
}
|
||||
|
||||
const MAPPED_OP_TYPES = new Set(["inline-style", "text-content", "attribute", "html-attribute"]);
|
||||
|
||||
// ─── Read-back helpers ────────────────────────────────────────────────────────
|
||||
|
||||
function kebabToCamel(prop: string): string {
|
||||
return prop.replace(/-([a-z])/g, (_, c: string) => c.toUpperCase());
|
||||
}
|
||||
|
||||
function normalizeText(v: string | null | undefined): string | null {
|
||||
if (v == null) return null;
|
||||
const t = v.trim();
|
||||
return t === "" ? null : t;
|
||||
}
|
||||
|
||||
type FlatEl = NonNullable<ReturnType<Composition["getElement"]>>;
|
||||
type AttrMap = Record<string, string | null>;
|
||||
|
||||
function checkStyleOp(
|
||||
op: PatchOperation,
|
||||
el: FlatEl,
|
||||
): { expected: string | null; actual: string | null } {
|
||||
return {
|
||||
expected: op.value ?? null,
|
||||
actual: el.inlineStyles[kebabToCamel(op.property)] ?? el.inlineStyles[op.property] ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function checkTextOp(
|
||||
op: PatchOperation,
|
||||
el: FlatEl,
|
||||
): { expected: string | null; actual: string | null } {
|
||||
return { expected: normalizeText(op.value), actual: normalizeText(el.text) };
|
||||
}
|
||||
|
||||
function checkAttrOp(
|
||||
op: PatchOperation,
|
||||
el: FlatEl,
|
||||
): { property: string; expected: string | null; actual: string | null } {
|
||||
const property =
|
||||
op.type === "attribute"
|
||||
? op.property.startsWith("data-")
|
||||
? op.property
|
||||
: `data-${op.property}`
|
||||
: op.property;
|
||||
return {
|
||||
property,
|
||||
expected: op.value ?? null,
|
||||
actual: (el.attributes as AttrMap)[property] ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function checkOpValue(op: PatchOperation, el: FlatEl, hfId: string): SdkResolverMismatch | null {
|
||||
let property: string;
|
||||
let expected: string | null;
|
||||
let actual: string | null;
|
||||
|
||||
if (op.type === "inline-style") {
|
||||
property = op.property;
|
||||
({ expected, actual } = checkStyleOp(op, el));
|
||||
} else if (op.type === "text-content") {
|
||||
property = "text";
|
||||
({ expected, actual } = checkTextOp(op, el));
|
||||
} else if (op.type === "attribute" || op.type === "html-attribute") {
|
||||
({ property, expected, actual } = checkAttrOp(op, el));
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (actual === expected) return null;
|
||||
return { kind: "value_mismatch", hfId, property, expected, actual };
|
||||
}
|
||||
|
||||
// ─── Core check (pure — testable without flag) ────────────────────────────────
|
||||
|
||||
/**
|
||||
* Run the resolver shadow check against an already-open SDK session.
|
||||
*
|
||||
* Returns an array of mismatches (empty = parity). The value-parity check
|
||||
* dispatches the ops into the session to read the result back, then UNDOES
|
||||
* those mutations via the captured inverse patches before returning — the
|
||||
* session ends exactly as it started. This is essential: the session is shared
|
||||
* with the cutover path, and a residual shadow mutation would make the
|
||||
* subsequent sdkCutoverPersist see before === after and silently fall back to
|
||||
* the server path. Telemetry-only; the server path stays authoritative on disk.
|
||||
*
|
||||
* Exported for unit tests; call `runResolverShadow` at call sites.
|
||||
*/
|
||||
export function sdkResolverShadowCheck(
|
||||
session: Composition,
|
||||
hfId: string,
|
||||
ops: PatchOperation[],
|
||||
): SdkResolverMismatch[] {
|
||||
if (!session.getElement(hfId)) {
|
||||
return [{ kind: "element_not_found", hfId }];
|
||||
}
|
||||
|
||||
const shadowable = ops.filter(isShadowableOp);
|
||||
if (shadowable.length === 0) return [];
|
||||
|
||||
// Silently skip op batches containing unmapped types — not a resolver bug.
|
||||
if (shadowable.some((op) => !MAPPED_OP_TYPES.has(op.type))) return [];
|
||||
|
||||
// Capture the inverse of the shadow dispatch so we can restore the session.
|
||||
// `batch` fires a single PatchEvent whose `inversePatches` are already in
|
||||
// reverse-apply order (session.ts reverses inside buildPatchEvent), so
|
||||
// applyPatches(inverse) undoes the dispatch with no further reordering. If a
|
||||
// future SDK refactor ever coalesces batch into a composite with no per-op
|
||||
// inverse, this restore breaks — keep batch emitting inverse patches.
|
||||
const inverse: JsonPatchOp[] = [];
|
||||
const stopCapture = session.on("patch", (e) => inverse.push(...e.inversePatches));
|
||||
// restore() runs in `finally` so the patch listener is always removed and the
|
||||
// session is always undone — even if checkOpValue throws between dispatch and
|
||||
// return. A residual mutation or leaked listener on the shared session is the
|
||||
// exact cutover-coupling failure mode this module exists to avoid.
|
||||
try {
|
||||
try {
|
||||
const editOps = patchOpsToSdkEditOps(hfId, shadowable);
|
||||
session.batch(() => {
|
||||
for (const op of editOps) session.dispatch(op);
|
||||
});
|
||||
} catch (err) {
|
||||
return [{ kind: "dispatch_error", hfId, error: String(err) }];
|
||||
}
|
||||
|
||||
const el = session.getElement(hfId);
|
||||
if (!el) return [{ kind: "element_not_found", hfId }];
|
||||
|
||||
return shadowable
|
||||
.map((op) => checkOpValue(op, el, hfId))
|
||||
.filter((m): m is SdkResolverMismatch => m !== null);
|
||||
} finally {
|
||||
stopCapture();
|
||||
if (inverse.length > 0) session.applyPatches(inverse);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Telemetry ────────────────────────────────────────────────────────────────
|
||||
|
||||
// Redact all user-content values before telemetry: style values and text both
|
||||
// carry user data. Keep only the length so we can detect truncation without
|
||||
// leaking the actual bytes.
|
||||
function redactValue(value: string | null | undefined): string | null | undefined {
|
||||
if (value == null) return value;
|
||||
return `[redacted len=${value.length}]`;
|
||||
}
|
||||
|
||||
function redactMismatches(mismatches: SdkResolverMismatch[]): SdkResolverMismatch[] {
|
||||
return mismatches.map((m) => ({
|
||||
...m,
|
||||
expected: redactValue(m.expected),
|
||||
actual: redactValue(m.actual),
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the resolver shadow and emit `sdk_resolver_shadow` telemetry.
|
||||
* No-op when `STUDIO_SDK_RESOLVER_SHADOW_ENABLED` is false.
|
||||
* Never throws — any exception inside the shadow is swallowed.
|
||||
*
|
||||
* Side-effect-free on the live session: sdkResolverShadowCheck dispatches into
|
||||
* the session to read values back, then undoes those mutations before returning
|
||||
* (see below). The session is shared with the cutover path, so it MUST end the
|
||||
* call exactly as it started.
|
||||
*/
|
||||
export function runResolverShadow(
|
||||
session: Composition,
|
||||
hfId: string | null | undefined,
|
||||
ops: PatchOperation[],
|
||||
): void {
|
||||
if (!STUDIO_SDK_RESOLVER_SHADOW_ENABLED) return;
|
||||
if (!hfId) return;
|
||||
try {
|
||||
const mismatches = sdkResolverShadowCheck(session, hfId, ops);
|
||||
// Emit only on divergence — parity is silent, matching recordResolverParity
|
||||
// and recordAnimationResolverParity. Otherwise this fires a PostHog event on
|
||||
// every style/text/attr edit (the editor's chattiest path) at default-ON.
|
||||
if (mismatches.length === 0) return;
|
||||
trackStudioEvent("sdk_resolver_shadow", {
|
||||
hfId,
|
||||
mismatchCount: mismatches.length,
|
||||
mismatches: JSON.stringify(redactMismatches(mismatches)),
|
||||
});
|
||||
} catch {
|
||||
// never propagate from the shadow path
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record element-resolution parity for an element-targeted op WITHOUT
|
||||
* dispatching. Read-only: emits a single `element_not_found` event when the SDK
|
||||
* can't resolve a target the server path is addressing. This extends the
|
||||
* tripwire beyond the DOM-edit path (runResolverShadow) to the other
|
||||
* element-targeted cutover chokepoints — timing, delete, GSAP-tween add — for
|
||||
* the headline resolver signal, without the cost/mutation of a value check.
|
||||
*
|
||||
* No-op when the shadow flag is off; never throws; never mutates the session.
|
||||
*/
|
||||
export function recordResolverParity(
|
||||
session: Composition | null | undefined,
|
||||
hfId: string | null | undefined,
|
||||
opLabel: string,
|
||||
): void {
|
||||
if (!STUDIO_SDK_RESOLVER_SHADOW_ENABLED) return;
|
||||
if (!session || !hfId) return;
|
||||
try {
|
||||
if (session.getElement(hfId)) return; // resolves — parity, nothing to record
|
||||
trackStudioEvent("sdk_resolver_shadow", {
|
||||
hfId,
|
||||
opLabel,
|
||||
mismatchCount: 1,
|
||||
mismatches: JSON.stringify([
|
||||
{ kind: "element_not_found", hfId } satisfies SdkResolverMismatch,
|
||||
]),
|
||||
});
|
||||
} catch {
|
||||
// never propagate from the shadow path
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record animation-resolution parity for an animationId-targeted GSAP op WITHOUT
|
||||
* dispatching. Read-only: emits `animation_not_found` when the SDK can't resolve
|
||||
* the animationId the server GSAP path is addressing — the GSAP-edit-surface
|
||||
* analogue of element_not_found. The SDK's resolvable animation ids are the
|
||||
* located ids attached to elements (buildAnimationIdMap), so a target absent
|
||||
* from every element's animationIds is a resolver divergence.
|
||||
*
|
||||
* No-op when the shadow flag is off; never throws; never mutates the session.
|
||||
*/
|
||||
export function recordAnimationResolverParity(
|
||||
session: Composition | null | undefined,
|
||||
animationId: string,
|
||||
opLabel: string,
|
||||
): void {
|
||||
if (!STUDIO_SDK_RESOLVER_SHADOW_ENABLED) return;
|
||||
if (!session || !animationId) return;
|
||||
try {
|
||||
const resolves = session.getElements().some((el) => el.animationIds.includes(animationId));
|
||||
if (resolves) return; // SDK locates the animation — parity
|
||||
trackStudioEvent("sdk_resolver_shadow", {
|
||||
animationId,
|
||||
opLabel,
|
||||
mismatchCount: 1,
|
||||
mismatches: JSON.stringify([
|
||||
{ kind: "animation_not_found", animationId } satisfies SdkResolverMismatch,
|
||||
]),
|
||||
});
|
||||
} catch {
|
||||
// never propagate from the shadow path
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Soak gate ────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Evaluate the soak-gate exit criterion.
|
||||
*
|
||||
* A clean soak window has zero `element_not_found` divergences. When that
|
||||
* condition holds, resolver parity is proven and the flag can be retired.
|
||||
*/
|
||||
export function evaluateSoakGate(divergenceCount: number): "parity-proven" | "divergence-detected" {
|
||||
return divergenceCount === 0 ? "parity-proven" : "divergence-detected";
|
||||
}
|
||||
Reference in New Issue
Block a user