mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
## What Studio's feedback prompt now fires when a render finishes or fails, instead of on a session counter, and the reports it collects carry enough context to act on. - Replaces the 32px inline bar with a card in the existing toast stack - Adds `studio_feedback_shown` / `studio_feedback_dismissed` / `studio_feedback_interview_click`, so the funnel is visible - A failed export and a crash skip the 0-10 score and ask what happened - Adds a crash prompt to the error boundary - Attaches a breadcrumb trail, render settings and outcome, and how the project was created ## Before / after <img width="1500" alt="Before: a 32px feedback strip pinned under the preview. After: the feedback card in the toast stack, with one-tap answers, and the red variant for a failed export." src="https://github.com/user-attachments/assets/29e2373f-ee96-42c4-b006-ab9a0e56a60a" /> The old bar's rating numbers are `neutral-600` on `neutral-900/80`, which is why it reads as a disabled row rather than a control. **In the running Studio** — bottom-right, sharing the toast stack, hovering a chip explains it on the line above the input: <img width="1500" alt="The feedback card in the bottom-right of the running Studio, with the rotated follow-up question and a hovered chip explained inline." src="https://github.com/user-attachments/assets/3969da50-cc7a-466a-bbf9-b151a1bc1d5d" /> **On the crash screen** — the prompt the error boundary renders, asking what the user was doing rather than what went wrong, since the stack trace already covers the latter: <img width="1200" alt="The Studio crash screen with the feedback card below the Try again and Reload Studio buttons." src="https://github.com/user-attachments/assets/4d287011-9c19-4b94-99d0-07a0388fb39c" /> | | Before | After | |---|---|---| | **Trigger** | Every 10th session | A render finishing, failing, or a crash | | **Placement** | 32px inline bar, pushes preview up | Card in the toast stack, no layout shift | | **Rating targets** | 11 bare buttons, `neutral-600` | Native radios, resting fill, `neutral-400` | | **Follow-up** | Free text or nothing | One rotated question, four to seven one-tap answers | | **Option help** | None | Inline hint on hover and focus | | **Press feedback** | None | `active:scale-[0.97]`, 150ms ease-out | | **Keyboard** | No exit path | Escape closes, Enter sends, arrows move the rating | | **Auto-dismiss** | 20s, always | 30s, cancelled the moment you interact | | **Failure case** | Same NPS question | Its own question, no score, error quoted back | | **Crash case** | Nothing | Prompt on the crash screen | | **Visibility** | Submissions only | Shown / dismissed (with reason) / submitted / interview click | | **Report content** | Rating, comment | Plus breadcrumbs, render settings and outcome, project provenance | ## Why The old bar fired on a session count, so it interrupted at a moment with no subject: nothing the user had just done, nothing to have an opinion about. It also emitted nothing when it appeared or when it was dismissed, which made the collection rate impossible to diagnose. A prompt nobody answers and a prompt that never renders looked identical from the outside. Visually it read as a disabled row: 11 buttons at 11px in `neutral-600` on a dark strip, with no resting affordance. And appearing mid-task pushed the whole preview stack up, which the old code carried a comment apologising for. Separately, the reports it did collect were not actionable. A comment says what went wrong; it almost never says how to get there. ## How **One trigger, one owner.** `feedbackTrigger` owns eligibility and nothing else does: once per tab, thirty days after an answer, seven after a dismissal, never when telemetry is off (prompting someone whose response we would then drop wastes their attention). `VITE_HYPERFRAMES_NO_FEEDBACK=1` still disables it entirely. **One hook, every failure path.** The trigger watches the render job list rather than each of the four places a render can finish (server rejection, unreachable server, SSE terminal event, SSE connection drop), so paths added later are covered without touching the trigger. Renders loaded from disk history never fire it. **Reuses what exists.** The card wears `StudioToast`'s glass treatment and joins its stack, so there is no second visual language and no new CSS. The rating row is native radios, which gives arrow-key navigation, grouping and labels for free. **One question each, rotated across users.** A corner card that asks three things gets answered by nobody. Each person gets one follow-up with one-tap answers, explained on a reserved line rather than a floating tooltip (the card is 340px in a corner; a bubble above the chips lands on the question, below lands on the input). Detractors are never given a rotated question, because they already have a specific complaint. Every option was checked against the code: an option naming a feature Studio already has would collect taps meaning "I could not find it", which is indistinguishable afterwards from "it does not exist". **Breadcrumbs cost one line.** Every studio event already flows through `trackEvent`, so recording the trail there needs no new instrumentation and stays correct as events are added. **Provenance lives outside React.** A crash unmounts the tree, so it is captured when the project loads and read from module scope when the crash prompt renders. ### Privacy Breadcrumbs and provenance carry names, enums and counts only. Values are copied from a fixed allowlist of short keys, and anything longer than a slug is dropped rather than truncated, so comments, file paths, stack traces and project titles cannot reach them even if a future event carries one. Tests assert this. ### Where these responses land Studio feedback goes to PostHog and nowhere else, which is what it did before this change too. Worth stating because the CLI behaves differently: `hyperframes feedback` also forwards to the backend feedback endpoint via `submitFeedback`, on top of its PostHog event. Studio has never used that path, before or after this PR, so if you read CLI feedback anywhere other than PostHog, Studio responses will not show up there. Nothing here changes that either way. Whether the two surfaces should share a delivery path is a product question, not a defect in this change, and closing it would need a field on the backend DTO: it is shaped around `cli_version`, and Studio reports from a crash or a failed export deliberately carry no rating. ## Test plan - [x] Unit tests added/updated - [x] Manual testing performed - [ ] Documentation updated (if applicable) **Unit** — 39 new tests: trigger eligibility and cooldowns, the detractor override, rotation, preset shape and the no-brands rule, breadcrumb rolling and privacy, provenance parsing and its failure modes, and the crash boundary rendering the prompt with no rating input. **Live** — both render paths driven end to end against a running Studio on a production bundle, with real renders. Every PostHog request was intercepted and dropped, so nothing reached the project. Verified the emitted payload for a finished render, a failed export, the rotated follow-ups, each chip's hint, and the interview link. **Not covered** — no live capture of a spontaneous crash. Three attempts to force one failed because Studio's guards held and it kept rendering, so the crash path is verified by component tests rather than by driving it. Touch devices see chip labels without hints, since the hint is revealed on hover and focus.
62 lines
2.6 KiB
JavaScript
62 lines
2.6 KiB
JavaScript
import { strict as assert } from "node:assert";
|
|
import { test } from "node:test";
|
|
|
|
import { ALLOWED_DELETIONS, classify, parseBase } from "./check-no-main-deletions.mjs";
|
|
|
|
test("a branch that only adds reports nothing", () => {
|
|
const { deleted, renamed } = classify("A\tpackages/cli/src/new.ts\nM\tpackages/cli/src/old.ts\n");
|
|
assert.deepEqual(deleted, []);
|
|
assert.deepEqual(renamed, []);
|
|
});
|
|
|
|
test("a deleted file is named", () => {
|
|
const { deleted } = classify("D\t.agents/skills/README.md\nA\tsrc/new.ts\n");
|
|
assert.deepEqual(deleted, [".agents/skills/README.md"]);
|
|
});
|
|
|
|
test("a rename is not reported as a deletion", () => {
|
|
// The false alarm worth avoiding: in a name-only diff a rename looks exactly
|
|
// like loss, and a check that cried wolf on every move would be turned off.
|
|
const { deleted, renamed } = classify("R096\tsrc/old/name.ts\tsrc/new/name.ts\n");
|
|
assert.deepEqual(deleted, []);
|
|
assert.deepEqual(renamed, [{ from: "src/old/name.ts", to: "src/new/name.ts" }]);
|
|
});
|
|
|
|
test("deletions and renames are separated in one diff", () => {
|
|
const { deleted, renamed } = classify(
|
|
"D\tdocs/gone.md\nR100\ta.ts\tb.ts\nM\tc.ts\nD\tdocs/also-gone.md\n",
|
|
);
|
|
assert.deepEqual(deleted, ["docs/gone.md", "docs/also-gone.md"]);
|
|
assert.equal(renamed.length, 1);
|
|
});
|
|
|
|
test("the base ref defaults, and an explicit one is honoured", () => {
|
|
assert.equal(parseBase([]), "origin/main");
|
|
assert.equal(parseBase(["--base", "origin/release"]), "origin/release");
|
|
});
|
|
|
|
test("a --base with no value fails rather than silently defaulting", () => {
|
|
// Silently falling back would diff against the wrong ref and report a pass.
|
|
assert.throws(() => parseBase(["--base"]), /needs a ref/);
|
|
assert.throws(() => parseBase(["--base", "--other"]), /needs a ref/);
|
|
});
|
|
|
|
test("every agreed deletion names a path and says why", () => {
|
|
// The guard has no blanket override on purpose: a flag or an env var would
|
|
// be reached for by the branch deleting something by accident. An entry has
|
|
// to be written down, so the removal shows up in review.
|
|
for (const [path, reason] of ALLOWED_DELETIONS) {
|
|
assert.ok(path.length > 0, "an allowed deletion needs a path");
|
|
assert.ok(
|
|
reason && reason.length > 10,
|
|
`${path} needs a reason, got ${JSON.stringify(reason)}`,
|
|
);
|
|
}
|
|
});
|
|
|
|
test("the allowlist does not silence an unrelated deletion", () => {
|
|
const { deleted } = classify("D\tpackages/cli/src/something-else.ts\n");
|
|
assert.deepEqual(deleted, ["packages/cli/src/something-else.ts"]);
|
|
assert.equal(ALLOWED_DELETIONS.has("packages/cli/src/something-else.ts"), false);
|
|
});
|