mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-20 12:25:51 +00:00
docs: document feedback collection — cadence, data, opt-out (#1111)
* docs: document feedback collection — cadence, data, opt-out Adds guides/feedback.mdx covering: when CLI and Studio prompts appear (render cadence, session cadence), what data is collected (PostHog survey fields, doctor_summary shape), what is not collected, the hyperframes feedback command for manual/agent submission, agent runtime detection and structured hint, config file fields, and all opt-out paths (HYPERFRAMES_NO_TELEMETRY, DO_NOT_TRACK, CI guard, --quiet). Also adds hyperframes feedback command entry to packages/cli.mdx (Utilities tab, alongside telemetry) and registers guides/feedback in the docs.json nav. — Magi * docs(feedback): fix cadence, agent env vars, docker gate, telemetry scope, why-we-ask - Cadence: 1st/16th/31st (not 15th/30th/45th) per actual code - Agent vars: CLAUDECODE/CLAUDE_CODE_ENTRYPOINT, CODEX_THREAD_ID/CODEX_CI, TERM_PROGRAM=cursor, Copilot value checks; add Hermes/openclaw/Pi - Remove docker gate claim (non-TTY only, not docker-specific) - Telemetry disable only suppresses CLI prompt, not Studio bar - Add why-we-ask opening section - Remove Studio 'skip' action (CLI-only); fix 'counter resets' phrasing - Fix 'values never read' — Cursor and Copilot do value comparisons Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(feedback): remove invented Studio opt-out flags; document localStorage workaround VITE_HYPERFRAMES_FEEDBACK_INTERVAL=0 falls through to default (n > 0 guard). VITE_HYPERFRAMES_FEEDBACK feature flag doesn't exist. Bar is mounted unconditionally. Document the localStorage key workaround instead and note that a proper flag is a follow-up to hf#1101. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(feedback): fix localStorage workaround — only lastPromptedAt needs to be large Setting both keys to the same value just delays 10 sessions before the bar reappears. Setting only lastPromptedAt to 9999999 keeps count - lastAt negative indefinitely. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(studio): add VITE_HYPERFRAMES_NO_FEEDBACK build-time disable flag Sets isFeedbackDisabled() guard in shouldShowFeedback() — when VITE_HYPERFRAMES_NO_FEEDBACK=1, bar never shows regardless of session count. Updates docs to document the flag and remove the localStorage workaround. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 4.6
parent
8cd74c1e8c
commit
55c4a11884
+2
-1
@@ -90,7 +90,8 @@
|
||||
"guides/timeline-editing",
|
||||
"guides/video-editor-cheatsheet",
|
||||
"guides/common-mistakes",
|
||||
"guides/troubleshooting"
|
||||
"guides/troubleshooting",
|
||||
"guides/feedback"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -0,0 +1,219 @@
|
||||
---
|
||||
title: Feedback Collection
|
||||
description: "How HyperFrames collects feedback, what data is collected, and how to opt out."
|
||||
---
|
||||
|
||||
HyperFrames occasionally asks how a render went or how a Studio session felt. This page explains why we do it, when prompts appear, what data is collected, and how to disable them.
|
||||
|
||||
## Why We Ask
|
||||
|
||||
We use anonymous satisfaction scores to understand whether the tool is actually working well — not just whether it runs without errors. A render that takes 10 minutes and produces a broken file counts as a success in logs but a failure in practice. The feedback prompt is the only signal we have for that gap.
|
||||
|
||||
No account, email, or identity is tied to responses. Each installation generates a random UUID at setup; that is the only identifier.
|
||||
|
||||
## How It Works
|
||||
|
||||
### CLI — post-render prompt
|
||||
|
||||
After a successful `hyperframes render`, a short prompt may appear:
|
||||
|
||||
```
|
||||
How was this render? [1=poor 5=great, enter to skip]
|
||||
Any details? (enter to skip)
|
||||
```
|
||||
|
||||
**When it shows:**
|
||||
|
||||
- First ever successful render
|
||||
- Then every 15 renders after that (16th, 31st, 46th...)
|
||||
- At most once per process — re-renders in the same session don't trigger a second prompt
|
||||
- Automatically suppressed in quiet mode (`--quiet`), non-TTY shells, and CI environments
|
||||
|
||||
The prompt has a **10-second auto-timeout** — if you don't respond, it silently disappears and the CLI continues normally.
|
||||
|
||||
The render interval is configurable:
|
||||
|
||||
```bash
|
||||
# Show prompt every 5 renders instead of 15 (useful for testing)
|
||||
HYPERFRAMES_FEEDBACK_INTERVAL=5 hyperframes render --output out.mp4
|
||||
```
|
||||
|
||||
### Studio — session feedback bar
|
||||
|
||||
A thin 32px bar slides in at the bottom of the preview area periodically:
|
||||
|
||||
- Never on the first session — only starting from the 10th
|
||||
- Then every 10 sessions (10th, 20th, 30th...)
|
||||
- Slides in 3 seconds after page load to avoid flash
|
||||
- Auto-dismisses after 20 seconds if ignored
|
||||
- After any interaction (dismiss or submit), the session counter resets — next prompt after 10 more sessions
|
||||
|
||||
The session interval is configurable at build time:
|
||||
|
||||
```
|
||||
VITE_HYPERFRAMES_FEEDBACK_INTERVAL=3
|
||||
```
|
||||
|
||||
Invalid values (non-integer, zero, negative) fall back to the default.
|
||||
|
||||
**Note:** Disabling CLI telemetry (`hyperframes telemetry disable`) does not suppress the Studio bar — Studio feedback is gated separately. The submitted data is still sent through the same anonymous PostHog pipeline, so the same privacy guarantees apply.
|
||||
|
||||
### `hyperframes feedback` command
|
||||
|
||||
You can submit feedback manually at any time:
|
||||
|
||||
```bash
|
||||
# Quick rating
|
||||
hyperframes feedback --rating 5
|
||||
|
||||
# Rating with details
|
||||
hyperframes feedback --rating 3 --comment "render succeeded but GSAP timeline didn't animate text overlay"
|
||||
```
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--rating` | Satisfaction score, 1–5 (required) |
|
||||
| `--comment` | Optional free-text details |
|
||||
|
||||
This command collects a doctor summary automatically, flushes telemetry, and exits. It appears under the **Settings** group in `hyperframes --help`.
|
||||
|
||||
## Agent Runtimes
|
||||
|
||||
When an AI agent is detected, HyperFrames **skips the interactive readline prompt** and prints a structured hint instead:
|
||||
|
||||
```
|
||||
[hyperframes] Agent feedback: hyperframes feedback --rating <1-5> --comment "..."
|
||||
```
|
||||
|
||||
Agents can then submit feedback using the `hyperframes feedback` command above.
|
||||
|
||||
The same cadence gate applies: the hint only appears on the first render, then every 15th.
|
||||
|
||||
**Detected agents and their markers:**
|
||||
|
||||
| Agent | Environment markers |
|
||||
|-------|---------------------|
|
||||
| Claude Code | `CLAUDECODE` present, or `CLAUDE_CODE_ENTRYPOINT` present |
|
||||
| Codex | `CODEX_THREAD_ID`, `CODEX_CI`, or `CODEX_SANDBOX_NETWORK_DISABLED` present |
|
||||
| Cursor | `TERM_PROGRAM` equals `cursor` |
|
||||
| GitHub Copilot Agent | `GITHUB_ACTIONS` equals `true` and (`COPILOT_AGENT_ID` present or `RUNNER_NAME` equals `Copilot`) |
|
||||
| Replit | `REPL_ID` or `REPLIT_USER` present |
|
||||
| Hermes | `HERMES_QUIET` present |
|
||||
| openclaw | `OPENCLAW_STATE_DIR` or `OPENCLAW_CONFIG_PATH` present |
|
||||
| Pi | `PI_CODING_AGENT` present |
|
||||
|
||||
Only the existence (or in some cases the value) of these variables is checked — API keys and secrets that happen to share a prefix are never read.
|
||||
|
||||
## What Is Collected
|
||||
|
||||
### CLI feedback
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| `$survey_id` | `render_satisfaction` |
|
||||
| `$survey_response` | Rating (1–5) |
|
||||
| `$survey_response_2` | Free-text comment (only when provided) |
|
||||
| `render_duration_ms` | Time the render took in milliseconds |
|
||||
| `doctor_summary` | System context (see below) |
|
||||
|
||||
The `doctor_summary` is a compact string with environment context — included automatically so you don't need to run `hyperframes doctor` when reporting a problem:
|
||||
|
||||
```
|
||||
os=darwin/arm64 node=v22.11.0 cpu=10cores mem=32GB ffmpeg=yes
|
||||
```
|
||||
|
||||
It may also include `wsl` or sandbox runtime flags when those environments are detected.
|
||||
|
||||
### Studio feedback
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| `$survey_id` | `studio_experience` |
|
||||
| `$survey_response` | Rating (1–5) |
|
||||
| `$survey_response_2` | Free-text comment (only when provided) |
|
||||
| `source` | `studio` |
|
||||
| `doctor_summary` | Browser context (platform, screen, CPU cores, device memory, network type) |
|
||||
|
||||
## What Is NOT Collected
|
||||
|
||||
- File paths or project names
|
||||
- Composition content, HTML, or video files
|
||||
- Environment variable values
|
||||
- Personally identifiable information
|
||||
- IP addresses or precise location
|
||||
|
||||
Feedback is anonymous. Each installation has a random UUID (`anonymousId`) — there is no account, login, or email association.
|
||||
|
||||
## Config File
|
||||
|
||||
The CLI persists feedback state in `~/.hyperframes/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"telemetryEnabled": true,
|
||||
"anonymousId": "a1b2c3d4-...",
|
||||
"telemetryNoticeShown": true,
|
||||
"commandCount": 47,
|
||||
"renderSuccessCount": 14,
|
||||
"lastFeedbackPromptAt": 1
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `renderSuccessCount` | Total successful renders across all sessions |
|
||||
| `lastFeedbackPromptAt` | The `renderSuccessCount` value when the prompt last appeared — used to compute whether 15 renders have passed |
|
||||
|
||||
Studio stores the equivalent state in `localStorage` under the `hyperframes-studio:` prefix.
|
||||
|
||||
## Opting Out
|
||||
|
||||
### CLI — disable telemetry entirely
|
||||
|
||||
Disabling telemetry suppresses the CLI feedback prompt and all other CLI usage tracking:
|
||||
|
||||
```bash
|
||||
# Via CLI command (persisted to ~/.hyperframes/config.json)
|
||||
hyperframes telemetry disable
|
||||
|
||||
# Or via environment variable (per-session)
|
||||
HYPERFRAMES_NO_TELEMETRY=1 hyperframes render --output out.mp4
|
||||
|
||||
# Or via DO_NOT_TRACK (respects the global standard)
|
||||
DO_NOT_TRACK=1 hyperframes render --output out.mp4
|
||||
```
|
||||
|
||||
Once disabled, the `hyperframes feedback` command will print `Telemetry is disabled. Feedback not sent.` and exit without sending anything.
|
||||
|
||||
### CLI — suppress output without disabling telemetry
|
||||
|
||||
```bash
|
||||
# Quiet mode: renders without any post-render output (including the feedback prompt)
|
||||
hyperframes render --quiet --output out.mp4
|
||||
```
|
||||
|
||||
### CI environments
|
||||
|
||||
The CLI feedback prompt is automatically suppressed when the `CI` environment variable is set (GitHub Actions, CircleCI, etc. set this by default).
|
||||
|
||||
### Studio
|
||||
|
||||
The Studio feedback bar is not affected by the CLI telemetry setting. To disable it at build time, set:
|
||||
|
||||
```
|
||||
VITE_HYPERFRAMES_NO_FEEDBACK=1
|
||||
```
|
||||
|
||||
When set to `"1"`, the bar never shows — `shouldShowFeedback()` returns `false` unconditionally regardless of session count or `localStorage` state.
|
||||
|
||||
The session interval is still configurable independently:
|
||||
|
||||
```
|
||||
VITE_HYPERFRAMES_FEEDBACK_INTERVAL=20
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [Telemetry](/packages/cli#telemetry) — full telemetry settings and what usage data is collected
|
||||
- [`hyperframes feedback`](/packages/cli#feedback) — CLI command reference
|
||||
- [Troubleshooting](/guides/troubleshooting) — if a prompt is blocking your pipeline unexpectedly
|
||||
@@ -803,6 +803,27 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
|
||||
|
||||
Available topics: `data-attributes`, `examples`, `rendering`, `gsap`, `troubleshooting`, `compositions`. Run without a topic to see the full list.
|
||||
|
||||
### `feedback`
|
||||
|
||||
Submit anonymous satisfaction feedback about your experience:
|
||||
|
||||
```bash
|
||||
# Quick rating (1 = poor, 5 = great)
|
||||
npx hyperframes feedback --rating 5
|
||||
|
||||
# Rating with optional details
|
||||
npx hyperframes feedback --rating 3 --comment "render succeeded but GSAP timeline didn't animate"
|
||||
```
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--rating` | Satisfaction score, 1–5 (required) |
|
||||
| `--comment` | Optional free-text details |
|
||||
|
||||
This command is also available to AI agents after a render — see [Feedback Collection](/guides/feedback#agent-runtimes) for how agent detection and the automatic post-render hint work.
|
||||
|
||||
No-op when telemetry is disabled — prints `Telemetry is disabled. Feedback not sent.` and exits cleanly.
|
||||
|
||||
### `telemetry`
|
||||
|
||||
Manage anonymous usage telemetry:
|
||||
@@ -815,6 +836,8 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
|
||||
|
||||
Telemetry collects command names, render performance, example choices, and system info — including a coarse environment fingerprint (OS, kernel string, CPU/memory shape, sandbox runtime such as gVisor or Docker, and the *name* of a coding agent driving the CLI when one is detected, e.g. `claude_code` / `codex` / `cursor`). The agent name is derived from the existence of well-known environment variables; their values are never read. Telemetry does **not** collect file paths, project names, video content, environment variable values, or personally identifiable information. Disable with `HYPERFRAMES_NO_TELEMETRY=1` or the command above.
|
||||
|
||||
See [Feedback Collection](/guides/feedback) for how the periodic post-render prompt and Studio feedback bar work, what data they collect, and how to opt out.
|
||||
|
||||
### `skills`
|
||||
|
||||
Install HyperFrames skills for AI coding tools, including first-party runtime adapter skills:
|
||||
|
||||
@@ -4,6 +4,14 @@ import { trackStudioFeedback } from "../telemetry/events";
|
||||
const DEFAULT_FEEDBACK_INTERVAL = 10;
|
||||
const AUTO_DISMISS_MS = 20_000;
|
||||
|
||||
function isFeedbackDisabled(): boolean {
|
||||
try {
|
||||
return import.meta.env.VITE_HYPERFRAMES_NO_FEEDBACK === "1";
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
function getFeedbackInterval(): number {
|
||||
try {
|
||||
@@ -25,6 +33,7 @@ const STORAGE_KEYS = {
|
||||
|
||||
// fallow-ignore-next-line complexity
|
||||
function shouldShowFeedback(): boolean {
|
||||
if (isFeedbackDisabled()) return false;
|
||||
try {
|
||||
const count = parseInt(localStorage.getItem(STORAGE_KEYS.sessionCount) || "0", 10) || 0;
|
||||
const lastAt = parseInt(localStorage.getItem(STORAGE_KEYS.lastPromptedAt) || "0", 10) || 0;
|
||||
|
||||
Reference in New Issue
Block a user