feat(telemetry): measure which lint rules fire, cost, and fail to converge (#3367)

* feat(telemetry): measure which lint rules fire, cost, and fail to converge

Lint rule changes are currently argued from anecdote. This adds the three
measurements needed to argue them from data.

`lint_report`, once per `hyperframes lint` or `hyperframes check`:
- `code_counts` / `codes` — which rules actually fire, and how often
- `rule_group_ms` — milliseconds per rule-source module (core, gsap, media, ...)
- `slowest_rule` / `slowest_rule_ms` — slowest single rule as `<group>#<index>`
- `rule_count` — how many rules this build ran

`lint_rule_streak`, once per finding that survives an edit to its file:
- `edits` — how many edits the finding survived
- `cleared` — whether it eventually went away

The streak event is the one that matters. A lint pass costs about 5ms, so
per-rule CPU is not what makes the authoring loop slow; a rule an agent cannot
satisfy is, because every failed attempt costs a full edit-and-relint cycle. A
single run cannot see that, so `lint_rule_streak` reconstructs it across runs:
high `edits` with `cleared: false` is a rule nobody can fix, and the
`cleared: true` distribution is the baseline to judge it against.

An iteration is counted only when the file's content digest CHANGED and the
finding is still there. Re-linting an untouched project is not an attempt,
which is what stops `check` (which lints on every invocation) from inflating
the numbers.

Rule identity is the source module plus an index within it. Naming all 86
rules would make the timings prettier but it is a refactor this measurement
does not need: the group locates the file, and the index locates the rule.

Version, agent runtime, CI flag, and invocation id are already attached to
every event by `trackEvent`, so lint pain can be split by CLI version and by
which agent produced it without adding anything here.

Privacy: only rule codes, counts, and timings are sent. Streak state lives in
~/.hyperframes/lint-streaks.json alongside config.json (so `rm -rf
~/.hyperframes` is still a full reset) and stores digests only — no file
paths, no project names, no composition source. Nothing is written and nothing
is emitted when telemetry is off. Entries expire after 14 days and are capped
at 500 files.

`EventProperties` gains string arrays and numeric maps. `codes` and
`code_counts` are inherently a set and a histogram; flattening them into
dynamic top-level keys would make them unqueryable. PostHog stores both
natively.

`trackLintRun` is the single call site shared by `lint` and `check`, and it
swallows every error — telemetry must never turn a green lint red.

* feat(telemetry): emit per-group rule counts so slowest_rule stays comparable

Review catch on #3367: `slowest_rule` is the one positional key in either
event. It is `<group>#<index>`, so adding or removing a rule renumbers every
later slot in that group and the same string means different rules in two
builds. #3366 does exactly that to 34 of 81 surviving slots, and `rule_count`
alone says only THAT the ruleset moved, not which groups.

`rule_group_counts` carries the per-group sizes alongside it, so a consumer
comparing two builds can tell which groups' indices still mean the same thing
without anyone having to remember which release dropped rules. `codes`,
`code_counts` and `rule_group_ms` are keyed by name and were never affected.

Also corrects the rule count in the RULE_GROUPS comment: 86, not ~60, as
LINT_RULE_COUNT in the same file computes.

packages/cli/src/commands/check.test.ts

@@ -58,6 +58,7 @@ function cleanLint(): ProjectLintResult {
     results: [
       {
         file: "index.html",
+        contentHash: "test",
         result: {
           ok: true,
           errorCount: 0,
@@ -82,6 +83,7 @@ function lintWith(
     results: [
       {
         file: "index.html",
+        contentHash: "test",
         result: {
           ok: severity !== "error",
           errorCount: severity === "error" ? 1 : 0,

packages/cli/src/commands/lint.ts

@@ -12,6 +12,8 @@ export const examples: Example[] = [
 import { formatLintFindings } from "../utils/lintFormat.js";
 import { lintProject } from "../utils/lintProject.js";
 import { resolveProject } from "../utils/project.js";
+import { trackLintRun } from "../telemetry/lintRun.js";
+import { getRunId } from "../telemetry/runId.js";
 import { withMeta } from "../utils/updateCheck.js";
 
 export default defineCommand({
@@ -45,7 +47,13 @@ export default defineCommand({
     // (publish/transcribe/upgrade/play/present) already use.
     try {
       const project = resolveProject(args.dir);
+      const startedAt = Date.now();
       const lintResult = await lintProject(project.dir);
+      trackLintRun(project.dir, lintResult, {
+        command: "lint",
+        durationMs: Date.now() - startedAt,
+        ...(getRunId() !== undefined ? { runId: getRunId() } : {}),
+      });
 
       if (args.json) {
         const allFindings = lintResult.results.flatMap((r) => r.result.findings);

packages/cli/src/commands/validate.test.ts

@@ -232,6 +232,7 @@ describe("extractCompositionErrorsFromLint", () => {
       results: [
         {
           file: "index.html",
+          contentHash: "test",
           result: {
             ok: findings.length === 0,
             errorCount: 0,
@@ -282,6 +283,7 @@ describe("extractCompositionErrorsFromLint", () => {
       results: [
         {
           file: "index.html",
+          contentHash: "test",
           result: {
             ok: false,
             errorCount: 1,
@@ -298,6 +300,7 @@ describe("extractCompositionErrorsFromLint", () => {
         },
         {
           file: "compositions/nested.html",
+          contentHash: "test",
           result: {
             ok: false,
             errorCount: 1,

packages/cli/src/telemetry/events.ts

@@ -819,3 +819,76 @@ export function trackCheckReport(props: {
     ...runIdField(props.runId),
   });
 }
+
+/**
+ * One lint pass over a project. `code_counts` is what makes "which rules
+ * actually fire" answerable; `rule_group_ms` and `slowest_rule` are what make
+ * "which rules are expensive" answerable. Only lint rule codes and timings are
+ * sent — never file paths, project names, or composition source.
+ */
+export function trackLintReport(props: {
+  /** The command that ran the lint: "lint" or "check". */
+  command: string;
+  durationMs: number;
+  filesScanned: number;
+  errorCount: number;
+  warningCount: number;
+  infoCount: number;
+  /** Finding count keyed by lint rule code. */
+  codeCounts: Record<string, number>;
+  /** Milliseconds spent per rule-source module, summed across files. */
+  ruleGroupMs: Record<string, number>;
+  /** Slowest single rule as `<group>#<index>`, across every file in the run. */
+  slowestRule: string;
+  slowestRuleMs: number;
+  /** How many rules this build ran, so a ruleset change is visible in the data. */
+  ruleCount: number;
+  /**
+   * Rule count per group. `slowest_rule` is positional, so a group that changed
+   * size between two builds has indices that no longer mean the same thing.
+   */
+  ruleGroupCounts: Record<string, number>;
+  runId?: string;
+}): void {
+  trackEvent("lint_report", {
+    command: props.command,
+    duration_ms: Math.round(props.durationMs),
+    files_scanned: props.filesScanned,
+    error_count: props.errorCount,
+    warning_count: props.warningCount,
+    info_count: props.infoCount,
+    codes: Object.keys(props.codeCounts).sort(),
+    code_counts: props.codeCounts,
+    rule_group_ms: props.ruleGroupMs,
+    slowest_rule: props.slowestRule,
+    slowest_rule_ms: Math.round(props.slowestRuleMs),
+    rule_count: props.ruleCount,
+    rule_group_counts: props.ruleGroupCounts,
+    ...runIdField(props.runId),
+  });
+}
+
+/**
+ * A finding that survived one or more edits to the file it was reported on.
+ *
+ * `cleared: false` with a high `edits` is the signal that matters most: a rule
+ * an agent kept trying and failing to satisfy. `cleared: true` gives the
+ * distribution to compare it against — how many edits a normal finding costs.
+ */
+export function trackLintRuleStreak(props: {
+  code: string;
+  severity: string;
+  edits: number;
+  cleared: boolean;
+  command: string;
+  runId?: string;
+}): void {
+  trackEvent("lint_rule_streak", {
+    code: props.code,
+    severity: props.severity,
+    edits: props.edits,
+    cleared: props.cleared,
+    command: props.command,
+    ...runIdField(props.runId),
+  });
+}

packages/cli/src/telemetry/lintRun.e2e.test.ts

@@ -0,0 +1,134 @@
+// End-to-end: a real project on disk -> lintProject -> the exact PostHog
+// payloads. Unit tests cover the streak arithmetic; this proves the wiring
+// and pins the event shape a dashboard will be built against.
+
+import { describe, it, expect, vi, beforeEach } from "vitest";
+import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+const HOME = mkdtempSync(join(tmpdir(), "hf-lintrun-"));
+vi.mock("node:os", async (importOriginal) => {
+  const actual = await importOriginal<typeof import("node:os")>();
+  return { ...actual, homedir: () => HOME };
+});
+
+// Capture at the transport boundary so everything client.ts adds
+// (cli_version, invocation_id, ...) is visible in the assertions.
+const enqueued: Array<{ event: string; properties: Record<string, unknown> }> = [];
+vi.mock("./transport.js", () => ({
+  enqueue: (event: string, properties: Record<string, unknown>) =>
+    enqueued.push({ event, properties }),
+  flush: () => Promise.resolve(),
+}));
+// shouldTrack() consults these two. A dev build disables telemetry by default,
+// which would make this test assert on an empty queue and pass for the wrong
+// reason.
+vi.mock("./policy.js", async (importOriginal) => {
+  const actual = await importOriginal<typeof import("./policy.js")>();
+  return { ...actual, telemetryRuntimeOverride: () => null };
+});
+vi.mock("./config.js", async (importOriginal) => {
+  const actual = await importOriginal<typeof import("./config.js")>();
+  return { ...actual, readConfig: () => ({ ...actual.readConfig(), telemetryEnabled: true }) };
+});
+
+const { trackLintRun } = await import("./lintRun.js");
+const { lintProject } = await import("@hyperframes/lint");
+
+const COMPOSITION = `<html><body>
+  <div id="scene" data-composition-id="main" data-width="1920" data-height="1080"
+       data-start="0" data-duration="4">
+    <video id="clip" src="clip.mp4"></video>
+  </div>
+  <script src="gsap.min.js"></script>
+  <script>
+    window.__timelines = window.__timelines || {};
+    window.__timelines["main"] = gsap.timeline({ paused: true });
+  </script>
+</body></html>`;
+
+function makeProject(html: string): string {
+  const dir = mkdtempSync(join(tmpdir(), "hf-proj-"));
+  mkdirSync(join(dir, "compositions"), { recursive: true });
+  writeFileSync(join(dir, "index.html"), html, "utf-8");
+  return dir;
+}
+
+beforeEach(() => {
+  enqueued.length = 0;
+  rmSync(join(HOME, ".hyperframes"), { recursive: true, force: true });
+});
+
+describe("trackLintRun end to end", () => {
+  it("emits one lint_report carrying codes, timings, and the ruleset fingerprint", async () => {
+    const dir = makeProject(COMPOSITION);
+    const result = await lintProject(dir);
+    trackLintRun(dir, result, { command: "lint", durationMs: 12 });
+
+    const reports = enqueued.filter((e) => e.event === "lint_report");
+    expect(reports).toHaveLength(1);
+    const props = reports[0]!.properties;
+
+    expect(props["command"]).toBe("lint");
+    expect(props["files_scanned"]).toBe(1);
+    expect(props["duration_ms"]).toBe(12);
+    // The real linter found real problems in this composition.
+    expect((props["codes"] as string[]).length).toBeGreaterThan(0);
+    expect(props["error_count"]).toBeGreaterThan(0);
+    // Timings are attributed per rule group and a slowest rule is identified.
+    expect(Object.keys(props["rule_group_ms"] as object)).toContain("gsap");
+    expect(props["slowest_rule"]).toMatch(/^[a-z]+#\d+$/);
+    // Version and ruleset fingerprint ride along.
+    expect(props["cli_version"]).toBeTruthy();
+    expect(props["rule_count"]).toBeGreaterThan(0);
+    // Per-group sizes make the positional `slowest_rule` index comparable
+    // across builds: a group that changed size renumbered its rules.
+    const groupCounts = props["rule_group_counts"] as Record<string, number>;
+    expect(groupCounts["gsap"]).toBeGreaterThan(0);
+    expect(Object.values(groupCounts).reduce((a, b) => a + b, 0)).toBe(props["rule_count"]);
+    // code_counts sums to the number of findings.
+    const counts = Object.values(props["code_counts"] as Record<string, number>);
+    expect(counts.reduce((a, b) => a + b, 0)).toBe(
+      result.results.flatMap((r) => r.result.findings).length,
+    );
+
+    rmSync(dir, { recursive: true, force: true });
+  });
+
+  it("emits lint_rule_streak with cleared:true once an edit removes the finding", async () => {
+    const dir = makeProject(COMPOSITION);
+
+    const first = await lintProject(dir);
+    trackLintRun(dir, first, { command: "lint", durationMs: 1 });
+    const codes = first.results[0]!.result.findings.map((f) => f.code);
+    expect(codes).toContain("media_missing_data_start");
+
+    // Fix exactly that finding and re-lint.
+    writeFileSync(
+      join(dir, "index.html"),
+      COMPOSITION.replace('<video id="clip"', '<video id="clip" data-start="0" data-duration="4"'),
+      "utf-8",
+    );
+    enqueued.length = 0;
+    const second = await lintProject(dir);
+    trackLintRun(dir, second, { command: "lint", durationMs: 1 });
+
+    const streaks = enqueued.filter((e) => e.event === "lint_rule_streak");
+    const cleared = streaks.find((e) => e.properties["code"] === "media_missing_data_start");
+    expect(cleared?.properties).toMatchObject({
+      code: "media_missing_data_start",
+      cleared: true,
+      edits: 1,
+      command: "lint",
+    });
+
+    rmSync(dir, { recursive: true, force: true });
+  });
+
+  it("never throws when the lint result is malformed", () => {
+    expect(() =>
+      trackLintRun("/nope", { results: [] } as never, { command: "lint", durationMs: 0 }),
+    ).not.toThrow();
+  });
+});

packages/cli/src/telemetry/lintRun.ts

@@ -0,0 +1,83 @@
+// One place that turns a ProjectLintResult into telemetry, so `lint` and
+// `check` report identically instead of drifting apart.
+
+import { LINT_RULE_COUNT, LINT_RULE_GROUP_COUNTS, type ProjectLintResult } from "@hyperframes/lint";
+import { trackLintReport, trackLintRuleStreak } from "./events.js";
+import { recordLintRun } from "./lintStreaks.js";
+
+/**
+ * Report one lint pass: aggregate counts and timings, plus any finding that
+ * survived an edit to its file.
+ *
+ * Never throws — a telemetry failure must not fail the command that lints.
+ */
+export function trackLintRun(
+  projectDir: string,
+  lintResult: ProjectLintResult,
+  options: { command: string; durationMs: number; runId?: string },
+): void {
+  try {
+    const runIdField = options.runId !== undefined ? { runId: options.runId } : {};
+
+    trackLintReport({
+      command: options.command,
+      durationMs: options.durationMs,
+      filesScanned: lintResult.results.length,
+      errorCount: lintResult.totalErrors,
+      warningCount: lintResult.totalWarnings,
+      infoCount: lintResult.totalInfos,
+      ruleCount: LINT_RULE_COUNT,
+      ruleGroupCounts: LINT_RULE_GROUP_COUNTS,
+      ...summarize(lintResult),
+      ...runIdField,
+    });
+
+    const streaks = recordLintRun(
+      projectDir,
+      lintResult.results.map(({ file, contentHash, result }) => ({
+        file,
+        contentHash,
+        findings: result.findings,
+      })),
+    );
+    for (const streak of streaks) {
+      trackLintRuleStreak({ ...streak, command: options.command, ...runIdField });
+    }
+  } catch {
+    // Telemetry is best-effort. A malformed result, an unwritable home
+    // directory, or a transport failure must never turn a green lint red.
+  }
+}
+
+/** Roll every file's findings and timings up into one run-level summary. */
+function summarize(lintResult: ProjectLintResult): {
+  codeCounts: Record<string, number>;
+  ruleGroupMs: Record<string, number>;
+  slowestRule: string;
+  slowestRuleMs: number;
+} {
+  const codeCounts: Record<string, number> = {};
+  const ruleGroupMs: Record<string, number> = {};
+  let slowestRule = "";
+  let slowestRuleMs = 0;
+
+  for (const { result } of lintResult.results) {
+    for (const finding of result.findings) {
+      codeCounts[finding.code] = (codeCounts[finding.code] ?? 0) + 1;
+    }
+    const timings = result.timings;
+    if (!timings) continue;
+    for (const [group, ms] of Object.entries(timings.groupMs)) {
+      ruleGroupMs[group] = (ruleGroupMs[group] ?? 0) + ms;
+    }
+    if (timings.slowestRuleMs > slowestRuleMs) {
+      slowestRuleMs = timings.slowestRuleMs;
+      slowestRule = timings.slowestRule;
+    }
+  }
+
+  for (const group of Object.keys(ruleGroupMs)) {
+    ruleGroupMs[group] = Math.round(ruleGroupMs[group]!);
+  }
+  return { codeCounts, ruleGroupMs, slowestRule, slowestRuleMs };
+}

packages/cli/src/telemetry/lintStreaks.test.ts

@@ -0,0 +1,161 @@
+import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
+import { mkdtempSync, rmSync, existsSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+// Point the module's ~/.hyperframes at a scratch dir before it is imported,
+// so a test run never touches the developer's real streak history.
+const HOME = mkdtempSync(join(tmpdir(), "hf-streaks-"));
+vi.mock("node:os", async (importOriginal) => {
+  const actual = await importOriginal<typeof import("node:os")>();
+  return { ...actual, homedir: () => HOME };
+});
+
+const shouldTrack = vi.fn(() => true);
+vi.mock("./client.js", () => ({ shouldTrack: () => shouldTrack() }));
+
+const { recordLintRun, LINT_STREAK_STATE_FILE } = await import("./lintStreaks.js");
+
+const PROJECT = "/tmp/project";
+const finding = (code: string, severity = "error" as const) => ({ code, severity });
+
+/** One lint run over a single file. */
+function run(contentHash: string, codes: string[]) {
+  return recordLintRun(PROJECT, [
+    { file: "index.html", contentHash, findings: codes.map((c) => finding(c)) },
+  ]);
+}
+
+beforeEach(() => {
+  shouldTrack.mockReturnValue(true);
+  rmSync(LINT_STREAK_STATE_FILE, { force: true });
+});
+
+afterEach(() => {
+  rmSync(LINT_STREAK_STATE_FILE, { force: true });
+});
+
+describe("recordLintRun", () => {
+  it("emits nothing on a first sighting — no edit has happened yet", () => {
+    expect(run("aaa", ["gsap_from_opacity_noop"])).toEqual([]);
+  });
+
+  it("does not advance a streak when the file was not touched between runs", () => {
+    run("aaa", ["gsap_from_opacity_noop"]);
+    // Same content digest: `hyperframes check` re-linting an untouched project
+    // must not look like an agent failing to fix something.
+    expect(run("aaa", ["gsap_from_opacity_noop"])).toEqual([]);
+    expect(run("aaa", ["gsap_from_opacity_noop"])).toEqual([]);
+  });
+
+  it("reports edits_to_clear when an edit removes the finding", () => {
+    run("aaa", ["gsap_from_opacity_noop"]);
+    const events = run("bbb", []);
+    expect(events).toEqual([
+      {
+        code: "gsap_from_opacity_noop",
+        severity: "error",
+        edits: 1,
+        cleared: true,
+      },
+    ]);
+  });
+
+  it("counts the edits a finding survives before it clears", () => {
+    run("a1", ["media_missing_id"]);
+    expect(run("a2", ["media_missing_id"])).toEqual([]); // 1 edit, under threshold
+    expect(run("a3", ["media_missing_id"])).toEqual([]); // 2 edits
+    const stuck = run("a4", ["media_missing_id"]); // 3 edits -> reported
+    expect(stuck).toEqual([
+      { code: "media_missing_id", severity: "error", edits: 3, cleared: false },
+    ]);
+    const cleared = run("a5", []);
+    expect(cleared).toEqual([
+      { code: "media_missing_id", severity: "error", edits: 4, cleared: true },
+    ]);
+  });
+
+  it("reports an unresolved streak only once, not on every subsequent edit", () => {
+    run("a1", ["media_missing_id"]);
+    run("a2", ["media_missing_id"]);
+    run("a3", ["media_missing_id"]);
+    expect(run("a4", ["media_missing_id"])).toHaveLength(1);
+    expect(run("a5", ["media_missing_id"])).toEqual([]);
+    expect(run("a6", ["media_missing_id"])).toEqual([]);
+  });
+
+  it("treats a code introduced by an edit as a fresh streak, not a survivor", () => {
+    run("a1", ["media_missing_id"]);
+    const events = run("a2", ["media_missing_id", "video_missing_muted"]);
+    // media_missing_id survived (1 edit, under threshold), video_missing_muted
+    // is brand new, so neither is reportable yet.
+    expect(events).toEqual([]);
+    // The new code needs its own three edits before it counts as stuck.
+    run("a3", ["video_missing_muted"]);
+    run("a4", ["video_missing_muted"]);
+    const stuck = run("a5", ["video_missing_muted"]);
+    expect(stuck).toEqual([
+      { code: "video_missing_muted", severity: "error", edits: 3, cleared: false },
+    ]);
+  });
+
+  it("tracks each file independently", () => {
+    recordLintRun(PROJECT, [
+      { file: "index.html", contentHash: "i1", findings: [finding("media_missing_id")] },
+      { file: "compositions/a.html", contentHash: "a1", findings: [finding("media_missing_id")] },
+    ]);
+    // Only index.html is edited and fixed.
+    const events = recordLintRun(PROJECT, [
+      { file: "index.html", contentHash: "i2", findings: [] },
+      { file: "compositions/a.html", contentHash: "a1", findings: [finding("media_missing_id")] },
+    ]);
+    expect(events).toEqual([
+      { code: "media_missing_id", severity: "error", edits: 1, cleared: true },
+    ]);
+  });
+
+  it("writes no state and emits nothing when telemetry is disabled", () => {
+    shouldTrack.mockReturnValue(false);
+    expect(run("aaa", ["media_missing_id"])).toEqual([]);
+    expect(existsSync(LINT_STREAK_STATE_FILE)).toBe(false);
+  });
+
+  it("starts clean rather than throwing when the state file is corrupt", () => {
+    run("aaa", ["media_missing_id"]);
+    const { writeFileSync } = require("node:fs") as typeof import("node:fs");
+    writeFileSync(LINT_STREAK_STATE_FILE, "{ not json", "utf-8");
+    expect(() => run("bbb", ["media_missing_id"])).not.toThrow();
+  });
+
+  it("stores no file paths or project names on disk", () => {
+    run("aaa", ["media_missing_id"]);
+    const { readFileSync } = require("node:fs") as typeof import("node:fs");
+    const raw = readFileSync(LINT_STREAK_STATE_FILE, "utf-8");
+    expect(raw).not.toContain("index.html");
+    expect(raw).not.toContain(PROJECT);
+    expect(raw).toContain("media_missing_id");
+  });
+
+  it("evicts entries that have gone stale", () => {
+    const now = Date.now();
+    recordLintRun(
+      PROJECT,
+      [{ file: "old.html", contentHash: "o1", findings: [finding("media_missing_id")] }],
+      now - 30 * 24 * 60 * 60 * 1000,
+    );
+    recordLintRun(
+      PROJECT,
+      [{ file: "new.html", contentHash: "n1", findings: [finding("media_missing_id")] }],
+      now,
+    );
+    const { readFileSync } = require("node:fs") as typeof import("node:fs");
+    const state = JSON.parse(readFileSync(LINT_STREAK_STATE_FILE, "utf-8")) as {
+      files: Record<string, unknown>;
+    };
+    expect(Object.keys(state.files)).toHaveLength(1);
+  });
+});
+
+afterEach(() => {
+  rmSync(HOME, { recursive: true, force: true });
+});

packages/cli/src/telemetry/transport.ts

@@ -22,8 +22,22 @@ const FLUSH_TIMEOUT_MS = 5_000;
 // (opt-out, system-metadata enrichment, first-run notice) lives in client.ts.
 // ---------------------------------------------------------------------------
 
+// Scalars cover almost every event. String arrays and numeric maps are allowed
+// too because some facts are inherently a set (which lint rule codes fired) or
+// a histogram (how many findings per code), and flattening those into dynamic
+// top-level keys would make them unqueryable. PostHog stores both natively:
+// `arrayJoin(properties.codes)` for the array, `JSONExtractInt` for the map.
+export type EventPropertyValue =
+  | string
+  | number
+  | boolean
+  | null
+  | undefined
+  | readonly string[]
+  | Readonly<Record<string, number>>;
+
 export interface EventProperties {
-  [key: string]: string | number | boolean | null | undefined;
+  [key: string]: EventPropertyValue;
 }
 
 interface QueuedEvent {

packages/cli/src/utils/checkPipeline.ts

@@ -1,6 +1,7 @@
 import { mkdirSync, writeFileSync } from "node:fs";
 import { join, relative } from "node:path";
 import { trackCheckReport, trackCommandFailure } from "../telemetry/events.js";
+import { trackLintRun } from "../telemetry/lintRun.js";
 import { getRunId } from "../telemetry/runId.js";
 import type { ProjectDir } from "./project.js";
 import { lintProject, shouldBlockRender, type ProjectLintResult } from "./lintProject.js";
@@ -1106,6 +1107,7 @@ export async function runCheckPipeline(
   dependencies: CheckDependencies = DEFAULT_DEPENDENCIES,
 ): Promise<CheckReport> {
   let lintResult: ProjectLintResult;
+  const lintStartedAt = Date.now();
   try {
     lintResult = await dependencies.lintProject(project.dir);
   } catch (error) {
@@ -1114,6 +1116,11 @@ export async function runCheckPipeline(
     // for a script problem that doesn't exist.
     return failureReport(options, runtimeFailure(error, "check_lint_failure"));
   }
+  trackLintRun(project.dir, lintResult, {
+    command: "check",
+    durationMs: Date.now() - lintStartedAt,
+    ...(getRunId() !== undefined ? { runId: getRunId() } : {}),
+  });
 
   const lint = buildLintSection(lintResult);
   if (shouldBlockRender(true, false, lintResult.totalErrors, lintResult.totalWarnings)) {

packages/cli/src/utils/lintFormat.test.ts

@@ -19,6 +19,7 @@ function project(
   return {
     results: files.map(({ file, findings }) => ({
       file,
+      contentHash: "test",
       result: {
         ok: findings.every((f) => f.severity !== "error"),
         errorCount: findings.filter((f) => f.severity === "error").length,

packages/lint/src/hyperframeLinter.ts

@@ -1,4 +1,11 @@
-import type { HyperframeLintFinding, HyperframeLintResult, HyperframeLinterOptions } from "./types";
+import type {
+  HyperframeLintFinding,
+  HyperframeLintResult,
+  HyperframeLinterOptions,
+  LintRule,
+  LintTimings,
+} from "./types";
+import type { LintContext } from "./context";
 import { buildLintContext } from "./context";
 import { parseHtmlStructure, readAttr, truncateSnippet } from "./utils";
 import { coreRules } from "./rules/core";
@@ -11,40 +18,117 @@ import { textureRules } from "./rules/textures";
 import { fontRules } from "./rules/fonts";
 import { slideshowRules } from "./rules/slideshow";
 
-const ALL_RULES = [
+// Rules are grouped by source module so a timing can be attributed to
-  ...coreRules,
+// something a human can act on. Individual rules stay anonymous: an
-  ...mediaRules,
+// index within its group ("gsap#7") is enough to locate a pathological
-  ...gsapRules,
+// rule, and naming all 86 of them is a refactor this measurement does
-  ...captionRules,
+// not need in order to point at the right file.
-  ...compositionRules,
+//
-  ...adapterRules,
+// The cost of that choice is that the index is POSITIONAL: adding or
-  ...textureRules,
+// removing a rule renumbers every later slot in its group, so "gsap#8"
-  ...fontRules,
+// can mean different rules in two builds. LINT_RULE_GROUP_COUNTS below
-  ...slideshowRules,
+// is what makes that detectable — a consumer comparing two builds can
+// see which groups changed size and therefore which numbering is no
+// longer comparable, without anyone having to remember what shipped when.
+const RULE_GROUPS: ReadonlyArray<{
+  group: string;
+  rules: ReadonlyArray<LintRule<LintContext>>;
+}> = [
+  { group: "core", rules: coreRules },
+  { group: "media", rules: mediaRules },
+  { group: "gsap", rules: gsapRules },
+  { group: "captions", rules: captionRules },
+  { group: "composition", rules: compositionRules },
+  { group: "adapters", rules: adapterRules },
+  { group: "textures", rules: textureRules },
+  { group: "fonts", rules: fontRules },
+  { group: "slideshow", rules: slideshowRules },
 ];
 
+/**
+ * How many rules this build runs. `cli_version` already identifies the release,
+ * but a rule added or removed inside one version is invisible without this —
+ * and comparing findings-per-run across a rule change is the whole point of
+ * measuring them.
+ */
+export const LINT_RULE_COUNT = RULE_GROUPS.reduce((n, g) => n + g.rules.length, 0);
+
+/**
+ * How many rules each group runs. `slowest_rule` is reported as
+ * `<group>#<index>`, and that index shifts whenever a rule is added to or
+ * removed from its group. Comparing these counts between two builds tells a
+ * consumer exactly which groups' indices still mean the same thing.
+ */
+export const LINT_RULE_GROUP_COUNTS: Readonly<Record<string, number>> = Object.fromEntries(
+  RULE_GROUPS.map(({ group, rules }) => [group, rules.length]),
+);
+
+/** Two rules reporting the same problem on the same element report it once. */
+function dedupeKeyFor(finding: HyperframeLintFinding): string {
+  return [
+    finding.code,
+    finding.severity,
+    finding.selector || "",
+    finding.elementId || "",
+    finding.message,
+  ].join("|");
+}
+
+/**
+ * Run every rule against one parsed context, timing each and deduping as it
+ * goes. Rules are timed individually but reported per group; the slowest
+ * single rule is kept so a pathological one stays locatable.
+ */
+async function runRules(
+  ctx: LintContext,
+  filePath: string | undefined,
+): Promise<{ findings: HyperframeLintFinding[]; timings: Omit<LintTimings, "totalMs"> }> {
+  const findings: HyperframeLintFinding[] = [];
+  const seen = new Set<string>();
+  const groupMs: Record<string, number> = {};
+  let slowestRule = "";
+  let slowestRuleMs = 0;
+
+  for (const { group, rules } of RULE_GROUPS) {
+    for (let index = 0; index < rules.length; index++) {
+      const ruleStartedAt = performance.now();
+      const produced = await Promise.resolve(rules[index]!(ctx));
+      const ruleMs = performance.now() - ruleStartedAt;
+
+      groupMs[group] = (groupMs[group] ?? 0) + ruleMs;
+      if (ruleMs > slowestRuleMs) {
+        slowestRuleMs = ruleMs;
+        slowestRule = `${group}#${index}`;
+      }
+
+      collectFindings(produced, seen, filePath, findings);
+    }
+  }
+
+  return { findings, timings: { groupMs, slowestRule, slowestRuleMs } };
+}
+
+function collectFindings(
+  produced: readonly HyperframeLintFinding[],
+  seen: Set<string>,
+  filePath: string | undefined,
+  into: HyperframeLintFinding[],
+): void {
+  for (const finding of produced) {
+    const dedupeKey = dedupeKeyFor(finding);
+    if (seen.has(dedupeKey)) continue;
+    seen.add(dedupeKey);
+    into.push(filePath ? { ...finding, file: filePath } : finding);
+  }
+}
+
 export async function lintHyperframeHtml(
   html: string,
   options: HyperframeLinterOptions = {},
 ): Promise<HyperframeLintResult> {
+  const startedAt = performance.now();
   const ctx = buildLintContext(html, options);
-  const findings: HyperframeLintFinding[] = [];
+  const { findings, timings } = await runRules(ctx, options.filePath);
-  const seen = new Set<string>();
-
-  for (const rule of ALL_RULES) {
-    for (const finding of await Promise.resolve(rule(ctx))) {
-      const dedupeKey = [
-        finding.code,
-        finding.severity,
-        finding.selector || "",
-        finding.elementId || "",
-        finding.message,
-      ].join("|");
-      if (seen.has(dedupeKey)) continue;
-      seen.add(dedupeKey);
-      findings.push(options.filePath ? { ...finding, file: options.filePath } : finding);
-    }
-  }
 
   const errorCount = findings.filter((f) => f.severity === "error").length;
   const warningCount = findings.filter((f) => f.severity === "warning").length;
@@ -56,6 +140,7 @@ export async function lintHyperframeHtml(
     warningCount,
     infoCount,
     findings,
+    timings: { totalMs: performance.now() - startedAt, ...timings },
   };
 }
 

packages/lint/src/index.ts

@@ -3,7 +3,13 @@ export type {
   HyperframeLintFinding,
   HyperframeLintResult,
   HyperframeLinterOptions,
+  LintTimings,
 } from "./types.js";
-export { lintHyperframeHtml, lintMediaUrls } from "./hyperframeLinter.js";
+export {
+  lintHyperframeHtml,
+  lintMediaUrls,
+  LINT_RULE_COUNT,
+  LINT_RULE_GROUP_COUNTS,
+} from "./hyperframeLinter.js";
 export { lintProject, shouldBlockRender } from "./project.js";
 export type { ProjectLintResult } from "./project.js";

packages/lint/src/project.ts

@@ -1,4 +1,5 @@
 export { shouldBlockRender } from "./shouldBlockRender.js";
+import { createHash } from "node:crypto";
 import { existsSync, readFileSync, readdirSync } from "node:fs";
 import { dirname, extname, join, relative, resolve } from "node:path";
 import { rewriteAssetPath } from "@hyperframes/parsers/asset-paths";
@@ -47,12 +48,23 @@ function querySelectorAllIncludingTemplates(root: ParentNode, selector: string):
 }
 
 export interface ProjectLintResult {
-  results: Array<{ file: string; result: HyperframeLintResult }>;
+  results: Array<{ file: string; result: HyperframeLintResult; contentHash: string }>;
   totalErrors: number;
   totalWarnings: number;
   totalInfos: number;
 }
 
+/**
+ * Short content digest of a linted file. Callers use it to tell "the author
+ * edited this file and the finding survived" (an iteration that did not
+ * converge) from "the same file was linted twice" (no attempt was made).
+ * Truncated because it is only ever compared against the previous run's digest
+ * for the same file, never used as a security boundary.
+ */
+function contentDigest(html: string): string {
+  return createHash("sha256").update(html).digest("hex").slice(0, 16);
+}
+
 const AUDIO_EXTENSIONS = new Set([".mp3", ".wav", ".aac", ".ogg", ".m4a", ".flac", ".opus"]);
 const MASK_IMAGE_URL_RE =
   /\b(?:-webkit-)?mask-image\s*:\s*[^;{}]*url\(\s*(?:"([^"]+)"|'([^']+)'|([^"')\s]+))\s*\)/gi;
@@ -151,7 +163,7 @@ export async function lintProject(
   }
   const rootFile = relative(resolve(projectDir), indexPath).replace(/\\/g, "/") || "index.html";
   const rootCompSrcPath = rootFile === "index.html" ? undefined : rootFile;
-  const results: Array<{ file: string; result: HyperframeLintResult }> = [];
+  const results: ProjectLintResult["results"] = [];
   let totalErrors = 0;
   let totalWarnings = 0;
   let totalInfos = 0;
@@ -161,7 +173,7 @@ export async function lintProject(
     filePath: indexPath,
     externalStyles: collectExternalStyles(projectDir, rootHtml, rootCompSrcPath),
   });
-  results.push({ file: rootFile, result: rootResult });
+  results.push({ file: rootFile, result: rootResult, contentHash: contentDigest(rootHtml) });
   totalErrors += rootResult.errorCount;
   totalWarnings += rootResult.warningCount;
   totalInfos += rootResult.infoCount;
@@ -201,7 +213,11 @@ export async function lintProject(
         isSubComposition: true,
         externalStyles: collectExternalStyles(projectDir, html, compSrcPath),
       });
-      results.push({ file: `compositions/${file}`, result });
+      results.push({
+        file: `compositions/${file}`,
+        result,
+        contentHash: contentDigest(html),
+      });
       totalErrors += result.errorCount;
       totalWarnings += result.warningCount;
       totalInfos += result.infoCount;

packages/lint/src/types.ts

@@ -11,12 +11,25 @@ export type HyperframeLintFinding = {
   snippet?: string;
 };
 
+/**
+ * Where a single lint pass spent its time. Attributed per rule-source module
+ * ("gsap", "core", ...) rather than per rule, plus the single slowest rule as
+ * `<group>#<index-within-group>` so a pathological rule is locatable.
+ */
+export type LintTimings = {
+  totalMs: number;
+  groupMs: Record<string, number>;
+  slowestRule: string;
+  slowestRuleMs: number;
+};
+
 export type HyperframeLintResult = {
   ok: boolean;
   errorCount: number;
   warningCount: number;
   infoCount: number;
   findings: HyperframeLintFinding[];
+  timings?: LintTimings;
 };
 
 export type HyperframeLinterOptions = {