fix(catalog): rank on where a word appears and how rare it is (#3312)

* fix(catalog): rank on where a word appears and how rare it is

Word search returned the right move in the top three for 87% of a
39-query eval set built from real catalog intents. Three defects, all in
the same 75-line scorer, and all found by running the queries rather than
by reading the code.

A token matching an item's NAME counted exactly as much as one buried in
a description. Searching "typewriter effect on a title" ranked the item
literally called `typewriter` seventh, behind entries that merely mention
typing. Name and title now carry three times the weight: an author who
types a move's name is giving the strongest signal available and it was
being averaged away.

Plurals shared no vocabulary with the singular. "a stat that counts up
and then pulses once" matched nothing in a description reading "lands
with a restrained scale pulse", because `counts` is not `count`. Adding
detail to a query made results strictly worse, which is the opposite of
what a search should do. Plurals now fold, and only plurals: Porter would
fold `counter` to `count` and `values` to `valu`, merging moves that mean
different things.

Field weighting alone made one case worse, which is why inverse document
frequency is here too. "reveal a headline one line at a time" put every
item merely NAMED `*-reveal` on top, because one strong hit on the
catalog's most common word outscored several weak hits on the words that
actually narrowed it down. Rarity now scales each term.

Separately: a query in a script this ranker cannot index no longer
reports itself as an empty catalog. Tokenising on [a-z]+ leaves nothing
of a Japanese query, and returning "no items match" told the author the
catalog lacked a move it may well have, then invited them to file a gap
report about it. That case now says what actually happened and withholds
the gap prompt, since nothing was searched.

Measured on the same 39 queries, before and after:
  top-1  31/39 (79%) -> 33/39 (85%)
  top-3  34/39 (87%) -> 39/39 (100%)

Test plan: 13 new tests, each a real failing query reduced to the
smallest fixture that still reproduces it. Existing tests migrated to the
fields API (two callers total). Full CLI suite 2643 passed, 2
pre-existing transcribe failures unchanged. Verified against the real
CLI: "typewriter effect on a title" now returns typewriter first, and
"chat conversation between a user and an assistant" returns chat-message,
chat-thread, ai-chat-reveal instead of transitions-blur.

* docs(skills): say to query the catalog in English

The runtime message added alongside this explains an unsearchable query
after the fact. Saying it up front is cheaper: an agent that never writes
the query in Japanese never sees the error, never wastes the turn, and
never files a gap report about a component that exists.

Worth stating rather than assuming, because the mistake is a reasonable
one. On a Japanese or Chinese project the brief, the narration and the
captions are all in that language and the query naturally follows. The
rule is that the query language and the video language are unrelated:
describe the move in English, write the on-screen copy in whatever the
video needs.

Both skills that own `catalog --query` carry it, and those are the only
two that mention the command at all.

* fix(catalog): fail a non-English query instead of returning nothing

The message explaining an unsearchable query went to stdout and the
command exited 0. An agent that checks the exit code, which is most of
them, read that as "searched successfully, the catalog has nothing" and
went off to hand-author a move that is sitting in the registry. The
explanation only helped a human who happened to be reading the terminal.

It is bad input, not an empty shelf, so it now behaves like one: the
guidance goes to stderr and the command exits 1, matching what an invalid
--type already does. A genuine empty result, where the query parsed fine
and the catalog simply has nothing, still exits 0 -- that distinction is
the whole point, and both halves are pinned by tests.

The wording now also says what to do rather than only what happened:
search in English, and let the on-screen copy of the video stay in
whatever language it needs. That was the part agents were getting wrong,
since a Japanese project makes a Japanese query feel natural.

Test plan: 3 new tests covering the exit code, the wording, and the
genuine-empty case that must stay at 0. Also asserts the gap-report line
is absent, since nothing was searched and a report there is noise in the
one signal that tells us what to build. catalog.test.ts 32 passed;
commands + registry suites 887 passed with the 2 pre-existing transcribe
failures unchanged. Verified against the real CLI: a CJK query exits 1, a
genuine miss exits 0.

packages/cli/src/registry/localSearch.test.ts

@@ -1,6 +1,6 @@
 import { describe, expect, it } from "vitest";
 
-import { rankByWords, searchByWords, tokenize } from "./localSearch.js";
+import { hasNoSearchableTokens, rankByWords, searchByWords, tokenize } from "./localSearch.js";
 
 interface Item {
   name: string;
@@ -13,7 +13,7 @@ const ITEMS: Item[] = [
   { name: "text-reveal", text: "Type reveals line by line beneath a mask." },
 ];
 
-const textOf = (item: Item) => item.text;
+const textOf = (item: Item) => ({ strong: "", weak: item.text });
 
 describe("tokenize", () => {
   it("drops stop words", () => {
@@ -25,7 +25,7 @@ describe("tokenize", () => {
   });
 
   it("lowercases and strips punctuation and digits", () => {
-    expect(tokenize("Camera, pushes 42 times!")).toEqual(["camera", "pushes", "times"]);
+    expect(tokenize("Camera, pushes 42 times!")).toEqual(["camera", "push", "time"]);
   });
 
   it("returns nothing for a query made only of stop words", () => {
@@ -95,3 +95,91 @@ describe("searchByWords", () => {
     expect(searchByWords("quantum entanglement", ITEMS, textOf)).toEqual([]);
   });
 });
+
+// ── The three ranking defects this scorer exists to fix ─────────────────────
+// Each case is a query that failed against the real catalog, reduced to the
+// smallest fixture that still reproduces it.
+
+const named = (name: string, title: string, body: string) => ({ name, title, body });
+const fieldsOf = (i: { name: string; title: string; body: string }) => ({
+  strong: `${i.name} ${i.title}`,
+  weak: i.body,
+});
+
+describe("name and title outrank description", () => {
+  const items = [
+    named("typewriter", "Typewriter", "Character-by-character text reveal with a blinking cursor."),
+    named("mk-emphasis-type", "Emphasis Type", "A phrase types on with emphasis on the key word."),
+    named("variable-axis-type", "Variable Axis Type", "Type that shifts along a variable axis."),
+  ];
+
+  it("puts the item actually called typewriter first", () => {
+    // Ranked seventh against the real catalog before field weighting: an author
+    // typing a move's name got entries that merely mention typing.
+    expect(rankByWords("typewriter effect on a title", items, fieldsOf)[0]?.item.name).toBe(
+      "typewriter",
+    );
+  });
+});
+
+describe("plurals fold to the singular", () => {
+  const items = [
+    named(
+      "count-up",
+      "Count Up",
+      "A stat counter that eases between values and lands with a pulse.",
+    ),
+    named("skeleton-reveal", "Skeleton Reveal", "Placeholder blocks resolve into content once."),
+  ];
+
+  it("matches a pluralised query against a singular description", () => {
+    // "counts" is not "count" and "pulses" is not "pulse", so adding detail to
+    // a query used to make the result strictly worse.
+    expect(
+      rankByWords("a stat that counts up and then pulses once", items, fieldsOf)[0]?.item.name,
+    ).toBe("count-up");
+  });
+
+  it("leaves a word that merely ends in s alone", () => {
+    expect(tokenize("press glass")).toEqual(["press", "glass"]);
+  });
+});
+
+describe("common words count for less than distinctive ones", () => {
+  const items = [
+    named("line-by-line-slide", "Line By Line Slide", "A headline arrives one line at a time."),
+    named("ui-3d-reveal", "UI 3D Reveal", "A panel reveals in three dimensions."),
+    named("panel-reveal", "Panel Reveal", "A panel reveals itself."),
+    named("pull-back-reveal", "Pull Back Reveal", "The camera pulls back to reveal the scene."),
+  ];
+
+  it("does not let one hit on a catalog-wide word beat several precise ones", () => {
+    // Field weighting alone made this worse: every item merely NAMED *-reveal
+    // outranked the one that does the thing, because "reveal" is the catalog's
+    // most common word. Inverse document frequency is what settles it.
+    expect(rankByWords("reveal a headline one line at a time", items, fieldsOf)[0]?.item.name).toBe(
+      "line-by-line-slide",
+    );
+  });
+});
+
+describe("hasNoSearchableTokens", () => {
+  it("flags a query in a script this ranker cannot index", () => {
+    // The catalog is written in English and tokenising on [a-z]+ leaves nothing
+    // of a Japanese query. Reporting that as "no items match" told the author
+    // the catalog lacked a move it may well have.
+    expect(hasNoSearchableTokens("実写写真のみ 9:16 生活ハック")).toBe(true);
+  });
+
+  it("does not flag an ordinary query that simply matches nothing", () => {
+    expect(hasNoSearchableTokens("quantum entanglement")).toBe(false);
+  });
+
+  it("does not flag an empty query, which is a listing rather than a search", () => {
+    expect(hasNoSearchableTokens("   ")).toBe(false);
+  });
+
+  it("flags a query that is only stop words and punctuation", () => {
+    expect(hasNoSearchableTokens("the and of !!!")).toBe(true);
+  });
+});

packages/cli/src/registry/localSearch.ts

@@ -6,12 +6,26 @@
  * so the honest result was zero matches. Scoring shared vocabulary answers it
  * partially, offline, with no model and no network.
  *
- * The scorer is the one the retrieval evaluation used, reproduced so the local
- * arm and the remote fallback rank identically rather than merely similarly:
- * lowercase alphabetic tokens, stop words dropped, anything three characters or
- * shorter dropped, and the shared-token count divided by the square root of the
- * entry's token count. That divisor is load-bearing. Without it the wordiest
- * entry wins every query on sheer surface area.
+ * The scorer is the one the retrieval evaluation used: lowercase tokens, stop
+ * words dropped, anything three characters or shorter dropped, and the shared
+ * token count divided by the square root of the entry's token count. That
+ * divisor is load-bearing. Without it the wordiest entry wins every query on
+ * sheer surface area.
+ *
+ * Two things sit on top of that, both measured against a fixed eval set of
+ * catalog queries rather than adjusted by feel:
+ *
+ * 1. A token matching an item's NAME or TITLE counts for more than one
+ *    matching its description. Without this, searching "typewriter effect on a
+ *    title" ranked the item literally called `typewriter` seventh, behind
+ *    entries that merely mention typing. An author who types a move's name is
+ *    giving the strongest signal available and it was being averaged away.
+ *
+ * 2. Plurals fold to their singular on both sides. "a stat that counts up and
+ *    then pulses once" shares no token with a description reading "lands with
+ *    a restrained scale pulse", because `counts` is not `count` and `pulses`
+ *    is not `pulse`. Adding detail to a query made results strictly worse,
+ *    which is the opposite of what a search should do.
  *
  * Ties break on descending name, matching the evaluation's sort.
  */
@@ -26,9 +40,51 @@ const STOP = new Set(
   ).split(" "),
 );
 
+/** How much more a name/title token is worth than a description token. */
+const STRONG_FIELD_WEIGHT = 3;
+
+/**
+ * Fold a plural to its singular, and nothing else.
+ *
+ * Deliberately not a real stemmer. Porter would fold `counter` to `count` and
+ * `values` to `valu`, which merges moves that mean different things and makes
+ * the failure harder to read when it happens. Plurals are the case that
+ * actually bit: every extra descriptive word an author adds tends to arrive
+ * pluralised while the catalog writes its descriptions in the singular.
+ */
+const PLURAL_RULES: ReadonlyArray<readonly [RegExp, string]> = [
+  [/([^aeiou])ies$/, "$1y"], // stories -> story
+  [/(ch|sh|ss|x)es$/, "$1"], // matches -> match, glasses -> glass
+  [/(ss|us|is)$/, "$&"], // press, status, axis: the trailing s is not a plural
+  [/s$/, ""], // counts -> count, pulses -> pulse
+];
+
+function singularize(word: string): string {
+  if (word.length <= 3) return word;
+  // First rule wins, so the guard above `s$` is what protects press and status.
+  for (const [pattern, replacement] of PLURAL_RULES) {
+    if (pattern.test(word)) return word.replace(pattern, replacement);
+  }
+  return word;
+}
+
 export function tokenize(text: string): string[] {
   const words = text.toLowerCase().match(/[a-z]+/g) ?? [];
-  return words.filter((word) => word.length > 2 && !STOP.has(word));
+  return words.filter((word) => word.length > 2 && !STOP.has(word)).map(singularize);
+}
+
+/**
+ * True when a query contains no token this ranker can search on.
+ *
+ * The catalog is written in English and tokenising on `[a-z]+` means a query
+ * in another script produces nothing at all. That used to be indistinguishable
+ * from "the catalog has nothing like this", so a Japanese query printed the
+ * same "no items match" as a genuine gap and sent the author off to report a
+ * missing move that may well exist. Callers use this to say which of the two
+ * actually happened.
+ */
+export function hasNoSearchableTokens(query: string): boolean {
+  return tokenize(query).length === 0 && query.trim().length > 0;
 }
 
 export interface Scored<T> {
@@ -36,6 +92,14 @@ export interface Scored<T> {
   score: number;
 }
 
+/** The searchable text of an item, split by how much a match in it counts. */
+export interface ItemText {
+  /** Name and title: what an author types when they know what they want. */
+  strong: string;
+  /** Description, tags, everything else. */
+  weak: string;
+}
+
 /**
  * Rank every item by shared vocabulary, best first.
  *
@@ -45,25 +109,51 @@ export interface Scored<T> {
 export function rankByWords<T>(
   query: string,
   items: T[],
-  textOf: (item: T) => string,
+  textOf: (item: T) => ItemText,
 ): Scored<T>[] {
   const want = new Set(tokenize(query));
   if (want.size === 0) return items.map((item) => ({ item, score: 0 }));
 
-  return items
-    .map((item) => {
-      const have = new Set(tokenize(textOf(item)));
+  const parsed = items.map((item) => {
+    const { strong, weak } = textOf(item);
+    const strongTokens = new Set(tokenize(strong));
+    return { item, strongTokens, allTokens: new Set([...strongTokens, ...tokenize(weak)]) };
+  });
+
+  // How rare each queried word is across the catalog. Without this a common
+  // word carries the same weight as a distinctive one, and field weighting
+  // makes that worse rather than better: searching "reveal a headline one line
+  // at a time" put every item merely NAMED `*-reveal` on top, because one
+  // strong hit on the catalog's most common word outscored several weak hits
+  // on the words that actually narrowed it down.
+  const idf = new Map<string, number>();
+  for (const token of want) {
+    const df = parsed.reduce((count, p) => count + (p.allTokens.has(token) ? 1 : 0), 0);
+    // +1 inside the log keeps a token present in every item at a small
+    // positive weight rather than exactly zero: still nearly worthless, but
+    // never able to flip a tie on its own.
+    idf.set(token, Math.log((parsed.length + 1) / (df + 1)) + 1);
+  }
+
+  return parsed
+    .map(({ item, strongTokens, allTokens }) => {
       let shared = 0;
-      for (const token of want) if (have.has(token)) shared += 1;
+      for (const token of want) {
+        const weight = idf.get(token) ?? 1;
+        if (strongTokens.has(token)) shared += STRONG_FIELD_WEIGHT * weight;
+        else if (allTokens.has(token)) shared += weight;
+      }
       // sqrt normalization: a longer entry has more chances to overlap, and
-      // without this the wordiest blurb ranks first for every query.
-      return { item, score: shared / (Math.sqrt(have.size) || 1) };
+      // without this the wordiest blurb ranks first for every query. Measured
+      // over the whole token set, so weighting a field cannot be gamed by
+      // moving words into the name.
+      return { item, score: shared / (Math.sqrt(allTokens.size) || 1) };
     })
     .sort((a, b) => b.score - a.score || nameOf(b.item).localeCompare(nameOf(a.item)));
 }
 
 /** Only items sharing at least one token, best first. */
-export function searchByWords<T>(query: string, items: T[], textOf: (item: T) => string): T[] {
+export function searchByWords<T>(query: string, items: T[], textOf: (item: T) => ItemText): T[] {
   return rankByWords(query, items, textOf)
     .filter((scored) => scored.score > 0)
     .map((scored) => scored.item);