Files
hyperframes/evals
Miguel Angel Simon Sierra a4e879f5ed feat(eval): report round 3 with intervals and a named stop reason
Round 3's arm ranking is decided by recall, which the retriever computes
before a token is bought, so the report that carries it should cost nothing
and be readable without the paid stage having run.

Three things drove the shape. Every table ships a clustered interval from
stats.py rather than a bare mean, because a recall number without one cannot
be read against the pre-registered rule; that forced the recall curve into
long form, since a wide arm-by-k grid has no room for error bars. The report
names which of the three stopping conditions fired, because an arm that won
on measurement, a round that ran out of corpus, and a round that ran out of
budget are three different verdicts and collapsing them into one is how a
tie gets reported as a win. And the summary file replaces infinite bounds
with null, since JSON has no infinity and a file only one language can read
back is not an auditable artifact.

VERDICT.md is deliberately pending. The paid confirmation has not run, so it
states the rule, the three stop reasons and the 424-entry shelf ceiling, and
lists every absent number beside the file that will produce it, rather than
estimating any of them. The ceiling is the load-bearing part: no corpus size
buys more than 424 independent things to be right about, so if the arms do
not separate the honest recommendation is a larger catalog, not a larger
corpus.

Tests are unittest, matching the rest of round 3, and parse the rendered
HTML rather than grepping it, since a grep passes on a caption that merely
mentions an interval.
2026-07-31 22:10:15 +02:00
..