Files
hyperframes/evals
Miguel Angel Simon Sierra 424e377990 feat(evals): confirm a grid of cells, one variable at a time (U6)
Round 4 confirms end to end at several list lengths and on two models, so
the paid stage takes a grid of (arm, k, model) cells rather than a pair.

Exactly one of the three may move between two compared cells. A pairing that
would move two is refused rather than computed and qualified in the write-up:
an arm effect and a list length effect land on the same difference and nothing
in the output can say which was which. The comparison stays paired brief by
brief, and a brief only one cell ran is dropped from that pairing and from no
other, so the shared set is recomputed per pairing rather than once for the
grid.

Fixes a real defect on this path. build_report2.cell_key was model blind while
run2._record_key and run2._cell_name already carried the model, so a round 4
checkpoint holding both models would have merged model A's answers with model
B's into one cell and reported the average as a reproduction. A record that
carries no model keeps round 2's published label exactly, because round 2's
report is a committed artifact.

The temperature confound survives into the report. gpt-5.6-luna rejects
temperature 0, which every round 2 and round 3 cell used, so a model B cell
differs from its model A twin in the model and in the sampling temperature. The
pairing, the printed comparison and the rendered table all carry it, and the
cell that will not sample at 0 is named from the temperatures rather than
assumed to be the leader. provider owns that fact now, and the two backends
that send a temperature read their default from it.

The dry run prices the whole grid, cells times models, with each family priced
from its own row and a grand total that is a sum and not an average. A paid run
that names more than one model is refused instead of quietly spending on one
and reporting the grid as done.

round3/report3.html still renders byte for byte at 37794 bytes from the
committed summary and confirmation, now asserted by a test.
2026-08-01 15:32:38 +02:00
..