mirror of
https://github.com/andrewyng/openworker.git
synced 2026-09-03 13:00:37 +00:00
Pin eval renderer to engine renderer with a parity test
render_known_world (the exam's prompt builder) promised to match KnownWorld.render() (production's) by comment only - format drift would silently grade the reviewer against a stale prompt shape. Now every corpus setup renders through both and must come out byte-identical (plus a fixed example incl. the empty-world collapse, and a corpus format pin: remotes must be 'name url' since the engine renderer has no name-only representation).
This commit is contained in:
@@ -85,7 +85,9 @@ def load_corpus(name: str) -> list[Row]:
|
||||
|
||||
def render_known_world(setup: dict[str, Any]) -> str:
|
||||
"""Reconstruct the reviewer's known-world block from a corpus row's `setup`, matching
|
||||
KnownWorld.render() — folders and remotes only, never hostnames (spec §2.4)."""
|
||||
KnownWorld.render() — folders and remotes only, never hostnames (spec §2.4). Parity is
|
||||
ENFORCED, not just intended: test_shadow_eval.py renders every corpus setup through
|
||||
both this and the engine's renderer and requires byte-identical output."""
|
||||
lines = ["KNOWN WORLD (frozen when this session started)"]
|
||||
for root in setup.get("roots", []):
|
||||
writable = "read-write" if root.get("writable") else "read-only"
|
||||
|
||||
Reference in New Issue
Block a user