The OPE-186 trial (15 long tasks) compacted 64 times and gave a usable summary on 7 of 103
summariser calls. Two independent causes, both fixed here.
1. The summariser was asked last and read the session last. `summarizer_messages`
sent the ~1,800-character instruction as the system message and then tens of
thousands of characters of transcript, ending wherever the session happened to
be — usually on a raw tool result. The model continued that voice instead of
summarising: 13 of 16 short replies were a next-action sentence ("The output
got truncated. Let me re-run the remaining checks." — 61 characters standing in
for 75 turns), each stopping voluntarily with most of its budget unused. The
transcript is now delimited and the instruction restated after it, in bold,
ending on the exact first heading to write.
2. The budget was 3,000 tokens, shared with the model's reasoning. On 60 of 103
calls Kimi K3 spent it thinking and the summary was cut off or never started.
SUMMARY_MAX_TOKENS is now 16,000 and is a setting: `compaction_summary_max_tokens`
in config.toml, COWORKER_COMPACTION_SUMMARY_MAX_TOKENS in the environment.
Measured by replaying 20 recorded compactions from that trial against Kimi K3,
scored by the gate below: the old shape at 3,000 tokens gives 4/20 usable; moving
the instruction alone gives 18/20; with the 16,000 budget, 20/20. Disabling
reasoning for the call adds nothing over that, so it is not done.
Also:
* A quality gate before a summary is accepted. "Not empty" let a 61-character
sentence become the coworker's whole memory of a span. `summary_quality_problem`
refuses empty, under 400 characters, or missing the section headings, and
`summarize_span` raises — so the engine's existing single retry runs before the
no-summary trim fallback.
* The first user message is pinned in the compacted block, whole, capped at 4,000
tokens. `_cap_user_messages` kept the newest 40, so any session with 41 or more
user messages silently dropped the opening mandate — the one message that never
goes stale. Omissions now come from the middle, and the block says so there.
* Per-message clipping is replaced by a token budget filled backwards from the
newest. Each preserved user message was cut at 600 characters; 8 of the 15 trial
task prompts are longer, up to 1,424, so after every compaction the agent's
verbatim record of its job was cut mid-sentence. On video-processing that
discarded the output path and both field names the task checks. The budget
is 8% of the compaction trigger, bounded to [2,000, 20,000] — a fraction, so
lowering the trigger to save tokens is not undone by the block, and 20,000 at
the default trigger is a generous allowance.
The trigger itself is unchanged: min(80% of the context window, cap), cap 250,000,
overridden by `compaction_cap_tokens`. Lowering the default should follow a working
summariser, not precede it.
Unrelated, included here to keep the tree clean: model-matrix rows for Claude Opus
4.6, Opus 4.5 and Sonnet 4.5. These are added so OpenWorker can be measured on
those models against published evaluations of the same models.
Tests: tests/test_compaction.py, tests/test_compaction_engine.py and
tests/test_compaction_transcript.py cover the new request shape, the gate, the pin,
the budget and the config/env plumbing. Their fake summarisers were returning
strings the gate now refuses, and were updated to real-shaped summaries.
Findings from a long multi-task Kimi K3 session: ten long tasks used 57% of
451 M tokens, tool results were 31% of the final context and whole-file
write_file bodies 14%, and compaction fired once. Three harness changes:
1. Tool-result cap. Every tool result is bounded on arrival at 10,000 bytes,
head + tail, with a marker naming the spill file that holds the full text
(coworker/toolresult.py; engine `_record_result`; config
`tool_result_max_bytes`, env COWORKER_TOOL_RESULT_MAX_BYTES, 0 = off).
The spill folder is the session's scratch root when there is one, else it
joins the session's directories read-only so read_file can open it.
shell.py's own 20k-char tail cut becomes a 2M safety net.
2. Edit steering. write_file, replace_in_file, apply_patch and
apply_unified_diff describe WHEN to use each (catalog.EDIT_TOOL_GUIDANCE),
and the Cowork prompt says to edit in place rather than rewrite.
3. Compaction. `compaction_cap_tokens` (config / env
COWORKER_COMPACTION_CAP_TOKENS) lowers the trigger; each compaction writes
the verbatim transcript of the compacted turns (no reasoning) to the spill
folder and the compacted block names the file (CompactionState.transcript_path,
compaction.render_transcript).
Trial on 15 long tasks x 3 attempts (Kimi K3 via Together, cap 60,000):
33/45 passed vs 54/75 for the unchanged harness on the same tasks, tokens per
attempt 3.30 M -> 1.40 M, cost down 44%. Observed and left for a
follow-up: Kimi's summariser reply lands mostly in reasoning within the
3,000-token summary budget (7 proper summaries in 103 calls), and the trial had
16 timeouts vs 9 in the baseline.
Tests: tests/test_tool_result_cap.py, tests/test_edit_tool_guidance.py,
tests/test_compaction_transcript.py; full suite unchanged against the Windows
baseline.
Devika Verma