The OPE-186 trial (15 long tasks) compacted 64 times and gave a usable summary on 7 of 103
summariser calls. Two independent causes, both fixed here.
1. The summariser was asked last and read the session last. `summarizer_messages`
sent the ~1,800-character instruction as the system message and then tens of
thousands of characters of transcript, ending wherever the session happened to
be — usually on a raw tool result. The model continued that voice instead of
summarising: 13 of 16 short replies were a next-action sentence ("The output
got truncated. Let me re-run the remaining checks." — 61 characters standing in
for 75 turns), each stopping voluntarily with most of its budget unused. The
transcript is now delimited and the instruction restated after it, in bold,
ending on the exact first heading to write.
2. The budget was 3,000 tokens, shared with the model's reasoning. On 60 of 103
calls Kimi K3 spent it thinking and the summary was cut off or never started.
SUMMARY_MAX_TOKENS is now 16,000 and is a setting: `compaction_summary_max_tokens`
in config.toml, COWORKER_COMPACTION_SUMMARY_MAX_TOKENS in the environment.
Measured by replaying 20 recorded compactions from that trial against Kimi K3,
scored by the gate below: the old shape at 3,000 tokens gives 4/20 usable; moving
the instruction alone gives 18/20; with the 16,000 budget, 20/20. Disabling
reasoning for the call adds nothing over that, so it is not done.
Also:
* A quality gate before a summary is accepted. "Not empty" let a 61-character
sentence become the coworker's whole memory of a span. `summary_quality_problem`
refuses empty, under 400 characters, or missing the section headings, and
`summarize_span` raises — so the engine's existing single retry runs before the
no-summary trim fallback.
* The first user message is pinned in the compacted block, whole, capped at 4,000
tokens. `_cap_user_messages` kept the newest 40, so any session with 41 or more
user messages silently dropped the opening mandate — the one message that never
goes stale. Omissions now come from the middle, and the block says so there.
* Per-message clipping is replaced by a token budget filled backwards from the
newest. Each preserved user message was cut at 600 characters; 8 of the 15 trial
task prompts are longer, up to 1,424, so after every compaction the agent's
verbatim record of its job was cut mid-sentence. On video-processing that
discarded the output path and both field names the task checks. The budget
is 8% of the compaction trigger, bounded to [2,000, 20,000] — a fraction, so
lowering the trigger to save tokens is not undone by the block, and 20,000 at
the default trigger is a generous allowance.
The trigger itself is unchanged: min(80% of the context window, cap), cap 250,000,
overridden by `compaction_cap_tokens`. Lowering the default should follow a working
summariser, not precede it.
Unrelated, included here to keep the tree clean: model-matrix rows for Claude Opus
4.6, Opus 4.5 and Sonnet 4.5. These are added so OpenWorker can be measured on
those models against published evaluations of the same models.
Tests: tests/test_compaction.py, tests/test_compaction_engine.py and
tests/test_compaction_transcript.py cover the new request shape, the gate, the pin,
the budget and the config/env plumbing. Their fake summarisers were returning
strings the gate now refuses, and were updated to real-shaped summaries.
Settings -> Models grows a Context compaction card next to Token savings:
the trigger % of the context window (10-95), the absolute token cap
(clamped 10k-2M), and the summarizer-model pin (default: the session's
own model). POST /v1/settings/compaction persists them; engines read the
knobs live per check, so changes apply to running sessions immediately.
The "context compacted" divider rides the existing notice machinery: the
persisted `compacted` notice replays on reload (itemsFromMessages) and
the live COMPACTED event appends the same info notice mid-turn. The
transcript itself stays intact - outbound-only by construction.
Covered by vitest (marker replay), a settings-card e2e (defaults +
clamped POSTs + model pin), and a mid-session divider e2e driven by the
fixtures' scripted `compacted` event.
Minimal engine footprint: a checkpoint at each iteration top (between tool
turns and before a new turn), the usage signal captured per round-trip
(context_tokens; chars/4 estimate when never reported), and
_outbound_messages consulting the boundary. The summarizer runs off-loop
through the normal provider router, so the Settings model pin is just an
id.
Failure policy per spec: retry once in both modes; attended sessions get
the Retry / Trim-oldest-10% prompt (via the ask_user plumbing, gated by an
is_attended callback the WS surface wires); unattended runs auto-trim and
continue — never parked on internal bookkeeping. Raw context-overflow 400s
from the main model route into the same policy, progress-guarded so a
still-overflowing model terminates in the error path.
CompactionState persists on the session record (new sqlite column, same
defensive parse as grants), so reloads keep the compacted view. A
persisted compacted notice + a new COMPACTED event mark the spot for
the GUI divider (rendered in commit 3).
Devika Verma