Audited the checks added in this PR by building the defects they are meant to
stop and confirming each one goes red. The ones that did not are closed here;
the changes are test-only.
The attachment assertions were one-sided. `est < 5_000` certifies an estimate
that collapsed to 26 exactly as readily as the correct ~1,500, and that
collapse is the inverse defect: a history that really is over the cap, held
permanently under it. Both attachment checks are now bracketed on both sides,
and a differential check asserts the estimate is *independent* of data-URL
length across the range attachments.py actually admits (12 M / 15 M), rather
than sampling one size and inferring the property.
Also newly covered: text and metadata inside a multi-part message (only
image_url and file parts get the flat charge; everything else must still be
measured), the `or` -> `is not None` boundary fix, which changed real behaviour
with nothing pinning it, and the `assistants[-1]` fallback.
The linearity claim now has a clock-free check that counts per-message
measurements (<= 3n; the quadratic version ran ~150x that), so the complexity
guarantee no longer rests on a wall-clock bound evaluated on a shared CI
runner. The timing test stays as a companion — it catches a regression that
keeps the call count linear but makes each call expensive.
The module now carries a written statement of what it does not cover — the
absolute accuracy of the 1,500 / 3,000 constants, which are bracketed to an
order of magnitude but not anchored to a published provider formula.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wevp5RAtiXBunmUuXfA6hk
compaction.py
- estimate_tokens charged image/PDF content parts their base64 length. A 1.5 MB
screenshot measured 375,052 tokens against a 160,000 trigger, so a single
attachment sat permanently above cap_tokens and re-fired compaction on every
iteration of the turn loop. Charge a flat per-part estimate instead; text-only
histories are byte-identical (asserted).
- pick_boundary measured each candidate with estimate_tokens(messages[i:]),
re-serializing the whole tail per candidate. One backward pass of suffix sums
gives the same boundary in 6.7 ms instead of 950 ms on a 1,201-message session.
- `_fit(assistants) or ...` discarded a legitimate index 0.
Full backend suite green (1092 passed). New tests verified to fail against
unpatched source.
The OPE-186 trial (15 long tasks) compacted 64 times and gave a usable summary on 7 of 103
summariser calls. Two independent causes, both fixed here.
1. The summariser was asked last and read the session last. `summarizer_messages`
sent the ~1,800-character instruction as the system message and then tens of
thousands of characters of transcript, ending wherever the session happened to
be — usually on a raw tool result. The model continued that voice instead of
summarising: 13 of 16 short replies were a next-action sentence ("The output
got truncated. Let me re-run the remaining checks." — 61 characters standing in
for 75 turns), each stopping voluntarily with most of its budget unused. The
transcript is now delimited and the instruction restated after it, in bold,
ending on the exact first heading to write.
2. The budget was 3,000 tokens, shared with the model's reasoning. On 60 of 103
calls Kimi K3 spent it thinking and the summary was cut off or never started.
SUMMARY_MAX_TOKENS is now 16,000 and is a setting: `compaction_summary_max_tokens`
in config.toml, COWORKER_COMPACTION_SUMMARY_MAX_TOKENS in the environment.
Measured by replaying 20 recorded compactions from that trial against Kimi K3,
scored by the gate below: the old shape at 3,000 tokens gives 4/20 usable; moving
the instruction alone gives 18/20; with the 16,000 budget, 20/20. Disabling
reasoning for the call adds nothing over that, so it is not done.
Also:
* A quality gate before a summary is accepted. "Not empty" let a 61-character
sentence become the coworker's whole memory of a span. `summary_quality_problem`
refuses empty, under 400 characters, or missing the section headings, and
`summarize_span` raises — so the engine's existing single retry runs before the
no-summary trim fallback.
* The first user message is pinned in the compacted block, whole, capped at 4,000
tokens. `_cap_user_messages` kept the newest 40, so any session with 41 or more
user messages silently dropped the opening mandate — the one message that never
goes stale. Omissions now come from the middle, and the block says so there.
* Per-message clipping is replaced by a token budget filled backwards from the
newest. Each preserved user message was cut at 600 characters; 8 of the 15 trial
task prompts are longer, up to 1,424, so after every compaction the agent's
verbatim record of its job was cut mid-sentence. On video-processing that
discarded the output path and both field names the task checks. The budget
is 8% of the compaction trigger, bounded to [2,000, 20,000] — a fraction, so
lowering the trigger to save tokens is not undone by the block, and 20,000 at
the default trigger is a generous allowance.
The trigger itself is unchanged: min(80% of the context window, cap), cap 250,000,
overridden by `compaction_cap_tokens`. Lowering the default should follow a working
summariser, not precede it.
Unrelated, included here to keep the tree clean: model-matrix rows for Claude Opus
4.6, Opus 4.5 and Sonnet 4.5. These are added so OpenWorker can be measured on
those models against published evaluations of the same models.
Tests: tests/test_compaction.py, tests/test_compaction_engine.py and
tests/test_compaction_transcript.py cover the new request shape, the gate, the pin,
the budget and the config/env plumbing. Their fake summarisers were returning
strings the gate now refuses, and were updated to real-shaped summaries.
Trigger math (usage signal, chars/4 estimate fallback, min(80% x window,
250k cap) with overridable knobs), boundary picking that never splits a
turn (user-message starts preferred, iteration starts inside a giant tool
loop), the 8-section summarizer prompt with the continuation contract,
mechanical working-state extraction from tool records, deterministic
user-message preservation, the trim-oldest fallback, outbound-view
application, and context-overflow detection. Injectable provider seam;
no engine changes yet.
