Compare commits

...
Author SHA1 Message Date
Teknium 8dd07bd517 feat(ci): plugin-validate reusable action + plugin-catalog admission gate
- scripts/validate_plugin_catalog.py: standalone stdlib+pyyaml structural
  validator for plugin-catalog entries and removed.yaml (no hermes install
  needed; runtime twin of hermes_cli/plugin_catalog.py). --json support,
  unknown top-level keys warn instead of failing for forward compat.
- .github/actions/plugin-validate: composite action plugin authors drop
  into their own repo's CI — installs hermes-agent from a chosen ref and
  runs 'hermes plugins validate <path>'.
- .github/workflows/plugin-catalog-ci.yml: admission gate on PRs touching
  plugin-catalog/** — structural job plus pinned-source job that clones
  each changed entry's repo, hard-fails on unreachable pinned sha
  (supply-chain gate), and validates the plugin at that exact commit.
2026-07-22 07:22:43 -07:00
Teknium d358280ad7 test(gateway): thread follow-ups survive a pending native clarify
Gateway-level regression coverage for #62034 on top of the
clarify_gateway prose-rejection fix: drives GatewayRunner
._handle_message with a pending native multi-choice clarify and proves

- arbitrary thread prose is NOT swallowed (falls through the clarify
  text-intercept and continues as a normal turn),
- typed numeric selections and exact choice labels still resolve,
- 'Other' text-capture mode and open-ended clarifies still accept
  free text.

Incident analysis and repro by @brandician (#62034).
2026-07-22 07:00:47 -07:00
Teknium 76283a9ee4 fix(gateway): suppress tool-progress bubble for clarify prompts
The adapter's send_clarify IS the user-facing rendering of a clarify
prompt (interactive buttons, or the numbered-text fallback). The
gateway's tool-progress callback additionally rendered a progress
bubble for the clarify tool.started event — in verbose mode that
bubble contains the raw tool-call args JSON
({"question": ..., "choices": [...]}), and because the progress
queue drains on a background task, the JSON landed right underneath
the rendered interactive prompt on Slack.

Skip clarify in the progress callback entirely: the prompt rendering
already covers every mode, so a progress line is pure duplication at
best and a raw-JSON leak at worst.

Regression test proves no clarify progress content (raw JSON, verb
line, or question text) reaches the chat in verbose or all modes,
while unrelated tools still render progress normally.

Reported by @alexgrama-dev.

Fixes #52374
2026-07-22 07:00:47 -07:00
liuhao1024 07cbb500b6 fix(clarify): reject arbitrary prose for native interactive multi-choice clarifies
In Slack threads, ordinary follow-up messages sent while a native
multi-choice clarify was pending were consumed as clarify answers —
_coerce_text_response accepted arbitrary text for any pending entry, so
the gateway text-intercept swallowed unrelated thread messages and the
user's messages appeared to be ignored.

Tighten resolve_text_response_for_session for native interactive
multi-choice prompts (buttons rendered, awaiting_text=False):

- numeric selections ("2") still resolve to the canonical choice
- exact choice-label matches (case-insensitive) still resolve
- arbitrary prose is now REJECTED (returns False) so the message
  continues as a normal turn instead of vanishing into the clarify

Behavior is preserved everywhere free text is legitimately the answer:
open-ended clarifies, explicit 'Other' text-capture mode, and the base
adapter's numbered-text fallback (which flips awaiting_text at send
time).

Salvaged from PR #62042 by @liuhao1024.

Fixes #62034
2026-07-22 07:00:47 -07:00
Eva 95aad9229b feat(slack): Block Kit buttons for clarify prompts
Slack now overrides send_clarify to render multi-choice clarify prompts
as native Block Kit buttons (one per choice + a final '✏️ Other…'
free-text button), mirroring the Telegram/Discord adapters and the
existing Slack approval-button pattern.

- Unique hermes_clarify_choice_<idx> action_ids (Slack rejects
  duplicate action_ids within one actions block); dispatch via a
  compiled-regex action matcher plus hermes_clarify_other.
- Chunks elements across actions blocks in groups of 5 so a larger
  choice list degrades gracefully instead of 400ing (invalid_blocks).
- Choice taps resolve through tools.clarify_gateway
  .resolve_gateway_clarify with the canonical registered choice text —
  the same applier the typed-reply path uses — then edit the message
  to show the outcome and drop the buttons.
- 'Other' flips the entry into text-capture via mark_awaiting_text
  (only on tap, never at send time) so the gateway text-intercept
  captures the next typed message.
- Auth-gated via _is_interactive_user_authorized; atomic-pop
  double-click guard mirrors _approval_resolved; late taps on evicted
  entries surface an honest expiry notice instead of a false ✓.
- Open-ended prompts delegate to the base plain-text render.

Salvaged from PR #61943 by @100yenadmin. Earliest implementation of
this feature was PR #28885 by @cypres0099; sibling implementations
#66606 (@jaaro-ai) and #51547 (@Mongol-Jimmi) are superseded.

Closes #52369
2026-07-22 07:00:47 -07:00
Teknium f944e84858 fix: close review gaps for per-model threshold overrides (#63020)
Follow-up to the salvaged contributor commit, closing the three gaps
flagged in the sweeper review:

1. Init ordering: assign compression.model_thresholds to a selected
   plugin context engine BEFORE the initial update_model() call in
   agent_init.py, so the initial model's override applies from init
   (previously it only took effect after the first /model switch).
   Base-class ContextEngine.update_model() now snapshots the
   pre-override percent once so repeated switches fall back to the
   engine's configured threshold, not a previous model's override.
2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to
   hermes_cli/config.py — additive key, no _config_version bump.
3. Docs: document the key in
   website/docs/developer-guide/context-compression-and-caching.md
   (yaml example, parameter table, dedicated section) and update the
   plugin-boundary note in context-engine-plugin.md to state the
   explicit context-engine contract for model_thresholds.

Adds tests/run_agent/test_per_model_threshold_init_ordering.py:
plugin-engine AIAgent init regression (override applies at init,
empty map unchanged), DEFAULT_CONFIG key presence, floor interaction
on the model-switch path (override below the small-context floor is
raised to the floor; above the floor wins), and base-class config
snapshot across repeated switches. Also maps @bennybuoy in
contributors/emails/.
2026-07-22 07:00:27 -07:00
Ben KamholtzandCopilot 5f2fdf66bf feat: per-model compression threshold overrides (v2, rebased on main)
Addresses teknium1 review feedback on PR #60781:

1. Gateway cache invalidation: added ('compression', 'model_thresholds')
   to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map
   invalidates the cached compressor (previously kept stale thresholds).

2. Integrated resolver with small-context floor: per-model overrides are
   resolved FIRST, then the existing 75% floor for <512K models is applied
   on top. The floor is no longer replaced — it stacks. An override below
   75% on a small-context model still gets floored to 75% (raise-only);
   an override above 75% wins.

3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing
   changes. Only the per-model threshold feature is added.

Changes:
- resolve_model_threshold() module-level helper (longest substring match)
- ContextCompressor.__init__ accepts model_thresholds dict
- _base_threshold_percent stores the per-model resolved value
- _config_threshold_percent stores the raw config value (fallback base)
- update_model() re-resolves on /model switch, falls back to config value
- ContextEngine base class update_model() applies overrides for plugin engines
- agent_init.py reads compression.model_thresholds from config, passes to ctor
- gateway/run.py cache busting key added
- cli-config.yaml.example documents the feature
- 17 tests covering resolve helper, compressor init (large/small context,
  override above/below floor), update_model (re-resolve, fallback), base class

Co-authored-by: Copilot <copilot@github.com>
2026-07-22 07:00:27 -07:00
Teknium f453c50b6f test(memory): behavioral check — memory tool handler works with skip_memory=True, provider stays skipped
Follow-up for salvaged PR #65453: extend the regression test to dispatch a
real memory_tool add through the store the tool executor wires in, assert the
write persists to memories/MEMORY.md, and assert the external memory provider
(MemoryManager) is still skipped under skip_memory=True. Also map the
contributor email for attribution CI.

Fixes #65429.
2026-07-22 07:00:16 -07:00
Ella CEO 45182401fa fix(agent): add regression test for #65429 (memory store with skip_memory + memory toolset)
Behavioral test constructing a real AIAgent with skip_memory=True and
enabled_toolsets=["memory"] asserts the built-in MemoryStore is created
(store is not None). Also covers the negative case (no memory toolset -> None)
and the normal case (skip_memory=False -> store created).
2026-07-22 07:00:16 -07:00
Ella CEO 596dda907f fix(agent): create built-in memory store when memory toolset is enabled despite skip_memory (#65429)
skip_memory=True was meant to skip the external memory *provider* for flush/
background agents, but it also suppressed creation of the built-in file-backed
MemoryStore. When a caller still enables the "memory" toolset, the memory tool
dispatched with store=None and every call failed with "Memory is not available",
silently losing the main automatic memory-capture path.

Now the built-in store is created whenever memory is enabled in config OR the
memory toolset is explicitly enabled, while the external-provider block stays
gated on skip_memory (preserving flush-agent intent).
2026-07-22 07:00:16 -07:00
liuhao1024 9bb253d4fa fix(tools): filter compaction summaries from session_search bookends and cap content length
Context-compaction handoff summaries (prefixed with [CONTEXT COMPACTION])
were being returned as normal bookend_start/bookend_end messages in
session_search discovery mode. A single compaction handoff could be 57K+
chars, immediately bloating a fresh session prompt to 73K+ chars from one
search hit.

Changes:
- Add _COMPACTION_PREFIXES and _is_compaction_summary() helper
- Filter compaction summaries from bookend_start and bookend_end in _discover()
- Cap bookend content to 1200 chars and window content to 4000 chars
- Add content_truncated/original_content_chars metadata when truncation occurs
- Add 6 regression tests covering prefix detection, bookend filtering,
  content capping, and legacy [CONTEXT SUMMARY] prefix

Fixes #43175
2026-07-22 07:00:04 -07:00
Teknium 75099ca0ef fix(state): inherit git_branch + gateway origin columns on compression children
Follow-ups on top of #64731's cwd/git_repo_root inheritance:

- git_branch joins the parent-row backfill (same NULL-only COALESCE hop):
  the Desktop sidebar branch chip otherwise vanishes at every compaction
  boundary even though the workspace didn't change.
- Belt-and-suspenders for #59527: compression forks (parent already ended
  with end_reason='compression') also inherit the gateway origin columns
  (user_id/session_key/chat_id/chat_type/thread_id/display_name/
  origin_json) at DB-level child creation. The gateway re-records the peer
  after rotation (d5b4879d4), but a hard crash in the window between child
  creation and that write left the child unrecoverable by
  find_latest_gateway_session_for_peer. Scoped to compression forks only —
  delegate/subagent children (parent still live) must NOT inherit routing
  keys, or peer recovery could repoint gateway traffic into a subagent's
  session.
- Behavioral test driving the real _compress_context rotation path,
  asserting the child row carries cwd/git_repo_root/git_branch and the
  origin columns.
2026-07-22 06:59:44 -07:00
Simplicio, Wesley (ext) 3c74c12554 fix(state): inherit cwd/git_repo_root on parent_session_id children
_insert_session_row never copied cwd/git_repo_root from a parent row when
parent_session_id was set, and git_repo_root wasn't even in the INSERT's
column list. The compression-fork path (and delegate/subagent spawns,
branch continuations) creates a child session without passing cwd/
git_repo_root at all, so the child's tip is born NULL — and since the
Desktop project sidebar groups sessions by cwd, the whole project silently
drops out of the sidebar every time a long conversation compresses. A
lineage that compresses repeatedly compounds this across generations.

Add git_repo_root to _insert_session_row's INSERT/COALESCE-on-conflict
column set, and backfill both cwd and git_repo_root from the immediate
parent row (single non-recursive hop, matching the existing COALESCE
"never overwrite an explicit value" contract) inside the same write
transaction whenever parent_session_id is set. A multi-generation chain
resolves correctly because each generation's own create_session call
already backfills from its (already-resolved) immediate parent.

Fixes #64709
2026-07-22 06:59:44 -07:00
Teknium b6a2b701c1 fix(anthropic): extend leading-user guard to the extracted-system-absent path
The salvaged guard from #52276 only fired when a system prompt was
present in messages[] (system is not None after extraction). The live
repro of #52160 is the auto path: the system prompt is passed outside
messages[], so after the second compaction messages[0] is the
assistant-role summary with system=None — and the guard never ran.

Make _ensure_leading_user_turn unconditional, exactly mirroring the
Bedrock Converse adapter ('Converse requires the first message to be
from the user' — convert_messages_to_converse). Add a regression test
building the exact post-double-compaction shape (no system in
messages, messages[0]=assistant summary) asserting the converted
payload leads with a user turn, and update existing fixtures that
started with a bare assistant message to locate roles instead of
indexing result[0].
2026-07-22 06:59:33 -07:00
fesalfayed e71c1137e8 fix(anthropic): prepend user turn when compaction leaves a leading assistant message
Anthropic extracts the system prompt into a separate `system` field and
requires messages[0] to be role="user"; a leading assistant turn is rejected
with HTTP 400. After a second context compaction the only surviving leading
anchor is the system prompt, so a summary/handoff message emitted as
role="assistant" becomes the first messages entry once the system prompt is
extracted. Anthropic reports this with a misleading error —
`messages.N: tool_use ids were found without tool_result blocks immediately
after: toolu_...` — even when every tool_use/tool_result pair is adjacent and
matched; the real structural defect is the leading assistant role (#52160).

This is engine-agnostic: it fires for any producer of a leading-assistant
transcript (built-in compressor, the DAG/LCM context engine, session
truncation), unlike the compressor-scoped fix in #52167. The native Bedrock
Converse adapter already guards the same invariant
(convert_messages_to_converse); this mirrors it for the native Anthropic path.

Add _ensure_leading_user_turn() to the convert_messages_to_anthropic
post-processing chain, scoped to the system-extracted case (the production
trigger) so bare assistant-only unit fixtures are unaffected. Adds regression
tests (leading-assistant, leading-assistant-with-adjacent-tool_use, and a
no-op negative control).
2026-07-22 06:59:33 -07:00
Teknium 17fa5910be chore: map contributor wernerhp 2026-07-22 06:59:14 -07:00
wernerhp cc84af9fad fix(memory): preserve genuine pre-delimiter content in merged compaction rows
teknium1 review on #57690: harvesting logic was skipping the ENTIRE merged
row when a compaction summary was appended to the tail message, discarding
real prior user content that context_compressor retains before the
_MERGED_SUMMARY_DELIMITER. Extract and harvest that pre-delimiter segment
instead of dropping it wholesale.

Revert-to-fail: reverting plugins/memory/holographic/__init__.py alone
drops test_merged_into_tail_preserves_genuine_pre_delimiter_preference
(19 passed, 1 failed); restoring the fix returns 20/20 passed.
2026-07-22 06:59:14 -07:00
wernerhp 004de13f14 fix(memory/holographic): don't harvest compaction summaries; honor auto_extract=false string
Two compounding defects in the holographic memory provider (#57682):

1. The on_session_end gate used plain truthiness on auto_extract, but the
   plugin's own config schema declares it as a string enum with default
   "false" — and not "false" is False, so extraction ran for users who
   had it configured off. Coerce with the shared utils.is_truthy_value
   (same fix class as the merged byterover no-op fix).

2. _auto_extract_facts scanned every role=user message. Context-compaction
   handoff summaries can be inserted as role=user messages and their prose
   reliably matches the decision patterns (we decided/agreed, the project
   uses), so the compactor's own output was persisted as a durable project
   fact on every rollover following a compaction — recreated even after
   manual deletion.

Adds agent.context_compressor.is_compaction_summary_message(), a public
helper that prefers the in-process COMPRESSED_SUMMARY_METADATA_KEY marker
and falls back to _is_context_summary_content() (covers merged-into-tail
and historical prefixes), since the metadata key is stripped by wire
sanitizers and doesn't survive all session-store round-trips. The plugin
skips summary messages before pattern matching.

Fixes #57682
2026-07-22 06:59:14 -07:00
Sora-blueskyandClaude Opus 4.7 cca7b93bfe chore(contributors): register Sora-bluesky email mapping
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-22 06:58:58 -07:00
Sora-blueskyandClaude Fable 5 19a59f7d7b fix(compression): mirror the full trigger recomputation in the suggestion guard
Review follow-up on #67431 (hermes-sweeper):

- The viability check compared the floored percentage against the raw
  context window, but the built-in trigger recomputation also applies the
  output-token reservation, the 64K floor, and the degenerate-window
  guard (_compute_threshold_tokens). Mirror that math exactly, so e.g. a
  200K window with max_tokens=120K recomputes to max(0.75*80K, 64K)=64K
  and the suggestion is correctly KEPT for an 80K aux model instead of
  being suppressed by the raw-window percentage.
- Gate the built-in policy behind isinstance(ContextCompressor): external
  context engines own compaction policy (#44439), so plugin engines keep
  the plain suggestion untouched.
- The non-viable explanation now names the recomputed trigger instead of
  hardcoding the 75%/512K wording, so it stays accurate when the
  reservation (not the percentage floor) is what makes the value
  unreachable.

Tests: reservation-viability regression and plugin-engine passthrough,
per the review; the floored-branch assertion updated to the recomputed
number.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 06:58:58 -07:00
Sora-blueskyandClaude Fable 5 dbc71fb6e4 fix(compression): don't recommend a threshold the small-context floor will ignore
The auxiliary-compression feasibility warning computes its
compression.threshold suggestion as aux_context / main_context,
independently of ContextCompressor._effective_threshold_percent()'s
raise-only small-context floor. For main windows under 512K the floor
raises any configured value below 75% back up, so a suggestion like
'threshold: 0.40' is silently ignored and the same warning returns every
session.

Derive the suggestion's viability through the compressor's own floor
logic: offer the 'lower the threshold' option only when the floored value
still fits the auxiliary model's context; otherwise recommend only a
larger compression model and explain the floor, so the guidance is always
actionable.

Tests: the updated auto-correct test pins the floored branch (no
threshold suggestion, floor explained); two new tests pin the surviving
suggestion at/above the floor on a small window and below 75% on a
512K+ window where no floor applies. The updated test fails against the
previous code.

Fixes #67422

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 06:58:58 -07:00
Teknium 295e20358c feat(delegation): let subagents use execute_code (#69325)
Children inherit the parent's env, repo, and toolsets but were denied
execute_code ('children should reason step-by-step, not write scripts').
That forces subagents doing mechanical multi-step work (batch file
reads, fetch-N-pages loops, filter-before-context reductions) to burn
reasoning iterations one tool call at a time.

- Remove execute_code from DELEGATE_BLOCKED_TOOLS
- Stop stripping the code_execution toolset from child bundles
- No recursion risk: the sandbox bridges only the 7 SANDBOX_ALLOWED_TOOLS
  (web/file/terminal) — delegate_task and execute_code itself are not
  reachable from inside a sandbox script
- Update schema text, AGENTS.md, and delegation docs
- Tests: blocked-constant, strip, and child-assembly tests updated;
  new test pins execute_code as intentionally unblocked
2026-07-22 06:58:45 -07:00
Teknium b2ba069cb4 test(slack): regression tests for Block Kit boundary sanitizer and oversized edits
- sanitize_blocks: null column_settings repair (#56615), >3000-char
  section clamp incl. HTML-escape-inflated approval updates (#62054 /
  #53693), empty-block drops, header clamp, 50-block cap, plain-text
  fallback when nothing valid remains, garbage never raises
- edit_message: oversized content truncates instead of failing with
  msg_too_long (#33224 behavior on the plugin adapter)
2026-07-22 06:58:34 -07:00
Teknium a43ac2af65 fix(slack): sanitize outbound Block Kit payloads at the API boundary
Consolidated defensive layer for the invalid_blocks / msg_too_long bug
class (#56615, #62054, #53693): one malformed or oversized block fails
the ENTIRE chat.postMessage / chat.update call, so approval cards never
update and messages silently drop.

New block_kit.sanitize_blocks() is applied at every call site that
attaches blocks (send/edit via _maybe_blocks, send_exec_approval,
send_slash_confirm, and the approval/slash-confirm chat.update button
handlers). It:

- truncates section/context text to the 3000-char cap with an ellipsis
  (covers interaction payloads where Slack's HTML-escaping of < > &
  inflates text past the limit budgeted at send time)
- truncates header text to its 150-char cap
- drops empty blocks (no text / elements / rows)
- replaces null table column_settings entries with {} and trims
  default trailing entries (Slack requires every entry be an object)
- caps the payload at Slack's 50-block maximum
- returns None when nothing valid remains so callers fall back to the
  plain-text payload; never raises

Also registers contributor mappings for the salvaged PRs in this
cluster (tw0316, kamonspecial, sowork-skills).
2026-07-22 06:58:34 -07:00
CJ Wang / SoWork 72da4f8657 fix(slack): truncate edit_message content to prevent msg_too_long
Slack's chat.update enforces the same ~40k character limit as
chat.postMessage, but edit_message sent the formatted text unchunked —
an oversized edit failed outright with msg_too_long and the message
was never updated. Unlike send() we cannot split an edit into multiple
messages, so truncate to MAX_MESSAGE_LENGTH via the shared chunker
(which preserves code-block boundaries) and keep the first chunk.

Reapplied from PR #33224 (targeted the pre-plugin gateway/platforms/slack.py
path) onto plugins/platforms/slack/adapter.py.
2026-07-22 06:58:34 -07:00
liuhao1024 a2d8b4b1f6 fix(slack): guard against empty text in chat.postMessage
Slack API returns `no_text` error when `chat.postMessage` is called
with empty or whitespace-only text. This happens when cron delivery
posts a response before the agent produces content (e.g. malformed
turn before [SILENT]).

Add early-return guards in both code paths:
- `_standalone_send()`: out-of-process cron delivery via Web API
- `send()`: in-process gateway adapter

Both paths now skip the API call and return success when the formatted
message is empty or whitespace-only.

Fixes #52663
2026-07-22 06:58:34 -07:00
liuhao1024 c13af09177 fix(slack): truncate inflated original_text in approval/confirm chat.update handlers
Slack re-escapes HTML entities in the interaction payload
(< -> &lt;, > -> &gt;, & -> &amp;), inflating the section text
past the 3000-char Block Kit limit when the button handler
rebuilds updated_blocks for chat.update.

The send path already budget-truncates, but the click handlers
used the echoed original_text verbatim. Cap it to 3000 chars
in both _handle_approval_action and _handle_slash_confirm_action.

Fixes #53693
2026-07-22 06:58:34 -07:00
AlexFucuson9 8fc0c086f5 fix: replace assert with runtime guard in Slack block_kit rich_text builder
assert statements are stripped when Python runs with -O flag. Replace
the assert cur is not None in _rich_text_list_block() with an explicit
if/continue guard. The assert is technically unreachable (first loop
iteration always sets cur), but using assert for invariant checking in
production code is fragile under optimization.
2026-07-22 06:58:34 -07:00
kamon d91a143977 fix(slack): guard rich_text builders against empty content rejected as invalid_blocks
Slack rejects a rich_text_section / rich_text_preformatted / rich_text_quote
whose elements list is empty or contains a zero-length text element, and a
header whose plain_text is empty. The Block Kit renderer emits exactly those
shapes for common inputs: a markdown table with a blank cell or a ragged
(short) row, an empty fenced code block, a blank quote line, an empty list
item, and an emphasis-only header ("# ***"). Any one of them poisons the
whole payload — chat.postMessage fails with invalid_blocks ("missing element"
/ "must be more than 0 characters") and the message loses its rich rendering
entirely.

Route every rich_text builder's child elements through _nonempty_elements
(drop zero-length text elements; substitute a single space when nothing
remains) and skip headers that reduce to empty after markdown-marker
stripping. Observed in production via live chat.postMessage rejections;
regression tests cover each case plus a well-formed-content control.

Complementary to #56618 (column_settings hardening + no-blocks retry): that
change makes Block Kit rejections recoverable, this one makes the common
empty-content cases render correctly in the first place. No overlapping hunks.
2026-07-22 06:58:34 -07:00
tw0316 870770b31e fix(slack): harden rich table block fallback 2026-07-22 06:58:34 -07:00
Teknium 961b832c11 test(sessions): cover branch-seed and compression timestamp round-trips
Follow-up to the salvaged #28840 change: forward original timestamps in
_persist_branch_seed (TUI first-turn branch persist), add behavioral
round-trip tests for branch copy and compression-style replace, TUI
protocol tests asserting session.branch and _persist_branch_seed forward
timestamps, and register the contributor email mapping.
2026-07-22 06:58:27 -07:00
tangyi 9d73006ade fix: forward timestamp in CLI, gateway, and TUI branch copy loops 2026-07-22 06:58:27 -07:00
Teknium 1927b60775 fix(tui): extend deferred context-engine finalize to the compute-host compress routes
The salvaged deferred-finalize wiring (#65670) covered cli.py, the gateway
slash command, and the three in-process tui_gateway/server.py compress
sites — but the dashboard compute-host isolated session.compress /
slash.compress routes run the compress mirror inside the host child, where
a control-handler exception after compress_context() queued the boundary
notification would leak it: never fired on success paths, or worse, fired
by a LATER compress against a boundary the host had rejected.

- tui_gateway/compute_host.py: _handle_control discards any pending
  context-engine compression notification (committed=False) when a
  session.compress / slash.compress control frame errors. finalize is
  exactly-once, so this is a no-op when the mirror already emitted or
  discarded it.
- tui_gateway/server.py: _mirror_slash_side_effects normalizes the
  /compact alias onto the compress branch so isolated-session compacts
  actually compress and hit the same finalize/discard wiring instead of
  silently no-oping.
- tests/tui_gateway/test_compute_host_phase1.py: compute-host route tests —
  commit-then-notify ordering, discard on host commit failure, /compact
  alias routing.
2026-07-22 06:58:16 -07:00
Teknium 7d4ed8e321 chore(release): map the3asic noreply email for contributor attribution 2026-07-22 06:58:16 -07:00
3ASiC d46f0fb2d5 fix(compression): notify context engine after commit 2026-07-22 06:58:16 -07:00
Teknium 15d33d5ab1 fix(desktop,tui,docs): dedup commands.catalog, route desktop /compact to /compress, update README
- tui_gateway commands.catalog: skip _TUI_EXTRA entries that collide with a
  registry command or alias (the /compact class of bug, #57133; also removes
  the pre-existing /sessions duplicate) — registry entry is canonical.
- apps/desktop: /compact now dispatches as an alias of /compress (matching
  the registry's canonical alias) instead of dead-ending; /density (the
  renamed TUI display toggle) is marked terminal-only so the desktop palette
  doesn't advertise a command its dispatcher can't run.
- ui-tui/README.md: document /density instead of the old /compact toggle.
- tests: commands.catalog regression test asserting no duplicate advertised
  names and no command shadowing another command's alias; desktop routing test.
2026-07-22 06:58:05 -07:00
liuhao1024 27a4e92802 fix(acp,tui): rename /compact to /compress and /density to resolve command collision
- acp_adapter/server.py: rename compact -> compress for context compression command
- tui_gateway/server.py: rename /compact -> /density for display density toggle
- ui-tui/core.ts: rename compact -> density for display density toggle
- Internal config keys (tui_compact) and UI state (ctx.ui.compact) unchanged
2026-07-22 06:58:05 -07:00
liuhao1024 cb39be92e4 fix: update test assertions for /compact -> /compress rename 2026-07-22 06:58:05 -07:00
Jakub Wolniewicz 4c2e34f07d fix(moa): measure advisor guidance before compression 2026-07-22 06:57:54 -07:00
Teknium 8a580e8e31 chore: add contributor mapping for tandixit95 2026-07-22 06:57:44 -07:00
Tanmay Dixit 2eb6c84bbb fix(codex): wait for compaction turn identity 2026-07-22 06:57:44 -07:00
Tanmay Dixit 63e363f306 fix(codex): scope app-server notifications to active turn 2026-07-22 06:57:44 -07:00
Teknium 453d0ee598 chore(contributors): map emails for slack block-text salvage (#29541, #61261, #52390) 2026-07-22 06:57:39 -07:00
Benjamin Ross 3865694cf9 fix(slack): surface Block Kit content in fetched thread context
Bot-posted alerts (Honeycomb, PagerDuty, Datadog, GitHub bot, etc.) carry
their actionable content — section text, button URLs — in Block Kit
blocks, while the plain text field holds only the alert title.
_fetch_thread_context and _fetch_thread_parent_text only read
msg.get('text'), so that content never reached the agent.

Add a _render_message_text helper that merges top-level text with
readable block content, section/header/context text, actionable URLs,
and (folded in from #61261 during conflict resolution) legacy
attachment fields, and use it for thread-context and parent-text
rendering.

Salvaged from #29541.
2026-07-22 06:57:39 -07:00
Pan Luo 1f92842c1c fix(slack): read thread context from attachments and blocks
`_fetch_thread_context` and `_fetch_thread_parent_text` only read each
message's plain `text` field, so messages posted by apps (Alertmanager,
Grafana, PagerDuty, CI bots) — which carry their content in legacy
`attachments` or Block Kit `blocks` with an empty `text` — were dropped
entirely. When such a message *starts* a thread (e.g. an alert), a bot
mentioned mid-thread to investigate sees an empty thread and can only ask
"what should I investigate?".

Fall back to the existing `_extract_text_from_slack_blocks` and a new
`_extract_text_from_slack_attachments` helper when `text` is empty, so
app-posted alerts and notifications are visible in fetched thread history.

Adds TestThreadContextAppMessages (attachment-only, blocks-only, and
empty-message cases).
2026-07-22 06:57:39 -07:00
Bartok9 1e0f5a6f07 fix(slack): detect Block-Kit-only @mentions in bot filter and router
Closes #52387

Slack messages can carry the bot @mention only inside Block Kit `blocks`
(a `rich_text` section with a `user` element), with the flat top-level
`text` containing just a fallback string. Both mention gates read only the
flat `text`:

- the `allow_bots: mentions` bot-message filter, and
- the `is_mentioned` channel router (`routing_text = original_text`)

so such messages were silently dropped — `allow_bots: mentions` was
effectively non-functional for Block-Kit senders, and the same blind spot
hit `require_mention` / `strict_mention`.

Add `_collect_slack_block_mentions()` (walks blocks, recovers `<@UID>`
tokens from non-quoted rich-text `user` elements) and
`_slack_mention_detection_text()` (flat text + recovered mentions). Both
gates now use the merged detection text. Mentions nested in
`rich_text_quote` are deliberately ignored, preserving the existing
contract that quoted/forwarded content can't trick the bot. Also emit a
debug line when a bot message is dropped so silent drops are diagnosable.

Tests: 4 new cases in tests/gateway/test_slack_mention.py (Block-Kit
mention recovered, flat-text passthrough, no-mention, quoted-mention
ignored). 275 slack tests pass.
2026-07-22 06:57:39 -07:00
Teknium 8fceaac143 test(compression): adapt strict-signature heartbeat test to signature-inspection dispatch
Main no longer catches TypeError to fall back — _supported_compression_kwargs
inspects the engine signature and calls once with only accepted kwargs.
Rework the salvaged test to assert a single correctly-shaped call while
keeping heartbeat start/stop and lock-release coverage.
2026-07-22 06:57:33 -07:00
Teknium 7f9956a679 chore: map contributor email for @WeiYusc 2026-07-22 06:57:33 -07:00
WeiYusc 928bcdde24 fix(compression): refresh gateway activity during compaction
Refresh the agent activity tracker while context compression is blocked in the auxiliary summarizer so gateway watchdogs do not report inactivity during long compactions.

Add regression coverage for successful heartbeats, exception cleanup, touch failures, and strict-signature compressor fallback.

(cherry picked from commit c09e58b7709cc60c5b454701f0ecf840e759222f)
2026-07-22 06:57:33 -07:00
Teknium ea0fd393db perf(compression): gate CJK-aware token estimation behind an ASCII fast path
The salvaged estimator ran a per-character Python loop on every
estimate_tokens_rough() call — a ~28,000,000x slowdown vs (len+3)//4 on a
1MB ASCII tool output (measured ~3.0s per call). Gate it:

- str.isascii() O(1) fast path keeps pure-ASCII text bit-identical to the
  classic (len+3)//4 rule at ~1.3x baseline cost (0.23us vs 0.17us per
  1MB call).
- Non-ASCII text counts dense CJK chars via a compiled character-class
  regex in C (len(text) - len(re.sub(''))): ~352ms/1MB hangul vs ~2.1s
  for the per-char loop.
- Non-ASCII-but-non-CJK text (accents, Cyrillic, emoji) keeps the classic
  rule.

Also: parity tests against the per-char reference implementation, and
updated two stale expectations that encoded the old behavior (CJK now
counted ~1 token/char; short string content now ceil-divided instead of
floored to 0). The continuity test now detects merged-into-tail summaries
via _is_context_summary_content.
2026-07-22 06:57:22 -07:00
miniadmin 3f33a1c5aa fix(compression): handle CJK token budgeting 2026-07-22 06:57:22 -07:00
Teknium a269a5b38c docs(compression): note acceptable false-positive path in zero-user provenance validator 2026-07-22 06:56:53 -07:00
embwl0x 7de7a073c6 fix(compression): preserve synthetic user provenance 2026-07-22 06:56:53 -07:00
embwl0x c93e4041b6 fix(compression): preserve zero-user provenance 2026-07-22 06:56:53 -07:00
Teknium 1c2faedd88 fix(compression): unify the attempt cap across every compression site
Follow-up to the salvaged #64010 (Kenmege) and #63870 (dombejar) commits,
making one resolved compression.max_attempts cap govern ALL per-turn
compression attempt sites:

- conversation_loop: resolve max_compression_attempts ONCE at turn start
  (it was previously re-resolved inside the API-call loop) and route the
  pre-API pressure gate through it — that gate still hardcoded
  'compression_attempts < 3' and logged 'attempt=%s/3'.
- conversation_loop: the salvaged post-tool compaction gate now uses the
  resolved cap instead of a hardcoded 3.
- turn_context: the preflight compaction loop was 'for _pass in range(3)';
  it now sizes itself from the same resolved cap.
- agent_init: harden the max_attempts parser — reject booleans (bool
  subclasses int; 'true' would coerce to 1), reject fractional floats
  instead of truncating them, keep accepting integral floats and numeric
  strings; anything else falls back to 3 (floor 1, ceiling 10 unchanged).
- tests: replace #63870's inspect.getsource source-shape test with
  behavioral loop tests (post-tool compaction fires <= cap times per turn,
  shares its budget with the pre-API gate, resets between turns); add an
  e2e test proving a 4th preflight pass runs at config cap=6 while the
  unset default still stops at 3; extend the #64010 config tests with the
  bool/float parser semantics.

Salvages #64010 by @Kenmege and #63870 by @dombejar.
2026-07-22 06:56:42 -07:00
Teknium cf433a33da chore: map dombejar contributor email for PR #63870 salvage 2026-07-22 06:56:42 -07:00
Dom Bejar fca883f06f fix(compaction): cap post-tool attempts per turn 2026-07-22 06:56:42 -07:00
Kennedy UmegeandClaude Fable 5 644b397fb2 fix(agent): make the compression retry cap config-driven (compression.max_attempts)
The conversation loop hardcodes max_compression_attempts = 3. Sessions
that legitimately need more rounds are stranded: on a restart history
reload, incompressible tool schemas can keep the per-request estimate
above the compressor threshold even though the message floor compresses
correctly, so three rounds cannot clear it and the turn dies with
"Context length exceeded: max compression attempts (3) reached" — the
same failure class as #62605, where the rough estimate similarly leaves
3 retries short.

Make the cap a config key, compression.max_attempts:

- default 3 = identical to today, so an unset key is behavior-neutral;
- parsed and validated in agent_init alongside the other compression.*
  keys (>= 1, hard-capped at 10, non-integer values fall back to 3),
  attached as agent.max_compression_attempts;
- the loop reads it via getattr(agent, "max_compression_attempts", 3),
  so objects without the attribute keep the prior behavior;
- documented in the DEFAULT_CONFIG compression block.

Tests pin the parse/validate/attach seam: default preserved, custom
value honored, floor and ceiling enforced, garbage tolerated, and the
loop-side getattr degradation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 06:56:42 -07:00
Teknium 28b6bd6570 fix(api): preserve compaction summaries at any history position during Responses auto-truncation
The gateway /compress path can force a user-leading layout that leaves
the compaction summary after a retained system head, so scanning only
the leading block misses it. Preserve marker-carrying messages wherever
they sit, filling the remaining budget with the most recent other
messages, and cover the non-leading position with a test.
2026-07-22 06:56:32 -07:00
luyifan 3807df7a06 fix(api-server): preserve compaction summaries on auto truncation
Fixes #55224
2026-07-22 06:56:32 -07:00
Teknium 1c21e96ed0 refactor(api): dedupe compressed-transcript persist sites, drop config opt-out
Rework on top of the salvaged #58133 commits:

- Remove the compression.persist_in_response_store config key — this is
  a bug fix (stored transcripts must reflect what the agent will actually
  replay), not behavior that should be opt-out-able.
- Drop the per-request load_config() imports the handler-level persist
  blocks added.
- Dedupe the two handler-level persist blocks: the compressed-transcript
  substitution already lives in _build_response_conversation_history
  (via result["_compressed"]), so the handlers only need to propagate
  the effective (possibly rotation-changed) session_id. The streaming
  path does this via a new session_id_snapshot arg on
  _persist_response_snapshot; the non-streaming path picks up
  result["session_id"] directly.
- Rotation propagation no longer gates on history-from-store: the first
  request in a chain can also rotate, and its stored session_id must be
  the child session or the next previous_response_id request resumes the
  pre-rotation session and re-compresses every turn.
2026-07-22 06:56:20 -07:00
Teknium 146a545491 chore(contributors): map LiangYang666 emails 2026-07-22 06:56:20 -07:00
尘漾 4166fdda6a fix(api_server): log warning when compressed response_store persist fails 2026-07-22 06:56:20 -07:00
尘漾 a26ed50e07 test(gateway): exercise real compression detection path with fake agents
Address review feedback (PR #58133): the original test mocked _run_agent
with _compressed=True directly, bypassing the detection logic.

New tests mock _create_agent instead, so _run_agent's detection path
runs naturally and reads agent.session_id / _last_compaction_in_place:

1. test_rotation_compression_exercises_detection_and_persists_rotated_session_id
   - Fake agent with rotated session_id -> verifies _compressed is set,
     compressed history is stored, and rotated session_id propagates to
     both response_store and X-Hermes-Session-Id header.

2. test_inplace_compression_exercises_detection_and_persists_compressed_history
   - Fake agent with _last_compaction_in_place=True, session_id unchanged
     -> verifies _compressed is set, compressed history is stored, and
     session_id does NOT rotate.

3. test_chained_rotation_propagates_effective_session_id
   - Two-request chain: first request triggers rotation, second request
     loads history using the rotated session_id stored by the first.
     Asserts the compressed transcript is loaded correctly for chaining.
2026-07-22 06:56:20 -07:00
尘漾 f66e773c44 fix(gateway): in-place compression not persisted in response_store
The persist logic only checked _result_sid != session_id (rotation),
missing in-place mode where session_id is unchanged but _compressed
flag is set. response_store history doubled every turn (11->26->55->110->225)
causing repeated re-compression.

Fix: detect compression via _did_compress or _rotated, and only update
_effective_session_id on actual rotation (not in-place).

Note: preflight loop break (turn_context.py) from original commit
eee64097a is excluded — it's an optimization, not a bug fix.

Cherry-picked from alidev eee64097a (api_server.py only)
2026-07-22 06:56:20 -07:00
尘漾 2ced02671e feat(api_server): persist compressed messages to prevent re-compression
- Detect when history is loaded from response_store (via previous_response_id)
- Add history_from_store parameter to distinguish history source
- When compression occurs, persist compressed messages instead of original
- Add persist_in_response_store config option (default True)
- Update session_id and response headers to reflect session rotation

Cherry-picked from alidev 2eb816f6b
2026-07-22 06:56:20 -07:00
liang 806b34ced9 test(gateway): add regression test for compressed Responses transcript storage 2026-07-22 06:56:20 -07:00
liang 85f04d4d7e fix(gateway): Responses API stores bloated context after compression
Compression produces a compact transcript in result['messages'],
but _build_response_conversation_history detected a prefix mismatch
and concatenated the original conversation_history on front.

Detect compression via _last_compaction_in_place / session_id
rotation and signal through result['_compressed'] so the builder
uses the compressed transcript directly.
2026-07-22 06:56:20 -07:00
99 changed files with 8812 additions and 312 deletions
@@ -0,0 +1,56 @@
name: Hermes Plugin Validate
description: >-
Validate a Hermes Agent plugin (plugin.yaml manifest schema AND
declared-vs-actually-registered capabilities) using
`hermes plugins validate`. Drop this into your plugin repo's CI:
- uses: actions/checkout@<sha>
- uses: NousResearch/hermes-agent/.github/actions/plugin-validate@main
with:
path: .
The caller's job owns checkout; this action installs Python + hermes-agent
(git install — a supported CI-context install route) and runs the
validator against your plugin directory.
inputs:
path:
description: Path to the plugin directory (containing plugin.yaml).
default: "."
hermes-ref:
description: hermes-agent git ref (branch/tag/sha) to install and validate with.
default: "main"
runs:
using: composite
steps:
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- name: Install hermes-agent
shell: bash
env:
_HERMES_REF: ${{ inputs.hermes-ref }}
run: |
set -euo pipefail
# CI-context install from git; the ref lets plugin authors validate
# against a pinned hermes release instead of main.
pip install "git+https://github.com/NousResearch/hermes-agent@${_HERMES_REF}"
- name: Validate plugin
shell: bash
env:
_PLUGIN_PATH: ${{ inputs.path }}
run: |
set -uo pipefail
# `hermes plugins validate` checks the plugin.yaml manifest schema
# and loads the plugin in a scratch subprocess to verify that the
# capabilities it DECLARES match what it actually registers.
if hermes plugins validate "$_PLUGIN_PATH"; then
echo "✅ PASS: plugin at '$_PLUGIN_PATH' validated cleanly"
else
echo "❌ FAIL: plugin at '$_PLUGIN_PATH' failed validation (see output above)"
exit 1
fi
+140
View File
@@ -0,0 +1,140 @@
name: Plugin Catalog CI
# Admission gate for plugin-catalog entries. Fires ONLY on PRs touching
# plugin-catalog/** so it can never go red on unrelated PRs.
#
# Two gates:
# structural — cheap schema check, no hermes install needed
# pinned-source-validate — supply-chain gate: the pinned sha MUST be
# reachable in the entry's repo, and the plugin
# at that exact commit must pass
# `hermes plugins validate`.
on:
pull_request:
paths:
- "plugin-catalog/**"
permissions:
contents: read
jobs:
structural:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- name: Install PyYAML
uses: ./.github/actions/retry
with:
command: pip install pyyaml==6.0.2
- name: Validate catalog files (structural)
run: |
set -euo pipefail
# Validating the whole directory is simpler than diffing and keeps
# the invariant that EVERYTHING in plugin-catalog/ stays valid.
python3 scripts/validate_plugin_catalog.py plugin-catalog/
pinned-source-validate:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0 # need the merge-base to diff changed catalog files
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- name: Find changed catalog entries
id: changed
run: |
set -euo pipefail
MERGE_BASE=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
# Added + modified entry files only; deletions and removed.yaml
# have nothing to clone.
CHANGED=$(git diff --name-only --diff-filter=AM "$MERGE_BASE"...HEAD \
-- 'plugin-catalog/*.yaml' 'plugin-catalog/*.yml' \
| grep -v '/removed\.yaml$' || true)
echo "Changed catalog entries:"
echo "${CHANGED:-<none>}"
{
echo 'files<<__EOF__'
echo "$CHANGED"
echo '__EOF__'
} >> "$GITHUB_OUTPUT"
- name: Install hermes-agent from the PR's own checkout
if: steps.changed.outputs.files != ''
uses: ./.github/actions/retry
with:
command: pip install -e .
- name: Clone each entry at its pinned sha and validate
if: steps.changed.outputs.files != ''
env:
CHANGED_FILES: ${{ steps.changed.outputs.files }}
run: |
set -euo pipefail
FAILED=0
while IFS= read -r entry; do
[ -z "$entry" ] && continue
echo "::group::validate $entry"
# Parse repo / sha / subdir from the entry yaml.
eval "$(python3 - "$entry" <<'PYEOF'
import shlex
import sys
import yaml
with open(sys.argv[1], encoding="utf-8") as fh:
data = yaml.safe_load(fh) or {}
print(f"REPO={shlex.quote(str(data.get('repo', '')))}")
print(f"SHA={shlex.quote(str(data.get('sha', '')))}")
print(f"SUBDIR={shlex.quote(str(data.get('subdir', '') or ''))}")
PYEOF
)"
echo "repo=$REPO sha=$SHA subdir=$SUBDIR"
CLONE_DIR=$(mktemp -d)
# Full clone (no --depth 1): the pinned sha may not be the branch tip.
if ! git clone "$REPO" "$CLONE_DIR"; then
echo "::error file=$entry::clone failed for $REPO"
FAILED=1; echo "::endgroup::"; continue
fi
# SUPPLY-CHAIN GATE: the pinned sha must be reachable in the repo.
if ! git -C "$CLONE_DIR" checkout --detach "$SHA"; then
echo "::error file=$entry::pinned sha $SHA is not reachable in $REPO"
FAILED=1; echo "::endgroup::"; continue
fi
PLUGIN_DIR="$CLONE_DIR${SUBDIR:+/$SUBDIR}"
if [ ! -f "$PLUGIN_DIR/plugin.yaml" ]; then
echo "::error file=$entry::no plugin.yaml at subdir '$SUBDIR' of $REPO@$SHA"
FAILED=1; echo "::endgroup::"; continue
fi
# Manifest schema + declared-vs-registered capability check.
if hermes plugins validate "$PLUGIN_DIR"; then
echo "✅ PASS: $entry"
else
echo "::error file=$entry::hermes plugins validate failed"
FAILED=1
fi
echo "::endgroup::"
done <<< "$CHANGED_FILES"
if [ "$FAILED" -ne 0 ]; then
echo "❌ FAIL: one or more catalog entries failed pinned-source validation"
exit 1
fi
echo "✅ PASS: all changed catalog entries validated at their pinned shas"
+2 -1
View File
@@ -998,7 +998,8 @@ Two shapes:
Roles:
- `role="leaf"` (default) — focused worker. Cannot call `delegate_task`,
`clarify`, `memory`, `send_message`, `execute_code`.
`clarify`, `memory`, `send_message`, `cronjob`. Retains `execute_code`
(programmatic tool calling).
- `role="orchestrator"` — retains `delegate_task` so it can spawn its
own workers. Gated by `delegation.orchestrator_enabled` (default true)
and bounded by `delegation.max_spawn_depth` (default 2).
+6 -6
View File
@@ -456,7 +456,7 @@ class HermesACPAgent(acp.Agent):
"tools": "List available tools",
"context": "Show conversation context info",
"reset": "Clear conversation history",
"compact": "Compress conversation context",
"compress": "Compress conversation context",
"steer": "Inject guidance into the currently running agent turn",
"queue": "Queue a prompt to run after the current turn finishes",
"version": "Show Hermes version",
@@ -485,7 +485,7 @@ class HermesACPAgent(acp.Agent):
"description": "Clear conversation history",
},
{
"name": "compact",
"name": "compress",
"description": "Compress conversation context",
},
{
@@ -1756,7 +1756,7 @@ class HermesACPAgent(acp.Agent):
"tools": self._cmd_tools,
"context": self._cmd_context,
"reset": self._cmd_reset,
"compact": self._cmd_compact,
"compress": self._cmd_compress,
"steer": self._cmd_steer,
"queue": self._cmd_queue,
"version": self._cmd_version,
@@ -1898,7 +1898,7 @@ class HermesACPAgent(acp.Agent):
lines.append(
f"Compression: due now (threshold ~{threshold_tokens:,}"
+ (f", {threshold_pct:.0f}%" if threshold_pct else "")
+ "). Run /compact."
+ "). Run /compress."
)
else:
lines.append(
@@ -1913,7 +1913,7 @@ class HermesACPAgent(acp.Agent):
if getattr(agent, "compression_enabled", True) is False:
lines.append("Compression is disabled for this agent.")
else:
lines.append("Tip: run /compact to compress manually before the threshold.")
lines.append("Tip: run /compress to compress manually before the threshold.")
return "\n".join(lines)
@@ -1933,7 +1933,7 @@ class HermesACPAgent(acp.Agent):
return "Conversation history cleared. Agent session state reset failed; see logs."
return "Conversation history cleared."
def _cmd_compact(self, args: str, state: SessionState) -> str:
def _cmd_compress(self, args: str, state: SessionState) -> str:
if not state.history:
return "Nothing to compress — conversation is empty."
try:
+59 -1
View File
@@ -1561,7 +1561,14 @@ def init_agent(
agent._memory_nudge_interval = 10
agent._turns_since_memory = 0
agent._iters_since_skill = 0
if not skip_memory:
# A flush/background agent may pass skip_memory=True to avoid spinning up an
# external memory *provider*, but if the caller also explicitly enables the
# "memory" toolset it still needs the built-in file-backed store — otherwise
# the memory tool dispatches with store=None and every call fails (#65429).
# So the built-in store is created unless memory is globally disabled, while
# the external-provider block below stays gated on skip_memory.
_memory_toolset_requested = "memory" in (agent.enabled_toolsets or [])
if not skip_memory or _memory_toolset_requested:
try:
mem_config = _agent_cfg.get("memory", {})
agent._memory_enabled = mem_config.get("memory_enabled", False)
@@ -1781,6 +1788,34 @@ def init_agent(
compression_enabled = str(_compression_cfg.get("enabled", True)).lower() in {"true", "1", "yes"}
compression_target_ratio = float(_compression_cfg.get("target_ratio", 0.20))
compression_protect_last = int(_compression_cfg.get("protect_last_n", 20))
# Cap on compression retry rounds before a turn gives up with "max
# compression attempts reached" (compression.max_attempts). Hardcoding 3
# strands sessions that legitimately need more rounds — e.g. a restart
# history reload whose incompressible tool schemas keep the request
# estimate above the threshold even though the messages compress fine
# (the #62605 failure class). Default 3 preserves current behavior, so
# an unset key is behavior-neutral; validated >= 1, hard-capped at 10,
# and any non-int-like value falls back to 3. Booleans are rejected
# (bool subclasses int, so int(True) would silently become 1) and
# fractional floats are rejected rather than truncated — "4.7 attempts"
# is a config mistake, not a request for 4.
_raw_max_attempts = _compression_cfg.get("max_attempts", 3)
if isinstance(_raw_max_attempts, bool):
compression_max_attempts = 3
elif isinstance(_raw_max_attempts, int):
compression_max_attempts = _raw_max_attempts
elif isinstance(_raw_max_attempts, float):
compression_max_attempts = (
int(_raw_max_attempts) if _raw_max_attempts.is_integer() else 3
)
else:
try:
compression_max_attempts = int(str(_raw_max_attempts).strip())
except (TypeError, ValueError):
compression_max_attempts = 3
if compression_max_attempts < 1:
compression_max_attempts = 3
compression_max_attempts = min(compression_max_attempts, 10)
# protect_first_n is the number of non-system messages to protect at
# the head, in addition to the system prompt (which is always
# implicitly protected by the compressor). Floor at 0 — a value of
@@ -1793,6 +1828,17 @@ def init_agent(
compression_abort_on_summary_failure = str(
_compression_cfg.get("abort_on_summary_failure", False)
).lower() in {"true", "1", "yes"}
# Per-model threshold overrides: keys are substring-matched against the
# model name (longest match wins). Empty dict = use the global threshold
# for all models (backward compatible).
_raw_model_thresholds = _compression_cfg.get("model_thresholds", {})
if isinstance(_raw_model_thresholds, dict):
compression_model_thresholds = {
str(k): float(v) for k, v in _raw_model_thresholds.items()
if isinstance(v, (int, float)) and not isinstance(v, bool)
}
else:
compression_model_thresholds = {}
# In-place compaction: when True, compress_context() rewrites the message
# list + rebuilds the system prompt WITHOUT rotating the session id (no
# parent_session_id chain, no `name #N` renumber). See #38763 and
@@ -2188,6 +2234,16 @@ def init_agent(
provider=agent.provider,
custom_providers=_custom_providers,
)
# Per-model threshold overrides are part of the explicit
# context-engine contract: assign them BEFORE the initial
# update_model() call so the first resolution (which derives
# threshold_percent/threshold_tokens for the initial model) already
# sees the overrides. Assigning after update_model() left the initial
# model on the engine's global threshold until the first /model
# switch. Engines that override update_model() own their own policy
# and may ignore the attribute.
if compression_model_thresholds:
agent.context_compressor.model_thresholds = compression_model_thresholds
agent.context_compressor.update_model(
model=agent.model,
context_length=_plugin_ctx_len,
@@ -2214,6 +2270,7 @@ def init_agent(
api_mode=agent.api_mode,
abort_on_summary_failure=compression_abort_on_summary_failure,
max_tokens=agent.max_tokens,
model_thresholds=compression_model_thresholds,
)
_bind_session_state = getattr(agent.context_compressor, "bind_session_state", None)
if callable(_bind_session_state):
@@ -2224,6 +2281,7 @@ def init_agent(
agent.compression_enabled = compression_enabled
agent.compression_in_place = compression_in_place
agent.codex_app_server_auto_compaction = codex_app_server_auto_compaction
agent.max_compression_attempts = compression_max_attempts
# Reject models whose context window is below the minimum required
# for reliable tool-calling workflows (64K tokens).
+19
View File
@@ -2412,6 +2412,24 @@ def _evict_old_screenshots(result: List[Dict[str, Any]]) -> None:
]
def _ensure_leading_user_turn(result: List[Dict[str, Any]]) -> None:
"""Anthropic requires messages[0] to have role=user.
After a second context compaction on the auto path the summary can be
emitted as role=assistant with nothing in front of it (the system prompt
lives outside messages[] or is extracted into the separate ``system``
param), so messages[0] ends up assistant and the Messages API rejects
the request with HTTP 400 — often masked by a misleading
"tool_use ids were found without tool_result blocks" error (#52160).
Mirror the Bedrock Converse adapter, which unconditionally prepends a
minimal user turn when the first message is not user
(convert_messages_to_converse).
"""
if result and result[0].get("role") != "user":
result.insert(0, {"role": "user", "content": [{"type": "text", "text": " "}]})
def convert_messages_to_anthropic(
messages: List[Dict],
base_url: str | None = None,
@@ -2470,6 +2488,7 @@ def convert_messages_to_anthropic(
_strip_orphaned_tool_blocks(result)
result = _merge_consecutive_roles(result)
_ensure_leading_user_turn(result)
_manage_thinking_signatures(result, base_url, model)
_evict_old_screenshots(result)
+298 -52
View File
@@ -31,9 +31,11 @@ from agent.model_metadata import (
MINIMUM_CONTEXT_LENGTH,
get_model_context_length,
estimate_messages_tokens_rough,
estimate_tokens_rough,
)
from agent.redact import redact_sensitive_text
from agent.turn_context import drop_stale_api_content
from tools.todo_tool import TODO_INJECTION_HEADER
logger = logging.getLogger(__name__)
@@ -130,8 +132,20 @@ LEGACY_SUMMARY_PREFIX = "[CONTEXT SUMMARY]:"
# poisoning every subsequent request in the session — a bare key like
# "is_compressed_summary" would reach the wire and trip exactly that.
COMPRESSED_SUMMARY_METADATA_KEY = "_compressed_summary"
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY = "_compressed_summary_has_user_turn"
_DB_PERSISTED_MARKER = "_db_persisted"
_NO_USER_TASK_SENTINEL = "None. This session contains no user-authored turns."
COMPRESSION_CONTINUATION_USER_CONTENT = (
"Continue from the compressed conversation context above. "
"This marker exists because no human user turn was available."
)
_LEGACY_COMPRESSION_CONTINUATION_USER_CONTENT = (
"Continue from the compressed conversation context above. "
"This marker exists because the compacted transcript contained "
"no preserved user turn."
)
def _fresh_compaction_message_copy(msg: Dict[str, Any]) -> Dict[str, Any]:
"""Copy a message for compaction assembly without persistence markers.
@@ -436,11 +450,15 @@ def _estimate_msg_budget_tokens(msg: dict) -> int:
compaction re-fires continuously (#55572). Accounting-only: replay
fields are never mutated or pruned here.
"""
content_len = _content_length_for_budget(msg.get("content") or "")
tokens = content_len // _CHARS_PER_TOKEN + 10 # +10 for role/key overhead
content = msg.get("content") or ""
if isinstance(content, str):
tokens = estimate_tokens_rough(content) + 10 # +10 for role/key overhead
else:
content_len = _content_length_for_budget(content)
tokens = content_len // _CHARS_PER_TOKEN + 10
for tc in msg.get("tool_calls") or []:
if isinstance(tc, dict):
tokens += len(str(tc)) // _CHARS_PER_TOKEN
tokens += estimate_tokens_rough(str(tc))
for key in _REPLAY_BUDGET_KEYS:
tokens += _serialized_length_for_budget(msg.get(key)) // _CHARS_PER_TOKEN
return tokens
@@ -856,6 +874,32 @@ def _summarize_tool_result_unguarded(tool_name: str, tool_args: str, tool_conten
return f"[{tool_name}]{first_arg} ({content_len:,} chars result)"
def resolve_model_threshold(
model: str,
model_thresholds: dict[str, float] | None,
default: float,
) -> float:
"""Resolve the effective compression threshold for a given model.
``model_thresholds`` maps substring keys to override fractions. The
longest matching key wins (so ``glm-5.2-1M`` beats ``glm-5.2`` when the
model is ``glm-5.2-1M``). When no override matches, or when
``model_thresholds`` is empty/None, ``default`` is returned unchanged.
This is a module-level helper so plugin context engines (e.g. LCM) can
import and reuse the same resolution logic as the built-in compressor.
"""
if not model_thresholds or not model:
return default
best_key = ""
for key in model_thresholds:
if key in model and len(key) > len(best_key):
best_key = key
if best_key:
return float(model_thresholds[best_key])
return default
class ContextCompressor(ContextEngine):
"""Default context engine — compresses conversation context via lossy summarization.
@@ -877,6 +921,7 @@ class ContextCompressor(ContextEngine):
self._context_probed = False
self._context_probe_persistable = False
self._previous_summary = None
self._summary_has_user_turn = None
self._last_summary_error = None
self._consecutive_timeout_failures = 0
self._last_summary_dropped_count = 0
@@ -917,6 +962,7 @@ class ContextCompressor(ContextEngine):
surface the moment the owning session ends.
"""
self._previous_summary = None
self._summary_has_user_turn = None
self._last_summary_error = None
self._consecutive_timeout_failures = 0
self._last_summary_dropped_count = 0
@@ -1163,17 +1209,19 @@ class ContextCompressor(ContextEngine):
self.provider = provider
self.api_mode = api_mode
self.context_length = context_length
# Re-apply the small-context threshold floor for the NEW window,
# starting from the originally-configured percent (not the possibly
# floored live value) so a small -> large switch drops back to the
# configured threshold and a large -> small switch gains the floor.
# Guard with getattr: compressors unpickled/constructed before this
# attribute existed fall back to the live value.
_configured_pct = getattr(
self, "_configured_threshold_percent", self.threshold_percent,
# Re-resolve per-model threshold for the NEW model, then re-apply the
# small-context threshold floor. Starting from _config_threshold_percent
# (the raw config value) so a switch from a model with an override to
# one without correctly falls back to the global threshold.
_config_pct = getattr(
self, "_config_threshold_percent", self.threshold_percent,
)
_new_base = resolve_model_threshold(
model, self.model_thresholds, _config_pct,
)
self._base_threshold_percent = _new_base
self.threshold_percent = self._effective_threshold_percent(
context_length, _configured_pct,
context_length, _new_base,
)
# max_tokens=None here means "caller didn't specify" → keep the existing
# output reservation. A switch that genuinely changes the output budget
@@ -1319,13 +1367,26 @@ class ContextCompressor(ContextEngine):
api_mode: str = "",
abort_on_summary_failure: bool = False,
max_tokens: int | None = None,
model_thresholds: dict[str, float] | None = None,
):
self.model = model
self.base_url = base_url
self.api_key = api_key
self.provider = provider
self.api_mode = api_mode
self.threshold_percent = threshold_percent
# Per-model threshold overrides (longest substring match wins).
# Stored as a plain dict; resolved in _resolve_threshold(), then the
# small-context floor is applied on top.
self.model_thresholds = model_thresholds or {}
# _config_threshold_percent is the raw config value (before per-model
# override or small-context floor). Used as the fallback when switching
# to a model with no matching override.
self._config_threshold_percent = threshold_percent
# Resolve per-model override first, then apply the small-context floor.
self._base_threshold_percent = resolve_model_threshold(
model, self.model_thresholds, threshold_percent,
)
self.threshold_percent = self._base_threshold_percent
self.protect_first_n = protect_first_n
self.protect_last_n = protect_last_n
self.summary_target_ratio = max(0.10, min(summary_target_ratio, 0.80))
@@ -1355,9 +1416,11 @@ class ContextCompressor(ContextEngine):
# resolved and BEFORE threshold_tokens is derived. The pre-floor
# value is kept so update_model() can re-derive for a new window
# (switching small -> large must drop back to the configured value).
# Note: _base_threshold_percent already has the per-model override
# applied, so the floor stacks on top of any model-specific threshold.
self._configured_threshold_percent = self.threshold_percent
self.threshold_percent = self._effective_threshold_percent(
self.context_length, self.threshold_percent,
self.context_length, self._base_threshold_percent,
)
threshold_percent = self.threshold_percent
# Floor: never compress below MINIMUM_CONTEXT_LENGTH tokens even if
@@ -1403,6 +1466,10 @@ class ContextCompressor(ContextEngine):
# Stores the previous compaction summary for iterative updates
self._previous_summary: Optional[str] = None
# Provenance for the rolling summary. A compaction handoff can carry
# role="user" solely to satisfy provider alternation, so role alone
# cannot prove that a human-authored turn ever existed.
self._summary_has_user_turn: Optional[bool] = None
# Anti-thrashing: track whether last compression was effective
self._last_compression_savings_pct: float = 100.0
self._ineffective_compression_count: int = 0
@@ -1990,6 +2057,9 @@ class ContextCompressor(ContextEngine):
role = msg.get("role", "unknown")
text = _compact_fallback_turn(msg.get("content"))
_collect_path_mentions(text, relevant_files)
synthetic_user = (
role == "user" and self._is_synthetic_compression_user_turn(msg)
)
turn_text = text
turn_tool_names: list[str] = []
@@ -2000,12 +2070,13 @@ class ContextCompressor(ContextEngine):
if turn_tool_names:
prefix = "tool calls: " + ", ".join(turn_tool_names[:6])
turn_text = f"{prefix}; {turn_text}" if turn_text else prefix
_remember_dropped_turn(str(role).upper(), turn_text)
turn_label = "INTERNAL CONTEXT" if synthetic_user else str(role).upper()
_remember_dropped_turn(turn_label, turn_text)
if len(text) > 600:
text = text[:420].rstrip() + " ... " + text[-160:].lstrip()
if role == "user" and text:
if role == "user" and text and not synthetic_user:
user_asks.append(text)
elif role == "assistant":
tool_names: list[str] = []
@@ -2051,7 +2122,7 @@ class ContextCompressor(ContextEngine):
active_task = (
f"User asked: {user_asks[-1]!r}"
if user_asks
else "Unknown from deterministic fallback."
else _NO_USER_TASK_SENTINEL
)
previous_summary_note = ""
if self._previous_summary:
@@ -2194,6 +2265,9 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
if _sanitized_memory_context
else ""
)
has_user_turn = getattr(self, "_summary_has_user_turn", None)
if has_user_turn is None:
has_user_turn = self._transcript_has_real_user_turn(turns_to_summarize)
# Current date for temporal anchoring (see ## Temporal Anchoring below).
# Date-only granularity matches system_prompt.py:337 (PR #20451) and the
@@ -2211,18 +2285,86 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
# Preamble shared by both first-compaction and iterative-update prompts.
# Keep the wording deliberately plain: Azure/OpenAI-compatible content
# filters have flagged stronger "injection" / "do not respond" framing.
if has_user_turn:
_language_and_provenance_rule = (
"Write the summary in the same language the user was using in the "
"conversation — do not translate or switch to English. "
)
_historical_task_instructions = """[THE SINGLE MOST IMPORTANT FIELD. Capture the user's most recent unfulfilled
input verbatim — the exact words they used. This includes:
- Explicit task assignments ("<specific user task>")
- Questions awaiting an answer ("<specific user question>")
- Decisions awaiting input ("<option A or B?>")
- Ongoing discussions where the assistant owes the next substantive reply
A conversation where the user just asked a question IS an active task — the
task is "answer that question with full context". Do NOT write "None" merely
because the user did not issue an imperative command; reserve "None" for the
rare case where the last exchange was fully resolved and the user said
something like "thanks, that's all".
If multiple items are outstanding, list only the ones NOT yet completed.
This historical snapshot must identify the latest unresolved user input precisely. Examples:
"User asked: '<exact latest user request>'"
"User asked: '<exact latest user question>' — needs investigation + answer"
"User chose <option>; awaiting implementation of <specific next step>"
If the user's most recent message was a reverse signal (stop, undo, roll
back, never mind, just verify, change of topic) that supersedes earlier
work, write the reverse signal verbatim and DO NOT carry forward the
cancelled task. Example: "User asked: '<exact reverse signal>' — earlier
in-flight work is cancelled."
If no outstanding task exists, write "None."]"""
_goal_instructions = "[What the user is trying to accomplish overall]"
_constraints_instructions = (
"[User preferences, coding style, constraints, important decisions]"
)
_resolved_questions_instructions = (
"[Questions the user asked that were ALREADY answered — include the "
"answer so it is not repeated]"
)
_pending_asks_instructions = (
"[Questions or requests from the user that have NOT yet been answered "
"or fulfilled. These are STALE — they were from the compacted turns. "
"Write them here for reference only. The agent must NOT act on them "
"unless the latest user message explicitly requests it. If none, "
'write "None."]'
)
else:
_language_and_provenance_rule = (
"This session contains no user-authored turns. Write the summary "
"in the dominant language of the source turns; if they are mixed, "
"use the language of the most recent natural-language assistant "
"turn. Do not translate, invent a user, or attribute any request "
"to a user. "
)
_historical_task_instructions = f"""[NO user-authored turn exists in this session. Write exactly:
{_NO_USER_TASK_SENTINEL}
Do not write "User asked:" or any translated equivalent anywhere in the summary.
Describe agent/tool work only as completed actions, state, or historical work.]"""
_goal_instructions = (
"[Historical cron/agent objective inferred only from assistant and "
"tool activity. Never call it a user goal.]"
)
_constraints_instructions = (
"[Runtime, configuration, and technical constraints only. Do not "
"invent user preferences.]"
)
_resolved_questions_instructions = (
"[Write exactly: None. No user-authored questions exist.]"
)
_pending_asks_instructions = (
"[Write exactly: None. No user-authored requests exist.]"
)
_summarizer_preamble = (
"You are a summarization agent creating a context checkpoint. "
"Treat the conversation turns below as source material for a "
"compact record of prior work. "
"Produce only the structured summary; do not add a greeting, "
"preamble, or prefix. "
"Write the summary in the same language the user was using in the "
"conversation — do not translate or switch to English. "
+ _language_and_provenance_rule +
"NEVER include API keys, tokens, passwords, secrets, credentials, "
"or connection strings in the summary — replace any that appear "
"with [REDACTED]. Note that the user had credentials present, but "
"do not preserve their values."
"with [REDACTED]. Note that credentials were present, but do not "
"preserve their values."
)
# Temporal anchoring directive. Rewrites relative / still-pending-sounding
@@ -2245,34 +2387,13 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
# Shared structured template (used by both paths).
_template_sections = f"""{HISTORICAL_TASK_HEADING}
[THE SINGLE MOST IMPORTANT FIELD. Capture the user's most recent unfulfilled
input verbatim — the exact words they used. This includes:
- Explicit task assignments ("<specific user task>")
- Questions awaiting an answer ("<specific user question>")
- Decisions awaiting input ("<option A or B?>")
- Ongoing discussions where the assistant owes the next substantive reply
A conversation where the user just asked a question IS an active task — the
task is "answer that question with full context". Do NOT write "None" merely
because the user did not issue an imperative command; reserve "None" for the
rare case where the last exchange was fully resolved and the user said
something like "thanks, that's all".
If multiple items are outstanding, list only the ones NOT yet completed.
This historical snapshot must identify the latest unresolved user input precisely. Examples:
"User asked: '<exact latest user request>'"
"User asked: '<exact latest user question>' — needs investigation + answer"
"User chose <option>; awaiting implementation of <specific next step>"
If the user's most recent message was a reverse signal (stop, undo, roll
back, never mind, just verify, change of topic) that supersedes earlier
work, write the reverse signal verbatim and DO NOT carry forward the
cancelled task. Example: "User asked: '<exact reverse signal>' — earlier
in-flight work is cancelled."
If no outstanding task exists, write "None."]
{_historical_task_instructions}
## Goal
[What the user is trying to accomplish overall]
{_goal_instructions}
## Constraints & Preferences
[User preferences, coding style, constraints, important decisions]
{_constraints_instructions}
## Completed Actions
[Numbered list of concrete actions taken — include tool used, target, and outcome.
@@ -2301,10 +2422,10 @@ Be specific with file paths, commands, line numbers, and results.]
[Important technical decisions and WHY they were made]
## Resolved Questions
[Questions the user asked that were ALREADY answered — include the answer so it is not repeated]
{_resolved_questions_instructions}
{HISTORICAL_PENDING_ASKS_HEADING}
[Questions or requests from the user that have NOT yet been answered or fulfilled. These are STALE — they were from the compacted turns. Write them here for reference only. The agent must NOT act on them unless the latest user message explicitly requests it. If none, write "None."]
{_pending_asks_instructions}
## Relevant Files
[Files read, modified, or created — with brief note on each]
@@ -2428,6 +2549,7 @@ This compaction should PRIORITISE preserving all information related to the focu
# ignore prompt instructions and echo back secrets verbatim.
summary = redact_sensitive_text(content.strip())
summary = self._ground_historical_task_snapshot(summary, turns_to_summarize)
self._validate_summary_user_provenance(summary, has_user_turn)
# Store for iterative updates on next compaction
self._previous_summary = summary
self._clear_compression_failure_cooldown()
@@ -2674,6 +2796,69 @@ This compaction should PRIORITISE preserving all information related to the focu
return False
return bool(message.get(COMPRESSED_SUMMARY_METADATA_KEY))
@classmethod
def _transcript_has_real_user_turn(cls, messages: List[Dict[str, Any]]) -> bool:
"""Return whether *messages* contain a user-authored turn.
Compaction summaries can deliberately carry ``role="user"`` to keep
strict provider transcripts valid. The metadata/content checks prevent
those synthetic transport rows from becoming evidence of a real user.
"""
for message in messages:
if not isinstance(message, dict) or message.get("role") != "user":
continue
if cls._is_synthetic_compression_user_turn(message):
continue
return True
return False
@classmethod
def _is_synthetic_compression_user_turn(cls, message: Any) -> bool:
"""Recognize internal user-role rows after SessionDB projection.
SessionDB preserves role/content but not underscore-prefixed metadata,
so the stable todo and continuation content markers are authoritative.
"""
if not isinstance(message, dict) or message.get("role") != "user":
return False
if cls._has_compressed_summary_metadata(message):
return True
content = message.get("content")
if cls._is_context_summary_content(content):
return True
text = _content_text_for_contains(content).strip()
return text in {
COMPRESSION_CONTINUATION_USER_CONTENT,
_LEGACY_COMPRESSION_CONTINUATION_USER_CONTENT,
} or text.startswith(
TODO_INJECTION_HEADER + "\n"
)
@staticmethod
def _validate_summary_user_provenance(summary: str, has_user_turn: bool) -> None:
"""Reject user attribution when the source transcript has no user."""
if has_user_turn:
return
match = re.search(
rf"(?ms)^{re.escape(HISTORICAL_TASK_HEADING)}\s*\n(.*?)(?=\n##\s|\Z)",
summary,
)
task_snapshot = match.group(1).strip() if match else ""
# NOTE: the "User asked:" scan covers the WHOLE summary, so tool output
# quoted verbatim in e.g. Completed Actions can false-positive in a
# zero-user session. That is acceptable: the RuntimeError only rides
# the existing retry/deterministic-fallback path (which emits the
# no-user sentinel itself), so a rare false positive costs one retry
# rather than letting fabricated user attribution persist.
if (
task_snapshot != _NO_USER_TASK_SENTINEL
or re.search(r"\bUser\s+asked\s*:", summary, re.IGNORECASE)
):
raise RuntimeError(
"Context compression summary invented user attribution for a "
"session with no user-authored turns"
)
@classmethod
def _derive_auto_focus_topic(
cls,
@@ -2685,9 +2870,9 @@ This compaction should PRIORITISE preserving all information related to the focu
msg = messages[idx]
if msg.get("role") != "user":
continue
content = msg.get("content")
if cls._is_context_summary_content(content):
if cls._is_synthetic_compression_user_turn(msg):
continue
content = msg.get("content")
text = redact_sensitive_text(_content_text_for_contains(content).strip())
if not text:
continue
@@ -2968,8 +3153,9 @@ This compaction should PRIORITISE preserving all information related to the focu
"""
for i in range(len(messages) - 1, head_end - 1, -1):
msg = messages[i]
if msg.get("role") == "user" and not self._is_context_summary_content(
msg.get("content")
if (
msg.get("role") == "user"
and not self._is_synthetic_compression_user_turn(msg)
):
return i
return -1
@@ -3444,9 +3630,24 @@ This compaction should PRIORITISE preserving all information related to the focu
summary_search_start,
compress_end,
)
real_user_present = self._transcript_has_real_user_turn(messages)
if summary_idx is not None:
if summary_body and not self._previous_summary:
self._previous_summary = summary_body
provenance = messages[summary_idx].get(
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY
)
if real_user_present:
self._summary_has_user_turn = True
elif isinstance(provenance, bool):
self._summary_has_user_turn = provenance
elif self._summary_has_user_turn is None:
# Legacy handoffs predate provenance metadata. Preserve their
# user context unless the exact no-user sentinel proves the
# stronger invariant; guessing False could erase a real ask.
self._summary_has_user_turn = not (
summary_body and _NO_USER_TASK_SENTINEL in summary_body
)
turns_to_summarize = messages[max(compress_start, summary_idx + 1):compress_end]
elif self._previous_summary:
# No handoff summary found in the current messages, but
@@ -3455,6 +3656,9 @@ This compaction should PRIORITISE preserving all information related to the focu
# it so _generate_summary() does not inject cross-session content
# into the summarizer prompt via the iterative-update path.
self._previous_summary = None
self._summary_has_user_turn = real_user_present
else:
self._summary_has_user_turn = real_user_present
if not self.quiet_mode:
logger.info(
@@ -3548,6 +3752,19 @@ This compaction should PRIORITISE preserving all information related to the focu
# Phase 4: Assemble compressed message list
compressed = []
for i in range(compress_start):
# If an earlier compaction handoff is in the protected head
# (common after resume / in-place compaction), do not carry it
# forward verbatim. It has already been rehydrated into
# _previous_summary above and _generate_summary() will emit the
# updated replacement below. Keeping both makes repeated
# compactions accumulate old summaries and prevents the live prompt
# from actually shrinking.
if (
summary_idx is not None
and i == summary_idx
and self._is_context_summary_content(messages[i].get("content"))
):
continue
msg = _fresh_compaction_message_copy(messages[i])
if i == 0 and msg.get("role") == "system":
existing = msg.get("content")
@@ -3574,7 +3791,7 @@ This compaction should PRIORITISE preserving all information related to the focu
)
_merge_summary_into_tail = False
last_head_role = messages[compress_start - 1].get("role", "user") if compress_start > 0 else "user"
last_head_role = compressed[-1].get("role", "user") if compressed else "user"
first_tail_role = messages[compress_end].get("role", "user") if compress_end < n_messages else "user"
# When the only protected head message is the system prompt, the
# summary becomes the first *visible* message in the API request
@@ -3645,6 +3862,9 @@ This compaction should PRIORITISE preserving all information related to the focu
"role": summary_role,
"content": summary,
COMPRESSED_SUMMARY_METADATA_KEY: True,
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY: bool(
self._summary_has_user_turn
),
})
for i in range(compress_end, n_messages):
@@ -3674,6 +3894,9 @@ This compaction should PRIORITISE preserving all information related to the focu
# Mark the merged message so frontends can identify it as
# containing a compression summary prefix.
msg[COMPRESSED_SUMMARY_METADATA_KEY] = True
msg[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] = bool(
self._summary_has_user_turn
)
# Content rewritten → the api_content sidecar (exact bytes
# previously sent) is stale; drop it so replay can't resend
# the pre-merge bytes without the summary.
@@ -3733,3 +3956,26 @@ This compaction should PRIORITISE preserving all information related to the focu
self._last_compression_made_progress = True
return compressed
def is_compaction_summary_message(message: Any) -> bool:
"""Return True when *message* is a context-compaction handoff summary.
Public API for consumers outside the compressor (memory providers,
frontends) that must not treat compaction summaries as real user or
assistant turns — e.g. fact extraction harvesting the compactor's own
output as user statements (#57682).
Prefers the in-process ``COMPRESSED_SUMMARY_METADATA_KEY`` marker and
falls back to the content heuristics in ``_is_context_summary_content``
(which cover the merged-into-tail and historical-prefix cases), because
the metadata key is stripped by the wire sanitizers and does not survive
all session-store round-trips.
"""
if isinstance(message, dict):
if message.get(COMPRESSED_SUMMARY_METADATA_KEY):
return True
content = message.get("content")
else:
content = message
return ContextCompressor._is_context_summary_content(content)
+15
View File
@@ -260,4 +260,19 @@ class ContextEngine(ABC):
(e.g. recalculate DAG budgets, switch summary models).
"""
self.context_length = context_length
# Apply per-model threshold overrides if set (longest substring match).
# Falls back to _config_threshold_percent (the raw config value) when
# no override matches. Plugin engines that override update_model() can
# call resolve_model_threshold() for the same logic.
from agent.context_compressor import resolve_model_threshold
if not hasattr(self, "_config_threshold_percent"):
# Snapshot the pre-override percent ONCE so repeated model
# switches fall back to the engine's configured value, not the
# previous model's override.
self._config_threshold_percent = self.threshold_percent
self._base_threshold_percent = resolve_model_threshold(
model, getattr(self, "model_thresholds", {}),
self._config_threshold_percent,
)
self.threshold_percent = self._base_threshold_percent
self.threshold_tokens = int(context_length * self.threshold_percent)
+214 -24
View File
@@ -31,6 +31,7 @@ from __future__ import annotations
import copy
import inspect
import logging
import math
import os
import tempfile
import uuid
@@ -231,6 +232,51 @@ def _supported_compression_kwargs(
return {name: value for name, value in candidates.items() if name in parameters}
class _CompressionActivityHeartbeat:
"""Refresh the agent inactivity tracker while compression blocks in an aux call."""
def __init__(self, agent: Any, interval_seconds: float | None = None) -> None:
self._agent = agent
if interval_seconds is None:
interval_seconds = getattr(agent, "_compression_activity_heartbeat_interval", 60.0)
try:
interval_seconds = float(interval_seconds or 60.0)
except (TypeError, ValueError):
interval_seconds = 60.0
if not math.isfinite(interval_seconds):
interval_seconds = 60.0
self._interval_seconds = max(0.1, interval_seconds)
self._stop = threading.Event()
self._thread = threading.Thread(
target=self._run,
name="compression-activity-heartbeat",
daemon=True,
)
def start(self) -> "_CompressionActivityHeartbeat":
self._touch("context compression started")
self._thread.start()
return self
def stop(self, desc: str = "context compression completed") -> None:
self._stop.set()
if self._thread.is_alive() and threading.current_thread() is not self._thread:
self._thread.join(timeout=1.0)
self._touch(desc)
def _touch(self, desc: str) -> None:
try:
touch = getattr(self._agent, "_touch_activity", None)
if callable(touch):
touch(desc)
except Exception:
logger.debug("compression activity heartbeat touch failed", exc_info=True)
def _run(self) -> None:
while not self._stop.wait(self._interval_seconds):
self._touch("context compression in progress")
class _CompressionLockLeaseRefresher:
def __init__(
self,
@@ -458,6 +504,29 @@ def check_compression_model_feasibility(agent: Any) -> None:
new_threshold / main_ctx
)
safe_pct = int((aux_context / main_ctx) * 100) if main_ctx else 50
# The "lower the threshold" suggestion must survive the built-in
# trigger recomputation (#67422): _effective_threshold_percent()
# raises sub-75% values back up for main windows under 512K, and
# _compute_threshold_tokens() further applies the output-token
# reservation, the 64K floor, and the degenerate-window guard.
# Recommending a value those would override is silently ignored
# and this warning would reappear every session — so mirror the
# compressor's own math and only offer the option when the
# recomputed trigger actually fits the auxiliary model's context.
# External engines own compaction policy (#44439); the built-in
# floor doesn't apply to them, so keep the plain suggestion.
from agent.context_compressor import ContextCompressor as _CC
recomputed_threshold = None
if main_ctx and isinstance(agent.context_compressor, _CC):
recomputed_threshold = _CC._compute_threshold_tokens(
main_ctx,
_CC._effective_threshold_percent(main_ctx, safe_pct / 100),
getattr(agent.context_compressor, "max_tokens", None),
)
threshold_suggestion_viable = (
recomputed_threshold is None or recomputed_threshold <= aux_context
)
# Build human-readable "model (provider)" labels for both
# the main model and the compression model so users can
# tell at a glance which provider each side is actually
@@ -491,15 +560,32 @@ def check_compression_model_feasibility(agent: Any) -> None:
f"{old_threshold:,} tokens. "
f"Auto-lowered this session's threshold to "
f"{new_threshold:,} tokens so compression can run.\n"
f" To make this permanent, edit config.yaml — either:\n"
f" 1. Use a larger compression model:\n"
f" auxiliary:\n"
f" compression:\n"
f" model: <model-with-{old_threshold:,}+-context>\n"
f" 2. Lower the compression threshold:\n"
f" compression:\n"
f" threshold: 0.{safe_pct:02d}"
)
if threshold_suggestion_viable:
msg += (
f" To make this permanent, edit config.yaml — either:\n"
f" 1. Use a larger compression model:\n"
f" auxiliary:\n"
f" compression:\n"
f" model: <model-with-{old_threshold:,}+-context>\n"
f" 2. Lower the compression threshold:\n"
f" compression:\n"
f" threshold: 0.{safe_pct:02d}"
)
else:
msg += (
f" To make this permanent, use a larger compression "
f"model in config.yaml:\n"
f" auxiliary:\n"
f" compression:\n"
f" model: <model-with-{old_threshold:,}+-context>\n"
f" (Lowering compression.threshold cannot help here — "
f"with {_main_label}'s {main_ctx:,}-token window, "
f"Hermes's small-context floor and output reservation "
f"would recompute the trigger to "
f"{recomputed_threshold:,} tokens, still above the "
f"compression model's {aux_context:,}.)"
)
agent._compression_warning = msg
agent._emit_status(msg)
logger.warning(
@@ -613,7 +699,7 @@ def _is_real_user_message(message: Any) -> bool:
return False
from agent.context_compressor import ContextCompressor
return not ContextCompressor._is_context_summary_content(text)
return not ContextCompressor._is_synthetic_compression_user_turn(message)
def _merge_anchor_into_user_message(target: dict, anchor: dict) -> None:
@@ -690,7 +776,10 @@ def _ensure_compressed_has_user_turn(original_messages: list, compressed: list)
"""Preserve human intent, not merely a synthetic user-role placeholder."""
if any(_is_real_user_message(message) for message in compressed):
return
from agent.context_compressor import _fresh_compaction_message_copy
from agent.context_compressor import (
COMPRESSION_CONTINUATION_USER_CONTENT,
_fresh_compaction_message_copy,
)
for message in reversed(original_messages):
if _is_real_user_message(message):
@@ -701,13 +790,77 @@ def _ensure_compressed_has_user_turn(original_messages: list, compressed: list)
return
compressed.append({
"role": "user",
"content": (
"Continue from the compressed conversation context above. "
"This marker exists because no human user turn was available."
),
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
})
_PENDING_CONTEXT_ENGINE_NOTIFICATION = (
"_pending_context_engine_compression_notification"
)
def _notify_context_engine_compression_complete(
agent: Any,
*,
new_session_id: str,
old_session_id: str,
) -> bool:
"""Notify the active context engine after a durable compression commit."""
callback = getattr(agent.context_compressor, "on_session_start", None)
if not callable(callback):
return False
try:
callback(
new_session_id,
boundary_reason="compression",
old_session_id=old_session_id,
platform=getattr(agent, "platform", None) or "cli",
conversation_id=getattr(agent, "_gateway_session_key", None),
)
except Exception:
# Context-engine hooks are observers. A callback failure must not undo
# history that the core or an outer host transaction already committed.
logger.debug(
"context engine on_session_start (compression) failed",
exc_info=True,
)
return False
return True
def _queue_context_engine_compression_notification(
agent: Any,
*,
new_session_id: str,
old_session_id: str,
) -> None:
"""Stage exactly one existing hook call for an outer host transaction."""
if callable(getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)):
raise RuntimeError("a compression notification is already pending")
def _notify() -> bool:
return _notify_context_engine_compression_complete(
agent,
new_session_id=new_session_id,
old_session_id=old_session_id,
)
setattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, _notify)
def finalize_context_engine_compression_notification(
agent: Any,
*,
committed: bool,
) -> bool:
"""Emit or discard a deferred notification; repeated calls are no-ops."""
pending = getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)
setattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)
if not committed or not callable(pending):
return False
return bool(pending())
def compress_context(
agent: Any,
messages: list,
@@ -717,6 +870,7 @@ def compress_context(
task_id: str = "default",
focus_topic: Optional[str] = None,
force: bool = False,
defer_context_engine_notification: bool = False,
) -> Tuple[list, str]:
"""Compress conversation context and split the session in SQLite.
@@ -734,6 +888,8 @@ def compress_context(
by the manual ``/compress`` slash command so users can retry
immediately after an auto-compress abort. Auto-compress
callers use the default ``False``.
defer_context_engine_notification: Delay the existing context-engine
hook until a manual host commits its outer history transaction.
Returns:
``(compressed_messages, new_system_prompt)`` tuple. When
@@ -742,6 +898,12 @@ def compress_context(
prompt — the session is NOT rotated. Callers should detect the
no-op via ``len(returned) == len(input)`` and stop the retry loop.
"""
if (
defer_context_engine_notification
and callable(getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None))
):
raise RuntimeError("a compression notification is already pending")
# Codex app-server sessions: the codex agent owns the real thread context;
# Hermes' summarizer would only rewrite a local mirror without shrinking
# the actual thread (#36801). Route compaction to the app server's own
@@ -1020,6 +1182,7 @@ def compress_context(
existing_prompt = agent._build_system_prompt(system_message)
return messages, existing_prompt
_activity_heartbeat: Optional[_CompressionActivityHeartbeat] = None
try:
if _lock_holder is not None:
_lock_refresher = _CompressionLockLeaseRefresher(
@@ -1070,13 +1233,20 @@ def compress_context(
)
messages_before_compression = copy.deepcopy(messages)
_activity_heartbeat = _CompressionActivityHeartbeat(agent).start()
compressed = compress_fn(messages, **compress_kwargs)
except BaseException:
# ANY exception after lock acquisition — memory hook, capability
# inspection, engine lookup, or compress() — must release the lock so
# the session isn't permanently blocked from future compression.
if _activity_heartbeat is not None:
_activity_heartbeat.stop("context compression failed")
_activity_heartbeat = None
_release_lock()
raise
finally:
if _activity_heartbeat is not None:
_activity_heartbeat.stop("context compression completed")
try:
# Capture boundary quality before session-rotation callbacks run. Built-in
@@ -1206,6 +1376,7 @@ def compress_context(
new_system_prompt = agent._build_system_prompt(system_message)
agent._cached_system_prompt = new_system_prompt
_session_commit_succeeded = False
if agent._session_db:
try:
# Trigger memory extraction on the current session before the
@@ -1386,6 +1557,7 @@ def compress_context(
for message in compressed
if isinstance(message, dict)
}
_session_commit_succeeded = True
except Exception as e:
# If the rotation rolled back to the parent (orphan-avoidance
# above), agent.session_id is the still-indexed parent and
@@ -1406,6 +1578,9 @@ def compress_context(
# id on rotation, the (unchanged) current id in-place.
_old_sid = locals().get("old_session_id")
_is_boundary = bool(_old_sid) or in_place
_context_engine_boundary_committed = _session_commit_succeeded and (
bool(_old_sid) or compacted_in_place
)
_boundary_parent = _old_sid or agent.session_id or ""
# Notify the context engine that a compaction boundary occurred. Plugin
@@ -1414,17 +1589,19 @@ def compress_context(
# re-initializing fresh. See hermes-lcm#68. Built-in ContextCompressor
# ignores kwargs. Fires in BOTH modes: rotation passes old→new ids; in-place
# passes the SAME id (the boundary is real even though the id didn't move).
try:
if _is_boundary and hasattr(agent.context_compressor, "on_session_start"):
agent.context_compressor.on_session_start(
agent.session_id or "",
boundary_reason="compression",
if _context_engine_boundary_committed:
if defer_context_engine_notification:
_queue_context_engine_compression_notification(
agent,
new_session_id=agent.session_id or "",
old_session_id=_boundary_parent,
)
else:
_notify_context_engine_compression_complete(
agent,
new_session_id=agent.session_id or "",
old_session_id=_boundary_parent,
platform=getattr(agent, "platform", None) or "cli",
conversation_id=getattr(agent, "_gateway_session_key", None),
)
except Exception as _ce_err:
logger.debug("context engine on_session_start (compression): %s", _ce_err)
# Notify memory providers of the compaction boundary so provider-cached
# per-session state (Hindsight's _document_id, accumulated turn buffers,
@@ -1594,7 +1771,20 @@ def _compress_context_via_codex_app_server(
except Exception:
pass
result = codex_session.compact_thread()
_activity_heartbeat: Optional[_CompressionActivityHeartbeat] = None
try:
_activity_heartbeat = _CompressionActivityHeartbeat(agent).start()
result = codex_session.compact_thread()
except BaseException:
if _activity_heartbeat is not None:
_activity_heartbeat.stop("context compression failed")
raise
if getattr(result, "interrupted", False) or getattr(result, "error", None):
_activity_heartbeat.stop("context compression failed")
else:
_activity_heartbeat.stop("context compression completed")
if getattr(result, "should_retire", False):
try:
codex_session.close()
+51 -4
View File
@@ -685,6 +685,13 @@ def run_conversation(
truncated_tool_call_retries = 0
truncated_response_parts: List[str] = []
compression_attempts = 0
# One resolved per-turn compression attempt cap, shared by every site that
# consumes ``compression_attempts``: the pre-API pressure gate, the
# overflow/413 retry handlers, and the post-tool compaction gate.
# Config-driven via compression.max_attempts (parsed + validated in
# agent_init); default 3 preserves the prior hardcoded behavior for
# objects without the attribute (older pickles / minimal stubs).
max_compression_attempts = getattr(agent, "max_compression_attempts", 3)
_last_preflight_pressure: Optional[int] = None
_preflight_compression_blocked = _ctx.preflight_compression_blocked
_turn_exit_reason = "unknown" # Diagnostic: why the loop ended
@@ -699,6 +706,10 @@ def run_conversation(
# reused as the final response — not merely because any interim was
# streamed. (#65919 review: response-loss blocker)
_pending_verification_response_previewed = False
# If pre-API compression fires after MoA advisors have produced guidance,
# retain that ephemeral output and rebase it onto the compacted transcript
# on the next loop iteration. This prevents a second advisor fan-out.
pending_moa_prepared_request = None
# Per-turn tally of consecutive successful credential-pool token refreshes,
# keyed by (provider, pool-entry-id). A persistent upstream 401 lets
@@ -1082,6 +1093,29 @@ def run_conversation(
# the OpenAI SDK. Sanitizing here prevents the 3-retry cycle.
_sanitize_messages_surrogates(api_messages)
# Build a persistent-MoA request before measuring compression pressure.
# MoA reference output is injected into the aggregator prompt, but it
# is deliberately ephemeral and therefore absent from ``messages``.
# Preparing here makes the pre-API guard measure the exact prompt the
# aggregator will receive; ``create()`` consumes this private prepared
# request later without running the advisors a second time.
_moa_prepared_request = None
if agent.provider == "moa":
_moa_completions = getattr(getattr(agent.client, "chat", None), "completions", None)
if pending_moa_prepared_request is not None:
_rebase_moa_request = getattr(_moa_completions, "rebase_prepared_request", None)
if callable(_rebase_moa_request):
_moa_prepared_request = _rebase_moa_request(
pending_moa_prepared_request, api_messages
)
pending_moa_prepared_request = None
if _moa_prepared_request is None:
_prepare_moa_request = getattr(_moa_completions, "prepare", None)
if callable(_prepare_moa_request):
_moa_prepared_request = _prepare_moa_request(api_messages)
if _moa_prepared_request is not None:
api_messages = _moa_prepared_request["messages"]
# One image-stripped message estimate feeds both figures. Was: a
# str(msg) char walk (re-serialized base64 every call) + a second
# messages walk inside estimate_request_tokens_rough. Tools added
@@ -1168,21 +1202,24 @@ def run_conversation(
if (
agent.compression_enabled
and len(messages) > 1
and compression_attempts < 3
and compression_attempts < max_compression_attempts
and not _preflight_compression_blocked
and not _defer_preflight(request_pressure_tokens)
and not _compression_cooldown
and _compressor.should_compress(request_pressure_tokens)
):
if _moa_prepared_request is not None:
pending_moa_prepared_request = _moa_prepared_request
compression_attempts += 1
logger.info(
"Pre-API compression: ~%s request tokens >= %s threshold "
"(context=%s, attempt=%s/3)",
"(context=%s, attempt=%s/%s)",
f"{request_pressure_tokens:,}",
f"{int(getattr(_compressor, 'threshold_tokens', 0) or 0):,}",
f"{int(getattr(_compressor, 'context_length', 0) or 0):,}"
if getattr(_compressor, "context_length", 0) else "unknown",
compression_attempts,
max_compression_attempts,
)
agent._emit_status(
f"📦 Pre-API compression: ~{request_pressure_tokens:,} tokens "
@@ -1252,7 +1289,6 @@ def run_conversation(
retry_count = 0
max_retries = agent._api_max_retries
_retry = TurnRetryState()
max_compression_attempts = 3
finish_reason = "stop"
response = None # Guard against UnboundLocalError if all retries fail
@@ -1421,6 +1457,12 @@ def run_conversation(
if env_var_enabled("HERMES_DUMP_REQUESTS"):
agent._dump_api_request_debug(api_kwargs, reason="preflight")
# This object is private to the in-process MoA facade. Add it
# only after middleware, hooks, and debug dumps so none of them
# attempts to serialize it as part of the provider payload.
if _moa_prepared_request is not None and agent.provider == "moa":
api_kwargs["_moa_prepared_request"] = _moa_prepared_request
# Always prefer the streaming path — even without stream
# consumers. Streaming gives us fine-grained health
# checking (90s stale-stream detection, 60s read timeout)
@@ -5175,7 +5217,12 @@ def run_conversation(
messages, tools=agent.tools or None
)
if agent.compression_enabled and _compressor.should_compress(_real_tokens):
if (
agent.compression_enabled
and compression_attempts < max_compression_attempts
and _compressor.should_compress(_real_tokens)
):
compression_attempts += 1
agent._safe_print(" ⟳ compacting context…")
messages, active_system_prompt = agent._compress_context(
messages, system_message,
+117 -63
View File
@@ -905,7 +905,114 @@ class MoAChatCompletions:
except Exception as exc: # pragma: no cover - display must never break the turn
logger.debug("MoA reference_callback failed for %s: %s", event, exc)
def prepare(self, messages: list[dict[str, Any]]) -> dict[str, Any]:
"""Run the advisor fan-out and return the exact aggregator request.
The normal agent loop needs to measure this augmented prompt before its
compression gate. ``create()`` also uses this method for direct callers;
when the loop supplies the returned private object back to ``create()``,
the advisor fan-out is not repeated.
"""
return self.create(messages=messages, _moa_prepare_only=True)
def rebase_prepared_request(
self, prepared: dict[str, Any], messages: list[dict[str, Any]]
) -> dict[str, Any]:
"""Apply already-generated advisor guidance to a rebuilt API transcript.
Context compression changes the persisted transcript but not the
ephemeral advisor result. Reusing the guidance avoids a second costly
fan-out while keeping the aggregator request aligned with the compacted
history.
"""
guidance = prepared.get("guidance")
agg_messages = [dict(message) for message in messages]
if guidance:
_attach_reference_guidance(agg_messages, str(guidance))
return {**prepared, "messages": agg_messages}
def _call_prepared_aggregator(
self, prepared: dict[str, Any], api_kwargs: dict[str, Any]
) -> Any:
"""Send an already prepared MoA aggregator request exactly once."""
agg_messages = prepared["messages"]
aggregator = prepared["aggregator"]
aggregator_temperature = prepared["aggregator_temperature"]
if aggregator.get("provider") == "moa":
raise RuntimeError("MoA aggregator cannot be another MoA preset")
agg_kwargs = dict(api_kwargs)
max_tokens: Any = agg_kwargs.get("max_tokens")
tools: Any = agg_kwargs.get("tools")
extra_body: Any = agg_kwargs.get("extra_body")
# Record the exact aggregator INPUT (incl. the injected reference
# context) into the pending trace so a trace captures what the
# aggregator actually saw, not a reconstruction.
if self._pending_trace is not None:
self._pending_trace["aggregator_input_messages"] = agg_messages
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
# The aggregator is the acting model. Resolve its slot to the provider's
# real runtime (base_url/api_key/api_mode) and call it through the same
# request-building path any model uses — so per-model wire-format
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
# temperature) applies identically to it. MoA imposes no output cap:
# max_tokens is passed through from the caller (normally None → omitted
# → the model's real maximum). The preset's old hardcoded 4096 default
# is gone — it truncated long syntheses.
# When the agent's streaming consumer calls us with stream=True, run the
# references first (above) and then return the aggregator's RAW token
# stream so the acting model's output reaches the user live. The consumer
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
# back to a non-streaming retry on error. The non-streaming path
# (stream=False) is unchanged — no stream/stream_options/timeout are
# forwarded, so its behavior is byte-for-byte identical to before.
stream = bool(api_kwargs.get("stream"))
stream_kwargs: dict[str, Any] = {}
if stream:
stream_kwargs["stream"] = True
stream_kwargs["stream_options"] = (
api_kwargs.get("stream_options") or {"include_usage": True}
)
# Forward the consumer's per-request (stream read) timeout so it
# actually governs the aggregator stream, not just call_llm's default.
if api_kwargs.get("timeout") is not None:
stream_kwargs["timeout"] = api_kwargs["timeout"]
_agg_response = call_llm(
task="moa_aggregator",
messages=agg_messages,
temperature=aggregator_temperature,
max_tokens=max_tokens,
tools=tools,
extra_body=extra_body,
# Prepared requests must retain the acting aggregator's reasoning
# policy exactly as the direct create() path does (#64187).
reasoning_config=_aggregator_reasoning_config(aggregator),
**stream_kwargs,
**_slot_runtime(aggregator),
)
# Non-streaming path (quiet mode / eval / subagents): the aggregator
# output is available inline, so capture it into the pending trace now.
# Streaming path: the aggregator's raw token stream is returned to the
# consumer live and its acting output lands as the turn's assistant
# message; the trace marks it streamed and points there.
if self._pending_trace is not None:
if stream:
self._pending_trace["aggregator_streamed"] = True
self._pending_trace["aggregator_output"] = None
else:
self._pending_trace["aggregator_streamed"] = False
try:
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
except Exception: # pragma: no cover - defensive
self._pending_trace["aggregator_output"] = None
return _agg_response
def create(self, **api_kwargs: Any) -> Any:
prepared_request = api_kwargs.pop("_moa_prepared_request", None)
if prepared_request is not None:
if not isinstance(prepared_request, dict):
raise TypeError("_moa_prepared_request must be a dict")
return self._call_prepared_aggregator(prepared_request, api_kwargs)
from hermes_cli.config import load_config
from hermes_cli.moa_config import resolve_moa_preset
@@ -1065,6 +1172,7 @@ class MoAChatCompletions:
ref_count=_ref_count,
)
guidance: str | None = None
agg_messages = [dict(m) for m in messages]
if reference_outputs:
joined = "\n\n".join(
@@ -1082,69 +1190,15 @@ class MoAChatCompletions:
)
_attach_reference_guidance(agg_messages, guidance)
if aggregator.get("provider") == "moa":
raise RuntimeError("MoA aggregator cannot be another MoA preset")
agg_kwargs = dict(api_kwargs)
agg_kwargs["messages"] = agg_messages
# Record the exact aggregator INPUT (incl. the injected reference
# context) into the pending trace so a trace captures what the
# aggregator actually saw, not a reconstruction.
if self._pending_trace is not None:
self._pending_trace["aggregator_input_messages"] = agg_messages
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
# The aggregator is the acting model. Resolve its slot to the provider's
# real runtime (base_url/api_key/api_mode) and call it through the same
# request-building path any model uses — so per-model wire-format
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
# temperature) applies identically to it. MoA imposes no output cap:
# max_tokens is passed through from the caller (normally None → omitted
# → the model's real maximum). The preset's old hardcoded 4096 default
# is gone — it truncated long syntheses.
# When the agent's streaming consumer calls us with stream=True, run the
# references first (above) and then return the aggregator's RAW token
# stream so the acting model's output reaches the user live. The consumer
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
# back to a non-streaming retry on error. The non-streaming path
# (stream=False) is unchanged — no stream/stream_options/timeout are
# forwarded, so its behavior is byte-for-byte identical to before.
stream = bool(api_kwargs.get("stream"))
stream_kwargs: dict[str, Any] = {}
if stream:
stream_kwargs["stream"] = True
stream_kwargs["stream_options"] = (
api_kwargs.get("stream_options") or {"include_usage": True}
)
# Forward the consumer's per-request (stream read) timeout so it
# actually governs the aggregator stream, not just call_llm's default.
if api_kwargs.get("timeout") is not None:
stream_kwargs["timeout"] = api_kwargs["timeout"]
_agg_response = call_llm(
task="moa_aggregator",
messages=agg_messages,
temperature=aggregator_temperature,
max_tokens=agg_kwargs.get("max_tokens"),
tools=agg_kwargs.get("tools"),
extra_body=agg_kwargs.get("extra_body"),
reasoning_config=_aggregator_reasoning_config(aggregator),
**stream_kwargs,
**_slot_runtime(aggregator),
)
# Non-streaming path (quiet mode / eval / subagents): the aggregator
# output is available inline, so capture it into the pending trace now.
# Streaming path: the aggregator's raw token stream is returned to the
# consumer live and its acting output lands as the turn's assistant
# message; the trace marks it streamed and points there.
if self._pending_trace is not None:
if stream:
self._pending_trace["aggregator_streamed"] = True
self._pending_trace["aggregator_output"] = None
else:
self._pending_trace["aggregator_streamed"] = False
try:
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
except Exception: # pragma: no cover - defensive
self._pending_trace["aggregator_output"] = None
return _agg_response
prepared_request = {
"messages": agg_messages,
"guidance": guidance,
"aggregator": aggregator,
"aggregator_temperature": aggregator_temperature,
}
if api_kwargs.pop("_moa_prepare_only", False):
return prepared_request
return self._call_prepared_aggregator(prepared_request, api_kwargs)
class MoAClient:
+80 -6
View File
@@ -2666,16 +2666,61 @@ async def get_model_context_length_async(
)
def _is_cjk_token_dense_char(ch: str) -> bool:
code = ord(ch)
return (
0x1100 <= code <= 0x11FF # Hangul Jamo
or 0x2E80 <= code <= 0x9FFF # CJK radicals/ideographs
or 0xA960 <= code <= 0xA97F # Hangul Jamo Extended-A
or 0xAC00 <= code <= 0xD7AF # Hangul Syllables
or 0xF900 <= code <= 0xFAFF # CJK compatibility ideographs
or 0xFF00 <= code <= 0xFFEF # Fullwidth forms / halfwidth kana
)
# Same codepoint ranges as _is_cjk_token_dense_char, as a compiled character
# class so dense-char counting runs in C (``len(text) - len(re.sub(...))``)
# instead of a per-char Python loop. MUST stay in sync with
# _is_cjk_token_dense_char.
_CJK_DENSE_RE = re.compile(
"[\u1100-\u11ff" # Hangul Jamo
"\u2e80-\u9fff" # CJK radicals/ideographs
"\ua960-\ua97f" # Hangul Jamo Extended-A
"\uac00-\ud7af" # Hangul Syllables
"\uf900-\ufaff" # CJK compatibility ideographs
"\uff00-\uffef]" # Fullwidth forms / halfwidth kana
)
def estimate_tokens_rough(text: str) -> int:
"""Rough token estimate (~4 chars/token) for pre-flight checks.
"""Rough token estimate for pre-flight checks.
Uses ceiling division so short texts (1-3 chars) never estimate as
0 tokens, which would cause the compressor and pre-flight checks to
systematically undercount when many short tool results are present.
CJK/Hangul/Kana text is much denser than English under common LLM
tokenizers, so count those codepoints as roughly one token each instead
of applying the English-centric ~4 chars/token rule.
Perf: this runs on every message in every preflight/compaction walk,
including MB-scale tool outputs, so the common all-ASCII case must stay
O(1). ``str.isascii()`` is a flag check on CPython's compact unicode
representation (no scan), and the CJK counting itself is a single
C-level ``re.findall`` rather than a per-character Python loop.
"""
if not text:
return 0
return (len(text) + 3) // 4
text = str(text)
if text.isascii():
# O(1) fast path — ASCII text cannot contain token-dense CJK chars.
return (len(text) + 3) // 4
dense = len(text) - len(_CJK_DENSE_RE.sub("", text))
if not dense:
# Non-ASCII but no CJK (accents, Cyrillic, emoji, ...): keep the
# classic ~4 chars/token rule.
return (len(text) + 3) // 4
sparse = len(text) - dense
return dense + ((sparse + 3) // 4)
def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
@@ -2687,12 +2732,12 @@ def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
estimated at ~250K tokens and trigger premature context compression.
"""
_IMAGE_TOKEN_COST = 1500
total_chars = 0
text_tokens = 0
image_tokens = 0
for msg in messages:
total_chars += _estimate_message_chars(msg)
text_tokens += _estimate_message_tokens_without_images(msg)
image_tokens += _count_image_tokens(msg, _IMAGE_TOKEN_COST)
return ((total_chars + 3) // 4) + image_tokens
return text_tokens + image_tokens
def _count_image_tokens(msg: Dict[str, Any], cost_per_image: int) -> int:
@@ -2754,6 +2799,35 @@ def _estimate_message_chars(msg: Dict[str, Any]) -> int:
return len(str(shadow))
def _estimate_message_tokens_without_images(msg: Dict[str, Any]) -> int:
"""Token estimate for a message shadow with image payloads stripped."""
if not isinstance(msg, dict):
return estimate_tokens_rough(str(msg))
shadow: Dict[str, Any] = {}
for k, v in msg.items():
if k == "_anthropic_content_blocks":
continue
if k == "content":
if isinstance(v, list):
cleaned = []
for part in v:
if isinstance(part, dict):
if part.get("type") in {"image", "image_url", "input_image"}:
cleaned.append({"type": part.get("type"), "image": "[stripped]"})
else:
cleaned.append(part)
else:
cleaned.append(part)
shadow[k] = cleaned
elif isinstance(v, dict) and v.get("_multimodal"):
shadow[k] = v.get("text_summary", "")
else:
shadow[k] = v
else:
shadow[k] = v
return estimate_tokens_rough(str(shadow))
def estimate_request_tokens_rough(
messages: List[Dict[str, Any]],
*,
@@ -2770,7 +2844,7 @@ def estimate_request_tokens_rough(
"""
total = 0
if system_prompt:
total += (len(system_prompt) + 3) // 4
total += estimate_tokens_rough(system_prompt)
if messages:
total += estimate_messages_tokens_rough(messages)
if tools:
@@ -92,6 +92,79 @@ class TurnResult:
_TURN_ABORTED_MARKERS = ("<turn_aborted>", "<turn_aborted/>")
def _notification_scope_ids(
note: dict,
) -> tuple[Optional[str], Optional[str]]:
"""Extract the thread/turn identity carried by a notification."""
if not isinstance(note, dict):
return None, None
params = note.get("params") or {}
if not isinstance(params, dict):
return None, None
nested_turn = params.get("turn") or {}
nested_item = params.get("item") or {}
observed_thread_id = params.get("threadId") or params.get("thread_id")
if observed_thread_id is None and isinstance(nested_turn, dict):
observed_thread_id = (
nested_turn.get("threadId")
or nested_turn.get("thread_id")
)
if observed_thread_id is None and isinstance(nested_item, dict):
observed_thread_id = (
nested_item.get("threadId")
or nested_item.get("thread_id")
)
observed_turn_id = params.get("turnId") or params.get("turn_id")
if observed_turn_id is None and isinstance(nested_turn, dict):
observed_turn_id = nested_turn.get("id") or nested_turn.get("turnId")
if observed_turn_id is None and isinstance(nested_item, dict):
observed_turn_id = (
nested_item.get("turnId")
or nested_item.get("turn_id")
)
return observed_thread_id, observed_turn_id
def _notification_belongs_to_turn(
note: dict,
*,
thread_id: Optional[str],
turn_id: Optional[str],
) -> bool:
"""Return whether a multiplexed notification belongs to this turn.
Codex app-server can carry parent and hosted subagent threads over one
JSON-RPC connection. An explicitly foreign child or
stale-turn event must not mutate the active parent's transcript or mark
its turn complete. Unscoped notifications remain accepted for protocol
compatibility.
"""
if not isinstance(note, dict):
return False
observed_thread_id, observed_turn_id = _notification_scope_ids(note)
if (
thread_id is not None
and observed_thread_id is not None
and str(observed_thread_id) != str(thread_id)
):
return False
if (
turn_id is not None
and observed_turn_id is not None
and str(observed_turn_id) != str(turn_id)
):
return False
return True
def _coerce_turn_input_text(user_input: Any) -> str:
"""Collapse Hermes/OpenAI rich content into app-server text input.
@@ -505,6 +578,17 @@ class CodexAppServerSession:
pending = self._client.take_notification(timeout=0)
if pending is None:
break
if not _notification_belongs_to_turn(
pending,
thread_id=self._thread_id,
turn_id=result.turn_id,
):
logger.debug(
"ignoring foreign codex notification while draining "
"server request: method=%s",
pending.get("method"),
)
continue
# Mirror the main notification-handling block below so
# display events surface and stay in step with projector
# state. Without this, item/started / item/completed
@@ -550,6 +634,16 @@ class CodexAppServerSession:
continue
method = note.get("method", "")
if not _notification_belongs_to_turn(
note,
thread_id=self._thread_id,
turn_id=result.turn_id,
):
logger.debug(
"ignoring foreign codex notification: method=%s", method
)
continue
if self._on_event is not None:
try:
self._on_event(note)
@@ -737,6 +831,48 @@ class CodexAppServerSession:
continue
method = note.get("method", "")
observed_thread_id, observed_turn_id = _notification_scope_ids(note)
if result.turn_id is None:
if method == "turn/started":
if (
observed_thread_id is not None
and str(observed_thread_id) != str(self._thread_id)
):
logger.debug(
"ignoring foreign compact turn/started: thread=%s",
observed_thread_id,
)
continue
if observed_turn_id is None:
logger.debug(
"ignoring compact turn/started without a turn id"
)
continue
result.turn_id = str(observed_turn_id)
elif observed_turn_id is not None or method in {
"item/completed",
"turn/completed",
}:
# thread/compact/start does not return a turn id. Until the
# new turn/started arrives, any terminal/projectable event
# is stale or cannot be safely attributed to this compaction.
logger.debug(
"ignoring codex notification before compact turn start: "
"method=%s",
method,
)
continue
if not _notification_belongs_to_turn(
note,
thread_id=self._thread_id,
turn_id=result.turn_id,
):
logger.debug(
"ignoring foreign codex notification: method=%s", method
)
continue
if self._on_event is not None:
try:
self._on_event(note)
+7 -1
View File
@@ -665,7 +665,13 @@ def build_turn_context(
f">= {_compressor.threshold_tokens:,} threshold. "
"This may take a moment."
)
for _pass in range(3):
# Preflight passes honor the same configured per-turn cap
# (compression.max_attempts) as the loop's compression sites;
# default 3 preserves the prior hardcoded behavior.
_max_preflight_passes = max(
1, int(getattr(agent, "max_compression_attempts", 3) or 3)
)
for _pass in range(_max_preflight_passes):
_orig_len = len(messages)
_orig_tokens = _preflight_tokens
messages, active_system_prompt = agent._compress_context(
@@ -31,7 +31,7 @@ describe('desktop slash command curation', () => {
it('hides terminal, messaging, and dedicated-UI commands from suggestions', () => {
expect(isDesktopSlashSuggestion('/clear')).toBe(false)
expect(isDesktopSlashSuggestion('/compact')).toBe(false)
expect(isDesktopSlashSuggestion('/density')).toBe(false)
expect(isDesktopSlashSuggestion('/redraw')).toBe(false)
expect(isDesktopSlashSuggestion('/approve')).toBe(false)
expect(isDesktopSlashSuggestion('/model')).toBe(false)
@@ -40,6 +40,14 @@ describe('desktop slash command curation', () => {
expect(isDesktopSlashSuggestion('/curator')).toBe(false)
})
it('routes /compact to /compress (context compression), not the TUI display toggle', () => {
expect(resolveDesktopCommand('/compact')?.name).toBe('/compress')
expect(isDesktopSlashCommand('/compact')).toBe(true)
// Alias stays out of the popover so /compress is the single visible entry.
expect(isDesktopSlashSuggestion('/compact')).toBe(false)
expect(isDesktopSlashSuggestion('/compress')).toBe(true)
})
it('surfaces /tools, /save, and /personality on the desktop', () => {
expect(isDesktopSlashSuggestion('/tools')).toBe(true)
expect(isDesktopSlashSuggestion('/save')).toBe(true)
@@ -148,7 +148,7 @@ const DESKTOP_COMMAND_SPECS: readonly DesktopCommandSpec[] = [
surface: exec()
},
{ name: '/background', description: 'Run a prompt in the background', aliases: ['/bg', '/btw'], surface: exec() },
{ name: '/compress', description: 'Compress this conversation context', surface: exec() },
{ name: '/compress', description: 'Compress this conversation context', aliases: ['/compact'], surface: exec() },
{ name: '/debug', description: 'Create a debug report', surface: exec() },
{ name: '/goal', description: 'Manage the standing goal for this session', surface: exec() },
{ name: '/personality', description: 'Switch personality for this session', surface: exec(), args: true },
@@ -187,10 +187,10 @@ const NO_DESKTOP_SURFACE: Record<DesktopUnavailableReason, readonly string[]> =
terminal: [
'/busy',
'/clear',
'/compact',
'/config',
'/copy',
'/cron',
'/density',
'/details',
'/exit',
'/footer',
+16
View File
@@ -414,6 +414,16 @@ compression:
# compaction doesn't fire with half the window still free; set above 0.75 to override.
threshold: 0.50
# Per-model threshold overrides: keys are substring-matched against the model
# name (longest match wins). Useful when some models need different compaction
# points — e.g. a 1M-context model can compress later (0.30) while a 128K
# model needs to compress earlier (0.60). The small-context floor (75% for
# <512K models) still applies on top of per-model overrides.
# model_thresholds:
# "glm-5.2": 0.40
# "claude-sonnet": 0.35
# "gpt-5": 0.30
# Existing Codex gpt-5.5 behavior: raise Hermes' compaction trigger to 85%
# for the ChatGPT Codex OAuth route. Set false to opt back down to threshold.
codex_gpt55_autoraise: true
@@ -429,6 +439,12 @@ compression:
# compression of older turns.
protect_last_n: 20
# Compression retry rounds before a turn gives up with "max compression
# attempts reached" (default: 3, same as the previous hardcoded value).
# Raise (e.g. 6) for tool-schema-heavy sessions where 3 rounds cannot bring
# the request estimate under the threshold. Validated >= 1, hard cap 10.
max_attempts: 3
# Codex app-server (codex CLI runtime) thread-compaction mode. The codex
# agent owns the real thread context on this runtime, so Hermes' summarizer
# cannot shrink it — compaction goes through the app server instead.
+12 -7
View File
@@ -9928,6 +9928,9 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
split_history_for_partial_compress,
summarize_compress_preview,
)
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
# Args after the command word (e.g. "/compress here 3" -> "here 3").
raw_args = ""
@@ -10027,14 +10030,8 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
approx_tokens=approx_tokens,
focus_topic=focus_topic or None,
force=True,
defer_context_engine_notification=True,
)
# Re-append the verbatim tail after the compressed head.
# The split guarantees `tail` begins on a user turn, so the
# compressed-head -> tail boundary is normally valid
# (the head's compressed output ends on assistant/tool).
# rejoin_compressed_head_and_tail() additionally guards the
# seam against any illegal user->user / assistant->assistant
# adjacency, defending provider role-alternation rules.
if partial and tail:
compressed = rejoin_compressed_head_and_tail(compressed, tail)
self.conversation_history = compressed
@@ -10054,6 +10051,10 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
# compressed handoff for the child session. Persist it from
# offset 0 so resume can recover the continuation after exit.
self.agent._flush_messages_to_session_db(self.conversation_history, None)
finalize_context_engine_compression_notification(
self.agent,
committed=True,
)
new_tokens = estimate_request_tokens_rough(
self.conversation_history,
system_prompt=_sys_prompt,
@@ -10078,6 +10079,10 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
print(f" {summary['note']}")
except Exception as e:
finalize_context_engine_compression_notification(
self.agent,
committed=False,
)
print(f" ❌ Compression failed: {e}")
+1
View File
@@ -0,0 +1 @@
WeiYusc
+2
View File
@@ -0,0 +1,2 @@
bpross
# PR #29541 salvage
+1
View File
@@ -0,0 +1 @@
bennybuoy
@@ -0,0 +1 @@
LiangYang666
+2
View File
@@ -0,0 +1,2 @@
sowork-skills
# PR #33224 salvage (slack: truncate edit_message to prevent msg_too_long)
@@ -0,0 +1 @@
tandixit95
@@ -0,0 +1 @@
dombejar
+1
View File
@@ -0,0 +1 @@
dsitmilis
+2
View File
@@ -0,0 +1,2 @@
kamonspecial
# PR #59621 salvage (slack: empty rich_text content guards)
@@ -0,0 +1 @@
LiangYang666
@@ -0,0 +1 @@
plainOldCode
+2
View File
@@ -0,0 +1,2 @@
tw0316
# PR #56618 salvage (slack: table column_settings + blocks rejection fallback)
+2
View File
@@ -0,0 +1,2 @@
xcompass
# PR #61261 salvage
@@ -0,0 +1 @@
Sora-bluesky
@@ -0,0 +1 @@
yitang
@@ -0,0 +1 @@
the3asic
@@ -0,0 +1 @@
wernerhp
+121 -6
View File
@@ -127,6 +127,8 @@ MAX_REQUEST_BYTES = 10_000_000 # 10 MB — accommodates long agent conversation
CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS = 30.0
MAX_NORMALIZED_TEXT_LENGTH = 65_536 # 64 KB cap for normalized content parts
MAX_CONTENT_LIST_SIZE = 1_000 # Max items when content is an array
RESPONSES_AUTO_TRUNCATION_HISTORY_LIMIT = 100
_COMPRESSED_SUMMARY_METADATA_KEY = "_compressed_summary"
def _coerce_port(value: Any, default: int = DEFAULT_PORT) -> int:
@@ -166,6 +168,71 @@ def _coerce_request_bool(value: Any, default: bool = False) -> bool:
return default
def _message_text_prefix(content: Any) -> str:
if isinstance(content, str):
return content[:128]
if not isinstance(content, list):
return ""
parts: List[str] = []
for item in content[:4]:
if isinstance(item, str):
parts.append(item)
elif isinstance(item, dict):
text = item.get("text")
if isinstance(text, str):
parts.append(text)
if sum(len(part) for part in parts) >= 128:
break
return "\n".join(parts)[:128]
def _is_compressed_summary_message(message: Any) -> bool:
if not isinstance(message, dict):
return False
if message.get(_COMPRESSED_SUMMARY_METADATA_KEY):
return True
prefix = _message_text_prefix(message.get("content"))
return prefix.startswith("[CONTEXT COMPACTION") or prefix.startswith("[CONTEXT SUMMARY]:")
def _auto_truncate_response_history(
conversation_history: List[Dict[str, Any]],
*,
limit: int = RESPONSES_AUTO_TRUNCATION_HISTORY_LIMIT,
) -> List[Dict[str, Any]]:
"""Keep recent Responses history without dropping the compaction handoff.
Compaction summaries are preserved wherever they sit in the history —
the gateway /compress path can leave them after a retained system head
(see ``context_compressor`` force-user-leading handling), so a
leading-block-only scan would silently drop them.
"""
if limit <= 0 or len(conversation_history) <= limit:
return conversation_history
summary_indices = [
index
for index, message in enumerate(conversation_history)
if _is_compressed_summary_message(message)
]
if not summary_indices:
return conversation_history[-limit:]
kept_indices = set(summary_indices[:limit])
remaining = limit - len(kept_indices)
if remaining > 0:
summary_index_set = set(summary_indices)
for index in range(len(conversation_history) - 1, -1, -1):
if index in summary_index_set:
continue
kept_indices.add(index)
remaining -= 1
if remaining <= 0:
break
return [conversation_history[index] for index in sorted(kept_indices)]
def _normalize_chat_content(
content: Any, *, _max_depth: int = 10, _depth: int = 0,
) -> str:
@@ -3310,6 +3377,7 @@ class APIServerAdapter(BasePlatformAdapter):
response_env: Dict[str, Any],
*,
conversation_history_snapshot: Optional[List[Dict[str, Any]]] = None,
session_id_snapshot: Optional[str] = None,
) -> None:
if not store:
return
@@ -3320,7 +3388,7 @@ class APIServerAdapter(BasePlatformAdapter):
"response": response_env,
"conversation_history": conversation_history_snapshot,
"instructions": instructions,
"session_id": session_id,
"session_id": session_id_snapshot or session_id,
})
if conversation:
self._response_store.set_conversation(conversation, response_id)
@@ -3723,9 +3791,15 @@ class APIServerAdapter(BasePlatformAdapter):
result,
final_response_text,
)
# Compression-aware transcript substitution happens inside
# _build_response_conversation_history (result["_compressed"]);
# here we only propagate a compression-rotated session_id so
# previous_response_id chaining resumes the child session.
_result_sid = result.get("session_id") if isinstance(result, dict) else None
_persist_response_snapshot(
completed_env,
conversation_history_snapshot=full_history,
session_id_snapshot=_result_sid if isinstance(_result_sid, str) and _result_sid else None,
)
terminal_snapshot_persisted = True
await _write_event("response.completed", {
@@ -3898,8 +3972,8 @@ class APIServerAdapter(BasePlatformAdapter):
return web.json_response(_openai_error("No user message found in input"), status=400)
# Truncation support
if body.get("truncation") == "auto" and len(conversation_history) > 100:
conversation_history = conversation_history[-100:]
if body.get("truncation") == "auto":
conversation_history = _auto_truncate_response_history(conversation_history)
# Reuse session from previous_response_id chain so the dashboard
# groups the entire conversation under one session entry.
@@ -4038,6 +4112,16 @@ class APIServerAdapter(BasePlatformAdapter):
final_response,
)
# Persist the effective session ID surfaced by _run_agent so that
# compression-triggered session rotations propagate to the stored
# response and the X-Hermes-Session-Id header. Without this,
# previous_response_id chaining keeps resuming the pre-rotation
# session and re-triggers compression on every subsequent request.
_effective_session_id = session_id
_result_sid = result.get("session_id") if isinstance(result, dict) else None
if isinstance(_result_sid, str) and _result_sid:
_effective_session_id = _result_sid
# Build output items from the current turn only. AIAgent returns a
# full transcript in result["messages"], while older/mocked paths may
# return only the current turn suffix.
@@ -4068,14 +4152,14 @@ class APIServerAdapter(BasePlatformAdapter):
"response": response_data,
"conversation_history": full_history,
"instructions": instructions,
"session_id": session_id,
"session_id": _effective_session_id,
})
# Update conversation mapping so the next request with the same
# conversation name automatically chains to this response
if conversation:
self._response_store.set_conversation(conversation, response_id)
response_headers = {"X-Hermes-Session-Id": session_id}
response_headers = {"X-Hermes-Session-Id": _effective_session_id}
if gateway_session_key:
response_headers["X-Hermes-Session-Key"] = gateway_session_key
return web.json_response(response_data, headers=response_headers)
@@ -4432,7 +4516,16 @@ class APIServerAdapter(BasePlatformAdapter):
result: Dict[str, Any],
final_response: Any,
) -> List[Dict[str, Any]]:
"""Build the stored Responses transcript without duplicating history."""
"""Build the stored Responses transcript without duplicating history.
When context compression occurs during a turn the agent returns a
compressed full transcript in ``result["messages"]`` (starting with a
summary) and sets ``result["_compressed"] = True``. Because the
compressed transcript does not share the input ``conversation_history``
prefix, the normal turn-start detection fails and old code would
concatenate the uncompressed history on front, bloating the stored
context and re-triggering compression on every subsequent request.
"""
prior = list(conversation_history)
current_user = {"role": "user", "content": user_message}
agent_messages = result.get("messages") if isinstance(result, dict) else None
@@ -4446,6 +4539,16 @@ class APIServerAdapter(BasePlatformAdapter):
if turn_start:
return list(agent_messages)
# turn_start == 0: agent_messages does not start with prior.
# This can happen because compression rewrote the transcript
# (summary prefix replaces original history), OR because
# agent_messages only carries the current turn without prior.
# The ``_compressed`` flag (set by _run_agent after compaction)
# distinguishes — skip the concatenation and use the compressed
# transcript directly.
if result.get("_compressed"):
return list(agent_messages)
full_history = prior
full_history.append(current_user)
full_history.extend(agent_messages)
@@ -4706,6 +4809,18 @@ class APIServerAdapter(BasePlatformAdapter):
_eff_sid = getattr(agent, "session_id", session_id)
if isinstance(_eff_sid, str) and _eff_sid:
result["session_id"] = _eff_sid
# Signal whether context compression occurred during this turn
# so _build_response_conversation_history can skip the
# prior-concatenation path and store the compressed transcript
# directly. Rotation mode changes agent.session_id; in-place
# mode sets _last_compaction_in_place (see #38763).
_compacted_in_place = bool(getattr(agent, "_last_compaction_in_place", False))
_session_rotated = (
isinstance(_eff_sid, str) and isinstance(session_id, str)
and _eff_sid != session_id
)
if _compacted_in_place or _session_rotated:
result["_compressed"] = True
return result, usage
finally:
clear_session_vars(tokens)
+12
View File
@@ -17643,6 +17643,7 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
("model", "max_tokens"),
("compression", "enabled"),
("compression", "threshold"),
("compression", "model_thresholds"),
("compression", "codex_gpt55_autoraise"),
("compression", "codex_app_server_auto"),
("compression", "target_ratio"),
@@ -19626,6 +19627,17 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
if event_type not in {"tool.started",}:
return
# Never render a progress bubble for the clarify tool. The
# adapter's send_clarify IS the user-facing rendering (interactive
# buttons or the numbered-text fallback), so a progress bubble is
# pure duplication — and in verbose mode it dumps the raw
# tool-call args JSON ({"question": ..., "choices": [...]}) into
# the chat. Because the progress queue drains on a background
# task, that raw JSON typically lands right underneath the
# rendered prompt (#52374).
if tool_name == "clarify":
return
# Suppress tool-progress bubbles once the user has sent `stop`.
# When the LLM response carries N parallel tool calls, the agent
# fires N "tool.started" events back-to-back before checking for
+20 -4
View File
@@ -3423,6 +3423,9 @@ class GatewaySlashCommandsMixin:
split_history_for_partial_compress,
summarize_compress_preview,
)
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
_raw_args = (event.get_command_args() or "").strip()
# Strip --preview/--dry-run/--aggressive before positional parsing
# so the flags coexist with 'here [N]' / focus-topic forms.
@@ -3551,11 +3554,15 @@ class GatewaySlashCommandsMixin:
loop = asyncio.get_running_loop()
compressed, _ = await loop.run_in_executor(
None,
lambda: tmp_agent._compress_context(head, "", approx_tokens=approx_tokens, focus_topic=focus_topic, force=True)
lambda: tmp_agent._compress_context(
head,
"",
approx_tokens=approx_tokens,
focus_topic=focus_topic,
force=True,
defer_context_engine_notification=True,
)
)
# Re-append the verbatim tail after the compressed head,
# guarding the seam against illegal role adjacency.
if partial and tail:
compressed = rejoin_compressed_head_and_tail(compressed, tail)
@@ -3625,6 +3632,10 @@ class GatewaySlashCommandsMixin:
await self.async_session_store.update_session(
session_entry.session_key, last_prompt_tokens=0
)
finalize_context_engine_compression_notification(
tmp_agent,
committed=True,
)
new_tokens = estimate_request_tokens_rough(
compressed, system_prompt=_sys_prompt, tools=_tools
)
@@ -3655,6 +3666,10 @@ class GatewaySlashCommandsMixin:
_aux_fail_model = getattr(compressor, "_last_aux_model_failure_model", None)
_aux_fail_err = getattr(compressor, "_last_aux_model_failure_error", None)
finally:
finalize_context_engine_compression_notification(
tmp_agent,
committed=False,
)
# Evict cached agent so next turn rebuilds system prompt
# from current files (SOUL.md, memory, etc.).
self._evict_cached_agent(session_key)
@@ -4151,6 +4166,7 @@ class GatewaySlashCommandsMixin:
# replays the parent's exact wire bytes (warm provider
# prompt cache) instead of a full cold prefill.
api_content=extract_api_content_sidecar(msg),
timestamp=msg.get("timestamp"),
)
except Exception:
pass # Best-effort copy
+1
View File
@@ -968,6 +968,7 @@ class CLICommandsMixin:
# replays the parent's exact wire bytes (warm provider
# prompt cache) instead of a full cold prefill.
api_content=extract_api_content_sidecar(msg),
timestamp=msg.get("timestamp"),
)
except Exception:
pass # Best-effort copy
+16
View File
@@ -1471,6 +1471,11 @@ DEFAULT_CONFIG = {
# set this above 0.75 to override the floor.
"target_ratio": 0.20, # fraction of threshold to preserve as recent tail
"protect_last_n": 20, # minimum recent messages to keep uncompressed
"max_attempts": 3, # compression retry rounds before a turn gives up
# with "max compression attempts reached". Raise
# (e.g. 6) for tool-schema-heavy sessions where 3
# rounds cannot clear the request estimate.
# Validated >= 1, hard-capped at 10.
"hygiene_hard_message_limit": 5000, # gateway session-hygiene force-compress threshold by message count
"protect_first_n": 3, # non-system head messages always preserved
# verbatim, in ADDITION to the system prompt
@@ -1529,6 +1534,17 @@ DEFAULT_CONFIG = {
# session_search and recoverable, not deleted.
# Default False during rollout; will flip on
# after live validation.
"model_thresholds": {}, # Per-model threshold overrides. Keys are
# substring-matched against the model name
# (longest match wins); values replace the
# global `threshold` for that model, e.g.
# model_thresholds:
# "glm-5.2": 0.40
# "claude-sonnet": 0.35
# The small-context floor (0.75 for <512K
# models) still applies on top of overrides
# (raise-only: an override above the floor
# wins; one below it is raised to the floor).
},
# Kanban subsystem (orchestrator workers + dispatcher-driven child tasks).
+79 -3
View File
@@ -2709,6 +2709,7 @@ class SessionDB:
parent_session_id: str = None,
cwd: str = None,
profile_name: str = None,
git_repo_root: str = None,
) -> None:
"""Insert a session row, enriching NULL metadata on conflict.
@@ -2727,14 +2728,30 @@ class SessionDB:
a persisted, now-inactive row belongs to the caller's chat/thread before
switching to it (IDOR scoping — without them the ``sessions`` table has
no chat/thread to compare).
When ``parent_session_id`` is set (compression fork, delegate/subagent
spawn, branch continuation) and this row's own ``cwd``/``git_repo_root``/
``git_branch`` are still NULL after the insert, they are backfilled from
the parent row. Callers of ``create_session`` for a child session
historically didn't propagate these fields themselves (e.g. the
compression-fork path), so a lineage could silently lose its working
directory and drop out of the project sidebar every time it forked
(#64709). This only fills NULLs — an explicit ``cwd``/``git_repo_root``
on the child is never overwritten. For compression forks specifically
(parent ended with ``end_reason='compression'``), the gateway origin
columns (``user_id``/``session_key``/``chat_id``/``chat_type``/
``thread_id``/``display_name``/``origin_json``) are inherited too, so a
crash before the gateway re-records the peer can't strand the child
without a recoverable routing mapping (#59527).
"""
def _do(conn):
conn.execute(
"""INSERT INTO sessions (
id, source, user_id, session_key, chat_id, chat_type, thread_id,
model, model_config, system_prompt, parent_session_id, cwd, profile_name, started_at
model, model_config, system_prompt, parent_session_id, cwd,
profile_name, git_repo_root, started_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
model = COALESCE(sessions.model, excluded.model),
model_config = COALESCE(sessions.model_config, excluded.model_config),
@@ -2745,7 +2762,8 @@ class SessionDB:
thread_id = COALESCE(sessions.thread_id, excluded.thread_id),
parent_session_id = COALESCE(sessions.parent_session_id, excluded.parent_session_id),
cwd = COALESCE(sessions.cwd, excluded.cwd),
profile_name = COALESCE(sessions.profile_name, excluded.profile_name)""",
profile_name = COALESCE(sessions.profile_name, excluded.profile_name),
git_repo_root = COALESCE(sessions.git_repo_root, excluded.git_repo_root)""",
(
session_id,
source,
@@ -2760,9 +2778,67 @@ class SessionDB:
parent_session_id,
cwd,
profile_name,
git_repo_root,
time.time(),
),
)
if parent_session_id:
conn.execute(
"""UPDATE sessions
SET cwd = COALESCE(sessions.cwd,
(SELECT p.cwd FROM sessions p
WHERE p.id = sessions.parent_session_id)),
git_repo_root = COALESCE(sessions.git_repo_root,
(SELECT p.git_repo_root FROM sessions p
WHERE p.id = sessions.parent_session_id)),
git_branch = COALESCE(sessions.git_branch,
(SELECT p.git_branch FROM sessions p
WHERE p.id = sessions.parent_session_id))
WHERE id = ? AND parent_session_id IS NOT NULL""",
(session_id,),
)
# Belt-and-suspenders for gateway routing metadata (#59527):
# the gateway re-records the peer on the child after rotation
# (d5b4879d4), but a hard crash between child creation and that
# write leaves the child row without origin columns, so
# ``find_latest_gateway_session_for_peer`` can't recover the
# mapping on restart. Inherit them from the parent at creation
# time — but ONLY for compression forks (parent already ended
# with end_reason='compression'). Delegate/subagent children
# are spawned while the parent is still live and must NOT
# inherit routing keys, or peer recovery could repoint gateway
# traffic into a subagent's session.
conn.execute(
"""UPDATE sessions
SET user_id = COALESCE(sessions.user_id,
(SELECT p.user_id FROM sessions p
WHERE p.id = sessions.parent_session_id)),
session_key = COALESCE(sessions.session_key,
(SELECT p.session_key FROM sessions p
WHERE p.id = sessions.parent_session_id)),
chat_id = COALESCE(sessions.chat_id,
(SELECT p.chat_id FROM sessions p
WHERE p.id = sessions.parent_session_id)),
chat_type = COALESCE(sessions.chat_type,
(SELECT p.chat_type FROM sessions p
WHERE p.id = sessions.parent_session_id)),
thread_id = COALESCE(sessions.thread_id,
(SELECT p.thread_id FROM sessions p
WHERE p.id = sessions.parent_session_id)),
display_name = COALESCE(sessions.display_name,
(SELECT p.display_name FROM sessions p
WHERE p.id = sessions.parent_session_id)),
origin_json = COALESCE(sessions.origin_json,
(SELECT p.origin_json FROM sessions p
WHERE p.id = sessions.parent_session_id))
WHERE id = ? AND parent_session_id IS NOT NULL
AND EXISTS (
SELECT 1 FROM sessions p
WHERE p.id = sessions.parent_session_id
AND p.end_reason = 'compression'
)""",
(session_id,),
)
self._execute_write(_do)
def create_session(self, session_id: str, source: str, **kwargs) -> str:
+48 -2
View File
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
from agent.memory_provider import MemoryProvider
from tools.registry import tool_error
from utils import is_truthy_value
from .store import MemoryStore
from .retrieval import FactRetriever
from hermes_cli.config import cfg_get
@@ -235,7 +236,10 @@ class HolographicMemoryProvider(MemoryProvider):
return tool_error(f"Unknown tool: {tool_name}")
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
if not self._config.get("auto_extract", False):
# is_truthy_value: the config schema declares auto_extract as a string
# enum ("false"/"true"), and a plain truthiness check treats the string
# "false" as enabled (#57682).
if not is_truthy_value(self._config.get("auto_extract", False)):
return
if not self._store or not messages:
return
@@ -368,6 +372,35 @@ class HolographicMemoryProvider(MemoryProvider):
# -- Auto-extraction (on_session_end) ------------------------------------
def _auto_extract_facts(self, messages: list) -> None:
# Local import (pattern used in initialize()): the compressor module is
# heavier than this plugin and is only needed when auto_extract is on.
from agent.context_compressor import (
_MERGED_PRIOR_CONTEXT_HEADER,
_MERGED_SUMMARY_DELIMITER,
is_compaction_summary_message,
)
def _pre_delimiter_user_segment(msg: dict):
"""Return the genuine user text preceding a merged-into-tail
compaction summary, or None when the whole message is a summary.
Merge-into-tail messages (agent/context_compressor.py ~3163-3190)
wrap real prior tail content BEFORE ``_MERGED_SUMMARY_DELIMITER``,
prefixed with ``_MERGED_PRIOR_CONTEXT_HEADER``, then append the
generated handoff summary AFTER the delimiter. Dropping the whole
row (as ``is_compaction_summary_message`` alone would suggest)
discards that genuine pre-delimiter content too (#57690 review).
Only the summary suffix must be excluded from harvesting.
"""
content = msg.get("content", "")
if not isinstance(content, str) or _MERGED_SUMMARY_DELIMITER not in content:
return None
pre = content.split(_MERGED_SUMMARY_DELIMITER, 1)[0]
if pre.startswith(_MERGED_PRIOR_CONTEXT_HEADER):
pre = pre[len(_MERGED_PRIOR_CONTEXT_HEADER):]
pre = pre.strip()
return pre or None
_PREF_PATTERNS = [
re.compile(r'\bI\s+(?:prefer|like|love|use|want|need)\s+(.+)', re.IGNORECASE),
re.compile(r'\bmy\s+(?:favorite|preferred|default)\s+\w+\s+is\s+(.+)', re.IGNORECASE),
@@ -382,7 +415,20 @@ class HolographicMemoryProvider(MemoryProvider):
for msg in messages:
if msg.get("role") != "user":
continue
content = msg.get("content", "")
# Compaction handoff summaries can be inserted as role="user"
# messages; their prose reliably matches the decision patterns, so
# without this guard the compactor's own output is stored as a
# durable "fact" on every rollover (#57682). A merge-into-tail
# summary also carries genuine pre-delimiter user content in the
# SAME row; harvest that segment instead of dropping the whole
# message (#57690 review).
pre_delimiter_segment = _pre_delimiter_user_segment(msg)
if pre_delimiter_segment is not None:
content = pre_delimiter_segment
elif is_compaction_summary_message(msg):
continue
else:
content = msg.get("content", "")
if not isinstance(content, str) or len(content) < 10:
continue
+541 -23
View File
@@ -56,9 +56,9 @@ from gateway.platforms.base import (
)
try: # sibling module; support both package and flat plugin-dir import
from .block_kit import render_blocks
from .block_kit import render_blocks, sanitize_blocks
except ImportError: # pragma: no cover - plugin loaded outside package context
from block_kit import render_blocks # type: ignore
from block_kit import render_blocks, sanitize_blocks # type: ignore
logger = logging.getLogger(__name__)
@@ -113,6 +113,63 @@ def check_slack_requirements() -> bool:
return ensure_and_bind("platform.slack", _import, globals(), prompt=False)
def _collect_slack_block_mentions(blocks: list) -> list:
"""Return ``<@UID>`` mention tokens authored in non-quoted Block Kit text.
Slack's flat top-level ``text`` field does NOT contain mentions that were
authored only inside Block Kit ``blocks`` (e.g. a ``rich_text_section`` with
a ``user`` element). This walker recovers those mentions so the gates can
see Block-Kit-only mentions instead of silently dropping them (#52387).
Mentions nested inside ``rich_text_quote`` (quoted/forwarded content) are
deliberately ignored, so quoted text cannot trick the bot into responding
(matches the existing channel-routing contract).
"""
mentions: list = []
def _walk(node, in_quote: bool) -> None:
if isinstance(node, list):
for item in node:
_walk(item, in_quote)
return
if not isinstance(node, dict):
return
node_type = node.get("type")
quoted = in_quote or node_type == "rich_text_quote"
if node_type == "user" and not quoted:
uid = node.get("user_id", "")
if uid:
mentions.append(f"<@{uid}>")
for key in ("elements", "element"):
child = node.get(key)
if child is not None:
_walk(child, quoted)
try:
_walk(blocks, False)
except Exception: # pragma: no cover - defensive, never break gating
return []
return mentions
def _slack_mention_detection_text(event: dict) -> str:
"""Return the text used for @mention detection on a Slack message event.
Combines the flat top-level ``text`` with any ``<@UID>`` mentions recovered
from non-quoted Block Kit blocks (#52387), so a genuine Block-Kit-only
mention reaches the gates while quoted/forwarded mentions stay ignored.
"""
flat = event.get("text", "") or ""
blocks = event.get("blocks")
if not blocks:
return flat
mentions = _collect_slack_block_mentions(blocks)
extra = [m for m in mentions if m not in flat]
if not extra:
return flat
return (flat.strip() + "\n" + " ".join(extra)).strip()
def _extract_text_from_slack_blocks(blocks: list) -> str:
"""Extract readable text from Slack Block Kit blocks, including quoted/forwarded content.
@@ -208,6 +265,47 @@ def _extract_text_from_slack_blocks(blocks: list) -> str:
return "\n".join(parts)
def _extract_text_from_slack_attachments(attachments: list) -> str:
"""Extract readable text from legacy Slack message ``attachments``.
Apps such as Alertmanager, Grafana, PagerDuty, and CI bots post messages
with an empty top-level ``text`` and the real content inside ``attachments``
(Slack's legacy secondary-content format) or nested Block Kit ``blocks``.
Without this, such messages are invisible when the agent reads thread
history — e.g. an alert that started the very thread the agent was asked to
investigate would come through blank.
Prefers structured fields (``pretext``/``title``/``text``/``fields``) and
only falls back to an attachment's ``fallback`` string when it carries
nothing else.
"""
if not attachments:
return ""
lines: list[str] = []
for att in attachments:
if not isinstance(att, dict):
continue
got: list[str] = [
str(att[key]) for key in ("pretext", "title", "text") if att.get(key)
]
for field in att.get("fields", []) or []:
if not isinstance(field, dict):
continue
got += [str(field[k]) for k in ("title", "value") if field.get(k)]
nested = att.get("blocks")
if nested:
block_text = _extract_text_from_slack_blocks(nested)
if block_text:
got.append(block_text)
# Only use the (often duplicative) fallback when nothing structured exists.
if not got and att.get("fallback"):
got.append(str(att["fallback"]))
lines += got
return "\n".join(line for line in lines if line).strip()
def _serialize_slack_blocks_for_agent(blocks: list, max_chars: int = 6000) -> str:
"""Return a compact, redacted JSON view of the current message's Block Kit payload."""
if not blocks:
@@ -275,6 +373,47 @@ def _serialize_slack_blocks_for_agent(blocks: list, max_chars: int = 6000) -> st
return f"[Slack Block Kit payload for this message]\n```json\n{payload}\n```"
def _extract_urls_from_slack_blocks(blocks: list) -> list[str]:
"""Walk a Block Kit ``blocks`` tree and return URLs found on any element.
Returns URLs preserving discovery order with duplicates removed. Used to
surface the actionable links (``View graph``, ``View incident``, etc.)
embedded in bot-posted alerts so an agent reading the thread can fetch
or click them. The companion serializer
:func:`_serialize_slack_blocks_for_agent` deliberately strips ``url`` to
keep the JSON view compact and to avoid exposing arbitrary URLs through
the generic payload dump; this helper is the targeted opt-in for
use sites where URLs are the whole point of the message.
"""
if not blocks:
return []
found: list[str] = []
seen: set[str] = set()
def _maybe_add(value: Any) -> None:
if isinstance(value, str) and value.startswith(("http://", "https://")):
if value not in seen:
seen.add(value)
found.append(value)
def _walk(node: Any) -> None:
if isinstance(node, dict):
# The common URL-bearing keys across Block Kit (buttons, link
# elements in rich_text, image accessories, etc.).
for key in ("url", "image_url", "external_url"):
if key in node:
_maybe_add(node[key])
for value in node.values():
_walk(value)
elif isinstance(node, list):
for item in node:
_walk(item)
_walk(blocks)
return found
def _apply_slack_proxy(client: Any, proxy_url: Optional[str]) -> None:
"""Apply a resolved proxy to a Slack SDK client or clear it explicitly."""
if hasattr(client, "proxy"):
@@ -459,6 +598,9 @@ class SlackAdapter(BasePlatformAdapter):
# Track pending approval message_ts → resolved flag to prevent
# double-clicks on approval buttons.
self._approval_resolved: Dict[str, bool] = {}
# Same guard for clarify prompts (interactive multiple-choice
# buttons); mirrors _approval_resolved.
self._clarify_resolved: Dict[str, bool] = {}
# Track timestamps of messages sent by the bot so we can respond
# to thread replies even without an explicit @mention.
self._bot_message_ts: set = set()
@@ -1221,6 +1363,15 @@ class SlackAdapter(BasePlatformAdapter):
self._app.action("hermes_feedback")(self._handle_feedback_action)
# Register Block Kit action handlers for clarify buttons
# (interactive multiple-choice prompts; see tools/clarify_gateway.py).
# Choice buttons use indexed action IDs so each ID is unique within
# its actions block, as required by Slack's Block Kit schema.
self._app.action(
_re.compile(r"^hermes_clarify_choice_\d+$")
)(self._handle_clarify_action)
self._app.action("hermes_clarify_other")(self._handle_clarify_action)
# Register plugin-provided Block Kit action handlers.
#
# Plugins call ``ctx.register_slack_action_handler(action_id, cb)``
@@ -1427,6 +1578,11 @@ class SlackAdapter(BasePlatformAdapter):
# Convert standard markdown → Slack mrkdwn
formatted = self.format_message(content)
# Guard against empty/whitespace-only messages — Slack API
# returns ``no_text`` for chat.postMessage with blank text.
if not formatted or not formatted.strip():
return SendResult(success=True)
# Split long messages, preserving code block boundaries
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
@@ -1458,9 +1614,23 @@ class SlackAdapter(BasePlatformAdapter):
if broadcast and i == 0:
kwargs["reply_broadcast"] = True
last_result = await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_postMessage(**kwargs)
try:
last_result = await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_postMessage(**kwargs)
except Exception as e:
if kwargs.get("blocks") and self._is_block_payload_rejection(e):
retry_kwargs = dict(kwargs)
retry_kwargs.pop("blocks", None)
logger.info(
"[Slack] Block Kit payload rejected; retrying send without blocks: %s",
e,
)
last_result = await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_postMessage(**retry_kwargs)
else:
raise
# Clear Slack Assistant status as soon as the final message is posted.
if thread_ts:
@@ -1543,6 +1713,12 @@ class SlackAdapter(BasePlatformAdapter):
return SendResult(success=False, error="Not connected")
try:
formatted = self.format_message(content)
# Slack's chat.update has the same ~40k char limit as postMessage.
# Unlike send() we can't split into multiple messages (we're
# editing an existing one), so truncate to fit — an oversized
# payload fails the whole edit with ``msg_too_long``.
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
formatted = chunks[0] if chunks else formatted
update_kwargs: Dict[str, Any] = {
"channel": chat_id,
"ts": message_id,
@@ -1556,9 +1732,26 @@ class SlackAdapter(BasePlatformAdapter):
blocks = self._maybe_blocks(content)
if blocks:
update_kwargs["blocks"] = blocks
await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_update(**update_kwargs)
try:
await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_update(**update_kwargs)
except Exception as e:
if update_kwargs.get("blocks") and self._is_block_payload_rejection(e):
retry_kwargs = dict(update_kwargs)
# Explicitly clear any stale blocks when falling back to the
# flat text update path; otherwise Slack can preserve the
# prior block layout for an edited message.
retry_kwargs["blocks"] = []
logger.info(
"[Slack] Block Kit payload rejected; retrying edit without blocks: %s",
e,
)
await self._get_client(
chat_id, team_id=self._metadata_team_id(metadata)
).chat_update(**retry_kwargs)
else:
raise
if finalize:
await self.stop_typing(chat_id, metadata=metadata)
return SendResult(success=True, message_id=message_id)
@@ -2010,6 +2203,31 @@ class SlackAdapter(BasePlatformAdapter):
# ----- Markdown → mrkdwn conversion -----
@staticmethod
def _is_block_payload_rejection(error: BaseException) -> bool:
"""Return True for Slack errors recoverable by removing ``blocks``.
Rich Block Kit output is a progressive enhancement over the plain
``text`` fallback. If Slack rejects the structured payload as invalid
or too large, retrying the same content without blocks is safe and
prevents a formatting bug from dropping the whole response.
"""
recoverable_codes = {
"invalid_blocks",
"msg_too_long",
"too_many_blocks",
}
response = getattr(error, "response", None)
response_get = getattr(response, "get", None)
if callable(response_get):
try:
if response_get("error") in recoverable_codes:
return True
except Exception:
pass
message = str(error)
return any(code in message for code in recoverable_codes)
def _rich_blocks_enabled(self) -> bool:
"""Whether to render outbound agent messages as Slack Block Kit blocks.
@@ -2078,7 +2296,7 @@ class SlackAdapter(BasePlatformAdapter):
return None
try:
blocks = render_blocks(content, mrkdwn_fn=self.format_message)
return self._append_feedback_block(blocks)
return sanitize_blocks(self._append_feedback_block(blocks))
except Exception: # pragma: no cover - renderer already guards itself
logger.debug("[Slack] block render failed; using plain text", exc_info=True)
return None
@@ -3127,8 +3345,14 @@ class SlackAdapter(BasePlatformAdapter):
if allow_bots == "none":
return
elif allow_bots == "mentions":
text_check = event.get("text", "")
# Include Block-Kit-only mentions, not just the flat text (#52387)
text_check = _slack_mention_detection_text(event)
if self._bot_user_id and f"<@{self._bot_user_id}>" not in text_check:
logger.debug(
"[Slack] Dropping bot message under allow_bots=mentions: "
"no <@%s> mention in flat text or blocks",
self._bot_user_id,
)
return
# "all" falls through to process the message
# Always ignore our own messages to prevent echo loops
@@ -3348,7 +3572,8 @@ class SlackAdapter(BasePlatformAdapter):
# 3. The message is in a thread where the bot was previously @mentioned, OR
# 4. There's an existing session for this thread (survives restarts)
bot_uid = self._team_bot_user_ids.get(team_id, self._bot_user_id)
routing_text = original_text or ""
# Detect mentions authored only inside Block Kit blocks too (#52387)
routing_text = _slack_mention_detection_text(event) or original_text or ""
is_mentioned = bool(
(bot_uid and f"<@{bot_uid}>" in routing_text)
or self._slack_message_matches_mention_patterns(routing_text)
@@ -3874,7 +4099,7 @@ class SlackAdapter(BasePlatformAdapter):
kwargs: Dict[str, Any] = {
"channel": chat_id,
"text": f"⚠️ Command approval required: {cmd_preview[:100]}",
"blocks": blocks,
"blocks": sanitize_blocks(blocks),
}
if thread_ts:
kwargs["thread_ts"] = thread_ts
@@ -3954,7 +4179,7 @@ class SlackAdapter(BasePlatformAdapter):
kwargs: Dict[str, Any] = {
"channel": chat_id,
"text": f"{title or 'Confirm'}: {body[:100]}",
"blocks": blocks,
"blocks": sanitize_blocks(blocks),
}
if thread_ts:
kwargs["thread_ts"] = thread_ts
@@ -3969,6 +4194,105 @@ class SlackAdapter(BasePlatformAdapter):
logger.error("[Slack] send_slash_confirm failed: %s", e, exc_info=True)
return SendResult(success=False, error=str(e))
async def send_clarify(
self,
chat_id: str,
question: str,
choices: Optional[list],
clarify_id: str,
session_key: str,
metadata: Optional[Dict[str, Any]] = None,
) -> SendResult:
"""Render a clarify prompt as Block Kit interactive buttons.
Multi-choice mode (``choices`` non-empty): one button per option
(unique ``hermes_clarify_choice_<idx>`` action_id, ``value`` packs
``clarify_id|idx``) plus a final "✏️ Other…" button
(``hermes_clarify_other``). A choice click resolves the clarify
primitive directly; the "Other" button flips the entry into
text-capture mode so the gateway's platform-agnostic text-intercept
(:meth:`GatewayRunner._handle_message`) picks up the next typed
message and resolves the clarify — no Slack-specific text machinery.
Open-ended mode (``choices`` empty): delegates to the base
implementation, which renders the plain question and arms the same
text-intercept.
"""
# Open-ended prompts have no buttons — the base implementation renders
# the plain question and arms the gateway text-intercept for us.
if not choices:
return await super().send_clarify(
chat_id=chat_id,
question=question,
choices=choices,
clarify_id=clarify_id,
session_key=session_key,
metadata=metadata,
)
if not self._app:
return SendResult(success=False, error="Not connected")
try:
thread_ts = self._resolve_thread_ts(None, metadata)
# Escape the Slack mrkdwn control chars (&, <, >) so a question
# containing them renders literally instead of as markup/mentions.
# Section text caps at 3000 chars — budget the question so the
# wrapper never pushes the block over the limit (overflow →
# invalid_blocks → no buttons).
q = (question or "").replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
body = f"❓ {q}"
budget = 3000 - len("...")
if len(body) > budget:
body = body[:budget] + "..."
# One button per choice + a free-text "Other" button. Slack caps
# an actions block at 5 elements; the clarify tool caps choices at
# 4 (+ Other = 5) so this is normally one block, but chunk anyway
# so a larger choice list degrades gracefully instead of 400ing.
elements = []
for idx, choice in enumerate(choices):
label = str(choice).strip() or f"Option {idx + 1}"
elements.append({
"type": "button",
"text": {"type": "plain_text", "text": label[:75], "emoji": True},
"action_id": f"hermes_clarify_choice_{idx}",
"value": f"{clarify_id}|{idx}",
})
elements.append({
"type": "button",
"text": {"type": "plain_text", "text": "✏️ Other…", "emoji": True},
"action_id": "hermes_clarify_other",
"value": f"{clarify_id}|other",
})
blocks: list = [
{"type": "section", "text": {"type": "mrkdwn", "text": body}},
]
for start in range(0, len(elements), 5):
blocks.append({"type": "actions", "elements": elements[start:start + 5]})
kwargs: Dict[str, Any] = {
"channel": chat_id,
"text": body,
"blocks": blocks,
}
if thread_ts:
kwargs["thread_ts"] = thread_ts
result = await self._get_client(chat_id).chat_postMessage(**kwargs)
msg_ts = result.get("ts", "")
if msg_ts:
# Mark unresolved so the action handler's atomic-pop guard can
# reject double-clicks (mirrors _approval_resolved).
self._clarify_resolved[msg_ts] = False
return SendResult(success=True, message_id=msg_ts, raw_response=result)
except Exception as e:
logger.error("[Slack] send_clarify failed: %s", e, exc_info=True)
return SendResult(success=False, error=str(e))
def _is_interactive_user_authorized(
self,
user_id: str,
@@ -4098,6 +4422,11 @@ class SlackAdapter(BasePlatformAdapter):
original_text = block.get("text", {}).get("text", "")
break
# Slack re-escapes HTML entities in the interaction payload
# (< → &lt;, > → &gt;, & → &amp;), which can inflate the text
# past the 3000-char section-block limit on chat.update.
original_text = original_text[:3000]
updated_blocks = [
{
"type": "section",
@@ -4119,7 +4448,7 @@ class SlackAdapter(BasePlatformAdapter):
channel=channel_id,
ts=msg_ts,
text=decision_text,
blocks=updated_blocks,
blocks=sanitize_blocks(updated_blocks),
)
except Exception as e:
logger.warning("[Slack] Failed to update slash-confirm message: %s", e)
@@ -4265,6 +4594,11 @@ class SlackAdapter(BasePlatformAdapter):
original_text = block.get("text", {}).get("text", "")
break
# Slack re-escapes HTML entities in the interaction payload
# (< → &lt;, > → &gt;, & → &amp;), which can inflate the text
# past the 3000-char section-block limit on chat.update.
original_text = original_text[:3000]
updated_blocks = [
{
"type": "section",
@@ -4286,15 +4620,197 @@ class SlackAdapter(BasePlatformAdapter):
channel=channel_id,
ts=msg_ts,
text=decision_text,
blocks=updated_blocks,
blocks=sanitize_blocks(updated_blocks),
)
except Exception as e:
logger.warning("[Slack] Failed to update approval message: %s", e)
# (approval already resolved above; state consumed by atomic pop)
async def _update_clarify_message(
self,
channel_id: str,
msg_ts: str,
question_text: str,
decision_text: str,
) -> None:
"""Rewrite a clarify message to show the outcome and drop the buttons."""
updated_blocks = [
{
"type": "section",
"text": {"type": "mrkdwn", "text": question_text or "Clarification"},
},
{
"type": "context",
"elements": [{"type": "mrkdwn", "text": decision_text}],
},
]
try:
await self._get_client(channel_id).chat_update(
channel=channel_id,
ts=msg_ts,
text=decision_text,
blocks=updated_blocks,
)
except Exception as e:
logger.warning("[Slack] Failed to update clarify message: %s", e)
async def _handle_clarify_action(self, ack, body, action) -> None:
"""Handle a clarify button click (a choice or "Other") from Block Kit."""
await ack()
action_id = action.get("action_id", "")
value = action.get("value", "")
message = body.get("message", {})
msg_ts = message.get("ts", "")
channel_id = body.get("channel", {}).get("id", "")
user_name = body.get("user", {}).get("name", "unknown")
user_id = body.get("user", {}).get("id", "")
if not self._is_interactive_user_authorized(
user_id,
channel_id=channel_id,
user_name=user_name,
):
logger.warning(
"[Slack] Unauthorized clarify click by %s (%s) - ignoring",
user_name, user_id,
)
return
# value packs ``clarify_id|<idx|other>``.
if "|" not in value:
logger.warning("[Slack] Malformed clarify value: %s", value)
return
clarify_id, token = value.split("|", 1)
# Double-click guard — atomic pop; first caller gets False (proceed),
# any later click gets the True default and bails (mirrors approval).
if self._clarify_resolved.pop(msg_ts, True):
return
# Preserve the original question so the resolved message keeps context.
original_text = ""
for block in message.get("blocks", []):
if block.get("type") == "section":
original_text = block.get("text", {}).get("text", "")
break
from tools import clarify_gateway as _clarify_mod
# "Other" → enter text-capture mode. The gateway's text-intercept
# resolves the clarify from the user's next typed message, so there is
# no Slack-side text bookkeeping: mark_awaiting_text flips the entry and
# GatewayRunner._handle_message does the rest.
if action_id == "hermes_clarify_other" or token == "other":
if not _clarify_mod.mark_awaiting_text(clarify_id):
# Entry evicted (clarify_timeout) or gateway restarted between
# ask and tap — a typed answer would go nowhere.
await self._update_clarify_message(
channel_id, msg_ts, original_text,
f"⏳ This prompt expired — please send a new request. (by {user_name})",
)
return
await self._update_clarify_message(
channel_id, msg_ts, original_text,
f"✏️ Awaiting typed answer from {user_name}…",
)
return
# Numeric choice → resolve immediately with the chosen option text.
try:
idx = int(token)
except (ValueError, TypeError):
logger.warning("[Slack] Invalid clarify choice token: %s", token)
return
# Look up the canonical choice text from the registered entry (mirrors
# the Telegram adapter); fall back to a positional label on a race with
# timeout / session reset.
resolved_text: Optional[str] = None
try:
entry = _clarify_mod._entries.get(clarify_id) # type: ignore[attr-defined]
if entry and entry.choices and 0 <= idx < len(entry.choices):
resolved_text = str(entry.choices[idx])
except Exception:
resolved_text = None
if resolved_text is None:
resolved_text = f"choice {idx + 1}"
if _clarify_mod.resolve_gateway_clarify(clarify_id, resolved_text):
await self._update_clarify_message(
channel_id, msg_ts, original_text,
f"✅ {user_name}: {resolved_text}",
)
logger.info(
"Slack button resolved clarify (id=%s, choice=%r, user=%s)",
clarify_id, resolved_text, user_name,
)
else:
# Entry evicted / gateway restarted — surface expiry instead of a
# misleading ✓ on a button the agent will never receive.
await self._update_clarify_message(
channel_id, msg_ts, original_text,
f"⏳ This prompt expired — please send a new request. (by {user_name})",
)
logger.warning(
"[Slack] clarify resolve returned False (id=%s) — expired/reset",
clarify_id,
)
# ----- Thread context fetching -----
@staticmethod
def _render_message_text(msg: dict, bot_uid: str = "") -> str:
"""Return bounded display text for a Slack message, surfacing Block Kit content.
Starts with ``text``, strips bot mentions, then appends rich-text
content and actionable URLs from ``blocks`` when present. Unlike
:func:`_serialize_slack_blocks_for_agent` (which can emit up to
6 000 chars of JSON per message), this helper produces only the
readable text and URL list needed by thread-context and parent-
text rendering — bounded by what the blocks actually contain,
not a JSON dump.
"""
msg_text = (msg.get("text") or "").strip()
if bot_uid:
msg_text = msg_text.replace(f"<@{bot_uid}>", "").strip()
blocks = msg.get("blocks")
extras: list[str] = []
if blocks:
rich_text = _extract_text_from_slack_blocks(blocks).strip()
if rich_text and rich_text not in msg_text:
extras.append(rich_text)
for block in blocks:
block_type = (block or {}).get("type", "")
if block_type in ("section", "header", "context"):
text_obj = block.get("text") or {}
if isinstance(text_obj, dict):
section_text = (text_obj.get("text") or "").strip()
if section_text and section_text not in msg_text and all(section_text not in e for e in extras):
extras.append(section_text)
# Legacy ``attachments`` (Alertmanager, Grafana, PagerDuty, CI bots):
# apps often post with an empty ``text`` and the real content in
# attachment fields or attachment-nested blocks.
attachments_text = _extract_text_from_slack_attachments(
msg.get("attachments") or []
).strip()
if attachments_text and attachments_text not in msg_text and all(
attachments_text not in e for e in extras
):
extras.append(attachments_text)
if blocks:
urls = _extract_urls_from_slack_blocks(blocks)
new_urls = [u for u in urls if u not in msg_text and all(u not in e for e in extras)]
if new_urls:
extras.append("URLs: " + ", ".join(new_urls))
if extras:
addendum = "\n".join(extras)
msg_text = (msg_text + "\n" + addendum).strip() if msg_text else addendum
return msg_text
async def _fetch_thread_context(
self,
channel_id: str,
@@ -4397,14 +4913,10 @@ class SlackAdapter(BasePlatformAdapter):
):
continue
msg_text = msg.get("text", "").strip()
msg_text = self._render_message_text(msg, bot_uid=bot_uid)
if not msg_text:
continue
# Strip bot mentions from context messages
if bot_uid:
msg_text = msg_text.replace(f"<@{bot_uid}>", "").strip()
prefix = "[thread parent] " if is_parent else ""
display_user = msg_user or "unknown"
# Prefer the bot's own name when the message is a bot post.
@@ -4502,9 +5014,7 @@ class SlackAdapter(BasePlatformAdapter):
if parent.get("ts", "") != thread_ts:
return ""
bot_uid = self._team_bot_user_ids.get(team_id, self._bot_user_id)
text = (parent.get("text") or "").strip()
if bot_uid:
text = text.replace(f"<@{bot_uid}>", "").strip()
text = self._render_message_text(parent, bot_uid=bot_uid or "")
return text
except Exception as exc: # pragma: no cover - defensive
logger.debug("[Slack] Failed to fetch thread parent text: %s", exc)
@@ -4946,6 +5456,14 @@ async def _standalone_send(
exc_info=True,
)
if not formatted or not formatted.strip():
logger.debug("[Slack] _standalone_send: skipping empty/whitespace message")
return {
"success": True,
"platform": "slack",
"skipped": "empty_text",
}
try:
import aiohttp
except ImportError:
+168 -13
View File
@@ -146,9 +146,31 @@ def _inline_elements(text: str) -> List[Dict[str, Any]]:
# ----------------------------------------------------------------------------
def _header_block(text: str) -> Block:
def _nonempty_elements(elements: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Make a rich_text child-element list safe for Slack.
Slack rejects any ``rich_text_section`` / ``rich_text_preformatted`` /
``rich_text_quote`` whose ``elements`` list is empty or contains a ``text``
element of zero length (``invalid_blocks``: "missing element" / "must be
more than 0 characters"). Empty content is common — ragged table rows are
padded with ``""``, agents emit empty code fences around empty tool output,
blank quote lines and empty list items occur in the wild — so drop
zero-length text elements and, if nothing remains, substitute a single
space, which renders as blank yet stays schema-valid. Used by every
rich_text builder so empty content can never poison the whole payload.
"""
els = [e for e in elements if not (e.get("type") == "text" and not e.get("text"))]
return els or [{"type": "text", "text": " "}]
def _header_block(text: str) -> Optional[Block]:
# header blocks are plain_text only, 150 char cap.
clean = re.sub(r"[*_~`]", "", text).strip()
if not clean:
# Emphasis-/whitespace-only header (e.g. "# ***" or "# ") reduces to
# empty; Slack rejects an empty plain_text with invalid_blocks. Skip it
# (caller drops None) rather than poison the whole payload.
return None
if len(clean) > MAX_HEADER_TEXT:
clean = clean[: MAX_HEADER_TEXT - 1] + "…"
return {"type": "header", "text": {"type": "plain_text", "text": clean, "emoji": True}}
@@ -165,7 +187,7 @@ def _preformatted_block(text: str) -> Block:
"elements": [
{
"type": "rich_text_preformatted",
"elements": [{"type": "text", "text": text.rstrip("\n")}],
"elements": _nonempty_elements([{"type": "text", "text": text.rstrip("\n")}]),
}
],
}
@@ -179,7 +201,7 @@ def _quote_block(lines: List[str]) -> Block:
section_children.extend(_inline_elements(ln))
return {
"type": "rich_text",
"elements": [{"type": "rich_text_quote", "elements": section_children}],
"elements": [{"type": "rich_text_quote", "elements": _nonempty_elements(section_children)}],
}
@@ -205,9 +227,13 @@ def _list_block(items: List[Tuple[int, bool, str]]) -> Block:
}
elements.append(cur)
cur_key = key
assert cur is not None
if cur is None:
# Defensive: should never happen (first iteration always enters
# the ``if key != cur_key`` block above), but guard explicitly
# so ``python -O`` doesn't silently drop the check.
continue
cur["elements"].append(
{"type": "rich_text_section", "elements": _inline_elements(text)}
{"type": "rich_text_section", "elements": _nonempty_elements(_inline_elements(text))}
)
return {"type": "rich_text", "elements": elements}
@@ -252,11 +278,16 @@ def _split_row(row: str) -> List[str]:
def _rich_text_cell(text: str) -> Dict[str, Any]:
"""A ``rich_text`` table cell carrying inline-formatted content."""
"""A ``rich_text`` table cell carrying inline-formatted content.
Empty cells are common (ragged rows are padded with ``""``); Slack rejects
a cell whose section is empty or carries a zero-length text element, so the
elements are routed through ``_nonempty_elements``.
"""
return {
"type": "rich_text",
"elements": [
{"type": "rich_text_section", "elements": _inline_elements(text)}
{"type": "rich_text_section", "elements": _nonempty_elements(_inline_elements(text))}
],
}
@@ -285,18 +316,25 @@ def _table_block(rows: List[str], sep_line: str) -> Optional[Block]:
return None
aligns = _parse_alignment(sep_line)
column_settings: List[Optional[Dict[str, Any]]] = []
# Slack requires every provided ``column_settings`` entry to be an object.
# Missing trailing entries inherit defaults, so only emit settings through
# the last non-default alignment. Earlier default-left placeholders still
# need explicit valid objects to preserve positional alignment.
last_non_default = -1
for c in range(min(ncols, MAX_TABLE_COLS)):
align = aligns[c] if c < len(aligns) else "left"
# Only emit a setting when it differs from the default (left, no wrap);
# use null to skip a column, per the Slack schema.
column_settings.append({"align": align} if align != "left" else None)
if align != "left":
last_non_default = c
column_settings: List[Dict[str, Any]] = []
for c in range(last_non_default + 1):
align = aligns[c] if c < len(aligns) else "left"
column_settings.append({"align": align})
block: Block = {
"type": "table",
"rows": [[_rich_text_cell(cell) for cell in row] for row in parsed],
}
if any(cs is not None for cs in column_settings):
if column_settings:
block["column_settings"] = column_settings
return block
@@ -402,7 +440,9 @@ def render_blocks(
hm = _HEADER_RE.match(line)
if hm:
flush_para()
blocks.append(_header_block(hm.group(2)))
header = _header_block(hm.group(2))
if header is not None:
blocks.append(header)
i += 1
continue
@@ -510,3 +550,118 @@ def _split_text(text: str, limit: int) -> List[str]:
if remaining:
out.append(remaining)
return out
# ----------------------------------------------------------------------------
# Outbound payload boundary — last-resort clamp before the Slack API
# ----------------------------------------------------------------------------
def _clamp_text_obj(text_obj: Dict[str, Any], limit: int) -> Dict[str, Any]:
"""Return ``text_obj`` with its ``text`` clamped to ``limit`` chars."""
txt = text_obj.get("text") or ""
if len(txt) <= limit:
return text_obj
clamped = dict(text_obj)
clamped["text"] = txt[: limit - 1].rstrip() + "…"
return clamped
def sanitize_blocks(blocks: Optional[List[Block]]) -> Optional[List[Block]]:
"""Clamp an outbound ``blocks`` payload to Slack's hard limits.
Defensive boundary applied wherever the adapter attaches ``blocks`` to
``chat.postMessage`` / ``chat.update``. One oversized or malformed block
fails the WHOLE call with ``invalid_blocks`` — approval cards then never
update and messages silently drop — so instead of trusting every builder,
the payload is normalized just before the API call:
* ``section`` / ``context`` text objects are truncated to the 3000-char
cap with an ellipsis (Slack HTML-escapes ``< > &`` on storage, so text
echoed back through interaction payloads can exceed the limit that the
send path originally budgeted for — see #53693 / #62054).
* ``header`` text is truncated to its 150-char cap.
* Empty blocks (no text / no elements / no rows) are dropped — Slack
rejects zero-length text objects and empty element lists.
* ``table.column_settings`` entries must all be objects; ``null`` entries
(emitted by older renderers, per the "use null to skip" misreading of
the schema) are replaced with ``{}`` and default trailing entries are
trimmed (#56615).
* The payload is capped at Slack's 50-block maximum.
Returns the sanitized list, or ``None`` when nothing valid remains — the
caller then sends the plain ``text`` fallback alone. Never raises.
"""
if not blocks:
return None
try:
out: List[Block] = []
for block in blocks:
if not isinstance(block, dict) or not block.get("type"):
continue
btype = block["type"]
if btype == "section":
text_obj = block.get("text")
has_body = bool(block.get("fields")) or bool(block.get("accessory"))
if isinstance(text_obj, dict):
if not (text_obj.get("text") or "").strip() and not has_body:
continue
clamped = _clamp_text_obj(text_obj, MAX_SECTION_TEXT)
if clamped is not text_obj:
block = dict(block)
block["text"] = clamped
elif not has_body:
continue
elif btype == "header":
text_obj = block.get("text")
if not isinstance(text_obj, dict) or not (text_obj.get("text") or "").strip():
continue
clamped = _clamp_text_obj(text_obj, MAX_HEADER_TEXT)
if clamped is not text_obj:
block = dict(block)
block["text"] = clamped
elif btype == "context":
elements = block.get("elements") or []
if not elements:
continue
clamped_els = [
_clamp_text_obj(el, MAX_SECTION_TEXT)
if isinstance(el, dict) and el.get("type") in ("mrkdwn", "plain_text")
else el
for el in elements
]
if any(c is not e for c, e in zip(clamped_els, elements)):
block = dict(block)
block["elements"] = clamped_els
elif btype in ("rich_text", "actions", "context_actions"):
if not block.get("elements"):
continue
elif btype == "table":
if not block.get("rows"):
continue
settings = block.get("column_settings")
if isinstance(settings, list) and any(
not isinstance(cs, dict) for cs in settings
):
fixed = [cs if isinstance(cs, dict) else {} for cs in settings]
while fixed and not fixed[-1]:
fixed.pop()
block = dict(block)
if fixed:
block["column_settings"] = fixed
else:
block.pop("column_settings", None)
out.append(block)
if not out:
return None
return out[:MAX_BLOCKS]
except Exception:
# A sanitizer bug must never take down the send path.
return None
+12 -1
View File
@@ -6162,7 +6162,17 @@ class AIAgent:
"""
return self.api_mode != "codex_responses"
def _compress_context(self, messages: list, system_message: str, *, approx_tokens: int = None, task_id: str = "default", focus_topic: str = None, force: bool = False) -> tuple:
def _compress_context(
self,
messages: list,
system_message: str,
*,
approx_tokens: int = None,
task_id: str = "default",
focus_topic: str = None,
force: bool = False,
defer_context_engine_notification: bool = False,
) -> tuple:
"""Forwarder — see ``agent.conversation_compression.compress_context``.
``force=True`` is passed by the manual ``/compress`` slash command
@@ -6175,6 +6185,7 @@ class AIAgent:
self, messages, system_message,
approx_tokens=approx_tokens, task_id=task_id, focus_topic=focus_topic,
force=force,
defer_context_engine_notification=defer_context_engine_notification,
)
def _set_tool_guardrail_halt(self, decision: ToolGuardrailDecision) -> None:
+271
View File
@@ -0,0 +1,271 @@
#!/usr/bin/env python3
"""Standalone structural validator for plugin-catalog entry files.
Validates ``plugin-catalog/*.yaml`` catalog entries and
``plugin-catalog/removed.yaml`` against the catalog contract schema, using
only stdlib + PyYAML so the admission CI (and third-party repos) can run it
WITHOUT installing hermes-agent.
NOTE: this script intentionally duplicates the schema rules instead of
importing ``hermes_cli`` — the whole point is the no-install requirement for
cheap cross-repo CI use. The runtime twin of this schema lives in
``hermes_cli/plugin_catalog.py``; if the contract changes there, update the
rules here in lockstep.
Usage:
python3 scripts/validate_plugin_catalog.py plugin-catalog/
python3 scripts/validate_plugin_catalog.py entry.yaml removed.yaml
python3 scripts/validate_plugin_catalog.py --json plugin-catalog/
Exit codes: 0 = all files valid (warnings allowed), 1 = at least one error.
Human output is one ``<file>: ERROR: ...`` / ``<file>: warning: ...`` line
per finding; ``--json`` emits a machine-readable report on stdout instead.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover - dependency guidance only
print(
"ERROR: PyYAML is required (pip install pyyaml)",
file=sys.stderr,
)
sys.exit(2)
NAME_RE = re.compile(r"^[a-z0-9_-]{1,64}$")
SHA_RE = re.compile(r"^[0-9a-f]{40}$")
TIERS = ("official", "community")
PLATFORMS = ("linux", "macos", "windows")
CAPABILITY_KEYS = (
"provides_tools",
"provides_hooks",
"provides_middleware",
"requires_env",
)
# Top-level keys the contract knows about. Unknown keys WARN (forward
# compatibility: newer catalogs must stay valid under older validators).
KNOWN_KEYS = {
"name",
"repo",
"sha",
"subdir",
"description",
"maintainer",
"tier",
"requires_hermes",
"docs_url",
"platforms",
"capabilities",
}
REQUIRED_KEYS = ("name", "repo", "sha", "description", "maintainer")
# One comparator clause of a requires_hermes spec, e.g. ">=0.19" or "!=1.2.3".
_COMPARATOR_RE = re.compile(r"^(>=|<=|==|!=|>|<)\s*\d+(\.\d+)*$")
def _is_nonempty_str(value: object) -> bool:
return isinstance(value, str) and value.strip() != ""
def _check_requires_hermes(spec: object, errors: list[str]) -> None:
if not isinstance(spec, str):
errors.append(f"requires_hermes must be a string, got {type(spec).__name__}")
return
if spec.strip() == "":
return # empty = no constraint
for clause in spec.split(","):
if not _COMPARATOR_RE.match(clause.strip()):
errors.append(
f"requires_hermes clause {clause.strip()!r} is not a valid "
"comparator spec (expected e.g. '>=0.19')"
)
def validate_entry(data: object) -> tuple[list[str], list[str]]:
"""Validate one catalog entry document. Returns (errors, warnings)."""
errors: list[str] = []
warnings: list[str] = []
if not isinstance(data, dict):
return ["top-level document must be a YAML mapping"], warnings
for key in REQUIRED_KEYS:
if key not in data:
errors.append(f"missing required key: {key}")
for key in sorted(set(data) - KNOWN_KEYS):
warnings.append(f"unknown top-level key {key!r} (ignored by this validator)")
name = data.get("name")
if "name" in data and (not isinstance(name, str) or not NAME_RE.match(name)):
errors.append(f"name {name!r} must match [a-z0-9_-]{{1,64}}")
repo = data.get("repo")
if "repo" in data and (
not isinstance(repo, str) or not repo.startswith("https://")
):
errors.append(f"repo {repo!r} must be an https:// URL")
sha = data.get("sha")
if "sha" in data and (not isinstance(sha, str) or not SHA_RE.match(sha)):
errors.append(f"sha {sha!r} must be exactly 40 lowercase hex characters")
for key in ("description", "maintainer"):
if key in data and not _is_nonempty_str(data[key]):
errors.append(f"{key} must be a non-empty string")
tier = data.get("tier", "community")
if tier not in TIERS:
errors.append(f"tier {tier!r} must be one of {list(TIERS)}")
if "requires_hermes" in data:
_check_requires_hermes(data["requires_hermes"], errors)
platforms = data.get("platforms", [])
if platforms is None:
platforms = []
if not isinstance(platforms, list):
errors.append("platforms must be a list")
else:
bad = [p for p in platforms if p not in PLATFORMS]
if bad:
errors.append(f"platforms {bad!r} not in allowed set {list(PLATFORMS)}")
caps = data.get("capabilities", {})
if caps is None:
caps = {}
if not isinstance(caps, dict):
errors.append("capabilities must be a mapping")
else:
for key in sorted(set(caps) - set(CAPABILITY_KEYS)):
warnings.append(f"unknown capabilities key {key!r}")
for key in CAPABILITY_KEYS:
if key not in caps:
continue
value = caps[key]
if not isinstance(value, list) or not all(
isinstance(item, str) for item in value
):
errors.append(f"capabilities.{key} must be a list of strings")
return errors, warnings
def validate_removed(data: object) -> tuple[list[str], list[str]]:
"""Validate the removed.yaml document. Returns (errors, warnings)."""
errors: list[str] = []
warnings: list[str] = []
if not isinstance(data, dict):
return ["top-level document must be a YAML mapping"], warnings
removed = data.get("removed")
if removed is None:
errors.append("missing required key: removed")
return errors, warnings
if not isinstance(removed, list):
errors.append("removed must be a list")
return errors, warnings
for i, item in enumerate(removed):
if not isinstance(item, dict):
errors.append(f"removed[{i}] must be a mapping")
continue
if not _is_nonempty_str(item.get("name")):
errors.append(f"removed[{i}] missing non-empty 'name'")
for key in ("repo", "reason", "date"):
if key in item and not isinstance(item[key], str):
errors.append(f"removed[{i}].{key} must be a string")
return errors, warnings
def validate_file(path: Path) -> tuple[list[str], list[str]]:
"""Validate one YAML file (dispatching on filename). Returns (errors, warnings)."""
try:
with open(path, encoding="utf-8") as fh:
data = yaml.safe_load(fh)
except OSError as exc:
return [f"cannot read file: {exc}"], []
except yaml.YAMLError as exc:
return [f"invalid YAML: {exc}"], []
if path.name == "removed.yaml":
return validate_removed(data)
return validate_entry(data)
def collect_paths(args: list[str]) -> list[Path]:
paths: list[Path] = []
for arg in args:
p = Path(arg)
if p.is_dir():
paths.extend(sorted(p.glob("*.yaml")))
paths.extend(sorted(p.glob("*.yml")))
else:
paths.append(p)
return paths
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Standalone structural validator for plugin-catalog entry files."
)
parser.add_argument(
"paths",
nargs="+",
help="catalog entry files, removed.yaml, or a directory of them",
)
parser.add_argument(
"--json",
action="store_true",
help="emit a machine-readable JSON report on stdout",
)
opts = parser.parse_args(argv)
files = collect_paths(opts.paths)
if not files:
print("ERROR: no YAML files found", file=sys.stderr)
return 1
report = []
any_errors = False
for path in files:
errors, warnings = validate_file(path)
any_errors = any_errors or bool(errors)
report.append(
{
"path": str(path),
"ok": not errors,
"errors": errors,
"warnings": warnings,
}
)
if opts.json:
print(json.dumps({"ok": not any_errors, "files": report}, indent=2))
else:
for entry in report:
for err in entry["errors"]:
print(f"{entry['path']}: ERROR: {err}")
for warn in entry["warnings"]:
print(f"{entry['path']}: warning: {warn}")
checked = len(report)
bad = sum(1 for e in report if not e["ok"])
if any_errors:
print(f"FAIL: {bad}/{checked} file(s) invalid")
else:
print(f"OK: {checked} file(s) valid")
return 1 if any_errors else 0
if __name__ == "__main__":
sys.exit(main())
+4 -4
View File
@@ -287,7 +287,7 @@ class TestSessionOps:
"tools",
"context",
"reset",
"compact",
"compress",
"steer",
"queue",
"version",
@@ -1527,7 +1527,7 @@ class TestSlashCommands:
assert "Context usage: ~25,000 / 100,000 tokens (25.0%)" in result
assert "Compression: ~55,000 tokens until threshold (~80,000, 80%)" in result
assert "Tip: run /compact" in result
assert "Tip: run /compress" in result
def test_context_says_compression_due_when_past_threshold(self, agent, mock_manager):
state = self._make_state(mock_manager)
@@ -1544,7 +1544,7 @@ class TestSlashCommands:
result = agent._handle_slash_command("/context", state)
assert "Context usage: ~82,000 / 100,000 tokens (82.0%)" in result
assert "Compression: due now (threshold ~80,000, 80%). Run /compact." in result
assert "Compression: due now (threshold ~80,000, 80%). Run /compress." in result
def test_reset_clears_history(self, agent, mock_manager):
state = self._make_state(mock_manager)
@@ -1620,7 +1620,7 @@ class TestSlashCommands:
side_effect=[40, 12],
),
):
result = agent._handle_slash_command("/compact", state)
result = agent._handle_slash_command("/compress", state)
assert "Context compressed: 4 -> 1 messages" in result
assert "~40 -> ~12 tokens" in result
+122 -12
View File
@@ -945,7 +945,7 @@ class TestConvertMessages:
{"role": "tool", "tool_call_id": "tc_1", "content": "search results"},
]
_, result = convert_messages_to_anthropic(messages)
blocks = result[0]["content"]
blocks = next(m for m in result if m["role"] == "assistant")["content"]
assert blocks[0] == {"type": "text", "text": "Let me search."}
assert blocks[1]["type"] == "tool_use"
assert blocks[1]["id"] == "tc_1"
@@ -963,8 +963,13 @@ class TestConvertMessages:
{"role": "tool", "tool_call_id": "tc_1", "content": "result data"},
]
_, result = convert_messages_to_anthropic(messages)
# tool result is in the second message (user role)
user_msg = [m for m in result if m["role"] == "user"][0]
# tool result is in the user message following the assistant turn
user_msg = next(
m for m in result
if m["role"] == "user"
and isinstance(m["content"], list)
and any(b.get("type") == "tool_result" for b in m["content"])
)
assert user_msg["content"][0]["type"] == "tool_result"
assert user_msg["content"][0]["tool_use_id"] == "tc_1"
@@ -983,7 +988,12 @@ class TestConvertMessages:
]
_, result = convert_messages_to_anthropic(messages)
# assistant + merged user (with 2 tool_results)
user_msgs = [m for m in result if m["role"] == "user"]
user_msgs = [
m for m in result
if m["role"] == "user"
and isinstance(m["content"], list)
and any(b.get("type") == "tool_result" for b in m["content"])
]
assert len(user_msgs) == 1
assert len(user_msgs[0]["content"]) == 2
@@ -1041,7 +1051,12 @@ class TestConvertMessages:
{"role": "tool", "tool_call_id": "tc_orphan", "content": "stale result"},
]
_, result = convert_messages_to_anthropic(messages)
user_msg = [m for m in result if m["role"] == "user"][0]
user_msg = next(
m for m in result
if m["role"] == "user"
and isinstance(m["content"], list)
and any(b.get("type") == "tool_result" for b in m["content"])
)
tool_results = [
b for b in user_msg["content"] if b.get("type") == "tool_result"
]
@@ -1096,7 +1111,12 @@ class TestConvertMessages:
_, result = convert_messages_to_anthropic(messages)
asst = [m for m in result if m["role"] == "assistant"][0]
assert any(b.get("type") == "tool_use" for b in asst["content"])
user = [m for m in result if m["role"] == "user"][0]
user = next(
m for m in result
if m["role"] == "user"
and isinstance(m["content"], list)
and any(b.get("type") == "tool_result" for b in m["content"])
)
assert any(b.get("type") == "tool_result" for b in user["content"])
def test_system_with_cache_control(self):
@@ -1121,7 +1141,8 @@ class TestConvertMessages:
])
_, result = convert_messages_to_anthropic(messages)
assistant_blocks = result[0]["content"]
assistant_msg = next(m for m in result if m["role"] == "assistant")
assistant_blocks = assistant_msg["content"]
assert assistant_blocks[0]["type"] == "text"
assert assistant_blocks[0]["text"] == "Hello from assistant"
@@ -1207,7 +1228,12 @@ class TestConvertMessages:
], native_anthropic=True)
_, result = convert_messages_to_anthropic(messages)
user_msg = [m for m in result if m["role"] == "user"][0]
user_msg = next(
m for m in result
if m["role"] == "user"
and isinstance(m["content"], list)
and any(b.get("type") == "tool_result" for b in m["content"])
)
tool_block = user_msg["content"][0]
assert tool_block["type"] == "tool_result"
@@ -1356,6 +1382,90 @@ class TestConvertMessages:
assert isinstance(result[0]["content"], list)
assert result[0]["content"] == [{"type": "text", "text": "(empty message)"}]
def test_leading_assistant_after_compaction_gets_user_turn_prepended(self):
"""The adapter backstops compactors that emit a leading assistant summary."""
messages = [
{"role": "system", "content": "You are helpful."},
{"role": "assistant", "content": "[Context compaction summary] earlier work…"},
{"role": "user", "content": "continue"},
]
system, result = convert_messages_to_anthropic(messages)
assert system == "You are helpful."
assert result[0]["role"] == "user"
assert result[0]["content"] == [{"type": "text", "text": " "}]
assert result[1]["role"] == "assistant"
assert any(
m["role"] == "assistant" and "Context compaction summary" in str(m["content"])
for m in result
)
def test_double_compaction_no_system_in_messages_leads_with_user(self):
"""Exact post-double-compaction shape on the auto path (#52160).
On the auto path the system prompt is NOT inside messages[] and after
the second compaction protect_head has decayed to 0, so the
assistant-role summary is messages[0]. The converted payload must
still lead with a user turn or Anthropic 400s.
"""
messages = [
{"role": "assistant", "content": "[Context compaction summary] earlier work…"},
{"role": "user", "content": "continue"},
]
system, result = convert_messages_to_anthropic(messages)
assert system is None
assert result[0]["role"] == "user"
assert result[0]["content"] == [{"type": "text", "text": " "}]
assert result[1]["role"] == "assistant"
assert "Context compaction summary" in str(result[1]["content"])
def test_leading_user_message_is_not_modified(self):
"""A normal transcript that already starts with user must be untouched."""
messages = [
{"role": "system", "content": "sys"},
{"role": "user", "content": "hello"},
{"role": "assistant", "content": "hi"},
]
_, result = convert_messages_to_anthropic(messages)
assert len(result) == 2
assert result[0]["role"] == "user"
assert result[0]["content"] == "hello"
def test_leading_assistant_with_tool_use_after_compaction_is_repaired(self):
"""Repair the leading role without disturbing adjacent tool pairs."""
messages = [
{"role": "system", "content": "sys"},
{
"role": "assistant",
"content": "running it",
"tool_calls": [
{"id": "toolu_1", "function": {"name": "terminal", "arguments": "{}"}},
],
},
{"role": "tool", "tool_call_id": "toolu_1", "content": "ok"},
{"role": "user", "content": "next"},
]
_, result = convert_messages_to_anthropic(messages)
assert result[0]["role"] == "user"
asst_idx = next(
i for i, m in enumerate(result)
if m["role"] == "assistant"
and any(b.get("type") == "tool_use" for b in m["content"] if isinstance(b, dict))
)
nxt = result[asst_idx + 1]
assert nxt["role"] == "user"
assert any(
isinstance(b, dict) and b.get("type") == "tool_result" and b.get("tool_use_id") == "toolu_1"
for b in nxt["content"]
)
# ---------------------------------------------------------------------------
# Build kwargs
@@ -2076,7 +2186,7 @@ class TestThinkingBlockSignatureManagement:
},
]
_, result = convert_messages_to_anthropic(messages)
blocks = result[0]["content"]
blocks = next(m for m in result if m["role"] == "assistant")["content"]
thinking = [b for b in blocks if b.get("type") == "thinking"]
assert len(thinking) == 1
assert thinking[0]["signature"] == "sig_valid"
@@ -2094,7 +2204,7 @@ class TestThinkingBlockSignatureManagement:
},
]
_, result = convert_messages_to_anthropic(messages)
blocks = result[0]["content"]
blocks = next(m for m in result if m["role"] == "assistant")["content"]
# No thinking blocks should remain
assert not any(b.get("type") == "thinking" for b in blocks)
@@ -2114,7 +2224,7 @@ class TestThinkingBlockSignatureManagement:
},
]
_, result = convert_messages_to_anthropic(messages)
blocks = result[0]["content"]
blocks = next(m for m in result if m["role"] == "assistant")["content"]
redacted = [b for b in blocks if b.get("type") == "redacted_thinking"]
assert len(redacted) == 1
assert redacted[0]["data"] == "opaque_signature_data"
@@ -2208,7 +2318,7 @@ class TestThinkingBlockSignatureManagement:
_, result = convert_messages_to_anthropic(messages)
# First assistant is non-last, so thinking is stripped completely.
# The original content was empty and thinking was unsigned → placeholder
first_assistant = result[0]
first_assistant = next(m for m in result if m["role"] == "assistant")
assert first_assistant["role"] == "assistant"
assert len(first_assistant["content"]) >= 1
+94
View File
@@ -0,0 +1,94 @@
from unittest.mock import patch
from agent.context_compressor import ContextCompressor, _estimate_msg_budget_tokens
from agent.model_metadata import (
_is_cjk_token_dense_char,
estimate_messages_tokens_rough,
estimate_tokens_rough,
)
def test_cjk_text_is_not_estimated_as_four_chars_per_token():
assert estimate_tokens_rough("a" * 400) == 100
assert estimate_tokens_rough("가" * 400) >= 400
def test_message_estimate_counts_korean_content_as_token_dense():
messages = [{"role": "user", "content": "압축 테스트 " + ("가" * 1000)}]
assert estimate_messages_tokens_rough(messages) >= 1000
def test_compressor_tail_budget_uses_cjk_aware_message_estimate():
korean_msg = {"role": "assistant", "content": "가" * 2000}
english_msg = {"role": "assistant", "content": "a" * 2000}
assert _estimate_msg_budget_tokens(korean_msg) > _estimate_msg_budget_tokens(english_msg)
def test_cjk_tail_does_not_expand_to_english_char_budget():
with patch("agent.context_compressor.get_model_context_length", return_value=65536):
compressor = ContextCompressor(
"test/model",
protect_first_n=3,
protect_last_n=20,
summary_target_ratio=0.2,
quiet_mode=True,
)
messages = [
{"role": "user", "content": "head 1"},
{"role": "assistant", "content": "head 2"},
{"role": "user", "content": "head 3"},
]
for idx in range(40):
role = "assistant" if idx % 2 else "user"
messages.append({"role": role, "content": "가" * 1200})
compress_start = compressor._align_boundary_forward(
messages,
compressor._protect_head_size(messages),
)
compress_end = compressor._find_tail_cut_by_tokens(messages, compress_start)
assert len(messages) - compress_end < 31
def _reference_per_char_estimate(text: str) -> int:
"""The pre-perf-gate per-character reference implementation."""
dense = 0
sparse = 0
for ch in text:
if _is_cjk_token_dense_char(ch):
dense += 1
else:
sparse += 1
return dense + ((sparse + 3) // 4)
def test_perf_gated_estimator_matches_per_char_reference():
samples = [
"",
"ab",
"a" * 400,
"가" * 400,
"압축 테스트 " + ("가" * 1000),
"café résumé naïve", # non-ASCII, no CJK
"hello 안녕 world",
"アイウエオ テスト", # halfwidth kana (fullwidth-forms block)
"漢字とかな交じり文です。",
"русский текст", # Cyrillic — non-ASCII, non-CJK
]
for text in samples:
assert estimate_tokens_rough(text) == _reference_per_char_estimate(text), repr(text)
def test_ascii_fast_path_keeps_classic_four_chars_per_token():
# Pure ASCII must be bit-identical to the historical (len+3)//4 rule.
for text in ("x", "xyz", "a" * 1000, "tool output\n" * 500):
assert estimate_tokens_rough(text) == (len(text) + 3) // 4
def test_non_ascii_non_cjk_keeps_classic_rule():
text = "café résumé " * 40
assert estimate_tokens_rough(text) == (len(text) + 3) // 4
@@ -17,6 +17,7 @@ from unittest.mock import MagicMock, patch
import pytest
from agent.context_compressor import (
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY,
COMPRESSED_SUMMARY_METADATA_KEY,
ContextCompressor,
)
@@ -75,6 +76,7 @@ class TestMetadataFlagNeverReachesWire:
with '_'. A bare key would reach strict gateways (Fireworks etc.)
and 400 with 'Extra inputs are not permitted'."""
assert COMPRESSED_SUMMARY_METADATA_KEY.startswith("_")
assert COMPRESSED_SUMMARY_HAS_USER_TURN_KEY.startswith("_")
def test_chat_completions_transport_strips_flag(self):
from agent.transports.chat_completions import ChatCompletionsTransport
@@ -83,7 +85,11 @@ class TestMetadataFlagNeverReachesWire:
out = _compress(cc, _make_messages())
wire = ChatCompletionsTransport().convert_messages(out, model="some-model")
assert not any(
isinstance(m, dict) and COMPRESSED_SUMMARY_METADATA_KEY in m
isinstance(m, dict)
and (
COMPRESSED_SUMMARY_METADATA_KEY in m
or COMPRESSED_SUMMARY_HAS_USER_TURN_KEY in m
)
for m in wire
)
# Sanitization must not destroy the in-process flag on the originals.
@@ -94,6 +94,148 @@ def _count_children(db: SessionDB, parent_sid: str) -> int:
return len(rows)
def _wait_for_touch(touch_calls: list[str], value: str, timeout: float = 1.0) -> None:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if value in touch_calls:
return
time.sleep(0.01)
pytest.fail(f"Timed out waiting for touch activity {value!r}; calls={touch_calls!r}")
def test_compression_activity_heartbeat_touches_agent_during_long_compress(tmp_path: Path) -> None:
"""Long compression must refresh agent activity so gateway watchdogs do not fire."""
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "HEARTBEAT_TEST"
db.create_session(session_id, source="test")
agent = _build_agent_with_db(db, session_id)
agent._compression_activity_heartbeat_interval = 0.1
touch_calls: list[str] = []
agent._touch_activity = lambda desc: touch_calls.append(desc)
def _slow_compress(*_a, **_kw):
_wait_for_touch(touch_calls, "context compression in progress")
return [
{"role": "user", "content": "[CONTEXT COMPACTION] summary"},
{"role": "user", "content": "tail"},
]
agent.context_compressor.compress.side_effect = _slow_compress
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
agent._compress_context(messages, "sys", approx_tokens=120_000)
assert touch_calls[0] == "context compression started"
assert "context compression in progress" in touch_calls
assert touch_calls[-1] == "context compression completed"
assert db.get_compression_lock_holder(session_id) is None
def test_compression_activity_heartbeat_stops_on_compress_exception(tmp_path: Path) -> None:
"""Exception paths must stop the heartbeat and release the compression lock."""
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "HEARTBEAT_FAIL_TEST"
db.create_session(session_id, source="test")
agent = _build_agent_with_db(db, session_id)
agent._compression_activity_heartbeat_interval = 0.1
touch_calls: list[str] = []
agent._touch_activity = lambda desc: touch_calls.append(desc)
def _failing_compress(*_a, **_kw):
_wait_for_touch(touch_calls, "context compression in progress")
raise RuntimeError("compress boom")
agent.context_compressor.compress.side_effect = _failing_compress
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
with pytest.raises(RuntimeError, match="compress boom"):
agent._compress_context(messages, "sys", approx_tokens=120_000)
assert touch_calls[0] == "context compression started"
assert "context compression in progress" in touch_calls
assert touch_calls[-1] == "context compression failed"
assert db.get_compression_lock_holder(session_id) is None
def test_compression_activity_heartbeat_ignores_touch_errors(tmp_path: Path) -> None:
"""Activity touch failures must not affect compression success semantics."""
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "HEARTBEAT_TOUCH_ERROR_TEST"
db.create_session(session_id, source="test")
agent = _build_agent_with_db(db, session_id)
agent._compression_activity_heartbeat_interval = 0.1
agent._touch_activity = lambda _desc: (_ for _ in ()).throw(RuntimeError("touch boom"))
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
compressed, _sp = agent._compress_context(messages, "sys", approx_tokens=120_000)
assert compressed[0]["content"] == "[CONTEXT COMPACTION] summary"
assert db.get_compression_lock_holder(session_id) is None
def test_compression_activity_heartbeat_strict_signature_fallback_releases_lock(tmp_path: Path) -> None:
"""Strict compressor signatures still compress while heartbeat cleanup runs.
Main inspects the engine signature up front (_supported_compression_kwargs)
instead of catching TypeError, so a strict-signature engine is invoked
exactly once with only the kwargs it accepts. The heartbeat (with a
non-numeric configured interval falling back to the default) must still
wrap the call and stop cleanly.
"""
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "HEARTBEAT_TYPEERROR_TEST"
db.create_session(session_id, source="test")
agent = _build_agent_with_db(db, session_id)
agent._compression_activity_heartbeat_interval = "not-a-number"
touch_calls: list[str] = []
agent._touch_activity = lambda desc: touch_calls.append(desc)
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
strict_calls: list[int | None] = []
def _strict_compress(messages, current_tokens=None):
strict_calls.append(current_tokens)
return [
{"role": "user", "content": "[CONTEXT COMPACTION] strict summary"},
{"role": "user", "content": "tail"},
]
agent.context_compressor.compress = _strict_compress
compressed, _sp = agent._compress_context(messages, "sys", approx_tokens=120_000)
assert compressed[0]["content"] == "[CONTEXT COMPACTION] strict summary"
assert touch_calls[0] == "context compression started"
assert touch_calls[-1] == "context compression completed"
assert db.get_compression_lock_holder(session_id) is None
assert strict_calls == [120_000]
def test_compression_activity_heartbeat_nonfinite_interval_falls_back(tmp_path: Path) -> None:
"""Non-finite heartbeat intervals must not reach Event.wait()."""
from agent.conversation_compression import _CompressionActivityHeartbeat
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "HEARTBEAT_NONFINITE_INTERVAL_TEST"
db.create_session(session_id, source="test")
agent = _build_agent_with_db(db, session_id)
touch_calls: list[str] = []
agent._touch_activity = lambda desc: touch_calls.append(desc)
heartbeat = _CompressionActivityHeartbeat(agent, interval_seconds=float("inf"))
assert heartbeat._interval_seconds == 60.0
heartbeat.start()
heartbeat.stop()
assert touch_calls == ["context compression started", "context compression completed"]
def test_concurrent_compression_does_not_fork_session(tmp_path: Path) -> None:
"""Two AIAgents that share a session_id MUST NOT both rotate it.
@@ -0,0 +1,119 @@
"""compression.max_attempts — config-driven compression retry cap.
The conversation loop's compression retry cap was hardcoded to 3, stranding
sessions that legitimately need more rounds — e.g. a restart history reload
whose incompressible tool schemas keep the request estimate above the
threshold while the messages themselves compress fine (the #62605 failure
class). The cap is now parsed from ``compression.max_attempts`` in
``agent_init`` and read by the loop via
``getattr(agent, "max_compression_attempts", 3)``.
These tests pin the parse/validate/attach seam: default preserved, custom
value honored, floor and ceiling enforced, garbage tolerated.
"""
from __future__ import annotations
import contextlib
import io
from pathlib import Path
from hermes_state import SessionDB
from run_agent import AIAgent
def _config(max_attempts=None) -> dict:
compression = {
"enabled": True,
"threshold": 0.50,
"target_ratio": 0.20,
"protect_first_n": 3,
"protect_last_n": 20,
}
if max_attempts is not None:
compression["max_attempts"] = max_attempts
return {
"compression": compression,
"prompt_caching": {"cache_ttl": "5m"},
"sessions": {},
"bedrock": {},
}
def _make_agent(monkeypatch, tmp_path: Path, *, max_attempts=None):
from hermes_cli import config as config_mod
monkeypatch.setattr(
config_mod, "load_config", lambda: _config(max_attempts=max_attempts)
)
db = SessionDB(db_path=tmp_path / "state.db")
with contextlib.redirect_stdout(io.StringIO()):
agent = AIAgent(
base_url="https://chatgpt.com/backend-api/codex",
api_key="test-key",
provider="openai-codex",
model="gpt-5.5",
enabled_toolsets=[],
disabled_toolsets=[],
quiet_mode=True,
skip_memory=True,
session_db=db,
session_id="max-attempts-test",
)
return agent
class TestCompressionMaxAttemptsConfig:
def test_default_is_three_when_unset(self, monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path)
assert agent.max_compression_attempts == 3
def test_custom_value_is_honored(self, monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path, max_attempts=6)
assert agent.max_compression_attempts == 6
def test_hard_capped_at_ten(self, monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path, max_attempts=25)
assert agent.max_compression_attempts == 10
def test_zero_and_negative_fall_back_to_default(self, monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path, max_attempts=0)
assert agent.max_compression_attempts == 3
agent = _make_agent(monkeypatch, tmp_path, max_attempts=-2)
assert agent.max_compression_attempts == 3
def test_non_integer_falls_back_to_default(self, monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path, max_attempts="lots")
assert agent.max_compression_attempts == 3
def test_boolean_is_rejected_not_coerced(self, monkeypatch, tmp_path):
# bool subclasses int: int(True) == 1 would silently near-disable
# compression retries. YAML `max_attempts: true` must fall back to 3.
agent = _make_agent(monkeypatch, tmp_path, max_attempts=True)
assert agent.max_compression_attempts == 3
agent = _make_agent(monkeypatch, tmp_path, max_attempts=False)
assert agent.max_compression_attempts == 3
def test_fractional_float_is_rejected_not_truncated(self, monkeypatch, tmp_path):
# "4.7 attempts" is a config mistake, not a request for 4.
agent = _make_agent(monkeypatch, tmp_path, max_attempts=4.7)
assert agent.max_compression_attempts == 3
def test_integral_float_and_numeric_string_are_accepted(
self, monkeypatch, tmp_path
):
agent = _make_agent(monkeypatch, tmp_path, max_attempts=5.0)
assert agent.max_compression_attempts == 5
agent = _make_agent(monkeypatch, tmp_path, max_attempts="6")
assert agent.max_compression_attempts == 6
def test_loop_pickup_degrades_to_default_when_attribute_missing(
self, monkeypatch, tmp_path
):
# The loop reads getattr(agent, "max_compression_attempts", 3): a
# configured agent exposes its value, and an object without the
# attribute (older pickle / minimal stub) degrades to the prior
# hardcoded behavior.
agent = _make_agent(monkeypatch, tmp_path, max_attempts=7)
assert getattr(agent, "max_compression_attempts", 3) == 7
assert getattr(object(), "max_compression_attempts", 3) == 3
@@ -143,6 +143,46 @@ class TestOrphanRollbackOnCreateFailure:
_ = real_create # silence unused
class TestWorkspaceMetadataFollowsRotation:
def test_child_row_inherits_cwd_repo_and_origin_on_rotation(self, tmp_path: Path):
"""Behavioral #64709/#59527: drive the REAL compression rotation path
and assert the child session row carries the parent's workspace and
gateway-origin metadata, so the project sidebar entry and the peer
routing mapping both survive the compaction boundary."""
db = SessionDB(db_path=tmp_path / "state.db")
parent = "PARENT_CWD_ROT"
db.create_session(
parent,
source="telegram",
user_id="u1",
session_key="telegram:u1:c1",
chat_id="c1",
chat_type="private",
)
db.update_session_cwd(
parent, "/work/repo", git_branch="main", git_repo_root="/work/repo"
)
agent = _build_agent_with_db(db, parent, platform="telegram")
agent._compress_context(_msgs(), "sys", approx_tokens=120_000)
child = agent.session_id
assert child != parent # rotation happened
row = db.get_session(child)
assert row is not None
assert row["parent_session_id"] == parent
# Workspace metadata (#64709): sidebar grouping keys must survive.
assert row["cwd"] == "/work/repo"
assert row["git_repo_root"] == "/work/repo"
assert row["git_branch"] == "main"
# Gateway origin metadata (#59527): routing keys must survive even if
# the gateway never gets to re-record the peer (crash window).
assert row["session_key"] == "telegram:u1:c1"
assert row["chat_id"] == "c1"
assert row["chat_type"] == "private"
assert row["user_id"] == "u1"
class TestPlatformForwardedAtBoundary:
def test_on_session_start_receives_platform(self, tmp_path: Path):
db = SessionDB(db_path=tmp_path / "state.db")
@@ -70,8 +70,10 @@ class TestToolCallEnvelopeEstimate:
def test_non_dict_tool_calls_do_not_crash(self):
msg = {"role": "assistant", "content": "hi", "tool_calls": ["weird", None]}
# Non-dict entries are ignored (as before) without raising.
assert _estimate_msg_budget_tokens(msg) == len("hi") // _CHARS_PER_TOKEN + 10
# Non-dict entries are ignored (as before) without raising. String
# content now goes through estimate_tokens_rough (ceiling division),
# so 2 chars estimate as 1 token instead of flooring to 0.
assert _estimate_msg_budget_tokens(msg) == (len("hi") + 3) // _CHARS_PER_TOKEN + 10
@pytest.fixture()
@@ -70,12 +70,14 @@ def _make_compressor():
c.last_rough_tokens_when_real_prompt_fit = 0
c.awaiting_real_usage_after_compression = False
c._previous_summary = None
c._summary_has_user_turn = None
return c
def _simulate_cron_session_state(c):
"""Simulate per-session state that a cron compaction would leave behind."""
c._previous_summary = "Cron session summary that must not leak"
c._summary_has_user_turn = False
c._last_summary_error = "Cron session summary error"
c._last_summary_dropped_count = 5
c._last_summary_fallback_used = True
@@ -166,6 +168,7 @@ def test_on_session_end_matches_on_session_reset_surface():
per_session_attrs = [
"_previous_summary",
"_summary_has_user_turn",
"_last_summary_error",
"_last_summary_dropped_count",
"_last_summary_fallback_used",
@@ -89,3 +89,26 @@ def test_handoff_in_protected_head_populates_previous_summary_before_update():
assert compressor._previous_summary == old_summary
assert seen_turns
assert all(old_summary not in str(msg.get("content", "")) for msg in seen_turns)
def test_handoff_in_protected_head_is_replaced_not_duplicated():
"""Re-compaction must replace a protected old handoff with the updated one."""
compressor = _compressor()
old_summary = "OLD-PROTECTED-HANDOFF unique old summary body"
with patch("agent.context_compressor.call_llm", return_value=_response("UPDATED summary body")):
compressed = compressor.compress(_messages_with_handoff(old_summary))
# The summary may be emitted standalone or merged into the first tail
# message (alternation corner case), so detect it the same way the
# compressor does rather than via a startswith(SUMMARY_PREFIX) check.
summary_messages = [
msg
for msg in compressed
if isinstance(msg, dict)
and ContextCompressor._is_context_summary_content(msg.get("content"))
]
assert len(summary_messages) == 1
assert "UPDATED summary body" in str(summary_messages[0]["content"])
assert old_summary not in str(summary_messages[0]["content"])
assert old_summary not in "\n".join(str(msg.get("content") or "") for msg in compressed)
@@ -0,0 +1,376 @@
"""Regression coverage for zero-user compaction integrity (#64539)."""
import os
from types import SimpleNamespace
from unittest.mock import patch
import pytest
from agent.context_compressor import (
COMPRESSION_CONTINUATION_USER_CONTENT,
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY,
COMPRESSED_SUMMARY_METADATA_KEY,
HISTORICAL_TASK_HEADING,
SUMMARY_PREFIX,
ContextCompressor,
_NO_USER_TASK_SENTINEL,
)
from agent.conversation_compression import (
_ensure_compressed_has_user_turn,
compress_context,
)
from hermes_state import SessionDB
from tools.todo_tool import TODO_INJECTION_HEADER
def _response(content: str) -> SimpleNamespace:
return SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content=content))]
)
def _valid_zero_user_summary(label: str = "Checked artifacts.") -> str:
return f"""{HISTORICAL_TASK_HEADING}
{_NO_USER_TASK_SENTINEL}
## Goal
Historical cron work only.
## Completed Actions
1. {label}
## Resolved Questions
None. No user-authored questions exist.
## Historical Pending User Asks
None. No user-authored requests exist.
"""
def _assistant_tool_turns(start: int, count: int) -> list[dict]:
turns: list[dict] = []
for idx in range(start, start + count):
turns.extend(
[
{
"role": "assistant",
"content": "Continuing scheduled work in English.",
"tool_calls": [
{
"id": f"call-{idx}",
"function": {
"name": "terminal",
"arguments": '{"command":"pwd"}',
},
}
],
},
{
"role": "tool",
"tool_call_id": f"call-{idx}",
"content": "/workspace/project\n" + ("x" * 300),
},
]
)
return turns
def _assistant_turns(start: int, count: int) -> list[dict]:
return [
{
"role": "assistant",
"content": f"Scheduled step {idx} completed. " + ("x" * 500),
}
for idx in range(start, start + count)
]
def _lifecycle_agent(db: SessionDB, session_id: str):
with patch.dict(os.environ, {"OPENROUTER_API_KEY": "test-key"}):
from run_agent import AIAgent
agent = AIAgent(
api_key="test-key",
base_url="https://openrouter.ai/api/v1",
model="test/model",
quiet_mode=True,
session_db=db,
session_id=session_id,
skip_context_files=True,
skip_memory=True,
)
agent.compression_in_place = True
agent.context_compressor.protect_first_n = 0
agent.context_compressor.protect_last_n = 2
agent.context_compressor.tail_token_budget = 80
agent._todo_store.write(
[{"id": "inspect", "content": "Inspect artifacts", "status": "pending"}]
)
return agent
@pytest.fixture()
def compressor() -> ContextCompressor:
with patch(
"agent.context_compressor.get_model_context_length",
return_value=100_000,
):
instance = ContextCompressor(
model="test/model",
threshold_percent=0.50,
protect_first_n=0,
protect_last_n=2,
quiet_mode=True,
)
instance.tail_token_budget = 80
return instance
def test_generate_summary_rejects_fabricated_user_ask(compressor):
fabricated = f"""{HISTORICAL_TASK_HEADING}
User asked: 'Waar zijn de bestanden gedownload?'
## Goal
Vind de bestanden.
"""
with patch(
"agent.context_compressor.call_llm",
return_value=_response(fabricated),
):
result = compressor._generate_summary(_assistant_tool_turns(0, 2))
assert result is None
assert compressor._previous_summary is None
assert "invented user attribution" in compressor._last_summary_error
def test_zero_user_prompt_anchors_source_language_and_exact_sentinel(compressor):
captured_prompt = ""
def fake_call_llm(**kwargs):
nonlocal captured_prompt
captured_prompt = kwargs["messages"][0]["content"]
return _response(_valid_zero_user_summary())
with patch("agent.context_compressor.call_llm", side_effect=fake_call_llm):
result = compressor._generate_summary(_assistant_tool_turns(0, 2))
assert result == f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary().strip()}"
assert "dominant language of the source turns" in captured_prompt
assert _NO_USER_TASK_SENTINEL in captured_prompt
assert "Do not write \"User asked:\"" in captured_prompt
def test_zero_user_provenance_survives_iterative_compaction(compressor):
messages = _assistant_tool_turns(0, 12)
first_summary = f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary('First pass').strip()}"
with patch.object(compressor, "_generate_summary", return_value=first_summary):
first = compressor.compress(messages, current_tokens=90_000)
first_handoffs = [
message
for message in first
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
]
assert len(first_handoffs) == 1
assert first_handoffs[0][COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
with patch(
"agent.context_compressor.get_model_context_length",
return_value=100_000,
):
resumed = ContextCompressor(
model="test/model",
threshold_percent=0.50,
protect_first_n=0,
protect_last_n=2,
quiet_mode=True,
)
resumed.tail_token_budget = 80
# SessionDB persists the summary content/role but not arbitrary internal
# message keys. Simulate that round trip: the exact sentinel must recover
# false provenance even when both in-process metadata keys are absent.
persisted_handoff = dict(first_handoffs[0])
persisted_handoff.pop(COMPRESSED_SUMMARY_METADATA_KEY)
persisted_handoff.pop(COMPRESSED_SUMMARY_HAS_USER_TURN_KEY)
second_input = [persisted_handoff, *_assistant_tool_turns(20, 12)]
def assert_provenance_then_summarize(*_args, **_kwargs):
assert resumed._summary_has_user_turn is False
return f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary('Second pass').strip()}"
with patch.object(
resumed,
"_generate_summary",
side_effect=assert_provenance_then_summarize,
):
second = resumed.compress(second_input, current_tokens=90_000)
second_handoffs = [
message
for message in second
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
]
assert len(second_handoffs) == 1
assert second_handoffs[0][COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
def test_compress_context_todo_snapshot_stays_synthetic_across_two_boundaries(
tmp_path, monkeypatch
):
hermes_home = tmp_path / "hermes-home"
monkeypatch.setenv("HERMES_HOME", str(hermes_home))
db = SessionDB(db_path=tmp_path / "state.db")
session_id = "zero-user-todo-lifecycle"
db.create_session(session_id, source="cron", model="test/model")
first_agent = _lifecycle_agent(db, session_id)
with patch(
"agent.context_compressor.call_llm",
return_value=_response(_valid_zero_user_summary("First boundary")),
):
first, _ = compress_context(
first_agent,
_assistant_turns(0, 24),
"system",
approx_tokens=90_000,
force=True,
)
first_handoff = next(
message
for message in first
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
)
assert first_handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
assert "First boundary" in first_handoff["content"]
assert any(
message.get("role") == "user"
and str(message.get("content") or "").startswith(TODO_INJECTION_HEADER)
for message in first
)
projected = db.get_messages_as_conversation(session_id)
assert projected
assert all(
COMPRESSED_SUMMARY_METADATA_KEY not in message
and COMPRESSED_SUMMARY_HAS_USER_TURN_KEY not in message
for message in projected
)
second_agent = _lifecycle_agent(db, session_id)
with patch(
"agent.context_compressor.call_llm",
return_value=_response(_valid_zero_user_summary("Second boundary")),
):
second, _ = compress_context(
second_agent,
[*projected, *_assistant_turns(30, 24)],
"system",
approx_tokens=90_000,
force=True,
)
handoff = next(
message
for message in second
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
)
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
assert "Second boundary" in handoff["content"]
assert "User asked:" not in handoff["content"]
db.close()
def test_continuation_user_marker_is_not_reused_as_real_provenance():
todo_snapshot = f"{TODO_INJECTION_HEADER}\n- [ ] inspect. Inspect artifacts (pending)"
compressed = [{"role": "assistant", "content": "Scheduled work completed."}]
_ensure_compressed_has_user_turn(
[{"role": "user", "content": todo_snapshot}],
compressed,
)
assert compressed[-1] == {
"role": "user",
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
}
projected = [{"role": row["role"], "content": row["content"]} for row in compressed]
assert ContextCompressor._transcript_has_real_user_turn(projected) is False
def test_continuation_markers_are_not_human_anchors():
from agent.conversation_compression import _is_real_user_message
legacy = (
"Continue from the compressed conversation context above. "
"This marker exists because the compacted transcript contained "
"no preserved user turn."
)
assert not _is_real_user_message(
{"role": "user", "content": COMPRESSION_CONTINUATION_USER_CONTENT}
)
assert not _is_real_user_message({"role": "user", "content": legacy})
def test_static_fallback_does_not_attribute_synthetic_rows_to_user(compressor):
todo_snapshot = f"{TODO_INJECTION_HEADER}\n- [ ] inspect. Inspect artifacts (pending)"
fallback = compressor._build_static_fallback_summary(
[
{"role": "user", "content": todo_snapshot},
{
"role": "user",
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
},
*_assistant_tool_turns(0, 2),
]
)
assert _NO_USER_TASK_SENTINEL in fallback
assert "User asked:" not in fallback
assert "INTERNAL CONTEXT:" in fallback
def test_zero_user_deterministic_fallback_uses_same_provenance(compressor):
messages = _assistant_tool_turns(0, 12)
with patch.object(compressor, "_generate_summary", return_value=None):
result = compressor.compress(messages, current_tokens=90_000)
handoff = next(
message
for message in result
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
)
assert _NO_USER_TASK_SENTINEL in handoff["content"]
assert "User asked:" not in handoff["content"]
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
def test_real_user_turn_sets_provenance_true(compressor):
messages = [
{"role": "user", "content": "Please inspect the build artifacts."},
*_assistant_tool_turns(0, 12),
]
summary = f"{SUMMARY_PREFIX}\n{HISTORICAL_TASK_HEADING}\nUser asked: 'Please inspect the build artifacts.'"
with patch.object(compressor, "_generate_summary", return_value=summary):
result = compressor.compress(messages, current_tokens=90_000)
handoff = next(
message
for message in result
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
)
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is True
def test_session_boundaries_clear_summary_provenance(compressor):
compressor._summary_has_user_turn = False
compressor.on_session_reset()
assert compressor._summary_has_user_turn is None
compressor._summary_has_user_turn = True
compressor.on_session_end("cron-session", [])
assert compressor._summary_has_user_turn is None
+2 -2
View File
@@ -58,9 +58,9 @@ class TestEstimateTokensRough:
assert long > short
def test_unicode_multibyte(self):
"""Unicode chars are still 1 Python char each — 4 chars/token holds."""
"""CJK chars are token-dense: counted ~1 token each, not 4 chars/token."""
text = "你好世界" # 4 CJK characters
assert estimate_tokens_rough(text) == 1
assert estimate_tokens_rough(text) == 4
class TestEstimateMessagesTokensRough:
+109
View File
@@ -0,0 +1,109 @@
"""Regression test for issue #65429.
An agent built with ``skip_memory=True`` AND ``enabled_toolsets=["memory"]``
used to get a memory tool wired to ``store=None`` (the built-in
``MemoryStore`` was skipped along with the external provider), so every
memory call failed silently and the main auto-capture path was dead.
The fix creates the built-in store whenever memory is enabled in config OR the
memory toolset is explicitly enabled, while the external-provider block stays
gated on ``skip_memory``.
"""
import pytest
from run_agent import AIAgent
class _FakeOpenAI:
def __init__(self, **kw):
self.api_key = kw.get("api_key", "test")
self.base_url = kw.get("base_url", "http://test")
def close(self):
pass
def _make_agent(monkeypatch, enabled_toolsets=None, skip_memory=True):
monkeypatch.setattr("run_agent.get_tool_definitions", lambda **kw: [])
monkeypatch.setattr("run_agent.check_toolset_requirements", lambda: {})
monkeypatch.setattr("run_agent.OpenAI", _FakeOpenAI)
return AIAgent(
api_key="test-key",
base_url="http://test",
provider="openrouter",
api_mode="chat_completions",
max_iterations=1,
quiet_mode=True,
skip_context_files=True,
skip_memory=skip_memory,
enabled_toolsets=enabled_toolsets,
)
def test_skip_memory_with_memory_toolset_creates_store(monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=True)
assert agent._memory_store is not None, (
"memory toolset enabled despite skip_memory=True must still build "
"the built-in store (#65429)"
)
def test_skip_memory_without_memory_toolset_has_no_store(monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
agent = _make_agent(monkeypatch, enabled_toolsets=None, skip_memory=True)
assert agent._memory_store is None, (
"flush/background agent with skip_memory and no memory toolset must "
"have no built-in store"
)
def test_memory_toolset_without_skip_memory_creates_store(monkeypatch, tmp_path):
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=False)
assert agent._memory_store is not None
def test_skip_memory_memory_tool_handler_works_and_provider_skipped(
monkeypatch, tmp_path
):
"""End-to-end behavioral check for #65429.
The memory tool handler must actually WORK (not return the
"Memory is not available" store=None error) on a skip_memory=True agent
with the memory toolset enabled, while the external memory provider
sync/prefetch stays skipped (no MemoryManager is created).
"""
import json
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=True)
# Provider sync/prefetch must remain skipped: skip_memory still gates the
# external memory provider block.
assert agent._memory_manager is None, (
"skip_memory=True must still skip the external memory provider"
)
# Dispatch through the same entry point the tool executor uses
# (agent/tool_executor.py wires store=agent._memory_store).
from tools.memory_tool import memory_tool
raw = memory_tool(
action="add",
target="memory",
content="User prefers concise answers.",
store=agent._memory_store,
)
result = json.loads(raw)
assert result.get("success") is True, (
f"memory tool handler must work with skip_memory=True + memory "
f"toolset (#65429), got: {raw}"
)
assert "Memory is not available" not in raw
# The write must actually persist to the profile-scoped memories dir.
memory_md = tmp_path / "hm" / "memories" / "MEMORY.md"
assert memory_md.exists()
assert "User prefers concise answers." in memory_md.read_text()
@@ -95,6 +95,17 @@ class FakeClient:
# Test helpers
def queue_notification(self, method: str, **params):
# Keep legacy fixture shorthand aligned with the IDs returned by the
# fake thread/start and turn/start responses.
if params.get("threadId") in {"t", "th"}:
params["threadId"] = "thread-fake-001"
if params.get("turnId") == "tu1":
params["turnId"] = "turn-fake-001"
turn = params.get("turn")
if isinstance(turn, dict) and turn.get("id") == "tu1":
turn = dict(turn)
turn["id"] = "turn-fake-001"
params["turn"] = turn
self._notifications.append({"method": method, "params": params})
def queue_server_request(self, method: str, request_id: Any = "srv-1", **params):
@@ -196,6 +207,173 @@ class TestRunTurn:
# turn_id propagated for downstream session-DB linkage
assert r.turn_id == "turn-fake-001"
def test_subagent_completion_does_not_end_parent_turn(self):
"""A child completion must not become the parent's response."""
client = FakeClient()
client.queue_notification(
"item/completed",
threadId="thread-child-001",
turnId="turn-child-001",
item={
"type": "agentMessage",
"id": "child-message-1",
"text": "child summary",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-child-001",
turn={
"id": "turn-child-001",
"status": "completed",
"error": None,
},
)
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="turn-fake-001",
item={
"type": "agentMessage",
"id": "parent-message-1",
"text": "parent synthesis",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "turn-fake-001",
"status": "completed",
"error": None,
},
)
result = make_session(client).run_turn("delegate this", turn_timeout=2.0)
assert result.final_text == "parent synthesis"
assert result.interrupted is False
assert result.error is None
assert result.projected_messages == [
{"role": "assistant", "content": "parent synthesis"}
]
def test_stale_completion_on_parent_thread_is_ignored(self):
"""A late completion from the previous parent turn is not terminal."""
client = FakeClient()
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="turn-previous",
item={
"type": "agentMessage",
"id": "stale-message",
"text": "stale previous answer",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "turn-previous",
"status": "completed",
"error": None,
},
)
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="turn-fake-001",
item={
"type": "agentMessage",
"id": "current-message",
"text": "current parent answer",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "turn-fake-001",
"status": "completed",
"error": None,
},
)
result = make_session(client).run_turn("new prompt", turn_timeout=2.0)
assert result.final_text == "current parent answer"
assert result.projected_messages == [
{"role": "assistant", "content": "current parent answer"}
]
def test_foreign_completion_in_server_request_drain_is_ignored(self):
"""Approval draining must not project a child result into the parent."""
client = FakeClient()
client.queue_server_request(
"item/commandExecution/requestApproval",
request_id="approval-1",
command="pwd",
cwd="/tmp",
)
client.queue_notification(
"item/completed",
threadId="thread-child-001",
turnId="turn-child-001",
item={
"type": "agentMessage",
"id": "child-message",
"text": "child drain summary",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-child-001",
turn={
"id": "turn-child-001",
"status": "completed",
"error": None,
},
)
original_respond = client.respond
def respond_and_release_parent(request_id, response):
original_respond(request_id, response)
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="turn-fake-001",
item={
"type": "agentMessage",
"id": "parent-message",
"text": "parent after approval",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "turn-fake-001",
"status": "completed",
"error": None,
},
)
client.respond = respond_and_release_parent
session = make_session(
client,
request_routing=_ServerRequestRouting(auto_approve_exec=True),
)
result = session.run_turn("delegate then continue", turn_timeout=2.0)
assert client.responses == [("approval-1", {"decision": "accept"})]
assert result.final_text == "parent after approval"
assert result.projected_messages == [
{"role": "assistant", "content": "parent after approval"}
]
def test_token_usage_notification_is_captured(self):
client = FakeClient()
client.queue_notification(
@@ -528,6 +706,121 @@ class TestCompactThread:
assert r.token_usage_last["totalTokens"] == 12
assert r.model_context_window == 200000
def test_compact_thread_ignores_foreign_child_completion(self):
client = FakeClient()
client.queue_notification(
"turn/started",
threadId="thread-child-001",
turn={"id": "child-compact-turn"},
)
client.queue_notification(
"item/completed",
threadId="thread-child-001",
turnId="child-compact-turn",
item={
"type": "agentMessage",
"id": "child-compact-message",
"text": "child compact summary",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-child-001",
turn={
"id": "child-compact-turn",
"status": "completed",
"error": None,
},
)
client.queue_notification(
"turn/started",
threadId="thread-fake-001",
turn={"id": "compact-turn-1"},
)
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="compact-turn-1",
item={
"type": "agentMessage",
"id": "parent-compact-message",
"text": "parent compacted",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "compact-turn-1",
"status": "completed",
"error": None,
},
)
result = make_session(client).compact_thread(turn_timeout=2.0)
assert result.error is None
assert result.turn_id == "compact-turn-1"
assert result.final_text == "parent compacted"
assert result.projected_messages == [
{"role": "assistant", "content": "parent compacted"}
]
def test_compact_thread_ignores_stale_same_thread_completion_before_start(self):
client = FakeClient()
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="previous-compact-turn",
item={
"type": "agentMessage",
"id": "stale-compact-message",
"text": "stale compact result",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "previous-compact-turn",
"status": "completed",
"error": None,
},
)
client.queue_notification(
"turn/started",
threadId="thread-fake-001",
turn={"id": "compact-turn-1"},
)
client.queue_notification(
"item/completed",
threadId="thread-fake-001",
turnId="compact-turn-1",
item={
"type": "agentMessage",
"id": "current-compact-message",
"text": "current compact result",
},
)
client.queue_notification(
"turn/completed",
threadId="thread-fake-001",
turn={
"id": "compact-turn-1",
"status": "completed",
"error": None,
},
)
result = make_session(client).compact_thread(turn_timeout=2.0)
assert result.error is None
assert result.turn_id == "compact-turn-1"
assert result.final_text == "current compact result"
assert result.projected_messages == [
{"role": "assistant", "content": "current compact result"}
]
def test_compact_thread_interrupted_returns_non_success(self):
client = FakeClient()
client.queue_notification(
+356
View File
@@ -2102,6 +2102,273 @@ class TestResponsesEndpoint:
assert stored_history.count(first_history[0]) == 1
assert stored_history.count({"role": "user", "content": "Now add 1 more"}) == 1
@pytest.mark.asyncio
async def test_previous_response_id_stores_compressed_transcript_directly(self, adapter):
"""After compression, stored history is the compressed transcript, not prior + compressed."""
prior_history = [
{"role": "user", "content": "What is 1+1?"},
{"role": "assistant", "content": "2"},
] * 10 # 20 messages — enough to simulate a long conversation
adapter._response_store.put(
"resp_prev",
{
"response": {"id": "resp_prev", "status": "completed"},
"conversation_history": list(prior_history),
"session_id": "api-test-session",
},
)
compressed_history = [
# Compressed transcript starts with summary, NOT with prior[0]
{"role": "user", "content": "[Compressed summary of earlier conversation]"},
{"role": "user", "content": "Now add 1 more"},
{"role": "assistant", "content": "3"},
]
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
mock_run.return_value = (
{
"final_response": "3",
"messages": list(compressed_history),
"_compressed": True,
"api_calls": 1,
},
{"input_tokens": 0, "output_tokens": 0, "total_tokens": 0},
)
resp = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "Now add 1 more",
"previous_response_id": "resp_prev",
},
)
assert resp.status == 200
data = await resp.json()
stored = adapter._response_store.get(data["id"])
stored_history = stored["conversation_history"]
# Must NOT contain the original prior_history messages
for msg in prior_history:
assert msg not in stored_history, (
f"Prior history message leaked into stored compressed transcript: {msg}"
)
# Must contain the compressed transcript
assert stored_history == compressed_history
@pytest.mark.asyncio
async def test_rotation_compression_exercises_detection_and_persists_rotated_session_id(
self, adapter
):
"""Fake-agent rotation: _run_agent detects session_id change, sets
_compressed, and the Responses handler persists the rotated session_id
so subsequent previous_response_id chaining loads the compressed child."""
prior_history = [
{"role": "user", "content": "What is 1+1?"},
{"role": "assistant", "content": "2"},
] * 10
adapter._response_store.put(
"resp_prev_rot",
{
"response": {"id": "resp_prev_rot", "status": "completed"},
"conversation_history": list(prior_history),
"session_id": "api-test-session",
},
)
compressed_history = [
{"role": "user", "content": "[Compressed summary]"},
{"role": "user", "content": "Now add 1 more"},
{"role": "assistant", "content": "3"},
]
# Fake agent whose session_id was rotated by compression
mock_agent = MagicMock()
mock_agent.session_id = "rotated-child-session"
mock_agent._last_compaction_in_place = False
mock_agent.session_prompt_tokens = 0
mock_agent.session_completion_tokens = 0
mock_agent.session_total_tokens = 0
mock_agent.run_conversation.return_value = {
"final_response": "3",
"messages": list(compressed_history),
"api_calls": 1,
}
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_create_agent", return_value=mock_agent):
resp = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "Now add 1 more",
"previous_response_id": "resp_prev_rot",
},
)
assert resp.status == 200
data = await resp.json()
# The detection path in _run_agent must have set _compressed
# _run_agent mutates the result dict in place -- verify via stored data
stored = adapter._response_store.get(data["id"])
assert stored is not None
# Stored history must be the compressed transcript, not prior + compressed
stored_history = stored["conversation_history"]
for msg in prior_history:
assert msg not in stored_history
assert stored_history == compressed_history
# Stored session_id must be the rotated child, not the original
assert stored["session_id"] == "rotated-child-session"
# Response header must also reflect the rotated session
assert resp.headers.get("X-Hermes-Session-Id") == "rotated-child-session"
@pytest.mark.asyncio
async def test_inplace_compression_exercises_detection_and_persists_compressed_history(
self, adapter
):
"""Fake-agent in-place: _run_agent detects _last_compaction_in_place,
sets _compressed, and the Responses handler persists compressed history
WITHOUT rotating the session_id."""
prior_history = [
{"role": "user", "content": "What is 1+1?"},
{"role": "assistant", "content": "2"},
] * 10
adapter._response_store.put(
"resp_prev_inplace",
{
"response": {"id": "resp_prev_inplace", "status": "completed"},
"conversation_history": list(prior_history),
"session_id": "api-test-session",
},
)
compressed_history = [
{"role": "user", "content": "[Compressed summary in-place]"},
{"role": "user", "content": "Continue"},
{"role": "assistant", "content": "42"},
]
# Fake agent: in-place compaction, session_id unchanged
mock_agent = MagicMock()
mock_agent.session_id = "api-test-session" # same as input -- no rotation
mock_agent._last_compaction_in_place = True
mock_agent.session_prompt_tokens = 0
mock_agent.session_completion_tokens = 0
mock_agent.session_total_tokens = 0
mock_agent.run_conversation.return_value = {
"final_response": "42",
"messages": list(compressed_history),
"api_calls": 1,
}
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_create_agent", return_value=mock_agent):
resp = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "Continue",
"previous_response_id": "resp_prev_inplace",
},
)
assert resp.status == 200
data = await resp.json()
stored = adapter._response_store.get(data["id"])
assert stored is not None
# Stored history must be the compressed transcript
stored_history = stored["conversation_history"]
for msg in prior_history:
assert msg not in stored_history
assert stored_history == compressed_history
# Session_id must NOT change for in-place compaction
assert stored["session_id"] == "api-test-session"
assert resp.headers.get("X-Hermes-Session-Id") == "api-test-session"
@pytest.mark.asyncio
async def test_chained_rotation_propagates_effective_session_id(self, adapter):
"""Two-request chain: first request triggers rotation, second request
loads history using the rotated session_id stored by the first."""
# First request -- no previous_response_id, establishes the conversation
mock_agent_1 = MagicMock()
mock_agent_1.session_id = "child-session-after-rotation"
mock_agent_1._last_compaction_in_place = False
mock_agent_1.session_prompt_tokens = 0
mock_agent_1.session_completion_tokens = 0
mock_agent_1.session_total_tokens = 0
compressed_msg_1 = [
{"role": "user", "content": "[Summary of turn 1]"},
{"role": "assistant", "content": "Hello back"},
]
mock_agent_1.run_conversation.return_value = {
"final_response": "Hello back",
"messages": list(compressed_msg_1),
"api_calls": 1,
}
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_create_agent", return_value=mock_agent_1):
resp1 = await cli.post(
"/v1/responses",
json={"model": "hermes-agent", "input": "Hello"},
)
assert resp1.status == 200
data1 = await resp1.json()
response_id_1 = data1["id"]
# Verify the rotated session_id was persisted
stored_1 = adapter._response_store.get(response_id_1)
assert stored_1["session_id"] == "child-session-after-rotation"
assert stored_1["conversation_history"] == compressed_msg_1
# Second request -- chains via previous_response_id
# _run_agent is called with the session_id from the stored response
mock_agent_2 = MagicMock()
mock_agent_2.session_id = "child-session-after-rotation"
mock_agent_2._last_compaction_in_place = False
mock_agent_2.session_prompt_tokens = 0
mock_agent_2.session_completion_tokens = 0
mock_agent_2.session_total_tokens = 0
mock_agent_2.run_conversation.return_value = {
"final_response": "Goodbye",
"messages": [
{"role": "user", "content": "[Summary of turn 1]"},
{"role": "assistant", "content": "Hello back"},
{"role": "user", "content": "Goodbye"},
{"role": "assistant", "content": "See you!"},
],
"api_calls": 1,
}
with patch.object(adapter, "_create_agent", return_value=mock_agent_2):
resp2 = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "Goodbye",
"previous_response_id": response_id_1,
},
)
assert resp2.status == 200
# The second _run_agent call must receive the rotated session_id
# (from the stored response), not the original request session_id
call_kwargs = mock_agent_2.run_conversation.call_args.kwargs
# conversation_history loaded from store should be the compressed transcript
assert call_kwargs["conversation_history"] == compressed_msg_1
@pytest.mark.asyncio
async def test_previous_response_id_outputs_only_current_turn_items(self, adapter):
"""Response output must not replay previous tool artifacts."""
@@ -3270,6 +3537,95 @@ class TestTruncation:
call_kwargs = mock_run.call_args.kwargs
# History should be truncated to 100
assert len(call_kwargs["conversation_history"]) <= 100
assert call_kwargs["conversation_history"][0]["content"] == "msg 50"
@pytest.mark.asyncio
async def test_truncation_auto_preserves_leading_compaction_summary(self, adapter):
"""truncation=auto must not discard the compacted-history handoff."""
mock_result = {"final_response": "OK", "messages": [], "api_calls": 1}
summary = {
"role": "user",
"content": "[CONTEXT COMPACTION — REFERENCE ONLY]\nEarlier work.",
"_compressed_summary": True,
}
long_history = [summary] + [
{"role": "user", "content": f"msg {i}"}
for i in range(149)
]
adapter._response_store.put("resp_summary", {
"response": {"id": "resp_summary", "object": "response"},
"conversation_history": long_history,
"instructions": None,
})
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
mock_run.return_value = (mock_result, {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0})
resp = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "follow up",
"previous_response_id": "resp_summary",
"truncation": "auto",
},
)
assert resp.status == 200
history = mock_run.call_args.kwargs["conversation_history"]
assert len(history) == 100
assert history[0] == summary
assert history[1]["content"] == "msg 50"
assert history[-1]["content"] == "msg 148"
@pytest.mark.asyncio
async def test_truncation_auto_preserves_non_leading_compaction_summary(self, adapter):
"""A summary sitting after a retained system head must survive too.
The gateway /compress path can force a user-leading layout that
leaves the compaction summary after a kept system message, so the
preservation predicate must not assume the summary is at index 0.
"""
mock_result = {"final_response": "OK", "messages": [], "api_calls": 1}
system_head = {"role": "system", "content": "You are a helpful agent."}
summary = {
"role": "user",
"content": "[CONTEXT COMPACTION — REFERENCE ONLY]\nEarlier work.",
"_compressed_summary": True,
}
long_history = [system_head, summary] + [
{"role": "user", "content": f"msg {i}"}
for i in range(148)
]
adapter._response_store.put("resp_summary_mid", {
"response": {"id": "resp_summary_mid", "object": "response"},
"conversation_history": long_history,
"instructions": None,
})
app = _create_app(adapter)
async with TestClient(TestServer(app)) as cli:
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
mock_run.return_value = (mock_result, {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0})
resp = await cli.post(
"/v1/responses",
json={
"model": "hermes-agent",
"input": "follow up",
"previous_response_id": "resp_summary_mid",
"truncation": "auto",
},
)
assert resp.status == 200
history = mock_run.call_args.kwargs["conversation_history"]
assert len(history) == 100
assert history[0] == summary
assert history[1]["content"] == "msg 49"
assert history[-1]["content"] == "msg 147"
@pytest.mark.asyncio
async def test_no_truncation_keeps_full_history(self, adapter):
+156
View File
@@ -0,0 +1,156 @@
"""Regression tests for #52374 — raw clarify tool-call JSON must never leak
into the chat as a tool-progress bubble.
The adapter's ``send_clarify`` is the user-facing rendering of a clarify
prompt (interactive buttons, or the numbered-text fallback). The gateway's
tool-progress callback used to also render a progress bubble for the
``clarify`` tool.started event — in verbose mode that bubble contains the raw
tool-call args JSON (``{"question": ..., "choices": [...]}``), and because the
progress queue drains on a background task the JSON landed right underneath
the rendered interactive prompt on Slack.
"""
import importlib
import sys
import time
import types
import pytest
from gateway.config import Platform, PlatformConfig
from gateway.platforms.base import BasePlatformAdapter, SendResult
from gateway.session import SessionSource
class ProgressCaptureAdapter(BasePlatformAdapter):
"""Records every send so the test can assert nothing leaked."""
def __init__(self, platform=Platform.SLACK):
super().__init__(PlatformConfig(enabled=True, token="***"), platform)
self.sent = []
self.edits = []
async def connect(self, *, is_reconnect: bool = False) -> bool:
return True
async def disconnect(self) -> None:
return None
async def send(self, chat_id, content, reply_to=None, metadata=None) -> SendResult:
self.sent.append({"chat_id": chat_id, "content": content})
return SendResult(success=True, message_id="m-1")
async def edit_message(self, chat_id, message_id, content) -> SendResult:
self.edits.append({"chat_id": chat_id, "message_id": message_id, "content": content})
return SendResult(success=True, message_id=message_id)
async def send_typing(self, chat_id, metadata=None) -> None:
return None
async def stop_typing(self, chat_id) -> None:
return None
async def get_chat_info(self, chat_id: str):
return {"id": chat_id}
class ClarifyThenToolAgent:
"""Emits a clarify tool.started (with raw args) then a normal tool."""
def __init__(self, **kwargs):
self.tool_progress_callback = kwargs.get("tool_progress_callback")
self.tools = []
def run_conversation(self, message, conversation_history=None, task_id=None):
cb = self.tool_progress_callback
if cb is not None:
cb(
"tool.started",
"clarify",
"Which environment?",
{"question": "Which environment?", "choices": ["staging", "production"]},
)
time.sleep(0.35)
cb("tool.started", "terminal", "pwd", {})
time.sleep(0.35)
return {"final_response": "done", "messages": [], "api_calls": 1}
def _make_runner(adapter):
gateway_run = importlib.import_module("gateway.run")
GatewayRunner = gateway_run.GatewayRunner
runner = object.__new__(GatewayRunner)
runner.adapters = {adapter.platform: adapter}
runner._voice_mode = {}
runner._prefill_messages = []
runner._ephemeral_system_prompt = ""
runner._reasoning_config = None
runner._provider_routing = {}
runner._fallback_model = None
runner._session_db = None
runner._running_agents = {}
runner._session_run_generation = {}
runner.hooks = types.SimpleNamespace(loaded_hooks=False)
runner.config = types.SimpleNamespace(
thread_sessions_per_user=False,
group_sessions_per_user=False,
stt_enabled=False,
)
return runner
def _install_fakes(monkeypatch, mode):
monkeypatch.setenv("HERMES_TOOL_PROGRESS_MODE", mode)
fake_dotenv = types.ModuleType("dotenv")
fake_dotenv.load_dotenv = lambda *a, **k: None
monkeypatch.setitem(sys.modules, "dotenv", fake_dotenv)
fake_run_agent = types.ModuleType("run_agent")
fake_run_agent.AIAgent = ClarifyThenToolAgent
monkeypatch.setitem(sys.modules, "run_agent", fake_run_agent)
import tools.terminal_tool # noqa: F401 — register terminal emoji
gateway_run = importlib.import_module("gateway.run")
monkeypatch.setattr(gateway_run, "_resolve_runtime_agent_kwargs", lambda: {"api_key": "***"})
return gateway_run
@pytest.mark.parametrize("mode", ["verbose", "all"])
@pytest.mark.asyncio
async def test_clarify_tool_never_renders_progress_bubble(monkeypatch, tmp_path, mode):
"""No progress bubble for clarify — in any mode, especially verbose.
Verbose mode used to dump the raw args JSON
(``{"question": ..., "choices": [...]}``) into the chat right under the
interactive prompt (#52374).
"""
adapter = ProgressCaptureAdapter()
runner = _make_runner(adapter)
gateway_run = _install_fakes(monkeypatch, mode)
monkeypatch.setattr(gateway_run, "_hermes_home", tmp_path)
source = SessionSource(platform=Platform.SLACK, chat_id="C1", chat_type="dm")
result = await runner._run_agent(
message="hello",
context_prompt="",
history=[],
source=source,
session_id="sess-clarify-leak",
session_key="agent:main:slack:dm:C1",
)
assert result["final_response"] == "done"
all_content = "\n".join(
[m["content"] for m in adapter.sent] + [e["content"] for e in adapter.edits]
)
# Raw clarify args JSON must not leak anywhere.
assert '"question"' not in all_content
assert '"choices"' not in all_content
assert "Which environment?" not in all_content
# No clarify progress line at all (verb "Asking" / tool name).
assert "clarify" not in all_content
assert "Asking" not in all_content
# The unrelated terminal tool still renders progress normally.
assert "pwd" in all_content
@@ -0,0 +1,214 @@
"""Regression tests for #62034 — pending multi-choice clarify prompts must not
swallow unrelated thread follow-up messages.
When a NATIVE interactive multi-choice clarify (buttons rendered,
``awaiting_text=False``) is pending, the gateway text-intercept used to
consume ANY non-command message in the session as the clarify answer —
arbitrary prose vanished into clarify resolution and the agent appeared to
ignore the user's thread messages.
After the fix (``tools/clarify_gateway._coerce_text_response`` rejects
arbitrary prose for native multi-choice prompts):
* numeric selections ("2") and exact choice labels still resolve, and
* arbitrary prose falls through the intercept and continues as a normal
message-handling turn.
Open-ended clarifies, explicit "Other" text-capture mode, and the base
adapter's numbered-text fallback (which flips ``awaiting_text`` at send time)
keep accepting free text.
"""
from unittest.mock import patch
import pytest
from gateway.config import Platform, PlatformConfig
from gateway.platforms.base import (
BasePlatformAdapter,
MessageEvent,
MessageType,
SendResult,
)
from gateway.session import SessionSource
SESSION_KEY = "agent:main:slack:dm:D123:1111.2222"
class _StubAdapter(BasePlatformAdapter):
def __init__(self):
super().__init__(PlatformConfig(enabled=True, token="test"), Platform.SLACK)
async def connect(self, *, is_reconnect: bool = False):
return True
async def disconnect(self):
pass
async def send(self, chat_id, content, reply_to=None, metadata=None):
return SendResult(success=True, message_id="m1")
async def get_chat_info(self, chat_id):
return {"id": chat_id, "type": "im"}
class _FellThroughIntercept(Exception):
"""Sentinel: _handle_message got PAST the clarify text-intercept."""
def _event(text):
return MessageEvent(
text=text,
message_type=MessageType.TEXT,
source=SessionSource(
platform=Platform.SLACK,
chat_id="D123",
chat_type="dm",
user_id="U1",
thread_id="1111.2222",
),
message_id="msg1",
)
def _clear_clarify_state():
from tools import clarify_gateway as cm
with cm._lock:
cm._entries.clear()
cm._session_index.clear()
cm._notify_cbs.clear()
def _make_runner(adapter):
from gateway.run import GatewayRunner
runner = GatewayRunner.__new__(GatewayRunner)
runner._startup_restore_in_progress = False
runner._scale_to_zero_note_real_inbound = lambda: None
runner._is_user_authorized = lambda source: True
runner._session_key_for_source = lambda source: SESSION_KEY
runner._adapter_for_source = lambda source: adapter
runner._update_prompt_pending = {}
return runner
async def _dispatch(runner, event):
"""Run _handle_message with a tripwire installed AFTER the clarify
intercept (the slash-confirm pending lookup is the next statement), so a
raised ``_FellThroughIntercept`` proves the message was NOT swallowed."""
import tools.slash_confirm as slash_confirm_mod
def _tripwire(_key):
raise _FellThroughIntercept()
with patch("hermes_cli.plugins.invoke_hook", return_value=[]), \
patch.object(slash_confirm_mod, "get_pending", _tripwire):
return await runner._handle_message(event)
@pytest.mark.asyncio
async def test_thread_prose_not_swallowed_by_native_multi_choice_clarify():
"""Arbitrary prose during a pending button-clarify continues as a normal turn."""
_clear_clarify_state()
from tools import clarify_gateway as cm
adapter = _StubAdapter()
runner = _make_runner(adapter)
# Native interactive multi-choice prompt: awaiting_text stays False.
entry = cm.register("cl-native", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
assert entry.awaiting_text is False
with pytest.raises(_FellThroughIntercept):
await _dispatch(runner, _event("just checking the visual UI, no need to pass any data"))
# The clarify entry must still be pending and unresolved.
with cm._lock:
entry = cm._entries.get("cl-native")
assert entry is not None
assert not entry.event.is_set()
_clear_clarify_state()
@pytest.mark.asyncio
async def test_numeric_reply_still_resolves_native_multi_choice_clarify():
"""Typed "2" keeps resolving the button prompt through the same intercept."""
_clear_clarify_state()
from tools import clarify_gateway as cm
adapter = _StubAdapter()
runner = _make_runner(adapter)
cm.register("cl-num", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
result = await _dispatch(runner, _event("2"))
assert result == "" # intercepted + acknowledged silently
with cm._lock:
entry = cm._entries.get("cl-num")
assert entry is not None
assert entry.event.is_set()
assert entry.response == "dropdown"
_clear_clarify_state()
@pytest.mark.asyncio
async def test_exact_label_reply_still_resolves_native_multi_choice_clarify():
_clear_clarify_state()
from tools import clarify_gateway as cm
adapter = _StubAdapter()
runner = _make_runner(adapter)
cm.register("cl-label", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
result = await _dispatch(runner, _event("Buttons"))
assert result == ""
with cm._lock:
entry = cm._entries.get("cl-label")
assert entry is not None
assert entry.event.is_set()
assert entry.response == "buttons"
_clear_clarify_state()
@pytest.mark.asyncio
async def test_prose_still_accepted_after_other_flips_text_capture():
"""After the user taps 'Other', free text IS the answer — must resolve."""
_clear_clarify_state()
from tools import clarify_gateway as cm
adapter = _StubAdapter()
runner = _make_runner(adapter)
cm.register("cl-other", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
assert cm.mark_awaiting_text("cl-other") is True
result = await _dispatch(runner, _event("a carousel actually"))
assert result == ""
with cm._lock:
entry = cm._entries.get("cl-other")
assert entry is not None
assert entry.event.is_set()
assert entry.response == "a carousel actually"
_clear_clarify_state()
@pytest.mark.asyncio
async def test_prose_still_accepted_for_open_ended_clarify():
_clear_clarify_state()
from tools import clarify_gateway as cm
adapter = _StubAdapter()
runner = _make_runner(adapter)
cm.register("cl-open", SESSION_KEY, "What should I name it?", None)
result = await _dispatch(runner, _event("call it hermes-ux"))
assert result == ""
with cm._lock:
entry = cm._entries.get("cl-open")
assert entry is not None
assert entry.event.is_set()
assert entry.response == "call it hermes-ux"
_clear_clarify_state()
+197
View File
@@ -939,6 +939,20 @@ class TestSlackProxyBehavior:
assert adapter._handler.proxy == "http://proxy.example.com:3128"
assert adapter._handler.client.proxy == "http://proxy.example.com:3128"
assert "hermes_feedback" in created_apps[0].registered_actions
assert "hermes_clarify_other" in created_apps[0].registered_actions
clarify_choice_patterns = [
action_id
for action_id in created_apps[0].registered_actions
if hasattr(action_id, "fullmatch")
]
assert any(
pattern.fullmatch("hermes_clarify_choice_0")
for pattern in clarify_choice_patterns
)
assert not any(
pattern.fullmatch("hermes_clarify_choice")
for pattern in clarify_choice_patterns
)
@pytest.mark.asyncio
async def test_connect_clears_proxy_when_no_proxy_matches_slack(self):
@@ -2912,6 +2926,15 @@ class TestEditMessage:
kwargs = adapter._app.client.chat_update.call_args.kwargs
assert kwargs["text"] == "AT&amp;T &lt; 5 &gt; 3"
@pytest.mark.asyncio
async def test_edit_message_truncates_oversized_content(self, adapter):
"""Oversized edits are truncated instead of failing with msg_too_long."""
adapter._app.client.chat_update = AsyncMock(return_value={"ok": True})
result = await adapter.edit_message("C123", "1234.5678", "x" * 45000)
assert result.success
kwargs = adapter._app.client.chat_update.call_args.kwargs
assert len(kwargs["text"]) <= adapter.MAX_MESSAGE_LENGTH
# ---------------------------------------------------------------------------
# TestEditMessageStreamingPipeline
@@ -4071,6 +4094,69 @@ class TestMessageSplitting:
assert "<https://en.wikipedia.org/wiki/Foo_(bar)|Foo>" in kwargs["text"]
class TestEmptyTextGuard:
"""Guard against Slack ``no_text`` errors when content is empty/whitespace."""
@pytest.mark.asyncio
async def test_send_skips_empty_string(self, adapter):
"""Empty content must not call chat_postMessage."""
adapter._app.client.chat_postMessage = AsyncMock(return_value={"ts": "ts1"})
result = await adapter.send("C123", "")
assert result.success is True
adapter._app.client.chat_postMessage.assert_not_called()
@pytest.mark.asyncio
async def test_send_skips_whitespace_only(self, adapter):
"""Whitespace-only content must not call chat_postMessage."""
adapter._app.client.chat_postMessage = AsyncMock(return_value={"ts": "ts1"})
result = await adapter.send("C123", " \n\t ")
assert result.success is True
adapter._app.client.chat_postMessage.assert_not_called()
@pytest.mark.asyncio
async def test_standalone_send_skips_empty(self, monkeypatch):
"""_standalone_send returns success without HTTP call on empty text."""
from plugins.platforms.slack.adapter import _standalone_send
from types import SimpleNamespace
pconfig = SimpleNamespace(token="xoxb-test", extra={})
# Patch aiohttp so the import succeeds, but it should never be used.
mock_session = MagicMock()
mock_session.__aenter__ = AsyncMock(return_value=mock_session)
mock_session.__aexit__ = AsyncMock(return_value=False)
monkeypatch.setattr(
"plugins.platforms.slack.adapter.aiohttp",
MagicMock(ClientSession=MagicMock(return_value=mock_session)),
)
result = await _standalone_send(pconfig, "C123", "")
assert result.get("success") is True
assert result.get("skipped") == "empty_text"
mock_session.post.assert_not_called()
@pytest.mark.asyncio
async def test_standalone_send_skips_whitespace(self, monkeypatch):
"""_standalone_send returns success without HTTP call on whitespace."""
from plugins.platforms.slack.adapter import _standalone_send
from types import SimpleNamespace
pconfig = SimpleNamespace(token="xoxb-test", extra={})
mock_session = MagicMock()
mock_session.__aenter__ = AsyncMock(return_value=mock_session)
mock_session.__aexit__ = AsyncMock(return_value=False)
monkeypatch.setattr(
"plugins.platforms.slack.adapter.aiohttp",
MagicMock(ClientSession=MagicMock(return_value=mock_session)),
)
result = await _standalone_send(pconfig, "C123", " \n ")
assert result.get("success") is True
assert result.get("skipped") == "empty_text"
mock_session.post.assert_not_called()
# ---------------------------------------------------------------------------
# TestReplyBroadcast
# ---------------------------------------------------------------------------
@@ -4995,3 +5081,114 @@ class TestThreadContextUnverifiedTagging:
# Renders successfully without trust tag (exception → unknown trust).
assert "U_X: hello" in content
assert "[unverified]" not in content
# ---------------------------------------------------------------------------
# TestThreadContextAppMessages
# ---------------------------------------------------------------------------
class TestThreadContextAppMessages:
"""App-posted messages (Alertmanager, Grafana, CI bots) frequently carry
their content in ``attachments``/``blocks`` with an empty top-level
``text``. Thread-context must fall back to those so, e.g., an alert that
started the thread the bot was asked to investigate is not dropped."""
@staticmethod
def _make_replies(messages):
return AsyncMock(return_value={"messages": messages})
@pytest.mark.asyncio
async def test_attachment_only_parent_is_included(self, adapter):
"""Alertmanager-style parent: empty text, content in a legacy attachment."""
adapter._thread_context_cache.clear()
messages = [
{ # parent posted by the Alertmanager app: text="" , content in attachment
"ts": "100.0",
"bot_id": "B_ALERTMGR",
"subtype": "bot_message",
"username": "Alertmanager",
"text": "",
"attachments": [
{
"fallback": "[FIRING:1] KubeJobFailed cluster-01 "
"batch-job-123456",
"color": "danger",
}
],
},
{"ts": "101.0", "user": "U_BOB", "text": "<@U_BOT> investigate"},
]
adapter._app.client.conversations_replies = self._make_replies(messages)
with patch.object(
adapter, "_resolve_user_name",
new=AsyncMock(side_effect=lambda uid, **_: uid),
):
content = await adapter._fetch_thread_context(
channel_id="C1", thread_ts="100.0", current_ts="999.0",
)
# The alert text (previously dropped) is now present in the context.
assert "KubeJobFailed" in content
assert "batch-job-123456" in content
assert "[thread parent]" in content
@pytest.mark.asyncio
async def test_blocks_only_message_is_included(self, adapter):
"""Block Kit message with empty text falls back to block text."""
adapter._thread_context_cache.clear()
messages = [
{"ts": "100.0", "user": "U_BOB", "text": "kickoff"},
{
"ts": "101.0",
"bot_id": "B_CI",
"subtype": "bot_message",
"username": "CI",
"text": "",
"blocks": [
{
"type": "rich_text",
"elements": [
{
"type": "rich_text_section",
"elements": [
{"type": "text", "text": "deploy #42 succeeded"}
],
}
],
}
],
},
]
adapter._app.client.conversations_replies = self._make_replies(messages)
with patch.object(
adapter, "_resolve_user_name",
new=AsyncMock(side_effect=lambda uid, **_: uid),
):
content = await adapter._fetch_thread_context(
channel_id="C1", thread_ts="100.0", current_ts="999.0",
)
assert "deploy #42 succeeded" in content
@pytest.mark.asyncio
async def test_message_without_any_text_is_skipped(self, adapter):
"""A message with no text/blocks/attachments is still skipped (no crash)."""
adapter._thread_context_cache.clear()
messages = [
{"ts": "100.0", "user": "U_BOB", "text": "hello"},
{"ts": "101.0", "bot_id": "B_X", "subtype": "bot_message", "text": ""},
]
adapter._app.client.conversations_replies = self._make_replies(messages)
with patch.object(
adapter, "_resolve_user_name",
new=AsyncMock(side_effect=lambda uid, **_: uid),
):
content = await adapter._fetch_thread_context(
channel_id="C1", thread_ts="100.0", current_ts="999.0",
)
assert "hello" in content # the real message survives; empty bot msg dropped
+204 -1
View File
@@ -275,6 +275,37 @@ class TestSlackApprovalAction:
update_kwargs = mock_client.chat_update.call_args[1]
assert "Denied by alice" in update_kwargs["text"]
@pytest.mark.asyncio
async def test_truncates_inflated_original_text(self):
"""Interaction payload re-escapes HTML entities; text must be capped."""
adapter = _make_adapter()
_attach_auth_runner(adapter)
adapter._approval_resolved["1.2"] = False
# Simulate Slack re-escaping: original was ~2990 chars, but & → &amp;
# etc. inflates it past 3000.
inflated_text = "a" * 2990 + "&amp;" * 10 # 2990 + 50 = 3040 chars
ack = AsyncMock()
body = {
"message": {"ts": "1.2", "blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": inflated_text}},
]},
"channel": {"id": "C1"},
"user": {"name": "alice", "id": "U_ALICE"},
}
action = {"action_id": "hermes_approve_once", "value": "session-key"}
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
with patch("tools.approval.resolve_gateway_approval", return_value=1):
await adapter._handle_approval_action(ack, body, action)
update_kwargs = mock_client.chat_update.call_args[1]
section_text = update_kwargs["blocks"][0]["text"]["text"]
assert len(section_text) <= 3000
@pytest.mark.asyncio
async def test_global_allowlist_blocks_unauthorized_click(self, monkeypatch):
adapter = _make_adapter()
@@ -410,6 +441,40 @@ class TestSlackSlashConfirmAction:
secondary_client.chat_postMessage.assert_awaited_once()
adapter._team_clients["T1"].chat_update.assert_not_called()
@pytest.mark.asyncio
async def test_truncates_inflated_original_text(self):
"""Interaction payload re-escapes HTML entities; text must be capped."""
adapter = _make_adapter()
_attach_auth_runner(adapter)
adapter._approval_resolved["2222.3333"] = False
# Simulate Slack re-escaping inflating text past 3000 chars.
inflated_text = "b" * 2990 + "&lt;" * 10 # 2990 + 40 = 3030 chars
ack = AsyncMock()
body = {
"message": {"ts": "2222.3333", "blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": inflated_text}},
]},
"channel": {"id": "C1"},
"user": {"name": "owner", "id": "U_OWNER"},
}
action = {
"action_id": "hermes_confirm_once",
"value": "agent:main:slack:group:C1:1111|confirm-1",
}
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
mock_client.chat_postMessage = AsyncMock()
with patch("tools.slash_confirm.resolve", new=AsyncMock(return_value="ok")):
await adapter._handle_slash_confirm_action(ack, body, action)
update_kwargs = mock_client.chat_update.call_args[1]
section_text = update_kwargs["blocks"][0]["text"]["text"]
assert len(section_text) <= 3000
# ===========================================================================
# _fetch_thread_context
@@ -547,7 +612,145 @@ class TestSlackThreadContext:
assert "メール要約: 本日の新着3件" in context
@pytest.mark.asyncio
async def test_fetch_thread_context_excludes_self_bot_replies(self):
async def test_fetch_thread_context_extracts_block_kit_parent(self):
"""Bot-posted parents that put their content in ``blocks`` (Honeycomb,
PagerDuty, Datadog, GitHub bot, etc.) used to be reduced to just the
``text`` field — typically only the alert title — which dropped the
URL/button payload that makes the alert useful to an agent replying
in the thread. The fetched context must now include bounded display
text and actionable URLs so section text and button URLs survive."""
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.conversations_replies = AsyncMock(return_value={
"messages": [
# Bot-posted alert: title in `text`, URL only in `blocks`.
# Mirrors what Honeycomb, PagerDuty, etc. actually send.
{
"ts": "1000.0",
"bot_id": "B_ALERT",
"subtype": "bot_message",
"username": "alertbot",
"text": "low_alerts (checkout)",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Trigger fired:* low_alerts",
},
},
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {"type": "plain_text", "text": "View graph"},
"url": "https://example.example/view/abc123",
},
],
},
],
},
# User reply that triggered the fetch.
{"ts": "1000.1", "user": "U1", "text": "what's going on?"},
]
})
adapter._user_name_cache = {"U1": "Alice"}
context = await adapter._fetch_thread_context(
channel_id="C1",
thread_ts="1000.0",
current_ts="1000.1",
team_id="T1",
)
# Title still present.
assert "low_alerts (checkout)" in context
# URL from the action button must now surface.
assert "https://example.example/view/abc123" in context
# Marked as the thread parent.
assert "[thread parent]" in context
@pytest.mark.asyncio
async def test_fetch_thread_context_includes_blocks_only_parent(self):
"""A parent message with empty ``text`` but non-empty ``blocks`` must
still be included — without this, alerts that put *everything* in
``blocks`` (some webhook integrations do this) are silently dropped
because the ``if not msg_text: continue`` guard fires."""
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.conversations_replies = AsyncMock(return_value={
"messages": [
{
"ts": "1000.0",
"bot_id": "B_ALERT",
"subtype": "bot_message",
"username": "alertbot",
"text": "",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Build failed: <https://example.example/build/9|#9>",
},
},
],
},
{"ts": "1000.1", "user": "U1", "text": "looking"},
]
})
adapter._user_name_cache = {"U1": "Alice"}
context = await adapter._fetch_thread_context(
channel_id="C1",
thread_ts="1000.0",
current_ts="1000.1",
team_id="T1",
)
assert "[thread parent]" in context
assert "https://example.example/build/9" in context
@pytest.mark.asyncio
async def test_fetch_thread_parent_text_surfaces_block_urls(self):
"""Cold-cache _fetch_thread_parent_text must use the same renderer as
_fetch_thread_context so a bot-posted parent with a URL only in
``blocks`` surfaces it in reply_to_text, not just in thread context."""
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.conversations_replies = AsyncMock(return_value={
"messages": [
{
"ts": "1000.0",
"bot_id": "B_ALERT",
"subtype": "bot_message",
"username": "alertbot",
"text": "Incident triggered",
"blocks": [
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {"type": "plain_text", "text": "View incident"},
"url": "https://example.example/incident/42",
},
],
},
],
},
]
})
text = await adapter._fetch_thread_parent_text(
channel_id="C1",
thread_ts="1000.0",
team_id="T1",
)
assert "Incident triggered" in text
assert "https://example.example/incident/42" in text
"""Parent (non-self bot) is kept, self-bot child replies are dropped,
user replies are kept."""
adapter = _make_adapter()
+203 -2
View File
@@ -5,6 +5,7 @@ from plugins.platforms.slack.block_kit import (
MAX_HEADER_TEXT,
MAX_SECTION_TEXT,
render_blocks,
sanitize_blocks,
)
@@ -159,11 +160,34 @@ class TestTables:
)
blocks = render_blocks(md)
cs = blocks[0]["column_settings"]
# left is default -> null; center/right emitted
assert cs[0] is None
# Every provided entry must be a valid Slack column-settings object.
# Left placeholders are explicit only when needed to preserve position.
assert cs[0] == {"align": "left"}
assert cs[1] == {"align": "center"}
assert cs[2] == {"align": "right"}
def test_default_trailing_column_settings_are_omitted(self):
md = (
"| L | R | L2 |\n"
"|---|---:|---|\n"
"| 1 | 2 | 3 |"
)
blocks = render_blocks(md)
assert blocks is not None
cs = blocks[0]["column_settings"]
assert cs == [{"align": "left"}, {"align": "right"}]
assert all(isinstance(item, dict) for item in cs)
def test_all_default_table_omits_column_settings(self):
md = (
"| A | B |\n"
"|---|---|\n"
"| 1 | 2 |"
)
blocks = render_blocks(md)
assert blocks is not None
assert "column_settings" not in blocks[0]
def test_inline_formatting_inside_cells(self):
md = (
"| Item | Link |\n"
@@ -228,3 +252,180 @@ class TestLimits:
for junk in ["```unterminated\ncode", "| broken | table", "> ", "#" * 10]:
# must not raise; either blocks or None
render_blocks(junk)
class TestEmptyContentGuards:
"""Empty content must never produce a Slack-rejected (invalid_blocks) payload.
Slack rejects a rich_text_section / rich_text_preformatted /
rich_text_quote whose ``elements`` is empty or contains a zero-length
``text`` element, and a ``header`` whose plain_text is empty. Each guard
below corresponds to a real chat.postMessage rejection observed in
production ("missing element" / "must be more than 0 characters").
"""
@staticmethod
def _assert_schema_valid(blocks):
def walk(o):
if isinstance(o, dict):
if o.get("type") in (
"rich_text_section", "rich_text_preformatted", "rich_text_quote"
):
assert o.get("elements"), f"empty {o['type']} elements"
if o.get("type") == "text":
assert len(o.get("text", "")) > 0, "zero-length text element"
if o.get("type") == "header":
assert o["text"]["text"], "empty plain_text header"
for v in o.values():
walk(v)
elif isinstance(o, list):
for v in o:
walk(v)
walk(blocks)
def test_ragged_and_empty_table_cells_are_schema_valid(self):
# Blank middle cell + ragged short row (padded with "") must not emit
# an empty section or a 0-char text element.
md = (
"| x | y | z |\n"
"| --- | --- | --- |\n"
"| 1 | | 3 |\n" # blank middle cell
"| 4 |" # ragged row -> padded with empty cells
)
blocks = render_blocks(md)
assert blocks[0]["type"] == "table"
self._assert_schema_valid(blocks)
def test_empty_code_fence_quote_and_list_item_are_schema_valid(self):
# Empty fenced code block (common around empty tool output), blank
# quote line, and empty list item must all stay schema-valid.
md = "```\n```\n\n> \n\n- \n- real item"
blocks = render_blocks(md)
assert blocks is not None
self._assert_schema_valid(blocks)
def test_multiline_quote_preserves_newline_separators(self):
# _quote_block separates lines with length-1 "\n" text elements; the
# guard must KEEP them so a multi-line blockquote stays multi-line.
blocks = render_blocks("> alpha\n> bravo")
quote = None
for b in blocks:
for el in b.get("elements", []):
if isinstance(el, dict) and el.get("type") == "rich_text_quote":
quote = el
assert quote is not None, "no rich_text_quote produced"
texts = [e.get("text") for e in quote["elements"] if e.get("type") == "text"]
assert "\n" in texts, "newline separator dropped from multi-line quote"
assert any("alpha" in (t or "") for t in texts)
assert any("bravo" in (t or "") for t in texts)
def test_emphasis_only_header_is_dropped_not_empty(self):
# "# ***" reduces to "" after marker-strip; an empty plain_text header
# is rejected by Slack, so the header is skipped entirely.
blocks = render_blocks("# ***\n\nreal body")
assert not any(b.get("type") == "header" for b in blocks)
self._assert_schema_valid(blocks)
def test_normal_content_unaffected(self):
# Guard must not alter well-formed content.
md = "# Title\n\n| a | b |\n| --- | --- |\n| 1 | 2 |\n\n> quoted\n\n- item"
blocks = render_blocks(md)
assert any(b.get("type") == "header" for b in blocks)
assert any(b.get("type") == "table" for b in blocks)
self._assert_schema_valid(blocks)
class TestSanitizeBlocks:
"""Outbound boundary clamp: one bad block must never fail the whole call.
Regression coverage for the invalid_blocks / msg_too_long bug class
(#56615 null column_settings, #62054 / #53693 >3000-char sections on
approval chat.update after HTML-escaping inflation).
"""
def test_none_and_empty_return_none(self):
assert sanitize_blocks(None) is None
assert sanitize_blocks([]) is None
def test_valid_payload_passes_through(self):
blocks = render_blocks("# Title\n\nbody text\n\n- item")
assert sanitize_blocks(blocks) == blocks
def test_oversized_section_text_is_clamped(self):
blocks = [
{"type": "section", "text": {"type": "mrkdwn", "text": "x" * 3500}},
]
out = sanitize_blocks(blocks)
assert len(out[0]["text"]["text"]) <= MAX_SECTION_TEXT
assert out[0]["text"]["text"].endswith("…")
def test_html_escape_inflated_approval_update_is_clamped(self):
# #53693 / #62054: send path budgeted the RAW text to <=3000, but the
# interaction payload echoes it back HTML-escaped (& -> &amp;) so the
# chat.update section exceeds the cap.
inflated = "a" * 2990 + "&amp;" * 10 # 3040 chars
blocks = [
{"type": "section", "text": {"type": "mrkdwn", "text": inflated}},
{"type": "context", "elements": [{"type": "mrkdwn", "text": "✅ ok"}]},
]
out = sanitize_blocks(blocks)
assert len(out[0]["text"]["text"]) <= MAX_SECTION_TEXT
# context block untouched
assert out[1] == blocks[1]
def test_null_column_settings_entries_are_fixed(self):
# #56615: Slack rejects null entries in table column_settings.
table = {
"type": "table",
"rows": [[{"type": "rich_text", "elements": []}]],
"column_settings": [None, {"align": "center"}, None],
}
out = sanitize_blocks([table])
cs = out[0]["column_settings"]
assert cs == [{}, {"align": "center"}]
assert all(isinstance(c, dict) for c in cs)
def test_all_null_column_settings_are_dropped(self):
table = {
"type": "table",
"rows": [[{"type": "rich_text", "elements": []}]],
"column_settings": [None, None],
}
out = sanitize_blocks([table])
assert "column_settings" not in out[0]
def test_empty_blocks_are_dropped(self):
blocks = [
{"type": "section", "text": {"type": "mrkdwn", "text": " "}},
{"type": "rich_text", "elements": []},
{"type": "actions", "elements": []},
{"type": "context", "elements": []},
{"type": "header", "text": {"type": "plain_text", "text": ""}},
{"type": "table", "rows": []},
{"type": "section", "text": {"type": "mrkdwn", "text": "keep me"}},
]
out = sanitize_blocks(blocks)
assert out == [blocks[-1]]
def test_all_invalid_returns_none_for_plain_text_fallback(self):
blocks = [{"type": "section", "text": {"type": "mrkdwn", "text": ""}}]
assert sanitize_blocks(blocks) is None
def test_oversized_header_is_clamped(self):
blocks = [
{"type": "header", "text": {"type": "plain_text", "text": "h" * 200}},
]
out = sanitize_blocks(blocks)
assert len(out[0]["text"]["text"]) <= MAX_HEADER_TEXT
def test_payload_capped_at_50_blocks(self):
blocks = [
{"type": "section", "text": {"type": "mrkdwn", "text": f"b{i}"}}
for i in range(60)
]
out = sanitize_blocks(blocks)
assert len(out) == MAX_BLOCKS
def test_never_raises_on_garbage(self):
assert sanitize_blocks([{"no_type": True}, "not-a-dict", 42]) is None
+62 -1
View File
@@ -7,7 +7,7 @@ Verifies the opt-in behaviour contract:
* multi-chunk (>39k) messages fall back to plain text
"""
from unittest.mock import AsyncMock, MagicMock
from unittest.mock import AsyncMock, MagicMock, call
import pytest
@@ -29,6 +29,17 @@ def _make_adapter(extra=None):
RICH_MD = "# Title\n\n- a\n - nested\n\n---\n\nbody text"
RICH_TABLE_MD = (
"| Item | Status | Note |\n"
"|---|---:|---|\n"
"| Hermes | ok | table |"
)
class SlackRejectedBlocks(Exception):
def __init__(self, error="invalid_blocks"):
super().__init__(f"Slack API rejected blocks: {error}")
self.response = {"error": error}
class TestSendMessageBlocks:
@@ -98,6 +109,29 @@ class TestSendMessageBlocks:
assert "blocks" not in client.chat_postMessage.await_args.kwargs
@pytest.mark.asyncio
async def test_block_rejection_retries_send_without_blocks_using_workspace_client(self):
adapter, client = _make_adapter({"rich_blocks": True})
client.chat_postMessage = AsyncMock(
side_effect=[SlackRejectedBlocks("invalid_blocks"), {"ts": "111.333"}]
)
result = await adapter.send(
"C1", RICH_TABLE_MD, metadata={"team_id": "T_SECONDARY"}
)
assert result.success is True
assert adapter._get_client.call_args_list == [
call("C1", team_id="T_SECONDARY"),
call("C1", team_id="T_SECONDARY"),
]
assert client.chat_postMessage.await_count == 2
first = client.chat_postMessage.await_args_list[0].kwargs
second = client.chat_postMessage.await_args_list[1].kwargs
assert "blocks" in first and first["blocks"]
assert "blocks" not in second
assert second["text"]
class TestEditMessageBlocks:
@pytest.mark.asyncio
@@ -128,3 +162,30 @@ class TestEditMessageBlocks:
adapter, client = _make_adapter() # rich_blocks off
await adapter.edit_message("C1", "111.222", RICH_MD, finalize=True)
assert "blocks" not in client.chat_update.await_args.kwargs
@pytest.mark.asyncio
async def test_block_rejection_retries_edit_without_blocks_using_workspace_client(self):
adapter, client = _make_adapter({"rich_blocks": True})
client.chat_update = AsyncMock(
side_effect=[SlackRejectedBlocks("invalid_blocks"), {"ts": "111.222"}]
)
result = await adapter.edit_message(
"C1",
"111.222",
RICH_TABLE_MD,
finalize=True,
metadata={"team_id": "T_SECONDARY"},
)
assert result.success is True
assert adapter._get_client.call_args_list == [
call("C1", team_id="T_SECONDARY"),
call("C1", team_id="T_SECONDARY"),
]
assert client.chat_update.await_count == 2
first = client.chat_update.await_args_list[0].kwargs
second = client.chat_update.await_args_list[1].kwargs
assert "blocks" in first and first["blocks"]
assert second["blocks"] == []
assert second["text"]
+475
View File
@@ -0,0 +1,475 @@
"""Tests for Slack Block Kit interactive clarify buttons.
Mirrors test_slack_approval_buttons.py (harness) and
test_telegram_clarify_buttons.py (semantics) for the ``send_clarify`` override
and the indexed ``hermes_clarify_choice_<idx>`` /
``hermes_clarify_other`` action dispatch.
"""
import sys
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock
import pytest
# ---------------------------------------------------------------------------
# Ensure the repo root is importable
# ---------------------------------------------------------------------------
_repo = str(Path(__file__).resolve().parents[2])
if _repo not in sys.path:
sys.path.insert(0, _repo)
# ---------------------------------------------------------------------------
# Minimal Slack SDK mock so SlackAdapter can be imported (mirrors
# test_slack_approval_buttons.py)
# ---------------------------------------------------------------------------
def _ensure_slack_mock():
if "slack_bolt" in sys.modules:
return
slack_bolt = MagicMock()
slack_bolt.async_app.AsyncApp = MagicMock
sys.modules["slack_bolt"] = slack_bolt
sys.modules["slack_bolt.async_app"] = slack_bolt.async_app
handler_mod = MagicMock()
handler_mod.AsyncSocketModeHandler = MagicMock
sys.modules["slack_bolt.adapter"] = MagicMock()
sys.modules["slack_bolt.adapter.socket_mode"] = MagicMock()
sys.modules["slack_bolt.adapter.socket_mode.async_handler"] = handler_mod
sdk_mod = MagicMock()
sdk_mod.web = MagicMock()
sdk_mod.web.async_client = MagicMock()
sdk_mod.web.async_client.AsyncWebClient = MagicMock
sys.modules["slack_sdk"] = sdk_mod
sys.modules["slack_sdk.web"] = sdk_mod.web
sys.modules["slack_sdk.web.async_client"] = sdk_mod.web.async_client
_ensure_slack_mock()
from plugins.platforms.slack.adapter import SlackAdapter
from gateway.config import PlatformConfig
def _make_adapter():
config = PlatformConfig(enabled=True, token="xoxb-test-token")
adapter = SlackAdapter(config)
adapter._app = MagicMock()
adapter._bot_user_id = "U_BOT"
adapter._team_clients = {"T1": AsyncMock()}
adapter._team_bot_user_ids = {"T1": "U_BOT"}
adapter._channel_team = {"C1": "T1"}
return adapter
class _AuthRunner:
def __init__(self, auth_fn=None):
self._auth_fn = auth_fn or (lambda _source: True)
async def handle(self, event):
return None
def _is_user_authorized(self, source):
return self._auth_fn(source)
def _attach_auth_runner(adapter, auth_fn=None):
adapter.set_message_handler(_AuthRunner(auth_fn=auth_fn).handle)
def _clear_clarify_state():
from tools import clarify_gateway as cm
with cm._lock:
cm._entries.clear()
cm._session_index.clear()
cm._notify_cbs.clear()
# ===========================================================================
# send_clarify — Block Kit render (a)
# ===========================================================================
class TestSlackSendClarify:
def setup_method(self):
_clear_clarify_state()
@pytest.mark.asyncio
async def test_multi_choice_renders_buttons_and_other(self):
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1234.5678"})
result = await adapter.send_clarify(
chat_id="C1",
question="Which environment?",
choices=["staging", "production"],
clarify_id="cid1",
session_key="sk1",
)
assert result.success is True
assert result.message_id == "1234.5678"
# ts recorded for the double-click guard
assert adapter._clarify_resolved.get("1234.5678") is False
kwargs = mock_client.chat_postMessage.call_args[1]
blocks = kwargs["blocks"]
assert blocks[0]["type"] == "section"
assert "Which environment?" in blocks[0]["text"]["text"]
assert blocks[1]["type"] == "actions"
elements = blocks[1]["elements"]
# 2 choices + Other
assert len(elements) == 3
assert elements[0]["action_id"] == "hermes_clarify_choice_0"
assert elements[0]["value"] == "cid1|0"
assert elements[1]["action_id"] == "hermes_clarify_choice_1"
assert elements[1]["value"] == "cid1|1"
assert elements[0]["text"]["text"] == "staging"
# Final button is the free-text "Other"
assert elements[2]["action_id"] == "hermes_clarify_other"
assert elements[2]["value"] == "cid1|other"
for block in blocks:
if block["type"] == "actions":
action_ids = [element["action_id"] for element in block["elements"]]
assert len(action_ids) == len(set(action_ids))
@pytest.mark.asyncio
async def test_open_ended_no_buttons(self):
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "9.9"})
result = await adapter.send_clarify(
chat_id="C1",
question="What should I name the branch?",
choices=None,
clarify_id="cid-open",
session_key="sk-open",
)
assert result.success is True
kwargs = mock_client.chat_postMessage.call_args[1]
# Open-ended delegates to the base plain-text path — no action blocks.
assert "blocks" not in kwargs or all(
b.get("type") != "actions" for b in (kwargs.get("blocks") or [])
)
assert "What should I name the branch?" in kwargs["text"]
@pytest.mark.asyncio
async def test_mrkdwn_escapes_question(self):
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.1"})
await adapter.send_clarify(
chat_id="C1",
question="Use <A> & <B>?",
choices=["yes"],
clarify_id="cid2",
session_key="sk2",
)
section_text = mock_client.chat_postMessage.call_args[1]["blocks"][0]["text"]["text"]
assert "<A>" not in section_text
assert "&lt;A&gt;" in section_text
assert "&amp;" in section_text
@pytest.mark.asyncio
async def test_sends_in_thread(self):
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.2"})
await adapter.send_clarify(
chat_id="C1",
question="?",
choices=["a"],
clarify_id="cid3",
session_key="sk3",
metadata={"thread_id": "8888.0000"},
)
assert mock_client.chat_postMessage.call_args[1].get("thread_ts") == "8888.0000"
@pytest.mark.asyncio
async def test_not_connected(self):
adapter = _make_adapter()
adapter._app = None
result = await adapter.send_clarify(
chat_id="C1", question="?", choices=["a"], clarify_id="c", session_key="s"
)
assert result.success is False
@pytest.mark.asyncio
async def test_five_choices_chunk_across_actions_blocks(self):
"""Slack caps 5 elements per actions block; 5 choices + Other = 6
buttons must spill into a second block instead of 400ing."""
adapter = _make_adapter()
mock_client = adapter._team_clients["T1"]
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.3"})
await adapter.send_clarify(
chat_id="C1",
question="?",
choices=["a", "b", "c", "d", "e"],
clarify_id="cid5",
session_key="sk5",
)
blocks = mock_client.chat_postMessage.call_args[1]["blocks"]
action_blocks = [b for b in blocks if b["type"] == "actions"]
assert len(action_blocks) == 2
for b in action_blocks:
assert len(b["elements"]) <= 5
# ===========================================================================
# _handle_clarify_action — choice click resolves (b)
# ===========================================================================
class TestSlackClarifyChoiceAction:
def setup_method(self):
_clear_clarify_state()
@pytest.mark.asyncio
async def test_choice_resolves_with_choice_text(self):
from tools import clarify_gateway as cm
adapter = _make_adapter()
_attach_auth_runner(adapter)
cm.register("cidA", "sk-cb", "Pick", ["red", "green", "blue"])
adapter._clarify_resolved["1234.5678"] = False
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {
"ts": "1234.5678",
"blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
{"type": "actions", "elements": []},
],
},
"channel": {"id": "C1"},
"user": {"name": "norbert", "id": "U_NORBERT"},
}
action = {"action_id": "hermes_clarify_choice_1", "value": "cidA|1"}
await adapter._handle_clarify_action(ack, body, action)
ack.assert_called_once()
with cm._lock:
entry = cm._entries.get("cidA")
assert entry is not None
assert entry.response == "green"
assert entry.event.is_set()
# Message updated with the answer, buttons dropped.
update_kwargs = mock_client.chat_update.call_args[1]
assert "green" in update_kwargs["text"]
assert all(b["type"] != "actions" for b in update_kwargs["blocks"])
@pytest.mark.asyncio
async def test_prevents_double_click(self):
from tools import clarify_gateway as cm
adapter = _make_adapter()
_attach_auth_runner(adapter)
cm.register("cidDup", "sk-dup", "Pick", ["x"])
adapter._clarify_resolved["1.1"] = True # already resolved
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {"ts": "1.1", "blocks": []},
"channel": {"id": "C1"},
"user": {"name": "n", "id": "U1"},
}
action = {"action_id": "hermes_clarify_choice", "value": "cidDup|0"}
await adapter._handle_clarify_action(ack, body, action)
ack.assert_called_once()
with cm._lock:
entry = cm._entries.get("cidDup")
assert entry is not None
assert not entry.event.is_set()
mock_client.chat_update.assert_not_called()
@pytest.mark.asyncio
async def test_unauthorized_click_ignored(self):
from tools import clarify_gateway as cm
adapter = _make_adapter()
_attach_auth_runner(adapter, auth_fn=lambda _s: False)
cm.register("cidAuth", "sk-auth", "Pick", ["a", "b"])
adapter._clarify_resolved["2.2"] = False
ack = AsyncMock()
body = {
"message": {"ts": "2.2", "blocks": []},
"channel": {"id": "C1"},
"user": {"name": "mallory", "id": "U_BAD"},
}
action = {"action_id": "hermes_clarify_choice", "value": "cidAuth|0"}
await adapter._handle_clarify_action(ack, body, action)
with cm._lock:
entry = cm._entries.get("cidAuth")
assert entry is not None
assert not entry.event.is_set()
@pytest.mark.asyncio
async def test_expired_choice_shows_notice(self):
"""Late tap after the entry was evicted must surface expiry, not a ✓."""
adapter = _make_adapter()
_attach_auth_runner(adapter)
# No entry registered → resolve returns False.
adapter._clarify_resolved["3.3"] = False
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {"ts": "3.3", "blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
]},
"channel": {"id": "C1"},
"user": {"name": "t", "id": "U_T"},
}
action = {"action_id": "hermes_clarify_choice", "value": "cidGone|0"}
await adapter._handle_clarify_action(ack, body, action)
assert "expired" in mock_client.chat_update.call_args[1]["text"].lower()
# ===========================================================================
# _handle_clarify_action — "Other" → text-capture → typed reply (c)
# ===========================================================================
class TestSlackClarifyOtherFlow:
def setup_method(self):
_clear_clarify_state()
@pytest.mark.asyncio
async def test_other_flips_to_text_mode_then_typed_reply_resolves(self):
from tools import clarify_gateway as cm
adapter = _make_adapter()
_attach_auth_runner(adapter)
cm.register("cidO", "sk-other", "Pick", ["x", "y"])
adapter._clarify_resolved["4.4"] = False
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {"ts": "4.4", "blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
{"type": "actions", "elements": []},
]},
"channel": {"id": "C1"},
"user": {"name": "norbert", "id": "U_N"},
}
action = {"action_id": "hermes_clarify_other", "value": "cidO|other"}
await adapter._handle_clarify_action(ack, body, action)
# Entry flipped to text-capture; NOT yet resolved.
pending = cm.get_pending_for_session("sk-other")
assert pending is not None and pending.clarify_id == "cidO"
assert pending.awaiting_text is True
with cm._lock:
entry = cm._entries.get("cidO")
assert not entry.event.is_set()
assert "awaiting" in mock_client.chat_update.call_args[1]["text"].lower()
# Now the gateway text-intercept (platform-agnostic) resolves from the
# user's next typed message. We exercise that leveraged path directly.
assert cm.resolve_text_response_for_session("sk-other", "my custom answer") is True
with cm._lock:
entry = cm._entries.get("cidO")
assert entry.response == "my custom answer"
assert entry.event.is_set()
@pytest.mark.asyncio
async def test_other_expired_shows_notice(self):
adapter = _make_adapter()
_attach_auth_runner(adapter)
# No entry → mark_awaiting_text returns False.
adapter._clarify_resolved["5.5"] = False
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {"ts": "5.5", "blocks": [
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
]},
"channel": {"id": "C1"},
"user": {"name": "t", "id": "U_T"},
}
action = {"action_id": "hermes_clarify_other", "value": "cidOtherGone|other"}
await adapter._handle_clarify_action(ack, body, action)
assert "expired" in mock_client.chat_update.call_args[1]["text"].lower()
@pytest.mark.asyncio
async def test_malformed_value_ignored(self):
adapter = _make_adapter()
_attach_auth_runner(adapter)
adapter._clarify_resolved["6.6"] = False
mock_client = adapter._team_clients["T1"]
mock_client.chat_update = AsyncMock()
ack = AsyncMock()
body = {
"message": {"ts": "6.6", "blocks": []},
"channel": {"id": "C1"},
"user": {"name": "t", "id": "U_T"},
}
action = {"action_id": "hermes_clarify_choice", "value": "no-delimiter"}
await adapter._handle_clarify_action(ack, body, action)
mock_client.chat_update.assert_not_called()
# ===========================================================================
# Base text-fallback unchanged for platforms without an override (e)
# ===========================================================================
class TestBaseAdapterClarifyFallbackUnchanged:
@pytest.mark.asyncio
async def test_base_numbered_text_fallback(self):
from gateway.platforms.base import BasePlatformAdapter, SendResult
class _Stub(BasePlatformAdapter):
name = "stub"
def __init__(self):
self.sent: list = []
async def connect(self, *, is_reconnect: bool = False): pass
async def disconnect(self): pass
async def send(self, chat_id, content, **kw):
self.sent.append(content)
return SendResult(success=True, message_id="1")
async def edit(self, *a, **k): return SendResult(success=False)
async def get_history(self, *a, **k): return []
async def get_chat_info(self, *a, **k): return {}
adapter = _Stub()
result = await adapter.send_clarify(
chat_id="c", question="Pick a fruit",
choices=["apple", "banana"], clarify_id="x", session_key="s",
)
assert result.success is True
text = adapter.sent[0]
assert "Pick a fruit" in text
assert "1." in text and "apple" in text
assert "2." in text and "banana" in text
+89
View File
@@ -903,3 +903,92 @@ def test_mention_patterns_trigger_in_channel_without_literal_mention():
assert _would_process(adapter, text="hey hermes what's the status") is True
# Unrelated channel chatter is still ignored.
assert _would_process(adapter, text="lunch anyone?") is False
# ---------------------------------------------------------------------------
# Tests: Block-Kit-only mention detection (#52387)
# ---------------------------------------------------------------------------
from plugins.platforms.slack.adapter import _slack_mention_detection_text # noqa: E402
def _blockkit_mention_event(bot_user_id=BOT_USER_ID, flat_text="Release notification"):
"""A Slack event whose @mention lives ONLY inside Block Kit blocks."""
return {
"text": flat_text,
"blocks": [
{
"type": "rich_text",
"elements": [
{
"type": "rich_text_section",
"elements": [
{"type": "text", "text": "Hey "},
{"type": "user", "user_id": bot_user_id},
{"type": "text", "text": "! I will do a release"},
],
}
],
}
],
}
def test_mention_detection_text_recovers_blockkit_mention():
event = _blockkit_mention_event()
merged = _slack_mention_detection_text(event)
# The flat text alone never contains the mention...
assert f"<@{BOT_USER_ID}>" not in event.get("text", "")
# ...but the merged detection text does.
assert f"<@{BOT_USER_ID}>" in merged
def test_mention_detection_text_no_blocks_returns_flat_text():
event = {"text": f"<@{BOT_USER_ID}> hello"}
assert _slack_mention_detection_text(event) == f"<@{BOT_USER_ID}> hello"
def test_mention_detection_text_no_mention_anywhere():
event = {
"text": "lunch anyone?",
"blocks": [
{
"type": "rich_text",
"elements": [
{
"type": "rich_text_section",
"elements": [{"type": "text", "text": "lunch anyone?"}],
}
],
}
],
}
assert f"<@{BOT_USER_ID}>" not in _slack_mention_detection_text(event)
def test_mention_detection_text_ignores_quoted_blockkit_mention():
"""A mention inside rich_text_quote (forwarded content) must NOT count."""
event = {
"text": "please review",
"blocks": [
{
"type": "rich_text",
"elements": [
{
"type": "rich_text_quote",
"elements": [
{
"type": "rich_text_section",
"elements": [
{"type": "text", "text": "Contains "},
{"type": "user", "user_id": BOT_USER_ID},
{"type": "text", "text": " in quoted text"},
],
}
],
}
],
}
],
}
assert f"<@{BOT_USER_ID}>" not in _slack_mention_detection_text(event)
@@ -0,0 +1,170 @@
"""Regression tests for #57682 — holographic auto_extract harvested
context-compaction handoff summaries into fact_store, and ran even when
configured off.
Two compounding defects:
1. Gate: the plugin's config schema declares ``auto_extract`` as a string enum
(``"false"``/``"true"``), and the ``on_session_end`` gate used plain
truthiness — ``not "false"`` is ``False`` — so extraction ran despite being
explicitly configured off.
2. Eligibility: ``_auto_extract_facts`` scanned every ``role == "user"``
message. Compaction handoff summaries can be inserted as ``role="user"``
messages, and their prose reliably matches the decision patterns
(``we decided/agreed/chose``, ``the project uses/needs/requires``), so the
compactor's own output was stored as a durable ``project`` fact on every
session rollover that followed a compaction.
"""
import pytest
from agent.context_compressor import (
COMPRESSED_SUMMARY_METADATA_KEY,
SUMMARY_PREFIX,
_MERGED_PRIOR_CONTEXT_HEADER,
_MERGED_SUMMARY_DELIMITER,
is_compaction_summary_message,
)
from plugins.memory.holographic import HolographicMemoryProvider
def _make_provider(tmp_path, **config):
base = {"db_path": str(tmp_path / "memory_store.db"), "hrr_dim": 64}
base.update(config)
provider = HolographicMemoryProvider(config=base)
provider.initialize(session_id="test-session")
return provider
def _fact_contents(provider):
return [f["content"] for f in provider._store.list_facts(limit=100)]
def _user(content, **extra):
msg = {"role": "user", "content": content}
msg.update(extra)
return msg
DECISION_MSG = "we decided to use PostgreSQL for the persistence layer"
SUMMARY_MSG = (
f"{SUMMARY_PREFIX}\n## Historical Task Snapshot\n"
"The project uses a kanban board for all dispatch. "
"We agreed to route reviews through the fan-in consumer."
)
# ---------------------------------------------------------------------------
# Defect 1 — string-boolean gate
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("off_value", [False, "false", "False", "no", "off", "0", None, ""])
def test_auto_extract_off_values_disable_extraction(tmp_path, off_value):
provider = _make_provider(tmp_path, auto_extract=off_value)
provider.on_session_end([_user(DECISION_MSG)])
assert _fact_contents(provider) == []
provider.shutdown()
@pytest.mark.parametrize("on_value", [True, "true", "True", "1", "yes", "on"])
def test_auto_extract_on_values_enable_extraction(tmp_path, on_value):
provider = _make_provider(tmp_path, auto_extract=on_value)
provider.on_session_end([_user(DECISION_MSG)])
facts = _fact_contents(provider)
assert len(facts) == 1
assert "PostgreSQL" in facts[0]
provider.shutdown()
# ---------------------------------------------------------------------------
# Defect 2 — compaction summaries harvested as facts
# ---------------------------------------------------------------------------
def test_compaction_summary_not_harvested(tmp_path):
"""The exact failure mode from #57682: summary prose matches the decision
patterns but must not become a fact."""
provider = _make_provider(tmp_path, auto_extract=True)
provider.on_session_end([_user(SUMMARY_MSG)])
assert _fact_contents(provider) == []
provider.shutdown()
def test_metadata_marked_summary_not_harvested(tmp_path):
"""In-process summaries carry COMPRESSED_SUMMARY_METADATA_KEY even if a
future prefix rewrite changes the content sentinel."""
provider = _make_provider(tmp_path, auto_extract=True)
marked = _user(DECISION_MSG, **{COMPRESSED_SUMMARY_METADATA_KEY: True})
provider.on_session_end([marked])
assert _fact_contents(provider) == []
provider.shutdown()
def test_merged_into_tail_summary_suffix_not_harvested_prefix_content_ignored(tmp_path):
"""Merge-into-tail summaries embed the handoff prefix after the delimiter,
not at the start of the message. The wrapped pre-delimiter segment here has
no fact-pattern match, so nothing is harvested from either side."""
provider = _make_provider(tmp_path, auto_extract=True)
merged = _user(
f"{_MERGED_PRIOR_CONTEXT_HEADER}\nplease fix the login bug\n"
f"{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}"
)
provider.on_session_end([merged])
assert _fact_contents(provider) == []
provider.shutdown()
def test_merged_into_tail_preserves_genuine_pre_delimiter_preference(tmp_path):
"""#57690 review: teknium1 noted the ENTIRE merged row was being skipped,
discarding genuine pre-delimiter user content (context_compressor.py
~3163-3190 retains real prior tail text before the summary). The fix must
extract and harvest that segment while still excluding the summary
suffix."""
provider = _make_provider(tmp_path, auto_extract=True)
merged = _user(
f"{_MERGED_PRIOR_CONTEXT_HEADER}\n"
"I prefer tabs over spaces for indentation\n"
f"{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}"
)
provider.on_session_end([merged])
facts = _fact_contents(provider)
assert len(facts) == 1
assert "tabs over spaces" in facts[0]
assert "kanban board" not in facts[0]
assert "PostgreSQL" not in facts[0]
provider.shutdown()
def test_real_user_messages_still_extracted_alongside_summary(tmp_path):
"""The guard must skip only the summary, not suppress extraction for the
genuine user turns around it."""
provider = _make_provider(tmp_path, auto_extract=True)
provider.on_session_end(
[
_user(SUMMARY_MSG),
_user("I prefer tabs over spaces for indentation"),
]
)
facts = _fact_contents(provider)
assert len(facts) == 1
assert "tabs over spaces" in facts[0]
provider.shutdown()
# ---------------------------------------------------------------------------
# is_compaction_summary_message — public helper contract
# ---------------------------------------------------------------------------
def test_helper_detects_prefix_metadata_and_merged_forms():
assert is_compaction_summary_message(_user(SUMMARY_MSG))
assert is_compaction_summary_message(
_user("anything", **{COMPRESSED_SUMMARY_METADATA_KEY: True})
)
assert is_compaction_summary_message(
_user(f"prior tail\n{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}")
)
assert not is_compaction_summary_message(_user(DECISION_MSG))
assert not is_compaction_summary_message(_user(""))
@@ -1,5 +1,8 @@
import time
from types import SimpleNamespace
import pytest
from agent.codex_runtime import _record_codex_app_server_compaction
from agent.conversation_compression import COMPACTION_STATUS, compress_context
from agent.transports.codex_app_server_session import TurnResult
@@ -19,6 +22,26 @@ class FakeCodexSession:
self.closed = True
class SlowCodexSession(FakeCodexSession):
def __init__(self, result, touch_calls):
super().__init__(result)
self.touch_calls = touch_calls
def compact_thread(self):
self.calls += 1
_wait_for_touch(self.touch_calls, "context compression in progress")
return self.result
def _wait_for_touch(touch_calls, desc, timeout=1.0):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if desc in touch_calls:
return
time.sleep(0.01)
pytest.fail(f"timed out waiting for touch {desc!r}; saw {touch_calls!r}")
class DummyAgent:
def __init__(
self,
@@ -43,6 +66,11 @@ class DummyAgent:
self.warnings = []
self.events = []
self.built_prompts = []
self.touch_calls = []
self._compression_activity_heartbeat_interval = 0.1
def _touch_activity(self, desc):
self.touch_calls.append(desc)
def _emit_status(self, message):
self.statuses.append(message)
@@ -79,6 +107,33 @@ def test_codex_app_server_native_auto_mode_leaves_thread_compaction_to_codex():
assert agent.events == []
def test_codex_app_server_compaction_heartbeat_refreshes_activity_while_waiting():
agent = DummyAgent(
TurnResult(thread_id="thread-1", turn_id="compact-turn-1")
)
agent._codex_session = SlowCodexSession(
agent._codex_session.result,
agent.touch_calls,
)
messages = [{"role": "user", "content": "hi"}]
returned, prompt = compress_context(
agent,
messages,
"system",
approx_tokens=100000,
task_id="test",
force=True,
)
assert returned is messages
assert prompt == "cached prompt"
assert agent._codex_session.calls == 1
assert "context compression started" in agent.touch_calls
assert "context compression in progress" in agent.touch_calls
assert agent.touch_calls[-1] == "context compression completed"
def test_codex_app_server_manual_compression_routes_to_codex_thread():
agent = DummyAgent(
TurnResult(thread_id="thread-1", turn_id="compact-turn-1")
@@ -162,6 +217,8 @@ def test_codex_app_server_compression_failure_preserves_bookkeeping():
assert agent.context_compressor.compression_count == 0
assert agent.context_compressor.last_prompt_tokens == 123
assert agent.warnings
assert agent.touch_calls[0] == "context compression started"
assert agent.touch_calls[-1] == "context compression failed"
def test_codex_app_server_native_compaction_notice_emits_status_and_event():
@@ -16,7 +16,11 @@ import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
class TestCompressionBoundaryHook:
def _make_agent(self, session_db):
@@ -92,6 +96,159 @@ class TestCompressionBoundaryHook:
f"Expected new session_id as first positional arg, got {call!r}"
assert call.kwargs.get("old_session_id") == original_sid, \
f"Expected old_session_id={original_sid!r}, got {call.kwargs!r}"
assert len(comp_calls) == 1
def test_automatic_notification_follows_core_persistence(self):
from hermes_state import SessionDB
events = []
with tempfile.TemporaryDirectory() as tmpdir:
db = SessionDB(db_path=Path(tmpdir) / "test.db")
agent = self._make_agent(db)
compressor = MagicMock()
compressor.compress.return_value = [
{"role": "user", "content": "summary"}
]
compressor.compression_count = 1
compressor.last_prompt_tokens = 0
compressor.last_completion_tokens = 0
compressor._last_summary_error = None
compressor._last_compress_aborted = False
compressor.on_session_start.side_effect = (
lambda *_args, **kwargs: events.append(
kwargs.get("boundary_reason")
)
)
agent.context_compressor = compressor
original_update = db.update_system_prompt
def _record_update(*args, **kwargs):
result = original_update(*args, **kwargs)
events.append("persist")
return result
with patch.object(db, "update_system_prompt", side_effect=_record_update):
agent._compress_context(
[{"role": "user", "content": "request"}],
"sys",
approx_tokens=100,
)
assert events == ["persist", "compression"]
def test_failure_before_persistence_does_not_notify(self):
from hermes_state import SessionDB
with tempfile.TemporaryDirectory() as tmpdir:
db = SessionDB(db_path=Path(tmpdir) / "test.db")
agent = self._make_agent(db)
compressor = MagicMock()
compressor.compress.side_effect = RuntimeError("synthetic compression failure")
agent.context_compressor = compressor
with pytest.raises(RuntimeError, match="synthetic compression failure"):
agent._compress_context(
[{"role": "user", "content": "request"}],
"sys",
approx_tokens=100,
)
compressor.on_session_start.assert_not_called()
def test_failure_during_persistence_does_not_notify(self):
from hermes_state import SessionDB
with tempfile.TemporaryDirectory() as tmpdir:
db = SessionDB(db_path=Path(tmpdir) / "test.db")
agent = self._make_agent(db)
compressor = MagicMock()
compressor.compress.return_value = [
{"role": "user", "content": "summary"}
]
compressor.compression_count = 1
compressor.last_prompt_tokens = 0
compressor.last_completion_tokens = 0
compressor._last_summary_error = None
compressor._last_compress_aborted = False
agent.context_compressor = compressor
with patch.object(
db,
"update_system_prompt",
side_effect=RuntimeError("synthetic commit failure"),
):
agent._compress_context(
[{"role": "user", "content": "request"}],
"sys",
approx_tokens=100,
)
boundary_calls = [
call
for call in compressor.on_session_start.call_args_list
if call.kwargs.get("boundary_reason") == "compression"
]
assert boundary_calls == []
def test_no_progress_does_not_notify(self):
from hermes_state import SessionDB
with tempfile.TemporaryDirectory() as tmpdir:
db = SessionDB(db_path=Path(tmpdir) / "test.db")
agent = self._make_agent(db)
compressor = MagicMock()
compressor.compress.side_effect = lambda messages, **_kwargs: messages
compressor._last_compress_aborted = False
agent.context_compressor = compressor
messages = [{"role": "user", "content": "request"}]
returned, _ = agent._compress_context(
messages,
"sys",
approx_tokens=100,
)
assert returned is messages
compressor.on_session_start.assert_not_called()
@pytest.mark.parametrize("committed", [True, False])
def test_deferred_notification_finishes_exactly_once(self, committed):
from hermes_state import SessionDB
events = []
with tempfile.TemporaryDirectory() as tmpdir:
db = SessionDB(db_path=Path(tmpdir) / "test.db")
agent = self._make_agent(db)
compressor = MagicMock()
compressor.compress.return_value = [
{"role": "user", "content": "summary"}
]
compressor.compression_count = 1
compressor.last_prompt_tokens = 0
compressor.last_completion_tokens = 0
compressor._last_summary_error = None
compressor._last_compress_aborted = False
compressor.on_session_start.side_effect = (
lambda *_args, **_kwargs: events.append("notify")
)
agent.context_compressor = compressor
agent._compress_context(
[{"role": "user", "content": "request"}],
"sys",
approx_tokens=100,
force=True,
defer_context_engine_notification=True,
)
assert events == []
assert finalize_context_engine_compression_notification(
agent, committed=committed
) is committed
assert finalize_context_engine_compression_notification(
agent, committed=True
) is False
assert events == (["notify"] if committed else [])
def test_no_hook_when_no_session_db(self):
"""Without session_db, session_id does not rotate and the hook is not fired."""
+100 -1
View File
@@ -95,7 +95,12 @@ def test_auto_corrects_threshold_when_aux_context_below_threshold(mock_get_clien
assert "config.yaml" in messages[0]
assert "auxiliary:" in messages[0]
assert "compression:" in messages[0]
assert "threshold:" in messages[0]
# 200K main is under the 512K small-context limit and 80K/200K = 40% sits
# below the 75% floor — a `threshold:` suggestion would be raised back to
# 75% and ignored (#67422), so the message must not offer one and must
# explain the recomputed trigger instead (0.75 * 200K = 150K).
assert "threshold:" not in messages[0]
assert "150,000" in messages[0]
# Warning stored for gateway replay
assert agent._compression_warning is not None
# Threshold on the live compressor was actually lowered to aux_context.
@@ -514,3 +519,97 @@ def test_run_conversation_clears_warning_after_replay(mock_get_client, mock_ctx_
agent._compression_warning = None
assert len(callback_events) == 0
# ── #67422: threshold suggestion must survive the small-context floor ────────
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
@patch("agent.auxiliary_client.get_text_auxiliary_client")
def test_threshold_suggestion_kept_when_at_or_above_floor(mock_get_client, mock_ctx_len):
"""Small-context main, but aux/main = 80% >= the 75% floor — the
`threshold:` suggestion is still viable and must be offered."""
agent = _make_agent(main_context=100_000, threshold_percent=0.85)
# threshold = 85,000 — aux has 80,000 (above 64K floor, below threshold)
mock_client = MagicMock()
mock_client.base_url = "https://openrouter.ai/api/v1"
mock_client.api_key = "sk-aux"
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
messages = []
agent._emit_status = lambda msg: messages.append(msg)
agent._check_compression_model_feasibility()
assert len(messages) == 1
assert "threshold: 0.80" in messages[0]
@patch("agent.model_metadata.get_model_context_length", return_value=300_000)
@patch("agent.auxiliary_client.get_text_auxiliary_client")
def test_threshold_suggestion_kept_for_large_context_main(mock_get_client, mock_ctx_len):
"""Main window >= 512K has no floor — any suggestion is honored, so the
`threshold:` option stays even below 75%."""
agent = _make_agent(main_context=1_000_000, threshold_percent=0.50)
# threshold = 500,000 — aux has 300,000
mock_client = MagicMock()
mock_client.base_url = "https://openrouter.ai/api/v1"
mock_client.api_key = "sk-aux"
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
messages = []
agent._emit_status = lambda msg: messages.append(msg)
agent._check_compression_model_feasibility()
assert len(messages) == 1
assert "threshold: 0.30" in messages[0]
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
@patch("agent.auxiliary_client.get_text_auxiliary_client")
def test_threshold_suggestion_kept_when_reservation_shrinks_trigger(mock_get_client, mock_ctx_len):
"""Output-token reservation can make a floored suggestion viable again:
with max_tokens=120K on a 200K window, the recomputed trigger is
max(0.75 * 80K, 64K) = 64K, which fits the 80K aux — so the suggestion
must NOT be suppressed by the raw-window percentage check (sweeper
regression for #67422)."""
agent = _make_agent(main_context=200_000, threshold_percent=0.50)
agent.context_compressor.max_tokens = 120_000
mock_client = MagicMock()
mock_client.base_url = "https://openrouter.ai/api/v1"
mock_client.api_key = "sk-aux"
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
messages = []
agent._emit_status = lambda msg: messages.append(msg)
agent._check_compression_model_feasibility()
assert len(messages) == 1
assert "threshold: 0.40" in messages[0]
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
@patch("agent.auxiliary_client.get_text_auxiliary_client")
def test_plugin_engine_keeps_plain_suggestion(mock_get_client, mock_ctx_len):
"""External context engines own compaction policy (#44439) — the built-in
small-context floor must not suppress the threshold suggestion when the
active compressor is not the built-in ContextCompressor."""
agent = _make_agent(main_context=200_000, threshold_percent=0.50)
plugin_engine = MagicMock() # not spec'd — fails isinstance(ContextCompressor)
plugin_engine.context_length = 200_000
plugin_engine.threshold_tokens = 100_000
agent.context_compressor = plugin_engine
mock_client = MagicMock()
mock_client.base_url = "https://openrouter.ai/api/v1"
mock_client.api_key = "sk-aux"
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
messages = []
agent._emit_status = lambda msg: messages.append(msg)
agent._check_compression_model_feasibility()
assert len(messages) == 1
assert "threshold: 0.40" in messages[0]
+114
View File
@@ -1265,3 +1265,117 @@ def test_reference_messages_drops_whitespace_only_string_user_turn():
assert view[0] == {"role": "assistant", "content": "a"}
assert view[-1] == {"role": "user", "content": "real"}
assert all(str(m["content"]).strip() for m in view)
def test_moa_pre_api_compression_includes_reference_guidance(monkeypatch, tmp_path):
"""The aggregator must not receive guidance that pushes it past compression.
The normal pre-API check sees only the persisted conversation. MoA adds
reference guidance later, inside ``MoAChatCompletions.create()``, so this
regression drives a raw request just below the threshold and makes the
injected guidance cross it. Compression must occur before the aggregator
request and leave the rebuilt request below the threshold.
"""
home = tmp_path / ".hermes"
home.mkdir()
(home / "config.yaml").write_text(
"""
moa:
default_preset: review
presets:
review:
reference_models:
- provider: openrouter
model: advisor
aggregator:
provider: openrouter
model: aggregator
""".strip(),
encoding="utf-8",
)
monkeypatch.setenv("HERMES_HOME", str(home))
events = []
compression_inputs = []
aggregator_request_tokens = []
def fake_estimate(messages, *args, **kwargs):
rendered = str(messages)
raw_tokens = 80 if "PRE_COMPACTION_HISTORY" in rendered else 20
guidance_tokens = 40 if "Mixture of Agents reference context" in rendered else 0
return raw_tokens + guidance_tokens
def fake_call_llm(**kwargs):
if kwargs["task"] == "moa_reference":
events.append("reference")
return _response("advisor guidance")
events.append("aggregator")
aggregator_request_tokens.append(fake_estimate(kwargs["messages"]))
return _response("aggregator acted")
monkeypatch.setattr("agent.moa_loop.call_llm", fake_call_llm)
monkeypatch.setattr("agent.turn_context.estimate_request_tokens_rough", fake_estimate)
monkeypatch.setattr("agent.conversation_loop.estimate_request_tokens_rough", fake_estimate)
agent = AIAgent(
api_key="moa-virtual-provider",
base_url="moa://local",
model="review",
provider="moa",
quiet_mode=True,
skip_context_files=True,
skip_memory=True,
enabled_toolsets=["file"],
max_iterations=3,
)
compressor = getattr(agent, "context_compressor")
compressor.threshold_tokens = 100
def fake_compress(messages, *_args, **_kwargs):
events.append("compress")
compression_inputs.append(messages)
return ([{"role": "user", "content": "SUMMARY"}], "system")
monkeypatch.setattr(agent, "_compress_context", fake_compress)
result = agent.run_conversation(
"PRE_COMPACTION_HISTORY",
conversation_history=[{"role": "assistant", "content": "prior response"}],
)
assert result["final_response"] == "aggregator acted"
assert events.index("compress") < events.index("aggregator")
assert events.count("reference") == 1
assert all("Mixture of Agents reference context" not in str(item) for item in compression_inputs)
assert aggregator_request_tokens == [60]
def test_prepared_aggregator_preserves_reasoning_config(monkeypatch):
"""Prepared MoA requests retain the acting aggregator reasoning policy."""
from agent import moa_loop
captured = {}
expected_reasoning = {"enabled": True, "effort": "high"}
def fake_call_llm(**kwargs):
captured.update(kwargs)
return _response("aggregator acted")
monkeypatch.setattr(moa_loop, "call_llm", fake_call_llm)
monkeypatch.setattr(moa_loop, "_aggregator_reasoning_config", lambda _slot: expected_reasoning)
monkeypatch.setattr(
moa_loop,
"_slot_runtime",
lambda slot: {"provider": slot["provider"], "model": slot["model"]},
)
facade = moa_loop.MoAChatCompletions("review")
facade._call_prepared_aggregator(
{
"messages": [{"role": "user", "content": "question"}],
"aggregator": {"provider": "openrouter", "model": "aggregator"},
"aggregator_temperature": None,
},
{},
)
assert captured["reasoning_config"] == expected_reasoning
@@ -0,0 +1,233 @@
"""Tests for per-model compression threshold overrides.
Users who swap between models with very different context windows (e.g. a
256K model and a 1M model) need different compaction trigger points.
``compression.model_thresholds`` in config.yaml lets them set per-model
overrides that are resolved by longest substring match. The small-context
floor (75% for <512K models) still applies on top of per-model overrides.
"""
from unittest.mock import patch
from agent.context_compressor import ContextCompressor, resolve_model_threshold
from agent.context_engine import ContextEngine
# ---------------------------------------------------------------------------
# resolve_model_threshold helper
# ---------------------------------------------------------------------------
class TestResolveModelThreshold:
def test_no_overrides_returns_default(self):
assert resolve_model_threshold("glm-5.2", None, 0.50) == 0.50
assert resolve_model_threshold("glm-5.2", {}, 0.50) == 0.50
def test_empty_model_returns_default(self):
assert resolve_model_threshold("", {"glm": 0.70}, 0.50) == 0.50
def test_exact_match(self):
overrides = {"glm-5.2": 0.70}
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.70
def test_substring_match(self):
overrides = {"glm-5.2": 0.70}
assert resolve_model_threshold("openai/glm-5.2", overrides, 0.50) == 0.70
def test_longest_match_wins(self):
overrides = {"glm-5.2": 0.70, "glm-5.2-1M": 0.25}
# "glm-5.2-1M" is a longer match than "glm-5.2"
assert resolve_model_threshold("glm-5.2-1M", overrides, 0.50) == 0.25
# "glm-5.2" alone still matches at 0.70
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.70
def test_no_match_returns_default(self):
overrides = {"claude-sonnet-4": 0.60}
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.50
def test_override_can_lower_threshold(self):
"""Per-model overrides work in both directions (raise and lower)."""
overrides = {"small-model": 0.30}
assert resolve_model_threshold("small-model", overrides, 0.50) == 0.30
# ---------------------------------------------------------------------------
# ContextCompressor integration
# ---------------------------------------------------------------------------
class TestContextCompressorModelThresholds:
@patch("agent.context_compressor.get_model_context_length", return_value=1_000_000)
def test_init_large_context_with_override(self, _mock):
"""Large context (>=512K) + per-model override: override applies directly."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"glm-5.2": 0.40},
quiet_mode=True,
)
# 1M context >= 512K, so no small-context floor — override wins
assert cc.threshold_percent == 0.40
assert cc.threshold_tokens == int(1_000_000 * 0.40)
@patch("agent.context_compressor.get_model_context_length", return_value=1_000_000)
def test_init_large_context_no_match(self, _mock):
"""Large context + no matching override: global threshold used."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"claude-sonnet-4": 0.60},
quiet_mode=True,
)
assert cc.threshold_percent == 0.50
assert cc.threshold_tokens == int(1_000_000 * 0.50)
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
def test_init_small_context_override_below_floor(self, _mock):
"""Small context (<512K) + override below 75%: floor wins (raise-only)."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"glm-5.2": 0.40},
quiet_mode=True,
)
# 256K < 512K → floor at 0.75; override 0.40 < 0.75, so floor wins
assert cc.threshold_percent == 0.75
assert cc.threshold_tokens == int(256_000 * 0.75)
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
def test_init_small_context_override_above_floor(self, _mock):
"""Small context + override above 75%: override wins."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"glm-5.2": 0.80},
quiet_mode=True,
)
# 256K < 512K → floor at 0.75; override 0.80 > 0.75, so override wins
assert cc.threshold_percent == 0.80
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
def test_init_no_model_thresholds_dict(self, _mock):
"""Empty model_thresholds dict = backward compatible."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
quiet_mode=True,
)
# 256K < 512K → floored at 0.75
assert cc.threshold_percent == 0.75
assert cc.model_thresholds == {}
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
def test_init_none_model_thresholds(self, _mock):
"""Passing None for model_thresholds is safe."""
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds=None,
quiet_mode=True,
)
assert cc.model_thresholds == {}
@patch("agent.context_compressor.get_model_context_length")
def test_update_model_re_resolves_threshold(self, mock_ctx):
"""Switching models re-resolves the per-model threshold + re-applies floor."""
mock_ctx.return_value = 256_000
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"glm-5.2": 0.80, "glm-5.2-1M": 0.25},
quiet_mode=True,
)
# 256K < 512K → floor at 0.75; override 0.80 > 0.75, so 0.80 wins
assert cc.threshold_percent == 0.80
# Switch to the 1M model (large context, no floor)
mock_ctx.return_value = 1_000_000
cc.update_model(
model="glm-5.2-1M",
context_length=1_000_000,
)
# 1M >= 512K → no floor; override 0.25 applies directly
assert cc.threshold_percent == 0.25
assert cc.threshold_tokens == int(1_000_000 * 0.25)
@patch("agent.context_compressor.get_model_context_length")
def test_update_model_falls_back_to_global(self, mock_ctx):
"""Switching to a model with no override uses the global threshold."""
mock_ctx.return_value = 1_000_000
cc = ContextCompressor(
model="glm-5.2",
threshold_percent=0.50,
model_thresholds={"glm-5.2": 0.40},
quiet_mode=True,
)
# 1M context, override 0.40
assert cc.threshold_percent == 0.40
# Switch to a model with no override (still large context)
mock_ctx.return_value = 1_000_000
cc.update_model(
model="some-other-model",
context_length=1_000_000,
)
# No override match → falls back to global 0.50; 1M >= 512K → no floor
assert cc.threshold_percent == 0.50
assert cc.threshold_tokens == int(1_000_000 * 0.50)
# ---------------------------------------------------------------------------
# ContextEngine base class
# ---------------------------------------------------------------------------
class TestContextEngineModelThresholds:
def test_base_class_update_model_applies_overrides(self):
"""The base-class update_model() applies model_thresholds if set."""
class TestEngine(ContextEngine):
@property
def name(self):
return "test"
def update_from_response(self, usage):
pass
def should_compress(self, prompt_tokens=None):
return False
def compress(self, messages, current_tokens=None, focus_topic=None):
return messages
engine = TestEngine()
engine.threshold_percent = 0.50
engine._config_threshold_percent = 0.50
engine.context_length = 0
engine.model_thresholds = {"glm-5.2-1M": 0.25}
engine.update_model(model="glm-5.2-1M", context_length=1_000_000)
assert engine.threshold_percent == 0.25
assert engine.threshold_tokens == int(1_000_000 * 0.25)
def test_base_class_update_model_no_overrides(self):
"""Without model_thresholds, the base class behaves as before."""
class TestEngine(ContextEngine):
@property
def name(self):
return "test"
def update_from_response(self, usage):
pass
def should_compress(self, prompt_tokens=None):
return False
def compress(self, messages, current_tokens=None, focus_topic=None):
return messages
engine = TestEngine()
engine.threshold_percent = 0.50
engine._base_threshold_percent = 0.50
engine.context_length = 0
engine.model_thresholds = {}
engine.update_model(model="glm-5.2", context_length=256_000)
assert engine.threshold_percent == 0.50
assert engine.threshold_tokens == int(256_000 * 0.50)
@@ -0,0 +1,186 @@
"""Follow-up regression tests for per-model compression threshold overrides.
Covers the gaps flagged in the review of PR #63020:
1. Plugin-engine init ordering — ``compression.model_thresholds`` must be
assigned to a selected plugin context engine BEFORE the initial
``update_model()`` call in agent init, so the initial model already gets
its override (previously the override only took effect after the first
``/model`` switch).
2. ``compression.model_thresholds`` is a public key in ``DEFAULT_CONFIG``.
3. Floor interaction on the ``update_model()`` (model-switch) path:
an override below the small-context floor is raised to the floor
(raise-only); an override above the floor wins.
4. Base-class ``update_model()`` snapshots the pre-override percent once,
so repeated switches fall back to the engine's configured threshold
rather than a previous model's override.
"""
from unittest.mock import patch
from agent.context_compressor import ContextCompressor
from agent.context_engine import ContextEngine
class _StubEngine(ContextEngine):
"""Minimal concrete context engine for init-ordering tests."""
@property
def name(self) -> str:
return "stub"
def update_from_response(self, usage):
pass
def should_compress(self, prompt_tokens=None):
return False
def compress(self, messages, current_tokens=None):
return messages
def test_plugin_engine_gets_model_thresholds_before_initial_update_model():
"""The initial model's override must apply during AIAgent init.
Regression test for the PR #63020 review finding: the plugin engine was
initialized through update_model() before model_thresholds was assigned,
so the initial model kept the global threshold until a /model switch.
"""
engine = _StubEngine()
engine.threshold_percent = 0.50
cfg = {
"context": {"engine": "stub"},
"agent": {},
"compression": {
"threshold": 0.50,
"model_thresholds": {"glm-5.2": 0.25},
},
}
with (
patch("hermes_cli.config.load_config", return_value=cfg),
patch("plugins.context_engine.load_context_engine", return_value=engine),
patch("agent.model_metadata.get_model_context_length", return_value=1_000_000),
patch("run_agent.get_tool_definitions", return_value=[]),
patch("run_agent.check_toolset_requirements", return_value={}),
patch("run_agent.OpenAI"),
):
from run_agent import AIAgent
agent = AIAgent(
model="glm-5.2",
api_key="test-key-1234567890",
base_url="https://openrouter.ai/api/v1",
quiet_mode=True,
skip_context_files=True,
skip_memory=True,
)
assert agent.context_compressor is engine
# The override map arrived before the initial update_model() call, so the
# very first resolution already used it.
assert engine.model_thresholds == {"glm-5.2": 0.25}
assert engine.threshold_percent == 0.25
assert engine.threshold_tokens == int(1_000_000 * 0.25)
def test_plugin_engine_without_overrides_keeps_global_threshold():
"""Empty model_thresholds leaves plugin-engine init behavior unchanged."""
engine = _StubEngine()
engine.threshold_percent = 0.50
cfg = {
"context": {"engine": "stub"},
"agent": {},
"compression": {"threshold": 0.50},
}
with (
patch("hermes_cli.config.load_config", return_value=cfg),
patch("plugins.context_engine.load_context_engine", return_value=engine),
patch("agent.model_metadata.get_model_context_length", return_value=1_000_000),
patch("run_agent.get_tool_definitions", return_value=[]),
patch("run_agent.check_toolset_requirements", return_value={}),
patch("run_agent.OpenAI"),
):
from run_agent import AIAgent
agent = AIAgent(
model="glm-5.2",
api_key="test-key-1234567890",
base_url="https://openrouter.ai/api/v1",
quiet_mode=True,
skip_context_files=True,
skip_memory=True,
)
assert agent.context_compressor is engine
assert getattr(engine, "model_thresholds", {}) == {}
assert engine.threshold_percent == 0.50
assert engine.threshold_tokens == int(1_000_000 * 0.50)
def test_model_thresholds_key_in_default_config():
"""compression.model_thresholds is a public DEFAULT_CONFIG key."""
from hermes_cli.config import DEFAULT_CONFIG
assert "model_thresholds" in DEFAULT_CONFIG["compression"]
assert DEFAULT_CONFIG["compression"]["model_thresholds"] == {}
class TestFloorInteractionOnModelSwitch:
"""The small-context floor stacks on per-model overrides at switch time."""
@patch("agent.context_compressor.get_model_context_length")
def test_switch_override_below_floor_is_raised_to_floor(self, mock_ctx):
"""Switching to a small-context model with a sub-floor override → floor."""
mock_ctx.return_value = 1_000_000
cc = ContextCompressor(
model="glm-5.2-1M",
threshold_percent=0.50,
model_thresholds={"glm-5.2-1M": 0.25, "small-model": 0.40},
quiet_mode=True,
)
assert cc.threshold_percent == 0.25 # large context: override direct
# Switch to a <512K model whose override (0.40) is below the 0.75 floor.
mock_ctx.return_value = 128_000
cc.update_model(model="small-model", context_length=128_000)
assert cc.threshold_percent == 0.75 # raise-only floor wins
assert cc.threshold_tokens == int(128_000 * 0.75)
@patch("agent.context_compressor.get_model_context_length")
def test_switch_override_above_floor_wins(self, mock_ctx):
"""Switching to a small-context model with an above-floor override → override."""
mock_ctx.return_value = 1_000_000
cc = ContextCompressor(
model="glm-5.2-1M",
threshold_percent=0.50,
model_thresholds={"glm-5.2-1M": 0.25, "small-model": 0.85},
quiet_mode=True,
)
mock_ctx.return_value = 128_000
cc.update_model(model="small-model", context_length=128_000)
assert cc.threshold_percent == 0.85 # above the 0.75 floor: override wins
assert cc.threshold_tokens == int(128_000 * 0.85)
class TestBaseEngineConfigSnapshot:
"""Base-class update_model() must not compound a previous override."""
def test_repeated_switches_fall_back_to_original_threshold(self):
engine = _StubEngine()
engine.threshold_percent = 0.50
engine.context_length = 0
engine.model_thresholds = {"glm-5.2-1M": 0.25}
# NOTE: _config_threshold_percent deliberately NOT pre-set — the base
# class must snapshot the original 0.50 on the first call, so the
# second switch (no matching override) falls back to 0.50, not 0.25.
engine.update_model(model="glm-5.2-1M", context_length=1_000_000)
assert engine.threshold_percent == 0.25
engine.update_model(model="some-other-model", context_length=1_000_000)
assert engine.threshold_percent == 0.50
assert engine.threshold_tokens == int(1_000_000 * 0.50)
@@ -0,0 +1,206 @@
"""Behavioral regression tests for the post-tool compression attempt cap.
The pre-API pressure gate, the overflow/413 error handlers, and the post-tool
compaction gate all share ``compression_attempts`` as a per-turn backstop,
bounded by the resolved ``compression.max_attempts`` cap (default 3). Before
the fix the post-tool path neither checked nor incremented the counter, so a
long tool loop could compact after every tool response for the lifetime of
the turn.
These tests drive ``run_conversation()`` through real tool iterations with a
compressor that always demands compression and assert ``_compress_context``
fires at most ``max_compression_attempts`` times per turn — no source
inspection, only observable behavior.
"""
from __future__ import annotations
import json
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from run_agent import AIAgent
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _tool_call(i: int):
return SimpleNamespace(
id=f"call_{i}",
type="function",
function=SimpleNamespace(name="web_search", arguments='{"query": "x"}'),
)
def _tool_response(i: int):
msg = SimpleNamespace(
content=None,
reasoning_content=None,
reasoning=None,
tool_calls=[_tool_call(i)],
)
choice = SimpleNamespace(message=msg, finish_reason="tool_calls")
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
def _stop_response():
msg = SimpleNamespace(
content="done",
reasoning_content=None,
reasoning=None,
tool_calls=None,
)
choice = SimpleNamespace(message=msg, finish_reason="stop")
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
def _make_tool_defs(*names: str) -> list:
return [
{
"type": "function",
"function": {
"name": n,
"description": f"{n} tool",
"parameters": {"type": "object", "properties": {}},
},
}
for n in names
]
def _pressured_compressor() -> MagicMock:
"""A compressor that always reports context pressure after tools run.
``should_defer_preflight_to_real_usage`` returns True so the turn-start
preflight and the pre-API pressure gate stand down — isolating the
post-tool gate as the only compression site under test.
"""
compressor = MagicMock()
compressor.protect_first_n = 3
compressor.protect_last_n = 20
compressor.threshold_tokens = 10_000
compressor.context_length = 200_000
compressor.last_prompt_tokens = 150_000
compressor.should_compress.return_value = True
compressor.should_defer_preflight_to_real_usage.return_value = True
compressor.get_active_compression_failure_cooldown.return_value = None
return compressor
@pytest.fixture()
def agent():
with (
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search")),
patch("run_agent.check_toolset_requirements", return_value={}),
patch("run_agent.OpenAI"),
):
a = AIAgent(
api_key="test-key-1234567890",
base_url="https://openrouter.ai/api/v1",
quiet_mode=True,
skip_context_files=True,
skip_memory=True,
max_iterations=10,
)
a.client = MagicMock()
a._cached_system_prompt = "You are helpful."
a._use_prompt_caching = False
a._disable_streaming = True
a.tool_delay = 0
a.save_trajectories = False
a.compression_enabled = True
a.context_compressor = _pressured_compressor()
return a
def _run_tool_loop(agent, n_tool_iterations: int):
"""Drive one turn: ``n_tool_iterations`` tool calls, then a stop."""
responses = [_tool_response(i) for i in range(n_tool_iterations)]
responses.append(_stop_response())
agent.client.chat.completions.create.side_effect = responses
compress_calls = []
def _fake_compress(messages, system_message, **_kwargs):
compress_calls.append(len(messages))
return messages, "compressed prompt"
with (
patch.object(agent, "_compress_context", side_effect=_fake_compress),
patch.object(agent, "_persist_session"),
patch.object(agent, "_save_trajectory"),
patch.object(agent, "_cleanup_task_resources"),
patch(
"run_agent.handle_function_call",
lambda name, args, task_id=None, **kwargs: json.dumps({"ok": True}),
),
):
result = agent.run_conversation("do a lot of tool work")
return result, compress_calls
# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
class TestPostToolCompressionAttemptCap:
def test_post_tool_compression_capped_at_default_three(self, agent):
"""7 tool iterations under constant pressure → exactly 3 compactions.
Before the fix the post-tool gate re-fired after every tool response
(7 compactions here); the shared per-turn counter caps it at the
resolved default of 3.
"""
assert agent.max_compression_attempts == 3 # config default
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=7)
assert result["completed"] is True
assert len(compress_calls) == 3, (
f"post-tool compression must stop at the per-turn cap (3), "
f"got {len(compress_calls)} compactions"
)
def test_post_tool_compression_honors_configured_cap(self, agent):
"""A raised compression.max_attempts cap lets more rounds run."""
agent.max_compression_attempts = 5
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=8)
assert result["completed"] is True
assert len(compress_calls) == 5
def test_post_tool_compression_shares_counter_with_pre_api_gate(self, agent):
"""Pre-API compactions consume the same per-turn budget.
Let the pre-API pressure gate fire once (defer disabled for the first
check), then keep the pressure on through tool iterations: the
combined total must still respect the cap.
"""
# First pre-API check does not defer → pre-API gate fires once;
# afterwards defer again so only the post-tool gate keeps firing.
defers = iter([False])
agent.context_compressor.should_defer_preflight_to_real_usage.side_effect = (
lambda _t: next(defers, True)
)
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=7)
assert result["completed"] is True
assert len(compress_calls) == 3, (
"pre-API and post-tool compactions must share one per-turn "
f"attempt budget, got {len(compress_calls)} total compactions"
)
def test_cap_is_per_turn_not_per_session(self, agent):
"""A fresh turn gets a fresh attempt budget."""
_result, first = _run_tool_loop(agent, n_tool_iterations=5)
agent.client.chat.completions.create.side_effect = None
_result, second = _run_tool_loop(agent, n_tool_iterations=5)
assert len(first) == 3
assert len(second) == 3
@@ -0,0 +1,186 @@
"""E2E: compression.max_attempts=6 drives a 4th+ preflight compaction pass.
The turn-start preflight loop in ``agent/turn_context.py`` was hardcoded to
``range(3)``: even when every pass made real progress and the request stayed
over threshold, the 4th pass never ran, regardless of configuration. The
loop now sizes itself from the same resolved ``compression.max_attempts`` cap
as the conversation loop's compression sites.
This test builds a real ``AIAgent`` from a config with
``compression.max_attempts: 6`` (the config-driven path through
``agent_init``), then drives a full ``run_conversation()`` turn in which the
estimated request size keeps shrinking ~10% per compaction but stays above
threshold — the exact "progress, but not enough yet" shape that legitimately
needs more than three rounds. With cap=6 the preflight must run a 4th pass
(and ultimately all six).
"""
from __future__ import annotations
import contextlib
import io
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
from hermes_state import SessionDB
from run_agent import AIAgent
def _config(max_attempts) -> dict:
return {
"compression": {
"enabled": True,
"threshold": 0.50,
"target_ratio": 0.20,
"protect_first_n": 3,
"protect_last_n": 20,
"max_attempts": max_attempts,
},
"prompt_caching": {"cache_ttl": "5m"},
"sessions": {},
"bedrock": {},
}
def _stop_response():
msg = SimpleNamespace(
content="done",
reasoning_content=None,
reasoning=None,
tool_calls=None,
)
choice = SimpleNamespace(message=msg, finish_reason="stop")
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
def _make_agent(monkeypatch, tmp_path: Path, *, max_attempts) -> AIAgent:
from hermes_cli import config as config_mod
monkeypatch.setattr(
config_mod, "load_config", lambda: _config(max_attempts)
)
db = SessionDB(db_path=tmp_path / "state.db")
with (
contextlib.redirect_stdout(io.StringIO()),
patch("run_agent.get_tool_definitions", return_value=[]),
patch("run_agent.check_toolset_requirements", return_value={}),
patch("run_agent.OpenAI"),
):
agent = AIAgent(
base_url="https://openrouter.ai/api/v1",
api_key="test-key",
model="test/model",
enabled_toolsets=[],
disabled_toolsets=[],
quiet_mode=True,
skip_memory=True,
skip_context_files=True,
session_db=db,
session_id="preflight-cap-e2e",
)
agent.client = MagicMock()
agent._cached_system_prompt = "You are helpful."
agent._use_prompt_caching = False
agent._disable_streaming = True
agent.tool_delay = 0
agent.save_trajectories = False
return agent
def test_preflight_runs_fourth_compaction_pass_at_cap_six(monkeypatch, tmp_path):
agent = _make_agent(monkeypatch, tmp_path, max_attempts=6)
# Config-driven attach seam (agent_init) resolved the raised cap.
assert agent.max_compression_attempts == 6
# Keep the request permanently over threshold while every compaction
# makes material (~10% > the 5% progress floor) headway.
compressor = agent.context_compressor
compressor.threshold_tokens = 50_000
estimate_state = {"tokens": 1_000_000.0, "calls": 0}
def _shrinking_estimate(*_args, **_kwargs):
if estimate_state["calls"]:
estimate_state["tokens"] *= 0.9
estimate_state["calls"] += 1
return int(estimate_state["tokens"])
compress_calls = []
def _fake_compress(messages, system_message, **_kwargs):
compress_calls.append(len(messages))
return messages, "compressed prompt"
# 60 messages > protect_first_n + protect_last_n + 1, so the cheap
# preflight count gate opens without patching internals.
history = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"msg {i}"}
for i in range(60)
]
agent.client.chat.completions.create.return_value = _stop_response()
with (
patch(
"agent.turn_context.estimate_request_tokens_rough",
side_effect=_shrinking_estimate,
),
patch.object(agent, "_compress_context", side_effect=_fake_compress),
patch.object(agent, "_persist_session"),
patch.object(agent, "_save_trajectory"),
patch.object(agent, "_cleanup_task_resources"),
):
result = agent.run_conversation("hello", conversation_history=history)
assert result["completed"] is True
# The old hardcoded range(3) made a 4th pass impossible; cap=6 must
# deliver it (and, with steady progress over threshold, all six).
assert len(compress_calls) >= 4, (
f"expected a 4th preflight compaction pass at cap=6, "
f"got {len(compress_calls)} passes"
)
assert len(compress_calls) == 6
def test_preflight_still_stops_at_default_three(monkeypatch, tmp_path):
"""Unset compression.max_attempts keeps the historical 3-pass behavior."""
agent = _make_agent(monkeypatch, tmp_path, max_attempts=None)
assert agent.max_compression_attempts == 3
compressor = agent.context_compressor
compressor.threshold_tokens = 50_000
estimate_state = {"tokens": 1_000_000.0, "calls": 0}
def _shrinking_estimate(*_args, **_kwargs):
if estimate_state["calls"]:
estimate_state["tokens"] *= 0.9
estimate_state["calls"] += 1
return int(estimate_state["tokens"])
compress_calls = []
def _fake_compress(messages, system_message, **_kwargs):
compress_calls.append(len(messages))
return messages, "compressed prompt"
history = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"msg {i}"}
for i in range(60)
]
agent.client.chat.completions.create.return_value = _stop_response()
with (
patch(
"agent.turn_context.estimate_request_tokens_rough",
side_effect=_shrinking_estimate,
),
patch.object(agent, "_compress_context", side_effect=_fake_compress),
patch.object(agent, "_persist_session"),
patch.object(agent, "_save_trajectory"),
patch.object(agent, "_cleanup_task_resources"),
):
result = agent.run_conversation("hello", conversation_history=history)
assert result["completed"] is True
assert len(compress_calls) == 3
@@ -0,0 +1,273 @@
"""Behavior tests for scripts/validate_plugin_catalog.py.
The script is the no-install structural validator used by the plugin-catalog
admission CI: it must run with only stdlib + pyyaml, take file paths or a
directory, exit 0/1, and support --json machine output. These tests exercise
the CLI contract via subprocess (the same way CI invokes it).
"""
import json
import subprocess
import sys
from pathlib import Path
import yaml
REPO_ROOT = Path(__file__).resolve().parents[2]
SCRIPT = REPO_ROOT / "scripts" / "validate_plugin_catalog.py"
VALID_ENTRY = {
"name": "example-plugin",
"repo": "https://github.com/NousResearch/hermes-example-plugins",
"sha": "38fe0fb53eff98d477f807432e965429e665ca33",
"subdir": "",
"description": "One-line description.",
"maintainer": "NousResearch",
"tier": "official",
"requires_hermes": ">=0.19",
"docs_url": "",
"platforms": [],
"capabilities": {
"provides_tools": ["example_tool"],
"provides_hooks": [],
"provides_middleware": [],
"requires_env": [],
},
}
def write_entry(tmp_path: Path, data: dict, filename: str | None = None) -> Path:
name = filename or f"{data.get('name', 'entry')}.yaml"
path = tmp_path / name
path.write_text(yaml.safe_dump(data), encoding="utf-8")
return path
def run_validator(*args: str) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(SCRIPT), *args],
capture_output=True,
text=True,
)
# ── valid input ────────────────────────────────────────────────────────
def test_valid_entry_passes(tmp_path):
path = write_entry(tmp_path, VALID_ENTRY)
result = run_validator(str(path))
assert result.returncode == 0, result.stdout + result.stderr
def test_valid_entry_without_optional_fields_passes(tmp_path):
entry = {
"name": "minimal-plugin",
"repo": "https://github.com/example/minimal",
"sha": "a" * 40,
"description": "Minimal.",
"maintainer": "someone",
}
path = write_entry(tmp_path, entry)
result = run_validator(str(path))
assert result.returncode == 0, result.stdout + result.stderr
# ── each malformed field fails with a pointed error ────────────────────
def _expect_error(tmp_path, mutation: dict, expected_substring: str, drop: str = ""):
entry = {**VALID_ENTRY, **mutation}
if drop:
entry.pop(drop, None)
path = write_entry(tmp_path, entry, filename="entry.yaml")
result = run_validator(str(path))
combined = result.stdout + result.stderr
assert result.returncode == 1, combined
assert expected_substring in combined, combined
assert "entry.yaml" in combined, combined
def test_bad_name_fails(tmp_path):
_expect_error(tmp_path, {"name": "Bad Name!"}, "name")
def test_name_too_long_fails(tmp_path):
_expect_error(tmp_path, {"name": "x" * 65}, "name")
def test_non_https_repo_fails(tmp_path):
_expect_error(tmp_path, {"repo": "git@github.com:evil/x.git"}, "repo")
def test_short_sha_fails(tmp_path):
_expect_error(tmp_path, {"sha": "abc123"}, "sha")
def test_non_hex_sha_fails(tmp_path):
_expect_error(tmp_path, {"sha": "z" * 40}, "sha")
def test_bad_tier_fails(tmp_path):
_expect_error(tmp_path, {"tier": "platinum"}, "tier")
def test_empty_description_fails(tmp_path):
_expect_error(tmp_path, {"description": ""}, "description")
def test_empty_maintainer_fails(tmp_path):
_expect_error(tmp_path, {"maintainer": ""}, "maintainer")
def test_missing_required_field_fails(tmp_path):
_expect_error(tmp_path, {}, "sha", drop="sha")
def test_capabilities_value_not_a_list_fails(tmp_path):
_expect_error(
tmp_path,
{"capabilities": {"provides_tools": "not-a-list"}},
"provides_tools",
)
def test_capabilities_list_of_non_strings_fails(tmp_path):
_expect_error(
tmp_path,
{"capabilities": {"requires_env": [1, 2]}},
"requires_env",
)
def test_bad_requires_hermes_spec_fails(tmp_path):
_expect_error(tmp_path, {"requires_hermes": "banana"}, "requires_hermes")
def test_comma_separated_requires_hermes_passes(tmp_path):
entry = {**VALID_ENTRY, "requires_hermes": ">=0.19, <2.0"}
path = write_entry(tmp_path, entry)
result = run_validator(str(path))
assert result.returncode == 0, result.stdout + result.stderr
def test_unknown_platform_fails(tmp_path):
_expect_error(tmp_path, {"platforms": ["linux", "amiga"]}, "platforms")
def test_entry_not_a_mapping_fails(tmp_path):
path = tmp_path / "entry.yaml"
path.write_text("- just\n- a\n- list\n", encoding="utf-8")
result = run_validator(str(path))
assert result.returncode == 1
assert "mapping" in (result.stdout + result.stderr)
# ── unknown top-level keys warn but do not fail ────────────────────────
def test_unknown_key_warns_but_passes(tmp_path):
entry = {**VALID_ENTRY, "future_field": "hello"}
path = write_entry(tmp_path, entry)
result = run_validator(str(path))
combined = result.stdout + result.stderr
assert result.returncode == 0, combined
assert "future_field" in combined
assert "warning" in combined.lower()
# ── removed.yaml shape ─────────────────────────────────────────────────
def test_valid_removed_yaml_passes(tmp_path):
path = tmp_path / "removed.yaml"
path.write_text(
yaml.safe_dump(
{
"removed": [
{
"name": "some-plugin",
"repo": "https://github.com/evil/some-plugin",
"reason": "Exfiltrated env vars",
"date": "2026-07-02",
}
]
}
),
encoding="utf-8",
)
result = run_validator(str(path))
assert result.returncode == 0, result.stdout + result.stderr
def test_removed_yaml_not_a_list_fails(tmp_path):
path = tmp_path / "removed.yaml"
path.write_text(yaml.safe_dump({"removed": "nope"}), encoding="utf-8")
result = run_validator(str(path))
assert result.returncode == 1
assert "removed" in (result.stdout + result.stderr)
def test_removed_item_missing_name_fails(tmp_path):
path = tmp_path / "removed.yaml"
path.write_text(
yaml.safe_dump({"removed": [{"reason": "bad", "date": "2026-01-01"}]}),
encoding="utf-8",
)
result = run_validator(str(path))
assert result.returncode == 1
assert "name" in (result.stdout + result.stderr)
# ── --json machine output ──────────────────────────────────────────────
def test_json_output_shape_on_failure(tmp_path):
bad = write_entry(tmp_path, {**VALID_ENTRY, "sha": "short"}, filename="bad.yaml")
result = run_validator("--json", str(bad))
assert result.returncode == 1
payload = json.loads(result.stdout)
assert payload["ok"] is False
assert isinstance(payload["files"], list)
entry = next(f for f in payload["files"] if f["path"].endswith("bad.yaml"))
assert entry["ok"] is False
assert any("sha" in e for e in entry["errors"])
def test_json_output_shape_on_success_with_warning(tmp_path):
good = write_entry(tmp_path, {**VALID_ENTRY, "future_field": 1})
result = run_validator("--json", str(good))
assert result.returncode == 0
payload = json.loads(result.stdout)
assert payload["ok"] is True
(entry,) = payload["files"]
assert entry["ok"] is True
assert entry["errors"] == []
assert any("future_field" in w for w in entry["warnings"])
# ── directory mode ─────────────────────────────────────────────────────
def test_directory_mode_validates_all_entries_and_removed(tmp_path):
write_entry(tmp_path, VALID_ENTRY)
write_entry(tmp_path, {**VALID_ENTRY, "name": "bad-one", "sha": "nope"})
(tmp_path / "removed.yaml").write_text(
yaml.safe_dump({"removed": [{"name": "gone", "reason": "test"}]}),
encoding="utf-8",
)
result = run_validator(str(tmp_path))
combined = result.stdout + result.stderr
assert result.returncode == 1
assert "bad-one.yaml" in combined
# the valid entry and removed.yaml must not produce errors
assert combined.count("ERROR") == combined.count("bad-one.yaml: ERROR")
def test_directory_mode_all_valid_exits_zero(tmp_path):
write_entry(tmp_path, VALID_ENTRY)
(tmp_path / "removed.yaml").write_text(
yaml.safe_dump({"removed": []}), encoding="utf-8"
)
result = run_validator(str(tmp_path))
assert result.returncode == 0, result.stdout + result.stderr
+62 -1
View File
@@ -1,5 +1,8 @@
from contextlib import nullcontext
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
)
from cli import HermesCLI
@@ -9,8 +12,33 @@ class DummyAgent:
self._cached_system_prompt = "FULL CACHED SYSTEM PROMPT SHOULD NOT BE NESTED"
self.session_id = "new-session"
self.calls = []
self.flush_calls = []
self.flush_error = None
self.host_events = []
self.boundary_calls = []
self.context_compressor = type("ContextEngineStub", (), {})()
self.context_compressor.on_session_start = self._record_boundary
def _compress_context(self, messages, system_message, *, approx_tokens=None, focus_topic=None, force=False):
def _record_boundary(self, session_id, **kwargs):
self.host_events.append("notify")
self.boundary_calls.append((session_id, kwargs))
def _flush_messages_to_session_db(self, messages, _session_id=None):
self.host_events.append("persist")
self.flush_calls.append((list(messages), _session_id))
if self.flush_error is not None:
raise self.flush_error
def _compress_context(
self,
messages,
system_message,
*,
approx_tokens=None,
focus_topic=None,
force=False,
defer_context_engine_notification=False,
):
self.calls.append(
{
"messages": messages,
@@ -18,8 +46,17 @@ class DummyAgent:
"approx_tokens": approx_tokens,
"focus_topic": focus_topic,
"force": force,
"defer_context_engine_notification": (
defer_context_engine_notification
),
}
)
if defer_context_engine_notification:
_queue_context_engine_compression_notification(
self,
new_session_id=self.session_id,
old_session_id="old-session",
)
return ([{"role": "user", "content": "[CONTEXT SUMMARY]: compacted"}], "new system prompt")
@@ -56,3 +93,27 @@ def test_manual_compress_does_not_pass_cached_system_prompt(monkeypatch):
assert call["focus_topic"] == "database schema"
assert cli.session_id == "new-session"
assert cli._pending_title is None
assert len(cli.agent.flush_calls) == 1
assert cli.agent.host_events == ["persist", "notify"]
assert len(cli.agent.boundary_calls) == 1
def test_manual_compress_flush_failure_discards_notification(monkeypatch):
cli = HermesCLI.__new__(HermesCLI)
cli.conversation_history = [
{"role": "user", "content": "one"},
{"role": "assistant", "content": "two"},
{"role": "user", "content": "three"},
{"role": "assistant", "content": "four"},
]
cli.agent = DummyAgent()
cli.agent.flush_error = RuntimeError("synthetic child flush failure")
cli.session_id = "old-session"
cli._pending_title = "old title"
cli._busy_command = lambda _message: nullcontext()
cli._manual_compress("/compress")
assert len(cli.agent.flush_calls) == 1
assert cli.agent.host_events == ["persist"]
assert cli.agent.boundary_calls == []
+341
View File
@@ -139,6 +139,117 @@ class TestSessionLifecycle:
assert session["cwd"] == "/work/repo"
assert session["git_branch"] == "pets-feature"
def test_child_session_inherits_cwd_and_git_repo_root_from_parent(self, db):
"""A parent_session_id child born without cwd/git_repo_root (e.g. the
compression-fork path) must inherit both from its parent, so it
doesn't silently drop out of the project sidebar (#64709)."""
db.create_session(session_id="parent", source="cli")
db.update_session_cwd("parent", "/work/repo", git_repo_root="/work/repo")
db.create_session(session_id="child", source="cli", parent_session_id="parent")
child = db.get_session("child")
assert child["cwd"] == "/work/repo"
assert child["git_repo_root"] == "/work/repo"
def test_child_session_explicit_cwd_is_not_overwritten_by_parent(self, db):
"""A child that explicitly sets its own cwd/git_repo_root keeps it —
parent inheritance only fills in NULLs, never clobbers."""
db.create_session(session_id="parent", source="cli")
db.update_session_cwd("parent", "/work/repo-a", git_repo_root="/work/repo-a")
db.create_session(
session_id="child", source="cli", parent_session_id="parent",
cwd="/work/repo-b", git_repo_root="/work/repo-b",
)
child = db.get_session("child")
assert child["cwd"] == "/work/repo-b"
assert child["git_repo_root"] == "/work/repo-b"
def test_multi_generation_lineage_inherits_cwd(self, db):
"""cwd/git_repo_root propagate through a multi-hop compression chain
(root -> gen1 -> gen2), mirroring the multi-generation lineage from
the reported issue where a single conversation forked repeatedly."""
db.create_session(session_id="root", source="cli")
db.update_session_cwd("root", "/work/repo", git_repo_root="/work/repo")
db.create_session(session_id="gen1", source="cli", parent_session_id="root")
db.create_session(session_id="gen2", source="cli", parent_session_id="gen1")
assert db.get_session("gen1")["cwd"] == "/work/repo"
assert db.get_session("gen2")["cwd"] == "/work/repo"
def test_child_session_inherits_git_branch_from_parent(self, db):
"""git_branch travels with cwd/git_repo_root — the Desktop sidebar
shows the branch chip per session, so a compression child born
without it loses the chip even though the workspace didn't change."""
db.create_session(session_id="parent", source="cli")
db.update_session_cwd(
"parent", "/work/repo", git_branch="feature-x", git_repo_root="/work/repo"
)
db.create_session(session_id="child", source="cli", parent_session_id="parent")
child = db.get_session("child")
assert child["git_branch"] == "feature-x"
def test_compression_child_inherits_gateway_origin_columns(self, db):
"""A compression fork's child inherits gateway routing metadata
(session_key/chat_id/...) from the ended parent, so a crash before
the gateway re-records the peer can't strand it (#59527)."""
db.create_session(
session_id="parent", source="telegram",
user_id="u1", session_key="telegram:u1:c1",
chat_id="c1", chat_type="private", thread_id="t1",
)
db.record_gateway_session_peer(
"parent", source="telegram", user_id="u1",
session_key="telegram:u1:c1", chat_id="c1", chat_type="private",
thread_id="t1", display_name="Chat One", origin_json='{"p":"telegram"}',
)
# Rotation path: parent is ended with 'compression' BEFORE the child
# row is created (agent/conversation_compression.py).
db.end_session("parent", "compression")
db.create_session(
session_id="child", source="telegram", parent_session_id="parent"
)
child = db.get_session("child")
assert child["user_id"] == "u1"
assert child["session_key"] == "telegram:u1:c1"
assert child["chat_id"] == "c1"
assert child["chat_type"] == "private"
assert child["thread_id"] == "t1"
assert child["display_name"] == "Chat One"
assert child["origin_json"] == '{"p":"telegram"}'
def test_live_parent_child_does_not_inherit_gateway_origin(self, db):
"""Delegate/subagent children (parent still live) must NOT inherit
routing keys — peer recovery could otherwise repoint gateway traffic
into a subagent's session."""
db.create_session(
session_id="parent", source="telegram",
user_id="u1", session_key="telegram:u1:c1",
chat_id="c1", chat_type="private",
)
db.create_session(
session_id="sub", source="telegram", parent_session_id="parent"
)
sub = db.get_session("sub")
assert sub["session_key"] is None
assert sub["chat_id"] is None
assert sub["user_id"] is None
# Workspace metadata still inherits — that part is safe for any child.
db.update_session_cwd("parent", "/work/repo", git_repo_root="/work/repo")
db.create_session(
session_id="sub2", source="telegram", parent_session_id="parent"
)
assert db.get_session("sub2")["cwd"] == "/work/repo"
def test_update_session_cwd_empty_branch_does_not_clobber(self, db):
"""A failed branch probe (empty string) must not wipe a branch we
already captured — only the cwd updates."""
@@ -1639,6 +1750,236 @@ class TestMessageStorage:
assert conv[0]["codex_reasoning_items"][0]["encrypted_content"] == "enc_blob_123"
# =========================================================================
# Timestamp preservation
# =========================================================================
class TestTimestampPreservation:
"""Tests for the timestamp preservation feature.
``append_message()`` and ``replace_messages()`` now accept/forward an
optional ``timestamp`` parameter. These tests verify custom timestamps
survive the round trip through the DB and fall back to ``time.time()``
when omitted.
"""
@staticmethod
def _build_messages(ts_list, contents=None, roles=None):
"""Build message dicts with explicit timestamps for testing."""
if contents is None:
contents = [f"msg-{i}" for i in range(len(ts_list))]
if roles is None:
roles = ["user", "assistant"] * (len(ts_list) // 2 + 1)
return [
{"role": roles[i], "content": contents[i], "timestamp": ts}
for i, ts in enumerate(ts_list)
]
def _raw_timestamps(self, db, session_id):
"""Query timestamp column directly from SQLite for verification."""
rows = db._conn.execute(
"SELECT timestamp FROM messages WHERE session_id = ? ORDER BY id",
(session_id,),
).fetchall()
return [r[0] for r in rows]
def test_append_message_with_explicit_timestamp(self, db):
"""A caller-supplied timestamp is stored and round-tripped."""
db.create_session(session_id="s1", source="cli")
ts = 1_234_567.0
mid = db.append_message("s1", role="user", content="hello",
timestamp=ts)
msgs = db.get_messages("s1")
assert len(msgs) == 1
assert msgs[0]["timestamp"] == ts
assert msgs[0]["id"] == mid
raw = self._raw_timestamps(db, "s1")
assert raw == [ts]
def test_append_message_multiple_timestamps(self, db):
"""Multiple messages with different explicit timestamps."""
db.create_session(session_id="s1", source="cli")
timestamps = [1_000_000.0, 2_000_000.0, 3_000_000.0]
for i, ts in enumerate(timestamps, 1):
db.append_message("s1", role="user", content=f"msg {i}",
timestamp=ts)
msgs = db.get_messages("s1")
assert [m["timestamp"] for m in msgs] == timestamps
assert self._raw_timestamps(db, "s1") == timestamps
def test_append_message_without_timestamp_defaults(self, db):
"""Omitting timestamp stores a recent time.time() value."""
db.create_session(session_id="s1", source="cli")
before = time.time()
db.append_message("s1", role="user", content="hello")
after = time.time()
msgs = db.get_messages("s1")
stored = msgs[0]["timestamp"]
assert before <= stored <= after, (
f"Expected timestamp between {before} and {after}, got {stored}"
)
raw_stored = self._raw_timestamps(db, "s1")[0]
assert before <= raw_stored <= after
def test_append_message_mixed_timestamps(self, db):
"""Messages with and without explicit timestamps — those without
get a current time, those with keep their value."""
db.create_session(session_id="s1", source="cli")
explicit_ts = 500_000.0
db.append_message("s1", role="user", content="explicit",
timestamp=explicit_ts)
before = time.time()
db.append_message("s1", role="user", content="default")
after = time.time()
msgs = db.get_messages("s1")
assert msgs[0]["timestamp"] == explicit_ts
assert before <= msgs[1]["timestamp"] <= after
raw = self._raw_timestamps(db, "s1")
assert raw[0] == explicit_ts
assert before <= raw[1] <= after
def test_replace_messages_preserves_timestamps(self, db):
"""Message dicts with ``timestamp`` passed to ``replace_messages``
retain those timestamps after the rewrite."""
db.create_session(session_id="s1", source="cli")
msgs_in = [
{"role": "user", "content": "first", "timestamp": 100.0},
{"role": "assistant", "content": "second", "timestamp": 200.0},
{"role": "user", "content": "third", "timestamp": 300.0},
]
db.replace_messages("s1", msgs_in)
msgs_out = db.get_messages("s1")
assert [m["timestamp"] for m in msgs_out] == [100.0, 200.0, 300.0]
assert self._raw_timestamps(db, "s1") == [100.0, 200.0, 300.0]
def test_replace_messages_fallback_when_no_timestamp(self, db):
"""Message dicts without ``timestamp`` get auto-incrementing
fallback values (starting from ~time.time())."""
db.create_session(session_id="s1", source="cli")
db.replace_messages("s1", [
{"role": "user", "content": "a"},
{"role": "user", "content": "b"},
])
msgs = db.get_messages("s1")
assert len(msgs) == 2
t0, t1 = msgs[0]["timestamp"], msgs[1]["timestamp"]
assert t1 > t0
raw = self._raw_timestamps(db, "s1")
assert len(raw) == 2
assert raw[1] > raw[0]
def test_replace_messages_mixed_timestamps(self, db):
"""Some messages with timestamp, some without — a message without
timestamp uses the fallback clock, which is larger than any
explicit historical timestamp."""
db.create_session(session_id="s1", source="cli")
old_ts = 1_000.0
db.replace_messages("s1", [
{"role": "user", "content": "old", "timestamp": old_ts},
{"role": "user", "content": "new"},
])
msgs = db.get_messages("s1")
assert msgs[0]["timestamp"] == old_ts
assert msgs[1]["timestamp"] > old_ts
raw = self._raw_timestamps(db, "s1")
assert raw[0] == old_ts
assert raw[1] > old_ts
def test_fork_chain_preserves_timestamps(self, db):
"""Simulate a /branch fork: copy messages from parent to child,
verify timestamps are identical in both via raw SQL."""
base_ts = 1_700_000_000.0
timestamps = [base_ts + i * 20 for i in range(5)]
contents = [
"how do I fix a TypeError?",
"show me the traceback",
"TypeError at line 42",
"issue in utils.py",
"try int(...)",
]
roles = ["user", "assistant", "tool", "user", "assistant"]
parent_msgs = self._build_messages(timestamps, contents, roles)
db.create_session(session_id="parent", source="cli")
for msg in parent_msgs:
db.append_message("parent", role=msg["role"],
content=msg["content"],
timestamp=msg["timestamp"])
db.create_session(session_id="child", source="cli",
parent_session_id="parent")
for msg in parent_msgs:
db.append_message("child", role=msg["role"],
content=msg["content"],
timestamp=msg["timestamp"])
parent_raw = self._raw_timestamps(db, "parent")
child_raw = self._raw_timestamps(db, "child")
assert parent_raw == timestamps
assert child_raw == timestamps
assert parent_raw == child_raw
def test_branch_copy_roundtrip_preserves_timestamps(self, db):
"""End-to-end branch copy: load the parent transcript via
get_messages_as_conversation (the dict shape the CLI/gateway/TUI
branch loops iterate) and re-append into a child forwarding
``msg.get("timestamp")`` — the copies must keep the originals
instead of being restamped with time.time() (#28841).
"""
timestamps = [1_600_000_000.0, 1_600_000_060.0, 1_600_000_120.0]
db.create_session(session_id="parent", source="cli")
for i, ts in enumerate(timestamps):
db.append_message(
"parent",
role="user" if i % 2 == 0 else "assistant",
content=f"msg-{i}",
timestamp=ts,
)
history = db.get_messages_as_conversation("parent")
assert [m.get("timestamp") for m in history] == timestamps
db.create_session(session_id="child", source="cli",
parent_session_id="parent")
# Mirrors the branch copy loops in gateway/slash_commands.py,
# hermes_cli/cli_commands_mixin.py and tui_gateway/server.py.
for msg in history:
db.append_message(
"child",
role=msg.get("role", "user"),
content=msg.get("content"),
timestamp=msg.get("timestamp"),
)
assert self._raw_timestamps(db, "child") == timestamps
def test_compression_replace_roundtrip_preserves_timestamps(self, db):
"""Compression-style rewrite: replace_messages with dicts loaded from
get_messages_as_conversation must keep the surviving messages'
original timestamps (#28841)."""
timestamps = [1_500_000_000.0, 1_500_000_100.0, 1_500_000_200.0]
db.create_session(session_id="s1", source="cli")
for i, ts in enumerate(timestamps):
db.append_message(
"s1",
role="user" if i % 2 == 0 else "assistant",
content=f"msg-{i}",
timestamp=ts,
)
history = db.get_messages_as_conversation("s1")
# Simulate a compression that keeps the last two turns verbatim and
# prepends a fresh summary message (no timestamp — falls back to now).
compressed = [{"role": "user", "content": "[summary]"}] + history[-2:]
db.replace_messages("s1", compressed)
raw = self._raw_timestamps(db, "s1")
assert len(raw) == 3
assert raw[1:] == timestamps[-2:]
assert raw[0] > timestamps[-1] # summary stamped with a current time
# =========================================================================
# FTS5 search
# =========================================================================
+94 -13
View File
@@ -5926,26 +5926,37 @@ def test_session_compress_reports_aborted_summary_without_success(monkeypatch):
def test_session_compress_syncs_session_key_after_rotation(monkeypatch):
"""When AIAgent._compress_context rotates session_id (compression split),
the gateway session_key must follow so subsequent approval routing,
DB title/history lookups, and slash worker resume target the new
continuation session — mirrors HermesCLI._manual_compress's
session_id sync (cli.py).
"""
agent = types.SimpleNamespace(session_id="rotated-id")
"""LCM notification follows the TUI's final session-key transition."""
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
)
events = []
agent = types.SimpleNamespace(
session_id="rotated-id",
context_compressor=types.SimpleNamespace(
on_session_start=lambda *_args, **_kwargs: events.append("notify")
),
)
server._sessions["sid"] = _session(agent=agent)
server._sessions["sid"]["session_key"] = "old-key"
server._sessions["sid"]["pending_title"] = "stale title"
monkeypatch.setattr(
server,
"_compress_session_history",
lambda session, focus_topic=None, **_kw: (2, {"total": 42}),
)
def _compress(session, focus_topic=None, **_kw):
_queue_context_engine_compression_notification(
session["agent"],
new_session_id="rotated-id",
old_session_id="old-key",
)
return 2, {"total": 42}
monkeypatch.setattr(server, "_compress_session_history", _compress)
monkeypatch.setattr(server, "_session_info", lambda _agent, *a: {"model": "x"})
restart_calls = []
monkeypatch.setattr(
server, "_restart_slash_worker", lambda sid, s: restart_calls.append(s)
server,
"_restart_slash_worker",
lambda sid, s: (restart_calls.append(s), events.append("sync")),
)
try:
@@ -5961,6 +5972,52 @@ def test_session_compress_syncs_session_key_after_rotation(monkeypatch):
assert server._sessions["sid"]["session_key"] == "rotated-id"
assert server._sessions["sid"]["pending_title"] is None
assert len(restart_calls) == 1
assert events == ["sync", "notify"]
finally:
server._sessions.pop("sid", None)
def test_session_compress_sync_failure_discards_lcm_notification(monkeypatch):
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
)
events = []
agent = types.SimpleNamespace(
session_id="rotated-id",
context_compressor=types.SimpleNamespace(
on_session_start=lambda *_args, **_kwargs: events.append("notify")
),
)
server._sessions["sid"] = _session(agent=agent)
server._sessions["sid"]["session_key"] = "old-key"
def _compress(session, focus_topic=None, **_kw):
_queue_context_engine_compression_notification(
session["agent"],
new_session_id="rotated-id",
old_session_id="old-key",
)
return 2, {"total": 42}
monkeypatch.setattr(server, "_compress_session_history", _compress)
monkeypatch.setattr(
server,
"_session_info",
lambda *_args: (_ for _ in ()).throw(RuntimeError("finalization failed")),
)
try:
with patch("tui_gateway.server._emit"):
resp = server.handle_request(
{
"id": "1",
"method": "session.compress",
"params": {"session_id": "sid"},
}
)
assert resp["error"]["code"] == 5005
assert events == []
finally:
server._sessions.pop("sid", None)
@@ -6418,6 +6475,30 @@ def test_commands_catalog_includes_tui_mouse_command():
assert "/mouse" in tui_pairs
def test_commands_catalog_has_no_duplicate_or_alias_colliding_names():
"""No command may be advertised twice, and no advertised command may
shadow an alias of a different command (e.g. the historical /compact
collision where the registry aliased compact -> compress while the TUI
also registered its own /compact display toggle; see #57133)."""
resp = server.handle_request(
{"id": "1", "method": "commands.catalog", "params": {}}
)
names = [name for name, _ in resp["result"]["pairs"]]
dupes = {n for n in names if names.count(n) > 1}
assert not dupes, f"duplicate commands advertised in catalog: {sorted(dupes)}"
canon = resp["result"]["canon"]
colliding = {
name
for name in names
if canon.get(name.lower(), name) != name
}
assert not colliding, (
f"catalog commands shadow aliases of other commands: {sorted(colliding)}"
)
def test_commands_catalog_filters_gateway_only_commands_and_keeps_status_visible():
resp = server.handle_request(
{"id": "1", "method": "commands.catalog", "params": {}}
+45 -1
View File
@@ -82,14 +82,58 @@ class TestClarifyPrimitive:
assert cm.wait_for_response("id3c", timeout=0.1) == "Y"
def test_resolve_text_response_accepts_custom_other_text(self):
"""Arbitrary typed text should resolve as a custom Other answer."""
"""Arbitrary typed text should resolve as a custom Other answer when awaiting_text is True."""
from tools import clarify_gateway as cm
cm.register("id3d", "sk3d", "Pick", ["X", "Y"])
# Flip to text-capture mode (user picked "Other")
cm.mark_awaiting_text("id3d")
custom = "None of those are valid options"
assert cm.resolve_text_response_for_session("sk3d", custom) is True
assert cm.wait_for_response("id3d", timeout=0.1) == custom
def test_resolve_text_rejects_arbitrary_prose_for_native_multi_choice(self):
"""Native interactive multi-choice clarifies reject arbitrary prose unless awaiting_text is True."""
from tools import clarify_gateway as cm
# Native multi-choice (buttons, not awaiting text)
cm.register("id-strict", "sk-strict", "Pick one", ["A", "B", "C"])
# Arbitrary prose should be rejected
assert cm.resolve_text_response_for_session("sk-strict", "just checking the visual UI") is False
assert cm.resolve_text_response_for_session("sk-strict", "present 3 buttons") is False
# Numeric choices should still work
assert cm.resolve_text_response_for_session("sk-strict", "2") is True
assert cm.wait_for_response("id-strict", timeout=0.1) == "B"
# Exact label match should still work
cm.register("id-strict2", "sk-strict2", "Pick", ["Option Alpha", "Option Beta"])
assert cm.resolve_text_response_for_session("sk-strict2", "Option Alpha") is True
assert cm.wait_for_response("id-strict2", timeout=0.1) == "Option Alpha"
def test_text_fallback_mode_allows_any_text(self):
"""Text fallback mode (after base send_clarify calls mark_awaiting_text) accepts any text."""
from tools import clarify_gateway as cm
entry = cm.register("id-tf", "sk-tf", "Pick one", ["A", "B", "C"])
assert entry.awaiting_text is False
# Simulate base send_clarify calling mark_awaiting_text
cm.mark_awaiting_text("id-tf")
assert entry.awaiting_text is True
# Now arbitrary text is accepted
custom = "I choose a custom answer"
assert cm.resolve_text_response_for_session("sk-tf", custom) is True
assert cm.wait_for_response("id-tf", timeout=0.1) == custom
# Numeric choices also work
cm.register("id-tf2", "sk-tf2", "Pick", ["X", "Y"])
cm.mark_awaiting_text("id-tf2")
assert cm.resolve_text_response_for_session("sk-tf2", "1") is True
assert cm.wait_for_response("id-tf2", timeout=0.1) == "X"
def test_other_button_flips_to_text_mode(self):
"""mark_awaiting_text makes get_pending_for_session find the entry."""
from tools import clarify_gateway as cm
+11 -3
View File
@@ -159,7 +159,7 @@ class TestChildSystemPrompt(unittest.TestCase):
class TestStripBlockedTools(unittest.TestCase):
def test_removes_blocked_toolsets(self):
result = _strip_blocked_tools(["terminal", "file", "delegation", "clarify", "memory", "code_execution"])
self.assertEqual(sorted(result), ["file", "terminal"])
self.assertEqual(sorted(result), ["code_execution", "file", "terminal"])
def test_preserves_allowed_toolsets(self):
result = _strip_blocked_tools(["terminal", "file", "web", "browser"])
@@ -235,10 +235,12 @@ class TestStripBlockedTools(unittest.TestCase):
"clarify",
"cronjob",
"delegation",
"code_execution",
"memory",
):
self.assertIn(toolset_name, disabled)
# code_execution is deliberately NOT denied — children keep
# execute_code for programmatic tool calling (Teknium, Jul 2026).
self.assertNotIn("code_execution", disabled)
definitions = model_tools.get_tool_definitions(
enabled_toolsets=kwargs["enabled_toolsets"],
@@ -1152,9 +1154,15 @@ class TestSubagentCostRollup(unittest.TestCase):
class TestBlockedTools(unittest.TestCase):
def test_blocked_tools_constant(self):
for tool in ["delegate_task", "clarify", "memory", "send_message", "execute_code"]:
for tool in ["delegate_task", "clarify", "memory", "send_message", "cronjob"]:
self.assertIn(tool, DELEGATE_BLOCKED_TOOLS)
def test_execute_code_not_blocked(self):
"""Children retain execute_code (programmatic tool calling) so they
can batch mechanical work instead of burning reasoning iterations
(Teknium, Jul 2026)."""
self.assertNotIn("execute_code", DELEGATE_BLOCKED_TOOLS)
def test_constants(self):
from tools.delegate_tool import (
_get_max_spawn_depth, _get_orchestrator_enabled,
+134
View File
@@ -638,3 +638,137 @@ class TestCronDemotion:
# Interactive rows first, in original relative order; cron last, in
# original relative order.
assert [r["id"] for r in ordered] == [2, 4, 5, 1, 3]
# =========================================================================
# Compaction summary filtering (#43175)
# =========================================================================
class TestCompactionSummaryFiltering:
"""session_search discovery must exclude compaction handoffs from bookends."""
def test_is_compaction_summary_detects_prefix(self):
from tools.session_search_tool import _is_compaction_summary
assert _is_compaction_summary("[CONTEXT COMPACTION — REFERENCE ONLY] foo")
assert _is_compaction_summary("[CONTEXT SUMMARY]: old summary")
assert not _is_compaction_summary("Hello, how can I help?")
assert not _is_compaction_summary("")
assert not _is_compaction_summary(None)
def test_compaction_summary_excluded_from_bookend_start(self, db):
"""Compaction handoff in bookend_start position must be filtered out."""
db.create_session("s_compact", source="cli")
# First message: a compaction handoff (should be filtered)
db.append_message("s_compact", role="user",
content="[CONTEXT COMPACTION — REFERENCE ONLY] "
"Earlier turns were compacted into the summary below. " + "x" * 50000)
# Second message: normal user message
db.append_message("s_compact", role="user", content="Fix the zorgblat rendering bug")
# Padding messages to push window away from session start (so bookend has room)
for i in range(10):
db.append_message("s_compact", role="user", content=f"setup step {i}")
db.append_message("s_compact", role="assistant", content=f"setup done {i}")
# Match target: uses a unique term so FTS5 anchors here, not at the start
db.append_message("s_compact", role="user", content="investigate the frobnitz mob spawning in KubeJS")
db.append_message("s_compact", role="assistant", content="I'll look into the frobnitz mob spawning issue.")
# Tail messages
for i in range(5):
db.append_message("s_compact", role="user", content=f"tail {i}")
db.append_message("s_compact", role="assistant", content=f"done tail {i}")
db._conn.commit()
result = json.loads(session_search(query="frobnitz mob spawning", db=db, limit=1))
assert result["success"] is True
assert len(result["results"]) >= 1
entry = result["results"][0]
# bookend_start must NOT contain the compaction handoff
for msg in entry.get("bookend_start", []):
assert "[CONTEXT COMPACTION" not in (msg.get("content") or "")
# The normal message should still be present in bookend_start
bookend_contents = [m.get("content", "") for m in entry.get("bookend_start", [])]
assert any("zorgblat" in c for c in bookend_contents)
def test_compaction_summary_excluded_from_bookend_end(self, db):
"""Compaction handoff in bookend_end position must be filtered out."""
db.create_session("s_compact_end", source="cli")
# Normal opening
db.append_message("s_compact_end", role="user", content="Build a website")
db.append_message("s_compact_end", role="assistant", content="Sure, let me scaffold it.")
# Match target (early in session so bookend_end has room)
db.append_message("s_compact_end", role="user", content="fix the zorgblat rendering bug")
db.append_message("s_compact_end", role="assistant", content="Investigating the zorgblat rendering issue.")
# Many messages to create distance from the end
for i in range(10):
db.append_message("s_compact_end", role="user", content=f"feature {i}")
db.append_message("s_compact_end", role="assistant", content=f"implemented {i}")
# Last message: compaction handoff (should be filtered from bookend_end)
db.append_message("s_compact_end", role="assistant",
content="[CONTEXT COMPACTION — REFERENCE ONLY] "
"Summary of all work done. " + "y" * 50000)
db._conn.commit()
result = json.loads(session_search(query="zorgblat rendering", db=db, limit=1))
assert result["success"] is True
assert len(result["results"]) >= 1
entry = result["results"][0]
# bookend_end must NOT contain the compaction handoff
for msg in entry.get("bookend_end", []):
assert "[CONTEXT COMPACTION" not in (msg.get("content") or "")
def test_bookend_content_is_capped(self, db):
"""Bookend messages must have content capped at 1200 chars."""
db.create_session("s_long_bookend", source="cli")
# First message: very long normal content
db.append_message("s_long_bookend", role="user",
content="Start the project. " + "z" * 5000)
# Match target
db.append_message("s_long_bookend", role="user", content="deploy to production")
db.append_message("s_long_bookend", role="assistant", content="Deploying now.")
for i in range(10):
db.append_message("s_long_bookend", role="user", content=f"step {i}")
db.append_message("s_long_bookend", role="assistant", content=f"done {i}")
db._conn.commit()
result = json.loads(session_search(query="deploy production", db=db, limit=1))
assert result["success"] is True
entry = result["results"][0]
for msg in entry.get("bookend_start", []):
content = msg.get("content", "")
# Content should be capped (1200 chars + "…" ellipsis)
assert len(content) <= 1210 # 1200 + ellipsis + margin
if msg.get("content_truncated"):
assert msg["original_content_chars"] > 1200
def test_window_content_is_capped(self, db):
"""Window messages must have content capped at 4000 chars."""
db.create_session("s_long_window", source="cli")
db.append_message("s_long_window", role="user", content="search keyword here")
# Very long assistant reply containing the keyword
db.append_message("s_long_window", role="assistant",
content="Found it! keyword " + "a" * 10000)
db._conn.commit()
result = json.loads(session_search(query="keyword", db=db, limit=1))
assert result["success"] is True
entry = result["results"][0]
for msg in entry.get("messages", []):
content = msg.get("content", "")
assert len(content) <= 4010 # 4000 + ellipsis + margin
def test_legacy_context_summary_filtered(self, db):
"""Legacy [CONTEXT SUMMARY]: prefix must also be filtered."""
db.create_session("s_legacy", source="cli")
db.append_message("s_legacy", role="user",
content="[CONTEXT SUMMARY]: old compacted summary here")
db.append_message("s_legacy", role="user", content="new task: build API")
db.append_message("s_legacy", role="assistant", content="Building REST API now.")
for i in range(10):
db.append_message("s_legacy", role="user", content=f"step {i}")
db.append_message("s_legacy", role="assistant", content=f"done {i}")
db._conn.commit()
result = json.loads(session_search(query="build API", db=db, limit=1))
assert result["success"] is True
entry = result["results"][0]
for msg in entry.get("bookend_start", []):
assert "[CONTEXT SUMMARY]" not in (msg.get("content") or "")
@@ -334,3 +334,232 @@ for raw in sys.stdin:
assert supervisor.is_running()
finally:
supervisor.shutdown()
def _make_compress_host_session(events: list) -> dict:
class _Agent:
model = "host-model"
provider = "host-provider"
tools = []
_cached_system_prompt = ""
session_input_tokens = 1
session_output_tokens = 1
session_prompt_tokens = 1
session_completion_tokens = 1
session_total_tokens = 2
session_api_calls = 1
session_id = "rotated-id"
agent = _Agent()
agent.context_compressor = type("ContextEngineStub", (), {})()
agent.context_compressor.on_session_start = (
lambda *_args, **_kwargs: events.append("notify")
)
return {
"agent": agent,
"session_key": "before-key",
"history": [
{"role": "user", "content": "before"},
{"role": "assistant", "content": "before"},
],
"history_lock": threading.Lock(),
"history_version": 2,
"running": False,
"manual_compression_lock": threading.Lock(),
}
def test_compute_host_compress_control_notifies_engine_after_commit(monkeypatch):
"""The compute-host slash.compress route must fire the context-engine
boundary hook exactly once, and only AFTER the host commits the compressed
history + session-key sync (salvaged #65670, extended to this route)."""
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
finalize_context_engine_compression_notification,
)
from tui_gateway import server
out = io.StringIO()
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
events: list[str] = []
session = _make_compress_host_session(events)
def _compress(sess, focus_topic=None, **_kwargs):
# Simulate agent._compress_context(defer_context_engine_notification=True)
_queue_context_engine_compression_notification(
sess["agent"],
new_session_id="rotated-id",
old_session_id="before-key",
)
with sess["history_lock"]:
sess["history"] = [{"role": "summary", "content": "compressed"}]
sess["history_version"] = 3
def _sync(sid, sess):
events.append("sync")
sess["session_key"] = "after-key"
server._sessions["sid"] = session
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
monkeypatch.setattr(server, "_compress_session_history", _compress)
monkeypatch.setattr(server, "_sync_session_key_after_compress", _sync)
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
monkeypatch.setattr(
server,
"_session_info",
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
)
try:
host.handle_frame(
{
"type": "control",
"sid": "sid",
"request_id": "compress-1",
"route_name": "slash.compress",
"command": "/compress",
}
)
ack = _wait_for_frame(
out,
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compress-1",
)
finally:
server._sessions.pop("sid", None)
host.close()
# Exactly one notification, after the session-key commit.
assert events == ["sync", "notify"]
assert ack["session_key"] == "after-key"
# Nothing pending leaks onto the agent for a later compress to misfire.
assert (
finalize_context_engine_compression_notification(
session["agent"], committed=True
)
is False
)
def test_compute_host_compress_control_failure_discards_notification(monkeypatch):
"""When the host-side compress mirror fails after compression queued the
boundary notification, the pending hook must be discarded — never left to
fire against a boundary the host rejected."""
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
finalize_context_engine_compression_notification,
)
from tui_gateway import server
out = io.StringIO()
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
events: list[str] = []
session = _make_compress_host_session(events)
def _compress(sess, focus_topic=None, **_kwargs):
_queue_context_engine_compression_notification(
sess["agent"],
new_session_id="rotated-id",
old_session_id="before-key",
)
def _boom(*_args, **_kwargs):
raise RuntimeError("synthetic host commit failure")
server._sessions["sid"] = session
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
monkeypatch.setattr(server, "_compress_session_history", _compress)
monkeypatch.setattr(server, "_sync_session_key_after_compress", _boom)
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
monkeypatch.setattr(
server,
"_session_info",
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
)
try:
host.handle_frame(
{
"type": "control",
"sid": "sid",
"request_id": "compress-2",
"route_name": "slash.compress",
"command": "/compress",
}
)
ack = _wait_for_frame(
out,
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compress-2",
)
finally:
server._sessions.pop("sid", None)
host.close()
assert events == []
assert "live session sync failed" in str(ack.get("output") or "")
# The pending notification was discarded, not left on the agent.
assert (
finalize_context_engine_compression_notification(
session["agent"], committed=True
)
is False
)
assert events == []
def test_compute_host_compact_alias_routes_to_compress_mirror(monkeypatch):
"""slash.compress control frames forward the user's raw alias verbatim;
/compact must reach the compress mirror (and its deferred-notification
finalize wiring), not silently no-op."""
from agent.conversation_compression import (
_queue_context_engine_compression_notification,
)
from tui_gateway import server
out = io.StringIO()
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
events: list[str] = []
session = _make_compress_host_session(events)
calls: dict[str, object] = {}
def _compress(sess, focus_topic=None, **_kwargs):
calls["focus"] = focus_topic
_queue_context_engine_compression_notification(
sess["agent"],
new_session_id="rotated-id",
old_session_id="before-key",
)
server._sessions["sid"] = session
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
monkeypatch.setattr(server, "_compress_session_history", _compress)
monkeypatch.setattr(
server, "_sync_session_key_after_compress", lambda *_a: events.append("sync")
)
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
monkeypatch.setattr(
server,
"_session_info",
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
)
try:
host.handle_frame(
{
"type": "control",
"sid": "sid",
"request_id": "compact-1",
"route_name": "slash.compress",
"command": "/compact focus topic",
}
)
ack = _wait_for_frame(
out,
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compact-1",
)
finally:
server._sessions.pop("sid", None)
host.close()
assert calls == {"focus": "focus topic"}
assert events == ["sync", "notify"]
assert ack["route_name"] == "slash.compress"
+95
View File
@@ -1289,6 +1289,101 @@ def test_session_branch_persists_branched_from_marker(server, monkeypatch):
assert kwargs["model_config"] == {"_branched_from": parent_key}
def test_session_branch_forwards_original_timestamps(server, monkeypatch):
"""TUI /branch must copy the parent's messages WITH their original
timestamps — append_message otherwise stamps time.time() at INSERT and
the branch's whole history silently appears authored "now" (#28841).
"""
append_calls = []
class _DB:
def get_session_title(self, _key):
return "parent-title"
def get_next_title_in_lineage(self, base):
return f"{base} 2"
def create_session(self, new_key, **kwargs):
return new_key
def append_message(self, **kwargs):
append_calls.append(kwargs)
return None
def set_session_title(self, _key, _title):
return None
monkeypatch.setattr(server, "_get_db", lambda: _DB())
monkeypatch.setattr(server, "_resolve_model", lambda: "test/model")
monkeypatch.setattr(server, "_new_session_key", lambda: "20260101_000001_child0")
monkeypatch.setattr(
server,
"_make_agent",
lambda _sid, key, session_id=None, session_db=None, **_kwargs: types.SimpleNamespace(
model="test/model", session_id=session_id or key
),
)
monkeypatch.setattr(server, "_init_session", lambda *_a, **_k: None)
monkeypatch.setattr(server, "_set_session_context", lambda *_a, **_k: [])
monkeypatch.setattr(server, "_clear_session_context", lambda *_a, **_k: None)
monkeypatch.setattr(server, "_session_cwd", lambda _s: "/tmp/branch-cwd")
original_ts = [1_700_000_000.0, 1_700_000_020.0]
parent_sid = "parent02"
server._sessions[parent_sid] = {
"session_key": "20260101_000000_parent",
"history": [
{"role": "user", "content": "hello", "timestamp": original_ts[0]},
{"role": "assistant", "content": "hi!", "timestamp": original_ts[1]},
],
"history_lock": threading.Lock(),
"cols": 80,
}
resp = server.handle_request(
{"id": "b2", "method": "session.branch", "params": {"session_id": parent_sid}}
)
assert "error" not in resp, resp
assert len(append_calls) == 2
assert [c.get("timestamp") for c in append_calls] == original_ts
def test_persist_branch_seed_forwards_original_timestamps(server, monkeypatch):
"""First-turn branch seed persist must carry each copied message's
original timestamp through to append_message (#28841)."""
import contextlib
append_calls = []
class _DB:
def append_message(self, **kwargs):
append_calls.append(kwargs)
return None
@contextlib.contextmanager
def _fake_session_db(_session):
yield _DB()
monkeypatch.setattr(server, "_session_db", _fake_session_db)
original_ts = [100.0, 200.0]
session = {
"session_key": "20260101_000002_seed00",
"parent_session_id": "20260101_000000_parent",
"history": [
{"role": "user", "content": "a", "timestamp": original_ts[0]},
{"role": "assistant", "content": "b", "timestamp": original_ts[1]},
],
"history_lock": threading.Lock(),
}
server._persist_branch_seed(session)
assert session.get("_branch_seed_persisted") is True
assert [c.get("timestamp") for c in append_calls] == original_ts
def test_make_agent_accepts_list_system_prompt(server, monkeypatch):
captured = {}
+53 -15
View File
@@ -187,30 +187,68 @@ def get_pending_for_session(
return None
def _coerce_text_response(entry: _ClarifyEntry, response: str) -> str:
"""Map typed choice replies to canonical choice text, otherwise keep custom text."""
def _coerce_text_response(entry: _ClarifyEntry, response: str) -> Optional[str]:
"""Map typed choice replies to canonical choice text, otherwise keep or reject custom text.
For native interactive multi-choice clarifies (button UI, awaiting_text=False):
- Accept numeric selections ("2" → choice[1])
- Accept exact choice label matches (case-insensitive)
- Reject arbitrary prose (return None) so the message continues as a normal turn
For text fallback or awaiting_text mode:
- Accept any text (numeric/label/custom) after passing through coercion
For open-ended clarifies (no choices):
- Accept any text
Returns None when the response should be rejected (arbitrary prose for native multi-choice).
"""
text = str(response).strip()
if entry.choices:
try:
idx = int(text) - 1
except ValueError:
idx = -1
if 0 <= idx < len(entry.choices):
return entry.choices[idx]
for choice in entry.choices:
if text.casefold() == str(choice).strip().casefold():
return str(choice).strip()
return text
if not entry.choices:
# Open-ended: accept any text
return text
# Try numeric selection first (always valid for multi-choice)
try:
idx = int(text) - 1
except ValueError:
idx = -1
if 0 <= idx < len(entry.choices):
return entry.choices[idx]
# Try exact choice label match (always valid for multi-choice)
for choice in entry.choices:
if text.casefold() == str(choice).strip().casefold():
return str(choice).strip()
# For text fallback or awaiting_text mode, accept custom text
# For native interactive multi-choice mode, reject arbitrary prose
if entry.awaiting_text:
return text
return None
def resolve_text_response_for_session(session_key: str, response: str) -> bool:
"""Resolve the oldest pending clarify in ``session_key`` from typed text."""
"""Resolve the oldest pending clarify in ``session_key`` from typed text.
Returns False if no pending clarify exists or if the response was rejected
(arbitrary prose for native interactive multi-choice clarifies).
"""
entry = get_pending_for_session(session_key, include_choice_prompts=True)
if entry is None:
return False
coerced = _coerce_text_response(entry, response)
if coerced is None:
# Response rejected: message should continue as a normal turn
return False
return resolve_gateway_clarify(
entry.clarify_id,
_coerce_text_response(entry, response),
coerced,
)
+3 -4
View File
@@ -49,7 +49,6 @@ DELEGATE_BLOCKED_TOOLS = frozenset(
"clarify", # no user interaction
"memory", # no writes to shared MEMORY.md
"send_message", # no cross-platform side effects
"execute_code", # children should reason step-by-step, not write scripts
"cronjob", # no scheduling more work in the parent's name
]
)
@@ -774,7 +773,7 @@ def _strip_blocked_tools(toolsets: List[str]) -> List[str]:
"""
# Composite toolsets that should never pass through to children, even
# though their individual tools aren't all in DELEGATE_BLOCKED_TOOLS.
_COMPOSITE_BLOCKED_TOOLSETS = frozenset({"delegation", "code_execution"})
_COMPOSITE_BLOCKED_TOOLSETS = frozenset({"delegation"})
blocked_toolset_names = {
name
for name, defn in TOOLSETS.items()
@@ -3427,10 +3426,10 @@ def _build_top_level_description() -> str:
"status) and verify it yourself — fetch the URL, stat the file, read "
"back the content — before telling the user the operation succeeded.\n"
"- Leaf subagents (role='leaf', the default) CANNOT call: "
"delegate_task, clarify, memory, send_message, execute_code.\n"
"delegate_task, clarify, memory, send_message.\n"
"- Orchestrator subagents (role='orchestrator') retain "
"delegate_task so they can spawn their own workers, but still "
"cannot use clarify, memory, send_message, or execute_code. "
"cannot use clarify, memory, or send_message. "
f"Orchestrators are bounded by max_spawn_depth={max_depth} for this "
f"user and can be disabled globally via "
"delegation.orchestrator_enabled=false.\n"
+54 -6
View File
@@ -55,6 +55,16 @@ _DEMOTED_SESSION_SOURCES = ("cron",)
# the handful of distinct sessions a typical query returns.
_DISCOVER_SCAN_LIMIT = 300
# Prefixes that identify generated context-compaction handoff summaries.
# These are inserted by agent/context_compressor.py as normal user/assistant
# messages but contain machine-generated summary metadata — not user content.
# They must be excluded from discovery bookends to avoid re-introducing huge
# compaction payloads into fresh sessions via session_search. (#43175)
_COMPACTION_PREFIXES = (
"[CONTEXT COMPACTION",
"[CONTEXT SUMMARY]:",
)
def _format_timestamp(ts: Union[int, float, str, None]) -> str:
"""Convert a Unix timestamp (float/int) or ISO string to a human-readable date.
@@ -81,6 +91,15 @@ def _format_timestamp(ts: Union[int, float, str, None]) -> str:
return str(ts)
def _is_compaction_summary(content: str) -> bool:
"""Return True if *content* looks like a generated compaction handoff."""
if not content:
return False
stripped = content.lstrip()
return any(stripped.startswith(p) for p in _COMPACTION_PREFIXES)
def _resolve_to_parent(db, session_id: str) -> str:
"""Walk parent_session_id chain to the lineage root. Falls back to input on errors."""
if not session_id:
@@ -142,12 +161,30 @@ def _order_for_recall(raw_results: List[Dict[str, Any]]) -> List[Dict[str, Any]]
)
def _shape_message(m: Dict[str, Any], anchor_id: Optional[int] = None) -> Dict[str, Any]:
"""Slim a message row for the tool response. Keeps content even if empty."""
def _shape_message(
m: Dict[str, Any],
anchor_id: Optional[int] = None,
max_content_len: Optional[int] = None,
) -> Dict[str, Any]:
"""Slim a message row for the tool response. Keeps content even if empty.
When *max_content_len* is set, ``content`` is truncated to that many
characters and ``content_truncated`` / ``original_content_chars`` metadata
is added so callers know the payload was bounded.
"""
raw_content = m.get("content")
if max_content_len and raw_content and len(raw_content) > max_content_len:
content = raw_content[:max_content_len] + "…"
truncated = True
original_chars = len(raw_content)
else:
content = raw_content
truncated = False
original_chars = None
entry = {
"id": m.get("id"),
"role": m.get("role"),
"content": m.get("content"),
"content": content,
"timestamp": m.get("timestamp"),
}
if m.get("tool_name"):
@@ -158,6 +195,9 @@ def _shape_message(m: Dict[str, Any], anchor_id: Optional[int] = None) -> Dict[s
entry["tool_call_id"] = m.get("tool_call_id")
if anchor_id is not None and m.get("id") == anchor_id:
entry["anchor"] = True
if truncated:
entry["content_truncated"] = True
entry["original_content_chars"] = original_chars
# Strip None values to keep payload tight, but always keep content
# (absent content is meaningful — tool-call-only assistant turns).
return {k: v for k, v in entry.items() if v is not None or k in ("content",)}
@@ -620,9 +660,17 @@ def _discover(
"matched_role": match_info.get("role"),
"match_message_id": msg_id,
"snippet": match_info.get("snippet") or "",
"bookend_start": [_shape_message(m) for m in (view.get("bookend_start") or [])],
"messages": [_shape_message(m, anchor_id=msg_id) for m in (view.get("window") or [])],
"bookend_end": [_shape_message(m) for m in (view.get("bookend_end") or [])],
"bookend_start": [
_shape_message(m, max_content_len=1200)
for m in (view.get("bookend_start") or [])
if not _is_compaction_summary(m.get("content", ""))
],
"messages": [_shape_message(m, anchor_id=msg_id, max_content_len=4000) for m in (view.get("window") or [])],
"bookend_end": [
_shape_message(m, max_content_len=1200)
for m in (view.get("bookend_end") or [])
if not _is_compaction_summary(m.get("content", ""))
],
"messages_before": view.get("messages_before", 0),
"messages_after": view.get("messages_after", 0),
}
+6 -1
View File
@@ -36,6 +36,11 @@ MAX_TODO_ITEMS = 256
# before it is parsed and re-injected (see AIAgent._hydrate_todo_store).
MAX_TODO_RESULT_CHARS = 512_000
_TRUNCATION_MARKER = "… [truncated]"
# Persisted as ordinary message content. ContextCompressor uses this stable
# header to distinguish the synthetic post-compaction row from a real user.
TODO_INJECTION_HEADER = (
"[Your active task list was preserved across context compression]"
)
class TodoStore:
@@ -135,7 +140,7 @@ class TodoStore:
if not active_items:
return None
lines = ["[Your active task list was preserved across context compression]"]
lines = [TODO_INJECTION_HEADER]
for item in active_items:
marker = markers.get(item["status"], "[?]")
lines.append(f"- {marker} {item['id']}. {item['content']} ({item['status']})")
+22
View File
@@ -567,6 +567,28 @@ class ComputeHost:
}
)
except Exception as exc:
if route_name in {"session.compress", "slash.compress"}:
# The compress mirror defers the context-engine boundary
# notification until the host commits. If anything raises
# between queueing and finalize (e.g. building the ack's
# session_info), discard the pending notification so it can't
# leak onto the agent and fire against a rejected boundary on
# a later compress. finalize is exactly-once, so this is a
# no-op when the mirror already emitted or discarded it.
try:
from tui_gateway import server as _server
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
_agent = (_server._sessions.get(sid) or {}).get("agent")
if _agent is not None:
finalize_context_engine_compression_notification(
_agent,
committed=False,
)
except Exception:
pass
self.emit({"type": "control.error", "sid": sid, "request_id": request_id, "message": str(exc)})
def _bump_progress(self) -> None:
+91 -13
View File
@@ -2067,7 +2067,15 @@ def _persist_branch_seed(session: dict) -> None:
return
try:
for msg in seed:
db.append_message(session_id=key, role=msg.get("role", "user"), content=msg.get("content"))
db.append_message(
session_id=key,
role=msg.get("role", "user"),
content=msg.get("content"),
# Preserve the parent's original message timestamps —
# append_message would otherwise stamp time.time() and the
# branch's copied history would all appear authored "now".
timestamp=msg.get("timestamp"),
)
session["_branch_seed_persisted"] = True
except Exception:
logger.debug("branch seed persist failed", exc_info=True)
@@ -3553,6 +3561,9 @@ def _compress_session_history(
before_messages: list | None = None,
history_version: int | None = None,
) -> tuple[int, dict]:
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
from agent.model_metadata import estimate_request_tokens_rough
agent = session["agent"]
@@ -3581,16 +3592,28 @@ def _compress_session_history(
# cached prompt (which already contains the agent identity block)
# makes the rebuild append the identity a second time. Mirrors the
# CLI's _manual_compress fix for issue #15281.
compressed, _ = agent._compress_context(
history,
None,
approx_tokens=approx_tokens,
focus_topic=focus_topic or None,
)
try:
compressed, _ = agent._compress_context(
history,
None,
approx_tokens=approx_tokens,
focus_topic=focus_topic or None,
defer_context_engine_notification=True,
)
except Exception:
finalize_context_engine_compression_notification(
agent,
committed=False,
)
raise
with session["history_lock"]:
if int(session.get("history_version", 0)) != history_version:
# External mutation during compaction — drop the compressed
# result so we don't clobber concurrent edits.
finalize_context_engine_compression_notification(
agent,
committed=False,
)
usage = _get_usage(agent)
return 0, usage
session["history"] = compressed
@@ -8992,6 +9015,10 @@ def _(rid, params: dict) -> dict:
return _err(
rid, 4009, "session busy — /interrupt the current turn before /compress"
)
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
sid = params.get("session_id", "")
focus_topic = str(params.get("focus_topic", "") or "").strip()
try:
@@ -9059,6 +9086,10 @@ def _(rid, params: dict) -> dict:
)
info = _session_info(agent, session)
_emit("session.info", sid, info)
finalize_context_engine_compression_notification(
agent,
committed=True,
)
return _ok(
rid,
{
@@ -9080,6 +9111,10 @@ def _(rid, params: dict) -> dict:
# no-op, or raised.
_status_update(sid, "ready")
except Exception as e:
finalize_context_engine_compression_notification(
session["agent"],
committed=False,
)
return _err(rid, 5005, str(e))
@@ -9200,6 +9235,7 @@ def _(rid, params: dict) -> dict:
session_id=new_key,
role=msg.get("role", "user"),
content=msg.get("content"),
timestamp=msg.get("timestamp"),
)
db.set_session_title(new_key, title)
except Exception as e:
@@ -11946,7 +11982,7 @@ def _(rid, params: dict) -> dict:
)
return _ok(rid, {"key": key, "value": nv})
if key == "compact":
if key == "density":
raw = str(value or "").strip().lower()
cfg0 = _load_cfg()
d0 = cfg0.get("display") if isinstance(cfg0.get("display"), dict) else {}
@@ -11958,7 +11994,7 @@ def _(rid, params: dict) -> dict:
elif raw == "off":
nv_b = False
else:
return _err(rid, 4002, f"unknown compact value: {value}")
return _err(rid, 4002, f"unknown density value: {value}")
_write_config_key("display.tui_compact", nv_b)
return _ok(rid, {"key": key, "value": "on" if nv_b else "off"})
@@ -12828,7 +12864,7 @@ def _(rid, params: dict) -> dict:
)
nv = "full" if dm == "expanded" else "collapsed"
return _ok(rid, {"value": nv})
if key == "compact":
if key == "density":
on = bool((_load_cfg().get("display") or {}).get("tui_compact", False))
return _ok(rid, {"value": "on" if on else "off"})
if key == "theme":
@@ -13273,7 +13309,7 @@ _TUI_HIDDEN: frozenset[str] = frozenset(
)
_TUI_EXTRA: list[tuple[str, str, str]] = [
("/compact", "Toggle compact display mode", "TUI"),
("/density", "Toggle compact display mode", "TUI"),
("/logs", "Show recent gateway log lines", "TUI"),
(
"/mouse",
@@ -13342,6 +13378,13 @@ def _(rid, params: dict) -> dict:
cat_map[cat].append([c, desc])
for name, desc, cat in _TUI_EXTRA:
# Dedup guard: skip TUI extras that collide with a registry
# command or one of its aliases (e.g. the historical /compact
# collision, #57133, or /sessions which the registry also
# advertises). The registry entry is canonical.
if name.lower() in canon:
continue
canon[name.lower()] = name
all_pairs.append([name, desc])
if cat not in cat_map:
cat_map[cat] = []
@@ -13943,6 +13986,10 @@ def _(rid, params: dict) -> dict:
return _err(
rid, 4009, "session busy — /interrupt the current turn before /compress"
)
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
sid = params.get("session_id", "")
if _session_uses_compute_host(session):
command = f"/{name}" + (f" {arg}" if arg else "")
@@ -14016,6 +14063,10 @@ def _(rid, params: dict) -> dict:
compression_state=getattr(_agent, "context_compressor", None),
)
_emit("session.info", sid, _session_info(session.get("agent"), session))
finalize_context_engine_compression_notification(
_agent,
committed=True,
)
return _ok(
rid,
{
@@ -14026,6 +14077,10 @@ def _(rid, params: dict) -> dict:
},
)
except Exception as exc:
finalize_context_engine_compression_notification(
session["agent"],
committed=False,
)
return _err(rid, 5009, f"compress failed: {exc}")
return _err(rid, 4018, f"not a quick/plugin/bundle/skill command: {name}")
@@ -14498,8 +14553,8 @@ def _(rid, params: dict) -> dict:
text_lower = text.lower()
extras = [
{
"text": "/compact",
"display": "/compact",
"text": "/density",
"display": "/density",
"meta": "Toggle compact display mode",
},
{
@@ -15006,6 +15061,13 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
(parts[1].strip() if len(parts) > 1 else ""),
session.get("agent"),
)
if name == "compact":
# /compact is an alias of /compress in every host. The compute-host
# slash.compress control forwards the user's raw alias verbatim, so
# without normalizing here the child mirror silently no-ops — the
# session never compresses and the deferred context-engine
# notification wiring below is never exercised for that route.
name = "compress"
# Reject agent-mutating commands during an in-flight turn. These
# all do read-then-mutate on live agent/session state that the
@@ -15051,6 +15113,9 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
# while CLI and gateway both did.
from agent.manual_compression_feedback import summarize_manual_compression
from agent.model_metadata import estimate_request_tokens_rough
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
with session["history_lock"]:
_before_messages = list(session.get("history", []))
@@ -15090,6 +15155,10 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
_lines = [_fb["headline"], _fb["token_line"]]
if _fb.get("note"):
_lines.append(_fb["note"])
finalize_context_engine_compression_notification(
agent,
committed=True,
)
return "\n".join(_lines)
elif name == "fast" and agent:
mode = arg.lower()
@@ -15105,6 +15174,15 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
process_registry.kill_all()
except Exception as e:
if name == "compress" and agent:
from agent.conversation_compression import (
finalize_context_engine_compression_notification,
)
finalize_context_engine_compression_notification(
agent,
committed=False,
)
return f"live session sync failed: {e}"
return ""
+1 -1
View File
@@ -225,7 +225,7 @@ The following commands are handled directly by the TUI client. Unrecognized comm
### Core (`core.ts`)
`/help`, `/quit` (alias `/exit`), `/update`, `/clear` (alias `/new`),
`/compact`, `/copy`, `/paste`, `/details` (alias `/detail`),
`/density`, `/copy`, `/paste`, `/details` (alias `/detail`),
`/statusbar` (alias `/sb`), `/queue` (alias `/q`), `/logs`, `/history`,
`/save`, `/undo`, `/retry`, `/steer`, `/mouse` (alias `/scroll`),
`/status`, `/title`, `/fortune`, `/redraw`, `/terminal-setup`
+5 -5
View File
@@ -278,19 +278,19 @@ export const coreCommands: SlashCommand[] = [
},
{
help: 'toggle compact transcript',
name: 'compact',
help: 'toggle compact display',
name: 'density',
run: (arg, ctx) => {
const next = flagFromArg(arg, ctx.ui.compact)
if (next === null) {
return ctx.transcript.sys('usage: /compact [on|off|toggle]')
return ctx.transcript.sys('usage: /density [on|off|toggle]')
}
patchUiState({ compact: next })
ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'compact', value: next ? 'on' : 'off' }).catch(() => {})
ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'density', value: next ? 'on' : 'off' }).catch(() => {})
queueMicrotask(() => ctx.transcript.sys(`compact ${next ? 'on' : 'off'}`))
queueMicrotask(() => ctx.transcript.sys(`density ${next ? 'on' : 'off'}`))
}
},
@@ -82,6 +82,9 @@ All compression settings are read from `config.yaml` under the `compression` key
compression:
enabled: true # Enable/disable compression (default: true)
threshold: 0.50 # Fraction of context window (default: 0.50 = 50%)
# model_thresholds: # Per-model threshold overrides (substring match,
# "glm-5.2": 0.40 # longest key wins). See "Per-model threshold
# "claude-sonnet": 0.35 # overrides" below.
target_ratio: 0.20 # How much of threshold to keep as tail (default: 0.20)
protect_last_n: 20 # Minimum protected tail messages (default: 20)
codex_gpt55_autoraise: true # gpt-5.5 on Codex OAuth: raise trigger to 85% (default: true)
@@ -101,6 +104,7 @@ auxiliary:
| Parameter | Default | Range | Description |
|-----------|---------|-------|-------------|
| `threshold` | `0.50` | 0.0-1.0 | Compression triggers when prompt tokens ≥ `threshold × context_length` |
| `model_thresholds` | `{}` | map | Per-model overrides of `threshold`. Keys are substring-matched against the model name (longest match wins). The small-context floor still applies on top (see below) |
| `target_ratio` | `0.20` | 0.10-0.80 | Controls tail protection token budget: `threshold_tokens × target_ratio` |
| `protect_last_n` | `20` | ≥1 | Minimum number of recent messages always preserved |
| `protect_first_n` | `3` | (hardcoded) | System prompt + first exchange always preserved |
@@ -108,6 +112,39 @@ auxiliary:
| `codex_gpt55_autoraise_notice` | `true` | bool | Show the one-time Codex gpt-5.5 autoraise notice. Set `false` to keep the 85% autoraise but suppress the banner |
| `codex_app_server_auto` | `native` | `native`, `hermes`, `off` | Thread-compaction mode for Codex app-server sessions (see below) |
### Per-model threshold overrides
`compression.model_thresholds` lets you trigger compaction at different points
depending on the active model — useful when you swap between models with very
different context windows (e.g. a 1M-context model can compress later while a
128K model should compress earlier):
```yaml
compression:
threshold: 0.50
model_thresholds:
"glm-5.2": 0.40
"glm-5.2-1M": 0.25
"claude-sonnet": 0.35
```
Resolution rules:
- Keys are **substring-matched** against the model name; the **longest
matching key wins** (`glm-5.2-1M` beats `glm-5.2` for model `glm-5.2-1M`).
- When no key matches (or the map is empty), the global `threshold` applies.
- The override is re-resolved on every `/model` switch; switching to a model
with no matching key falls back to the global `threshold`.
- The **small-context floor still applies on top** of overrides (raise-only):
models with context windows below 512K are floored at `0.75`, so an
override below the floor is raised to `0.75`, while an override above it
(e.g. `0.80`) wins.
Plugin context engines can reuse the same resolution logic via
`from agent.context_compressor import resolve_model_threshold`; engines that
override `update_model()` own their own compaction policy and may ignore the
map.
### Codex gpt-5.5 threshold autoraise
The ChatGPT Codex OAuth backend hard-caps gpt-5.5 at a **272K** context window
@@ -165,7 +165,7 @@ context:
engine: "lcm" # must match your engine's name property
```
The `compression` config block (`compression.threshold`, `compression.protect_last_n`, etc.) is specific to the built-in `ContextCompressor`. Your engine should define its own config format if needed, reading from `config.yaml` during initialization.
The `compression` config block (`compression.threshold`, `compression.protect_last_n`, etc.) is specific to the built-in `ContextCompressor`, with one explicit exception: `compression.model_thresholds` (per-model threshold overrides) is part of the context-engine contract. The host assigns the resolved map to `engine.model_thresholds` *before* the initial `update_model()` call, and the base-class `update_model()` applies it (longest substring match, falling back to the engine's configured threshold). Engines that override `update_model()` own their own compaction policy and may honor or ignore the map — `from agent.context_compressor import resolve_model_threshold` to reuse the same resolution logic. For everything else, your engine should define its own config format if needed, reading from `config.yaml` during initialization.
## Testing
+1 -1
View File
@@ -194,7 +194,7 @@ This is often the most efficient pattern: `execute_code` handles the 10+ sequent
## Inherited Tool Access
Subagents inherit the parent's enabled toolsets. `delegate_task` does not accept a model-facing `toolsets` parameter, so delegated work cannot grant itself capabilities that the parent does not have. Configure the parent's tools before starting the conversation when a delegated task needs web, terminal, file, or other access. Hermes still strips child-blocked tools such as `clarify`, `memory`, and `execute_code`.
Subagents inherit the parent's enabled toolsets. `delegate_task` does not accept a model-facing `toolsets` parameter, so delegated work cannot grant itself capabilities that the parent does not have. Configure the parent's tools before starting the conversation when a delegated task needs web, terminal, file, or other access. Hermes still strips child-blocked tools such as `clarify`, `memory`, and `send_message`; children keep `execute_code` for programmatic tool calling.
---
@@ -264,7 +264,7 @@ For **durable execution** that must survive session closure or process restart,
- Each subagent gets its **own terminal session** (separate from the parent)
- Subagents inherit the parent's enabled toolsets; the model cannot select or widen them per call
- **Nested delegation is opt-in** — only `role="orchestrator"` children can delegate further, and only when `max_spawn_depth` is raised from its default of 1 (flat). Disable globally with `orchestrator_enabled: false`.
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `execute_code`. Orchestrator subagents retain `delegate_task` but still cannot use the other three.
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `send_message`, `cronjob`. Orchestrator subagents retain `delegate_task` but keep the other blocks. Both roles retain `execute_code` (programmatic tool calling) so children can batch mechanical work instead of burning reasoning iterations.
- **Cancellation follows ownership** — `/stop` or closing/resetting the owning session cancels its background children; synchronous descendants under orchestrators follow their parent's interrupt state
- Only the final summary enters the parent's context, keeping token usage efficient
- Subagents inherit the parent's **API key, provider configuration, and credential pool** (enabling key rotation on rate limits)