Compare commits
68
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8dd07bd517 | ||
|
|
d358280ad7 | ||
|
|
76283a9ee4 | ||
|
|
07cbb500b6 | ||
|
|
95aad9229b | ||
|
|
f944e84858 | ||
|
|
5f2fdf66bf | ||
|
|
f453c50b6f | ||
|
|
45182401fa | ||
|
|
596dda907f | ||
|
|
9bb253d4fa | ||
|
|
75099ca0ef | ||
|
|
3c74c12554 | ||
|
|
b6a2b701c1 | ||
|
|
e71c1137e8 | ||
|
|
17fa5910be | ||
|
|
cc84af9fad | ||
|
|
004de13f14 | ||
|
|
cca7b93bfe | ||
|
|
19a59f7d7b | ||
|
|
dbc71fb6e4 | ||
|
|
295e20358c | ||
|
|
b2ba069cb4 | ||
|
|
a43ac2af65 | ||
|
|
72da4f8657 | ||
|
|
a2d8b4b1f6 | ||
|
|
c13af09177 | ||
|
|
8fc0c086f5 | ||
|
|
d91a143977 | ||
|
|
870770b31e | ||
|
|
961b832c11 | ||
|
|
9d73006ade | ||
|
|
1927b60775 | ||
|
|
7d4ed8e321 | ||
|
|
d46f0fb2d5 | ||
|
|
15d33d5ab1 | ||
|
|
27a4e92802 | ||
|
|
cb39be92e4 | ||
|
|
4c2e34f07d | ||
|
|
8a580e8e31 | ||
|
|
2eb6c84bbb | ||
|
|
63e363f306 | ||
|
|
453d0ee598 | ||
|
|
3865694cf9 | ||
|
|
1f92842c1c | ||
|
|
1e0f5a6f07 | ||
|
|
8fceaac143 | ||
|
|
7f9956a679 | ||
|
|
928bcdde24 | ||
|
|
ea0fd393db | ||
|
|
3f33a1c5aa | ||
|
|
a269a5b38c | ||
|
|
7de7a073c6 | ||
|
|
c93e4041b6 | ||
|
|
1c2faedd88 | ||
|
|
cf433a33da | ||
|
|
fca883f06f | ||
|
|
644b397fb2 | ||
|
|
28b6bd6570 | ||
|
|
3807df7a06 | ||
|
|
1c21e96ed0 | ||
|
|
146a545491 | ||
|
|
4166fdda6a | ||
|
|
a26ed50e07 | ||
|
|
f66e773c44 | ||
|
|
2ced02671e | ||
|
|
806b34ced9 | ||
|
|
85f04d4d7e |
@@ -0,0 +1,56 @@
|
||||
name: Hermes Plugin Validate
|
||||
description: >-
|
||||
Validate a Hermes Agent plugin (plugin.yaml manifest schema AND
|
||||
declared-vs-actually-registered capabilities) using
|
||||
`hermes plugins validate`. Drop this into your plugin repo's CI:
|
||||
|
||||
- uses: actions/checkout@<sha>
|
||||
- uses: NousResearch/hermes-agent/.github/actions/plugin-validate@main
|
||||
with:
|
||||
path: .
|
||||
|
||||
The caller's job owns checkout; this action installs Python + hermes-agent
|
||||
(git install — a supported CI-context install route) and runs the
|
||||
validator against your plugin directory.
|
||||
|
||||
inputs:
|
||||
path:
|
||||
description: Path to the plugin directory (containing plugin.yaml).
|
||||
default: "."
|
||||
hermes-ref:
|
||||
description: hermes-agent git ref (branch/tag/sha) to install and validate with.
|
||||
default: "main"
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Install hermes-agent
|
||||
shell: bash
|
||||
env:
|
||||
_HERMES_REF: ${{ inputs.hermes-ref }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# CI-context install from git; the ref lets plugin authors validate
|
||||
# against a pinned hermes release instead of main.
|
||||
pip install "git+https://github.com/NousResearch/hermes-agent@${_HERMES_REF}"
|
||||
|
||||
- name: Validate plugin
|
||||
shell: bash
|
||||
env:
|
||||
_PLUGIN_PATH: ${{ inputs.path }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# `hermes plugins validate` checks the plugin.yaml manifest schema
|
||||
# and loads the plugin in a scratch subprocess to verify that the
|
||||
# capabilities it DECLARES match what it actually registers.
|
||||
if hermes plugins validate "$_PLUGIN_PATH"; then
|
||||
echo "✅ PASS: plugin at '$_PLUGIN_PATH' validated cleanly"
|
||||
else
|
||||
echo "❌ FAIL: plugin at '$_PLUGIN_PATH' failed validation (see output above)"
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,140 @@
|
||||
name: Plugin Catalog CI
|
||||
|
||||
# Admission gate for plugin-catalog entries. Fires ONLY on PRs touching
|
||||
# plugin-catalog/** so it can never go red on unrelated PRs.
|
||||
#
|
||||
# Two gates:
|
||||
# structural — cheap schema check, no hermes install needed
|
||||
# pinned-source-validate — supply-chain gate: the pinned sha MUST be
|
||||
# reachable in the entry's repo, and the plugin
|
||||
# at that exact commit must pass
|
||||
# `hermes plugins validate`.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "plugin-catalog/**"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
structural:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Install PyYAML
|
||||
uses: ./.github/actions/retry
|
||||
with:
|
||||
command: pip install pyyaml==6.0.2
|
||||
|
||||
- name: Validate catalog files (structural)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# Validating the whole directory is simpler than diffing and keeps
|
||||
# the invariant that EVERYTHING in plugin-catalog/ stays valid.
|
||||
python3 scripts/validate_plugin_catalog.py plugin-catalog/
|
||||
|
||||
pinned-source-validate:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0 # need the merge-base to diff changed catalog files
|
||||
|
||||
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Find changed catalog entries
|
||||
id: changed
|
||||
run: |
|
||||
set -euo pipefail
|
||||
MERGE_BASE=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
|
||||
# Added + modified entry files only; deletions and removed.yaml
|
||||
# have nothing to clone.
|
||||
CHANGED=$(git diff --name-only --diff-filter=AM "$MERGE_BASE"...HEAD \
|
||||
-- 'plugin-catalog/*.yaml' 'plugin-catalog/*.yml' \
|
||||
| grep -v '/removed\.yaml$' || true)
|
||||
echo "Changed catalog entries:"
|
||||
echo "${CHANGED:-<none>}"
|
||||
{
|
||||
echo 'files<<__EOF__'
|
||||
echo "$CHANGED"
|
||||
echo '__EOF__'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Install hermes-agent from the PR's own checkout
|
||||
if: steps.changed.outputs.files != ''
|
||||
uses: ./.github/actions/retry
|
||||
with:
|
||||
command: pip install -e .
|
||||
|
||||
- name: Clone each entry at its pinned sha and validate
|
||||
if: steps.changed.outputs.files != ''
|
||||
env:
|
||||
CHANGED_FILES: ${{ steps.changed.outputs.files }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
FAILED=0
|
||||
while IFS= read -r entry; do
|
||||
[ -z "$entry" ] && continue
|
||||
echo "::group::validate $entry"
|
||||
|
||||
# Parse repo / sha / subdir from the entry yaml.
|
||||
eval "$(python3 - "$entry" <<'PYEOF'
|
||||
import shlex
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as fh:
|
||||
data = yaml.safe_load(fh) or {}
|
||||
print(f"REPO={shlex.quote(str(data.get('repo', '')))}")
|
||||
print(f"SHA={shlex.quote(str(data.get('sha', '')))}")
|
||||
print(f"SUBDIR={shlex.quote(str(data.get('subdir', '') or ''))}")
|
||||
PYEOF
|
||||
)"
|
||||
echo "repo=$REPO sha=$SHA subdir=$SUBDIR"
|
||||
|
||||
CLONE_DIR=$(mktemp -d)
|
||||
# Full clone (no --depth 1): the pinned sha may not be the branch tip.
|
||||
if ! git clone "$REPO" "$CLONE_DIR"; then
|
||||
echo "::error file=$entry::clone failed for $REPO"
|
||||
FAILED=1; echo "::endgroup::"; continue
|
||||
fi
|
||||
|
||||
# SUPPLY-CHAIN GATE: the pinned sha must be reachable in the repo.
|
||||
if ! git -C "$CLONE_DIR" checkout --detach "$SHA"; then
|
||||
echo "::error file=$entry::pinned sha $SHA is not reachable in $REPO"
|
||||
FAILED=1; echo "::endgroup::"; continue
|
||||
fi
|
||||
|
||||
PLUGIN_DIR="$CLONE_DIR${SUBDIR:+/$SUBDIR}"
|
||||
if [ ! -f "$PLUGIN_DIR/plugin.yaml" ]; then
|
||||
echo "::error file=$entry::no plugin.yaml at subdir '$SUBDIR' of $REPO@$SHA"
|
||||
FAILED=1; echo "::endgroup::"; continue
|
||||
fi
|
||||
|
||||
# Manifest schema + declared-vs-registered capability check.
|
||||
if hermes plugins validate "$PLUGIN_DIR"; then
|
||||
echo "✅ PASS: $entry"
|
||||
else
|
||||
echo "::error file=$entry::hermes plugins validate failed"
|
||||
FAILED=1
|
||||
fi
|
||||
echo "::endgroup::"
|
||||
done <<< "$CHANGED_FILES"
|
||||
|
||||
if [ "$FAILED" -ne 0 ]; then
|
||||
echo "❌ FAIL: one or more catalog entries failed pinned-source validation"
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ PASS: all changed catalog entries validated at their pinned shas"
|
||||
@@ -998,7 +998,8 @@ Two shapes:
|
||||
Roles:
|
||||
|
||||
- `role="leaf"` (default) — focused worker. Cannot call `delegate_task`,
|
||||
`clarify`, `memory`, `send_message`, `execute_code`.
|
||||
`clarify`, `memory`, `send_message`, `cronjob`. Retains `execute_code`
|
||||
(programmatic tool calling).
|
||||
- `role="orchestrator"` — retains `delegate_task` so it can spawn its
|
||||
own workers. Gated by `delegation.orchestrator_enabled` (default true)
|
||||
and bounded by `delegation.max_spawn_depth` (default 2).
|
||||
|
||||
@@ -456,7 +456,7 @@ class HermesACPAgent(acp.Agent):
|
||||
"tools": "List available tools",
|
||||
"context": "Show conversation context info",
|
||||
"reset": "Clear conversation history",
|
||||
"compact": "Compress conversation context",
|
||||
"compress": "Compress conversation context",
|
||||
"steer": "Inject guidance into the currently running agent turn",
|
||||
"queue": "Queue a prompt to run after the current turn finishes",
|
||||
"version": "Show Hermes version",
|
||||
@@ -485,7 +485,7 @@ class HermesACPAgent(acp.Agent):
|
||||
"description": "Clear conversation history",
|
||||
},
|
||||
{
|
||||
"name": "compact",
|
||||
"name": "compress",
|
||||
"description": "Compress conversation context",
|
||||
},
|
||||
{
|
||||
@@ -1756,7 +1756,7 @@ class HermesACPAgent(acp.Agent):
|
||||
"tools": self._cmd_tools,
|
||||
"context": self._cmd_context,
|
||||
"reset": self._cmd_reset,
|
||||
"compact": self._cmd_compact,
|
||||
"compress": self._cmd_compress,
|
||||
"steer": self._cmd_steer,
|
||||
"queue": self._cmd_queue,
|
||||
"version": self._cmd_version,
|
||||
@@ -1898,7 +1898,7 @@ class HermesACPAgent(acp.Agent):
|
||||
lines.append(
|
||||
f"Compression: due now (threshold ~{threshold_tokens:,}"
|
||||
+ (f", {threshold_pct:.0f}%" if threshold_pct else "")
|
||||
+ "). Run /compact."
|
||||
+ "). Run /compress."
|
||||
)
|
||||
else:
|
||||
lines.append(
|
||||
@@ -1913,7 +1913,7 @@ class HermesACPAgent(acp.Agent):
|
||||
if getattr(agent, "compression_enabled", True) is False:
|
||||
lines.append("Compression is disabled for this agent.")
|
||||
else:
|
||||
lines.append("Tip: run /compact to compress manually before the threshold.")
|
||||
lines.append("Tip: run /compress to compress manually before the threshold.")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
@@ -1933,7 +1933,7 @@ class HermesACPAgent(acp.Agent):
|
||||
return "Conversation history cleared. Agent session state reset failed; see logs."
|
||||
return "Conversation history cleared."
|
||||
|
||||
def _cmd_compact(self, args: str, state: SessionState) -> str:
|
||||
def _cmd_compress(self, args: str, state: SessionState) -> str:
|
||||
if not state.history:
|
||||
return "Nothing to compress — conversation is empty."
|
||||
try:
|
||||
|
||||
+59
-1
@@ -1561,7 +1561,14 @@ def init_agent(
|
||||
agent._memory_nudge_interval = 10
|
||||
agent._turns_since_memory = 0
|
||||
agent._iters_since_skill = 0
|
||||
if not skip_memory:
|
||||
# A flush/background agent may pass skip_memory=True to avoid spinning up an
|
||||
# external memory *provider*, but if the caller also explicitly enables the
|
||||
# "memory" toolset it still needs the built-in file-backed store — otherwise
|
||||
# the memory tool dispatches with store=None and every call fails (#65429).
|
||||
# So the built-in store is created unless memory is globally disabled, while
|
||||
# the external-provider block below stays gated on skip_memory.
|
||||
_memory_toolset_requested = "memory" in (agent.enabled_toolsets or [])
|
||||
if not skip_memory or _memory_toolset_requested:
|
||||
try:
|
||||
mem_config = _agent_cfg.get("memory", {})
|
||||
agent._memory_enabled = mem_config.get("memory_enabled", False)
|
||||
@@ -1781,6 +1788,34 @@ def init_agent(
|
||||
compression_enabled = str(_compression_cfg.get("enabled", True)).lower() in {"true", "1", "yes"}
|
||||
compression_target_ratio = float(_compression_cfg.get("target_ratio", 0.20))
|
||||
compression_protect_last = int(_compression_cfg.get("protect_last_n", 20))
|
||||
# Cap on compression retry rounds before a turn gives up with "max
|
||||
# compression attempts reached" (compression.max_attempts). Hardcoding 3
|
||||
# strands sessions that legitimately need more rounds — e.g. a restart
|
||||
# history reload whose incompressible tool schemas keep the request
|
||||
# estimate above the threshold even though the messages compress fine
|
||||
# (the #62605 failure class). Default 3 preserves current behavior, so
|
||||
# an unset key is behavior-neutral; validated >= 1, hard-capped at 10,
|
||||
# and any non-int-like value falls back to 3. Booleans are rejected
|
||||
# (bool subclasses int, so int(True) would silently become 1) and
|
||||
# fractional floats are rejected rather than truncated — "4.7 attempts"
|
||||
# is a config mistake, not a request for 4.
|
||||
_raw_max_attempts = _compression_cfg.get("max_attempts", 3)
|
||||
if isinstance(_raw_max_attempts, bool):
|
||||
compression_max_attempts = 3
|
||||
elif isinstance(_raw_max_attempts, int):
|
||||
compression_max_attempts = _raw_max_attempts
|
||||
elif isinstance(_raw_max_attempts, float):
|
||||
compression_max_attempts = (
|
||||
int(_raw_max_attempts) if _raw_max_attempts.is_integer() else 3
|
||||
)
|
||||
else:
|
||||
try:
|
||||
compression_max_attempts = int(str(_raw_max_attempts).strip())
|
||||
except (TypeError, ValueError):
|
||||
compression_max_attempts = 3
|
||||
if compression_max_attempts < 1:
|
||||
compression_max_attempts = 3
|
||||
compression_max_attempts = min(compression_max_attempts, 10)
|
||||
# protect_first_n is the number of non-system messages to protect at
|
||||
# the head, in addition to the system prompt (which is always
|
||||
# implicitly protected by the compressor). Floor at 0 — a value of
|
||||
@@ -1793,6 +1828,17 @@ def init_agent(
|
||||
compression_abort_on_summary_failure = str(
|
||||
_compression_cfg.get("abort_on_summary_failure", False)
|
||||
).lower() in {"true", "1", "yes"}
|
||||
# Per-model threshold overrides: keys are substring-matched against the
|
||||
# model name (longest match wins). Empty dict = use the global threshold
|
||||
# for all models (backward compatible).
|
||||
_raw_model_thresholds = _compression_cfg.get("model_thresholds", {})
|
||||
if isinstance(_raw_model_thresholds, dict):
|
||||
compression_model_thresholds = {
|
||||
str(k): float(v) for k, v in _raw_model_thresholds.items()
|
||||
if isinstance(v, (int, float)) and not isinstance(v, bool)
|
||||
}
|
||||
else:
|
||||
compression_model_thresholds = {}
|
||||
# In-place compaction: when True, compress_context() rewrites the message
|
||||
# list + rebuilds the system prompt WITHOUT rotating the session id (no
|
||||
# parent_session_id chain, no `name #N` renumber). See #38763 and
|
||||
@@ -2188,6 +2234,16 @@ def init_agent(
|
||||
provider=agent.provider,
|
||||
custom_providers=_custom_providers,
|
||||
)
|
||||
# Per-model threshold overrides are part of the explicit
|
||||
# context-engine contract: assign them BEFORE the initial
|
||||
# update_model() call so the first resolution (which derives
|
||||
# threshold_percent/threshold_tokens for the initial model) already
|
||||
# sees the overrides. Assigning after update_model() left the initial
|
||||
# model on the engine's global threshold until the first /model
|
||||
# switch. Engines that override update_model() own their own policy
|
||||
# and may ignore the attribute.
|
||||
if compression_model_thresholds:
|
||||
agent.context_compressor.model_thresholds = compression_model_thresholds
|
||||
agent.context_compressor.update_model(
|
||||
model=agent.model,
|
||||
context_length=_plugin_ctx_len,
|
||||
@@ -2214,6 +2270,7 @@ def init_agent(
|
||||
api_mode=agent.api_mode,
|
||||
abort_on_summary_failure=compression_abort_on_summary_failure,
|
||||
max_tokens=agent.max_tokens,
|
||||
model_thresholds=compression_model_thresholds,
|
||||
)
|
||||
_bind_session_state = getattr(agent.context_compressor, "bind_session_state", None)
|
||||
if callable(_bind_session_state):
|
||||
@@ -2224,6 +2281,7 @@ def init_agent(
|
||||
agent.compression_enabled = compression_enabled
|
||||
agent.compression_in_place = compression_in_place
|
||||
agent.codex_app_server_auto_compaction = codex_app_server_auto_compaction
|
||||
agent.max_compression_attempts = compression_max_attempts
|
||||
|
||||
# Reject models whose context window is below the minimum required
|
||||
# for reliable tool-calling workflows (64K tokens).
|
||||
|
||||
@@ -2412,6 +2412,24 @@ def _evict_old_screenshots(result: List[Dict[str, Any]]) -> None:
|
||||
]
|
||||
|
||||
|
||||
def _ensure_leading_user_turn(result: List[Dict[str, Any]]) -> None:
|
||||
"""Anthropic requires messages[0] to have role=user.
|
||||
|
||||
After a second context compaction on the auto path the summary can be
|
||||
emitted as role=assistant with nothing in front of it (the system prompt
|
||||
lives outside messages[] or is extracted into the separate ``system``
|
||||
param), so messages[0] ends up assistant and the Messages API rejects
|
||||
the request with HTTP 400 — often masked by a misleading
|
||||
"tool_use ids were found without tool_result blocks" error (#52160).
|
||||
|
||||
Mirror the Bedrock Converse adapter, which unconditionally prepends a
|
||||
minimal user turn when the first message is not user
|
||||
(convert_messages_to_converse).
|
||||
"""
|
||||
if result and result[0].get("role") != "user":
|
||||
result.insert(0, {"role": "user", "content": [{"type": "text", "text": " "}]})
|
||||
|
||||
|
||||
def convert_messages_to_anthropic(
|
||||
messages: List[Dict],
|
||||
base_url: str | None = None,
|
||||
@@ -2470,6 +2488,7 @@ def convert_messages_to_anthropic(
|
||||
|
||||
_strip_orphaned_tool_blocks(result)
|
||||
result = _merge_consecutive_roles(result)
|
||||
_ensure_leading_user_turn(result)
|
||||
_manage_thinking_signatures(result, base_url, model)
|
||||
_evict_old_screenshots(result)
|
||||
|
||||
|
||||
+298
-52
@@ -31,9 +31,11 @@ from agent.model_metadata import (
|
||||
MINIMUM_CONTEXT_LENGTH,
|
||||
get_model_context_length,
|
||||
estimate_messages_tokens_rough,
|
||||
estimate_tokens_rough,
|
||||
)
|
||||
from agent.redact import redact_sensitive_text
|
||||
from agent.turn_context import drop_stale_api_content
|
||||
from tools.todo_tool import TODO_INJECTION_HEADER
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -130,8 +132,20 @@ LEGACY_SUMMARY_PREFIX = "[CONTEXT SUMMARY]:"
|
||||
# poisoning every subsequent request in the session — a bare key like
|
||||
# "is_compressed_summary" would reach the wire and trip exactly that.
|
||||
COMPRESSED_SUMMARY_METADATA_KEY = "_compressed_summary"
|
||||
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY = "_compressed_summary_has_user_turn"
|
||||
_DB_PERSISTED_MARKER = "_db_persisted"
|
||||
|
||||
_NO_USER_TASK_SENTINEL = "None. This session contains no user-authored turns."
|
||||
COMPRESSION_CONTINUATION_USER_CONTENT = (
|
||||
"Continue from the compressed conversation context above. "
|
||||
"This marker exists because no human user turn was available."
|
||||
)
|
||||
_LEGACY_COMPRESSION_CONTINUATION_USER_CONTENT = (
|
||||
"Continue from the compressed conversation context above. "
|
||||
"This marker exists because the compacted transcript contained "
|
||||
"no preserved user turn."
|
||||
)
|
||||
|
||||
|
||||
def _fresh_compaction_message_copy(msg: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Copy a message for compaction assembly without persistence markers.
|
||||
@@ -436,11 +450,15 @@ def _estimate_msg_budget_tokens(msg: dict) -> int:
|
||||
compaction re-fires continuously (#55572). Accounting-only: replay
|
||||
fields are never mutated or pruned here.
|
||||
"""
|
||||
content_len = _content_length_for_budget(msg.get("content") or "")
|
||||
tokens = content_len // _CHARS_PER_TOKEN + 10 # +10 for role/key overhead
|
||||
content = msg.get("content") or ""
|
||||
if isinstance(content, str):
|
||||
tokens = estimate_tokens_rough(content) + 10 # +10 for role/key overhead
|
||||
else:
|
||||
content_len = _content_length_for_budget(content)
|
||||
tokens = content_len // _CHARS_PER_TOKEN + 10
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if isinstance(tc, dict):
|
||||
tokens += len(str(tc)) // _CHARS_PER_TOKEN
|
||||
tokens += estimate_tokens_rough(str(tc))
|
||||
for key in _REPLAY_BUDGET_KEYS:
|
||||
tokens += _serialized_length_for_budget(msg.get(key)) // _CHARS_PER_TOKEN
|
||||
return tokens
|
||||
@@ -856,6 +874,32 @@ def _summarize_tool_result_unguarded(tool_name: str, tool_args: str, tool_conten
|
||||
return f"[{tool_name}]{first_arg} ({content_len:,} chars result)"
|
||||
|
||||
|
||||
def resolve_model_threshold(
|
||||
model: str,
|
||||
model_thresholds: dict[str, float] | None,
|
||||
default: float,
|
||||
) -> float:
|
||||
"""Resolve the effective compression threshold for a given model.
|
||||
|
||||
``model_thresholds`` maps substring keys to override fractions. The
|
||||
longest matching key wins (so ``glm-5.2-1M`` beats ``glm-5.2`` when the
|
||||
model is ``glm-5.2-1M``). When no override matches, or when
|
||||
``model_thresholds`` is empty/None, ``default`` is returned unchanged.
|
||||
|
||||
This is a module-level helper so plugin context engines (e.g. LCM) can
|
||||
import and reuse the same resolution logic as the built-in compressor.
|
||||
"""
|
||||
if not model_thresholds or not model:
|
||||
return default
|
||||
best_key = ""
|
||||
for key in model_thresholds:
|
||||
if key in model and len(key) > len(best_key):
|
||||
best_key = key
|
||||
if best_key:
|
||||
return float(model_thresholds[best_key])
|
||||
return default
|
||||
|
||||
|
||||
class ContextCompressor(ContextEngine):
|
||||
"""Default context engine — compresses conversation context via lossy summarization.
|
||||
|
||||
@@ -877,6 +921,7 @@ class ContextCompressor(ContextEngine):
|
||||
self._context_probed = False
|
||||
self._context_probe_persistable = False
|
||||
self._previous_summary = None
|
||||
self._summary_has_user_turn = None
|
||||
self._last_summary_error = None
|
||||
self._consecutive_timeout_failures = 0
|
||||
self._last_summary_dropped_count = 0
|
||||
@@ -917,6 +962,7 @@ class ContextCompressor(ContextEngine):
|
||||
surface the moment the owning session ends.
|
||||
"""
|
||||
self._previous_summary = None
|
||||
self._summary_has_user_turn = None
|
||||
self._last_summary_error = None
|
||||
self._consecutive_timeout_failures = 0
|
||||
self._last_summary_dropped_count = 0
|
||||
@@ -1163,17 +1209,19 @@ class ContextCompressor(ContextEngine):
|
||||
self.provider = provider
|
||||
self.api_mode = api_mode
|
||||
self.context_length = context_length
|
||||
# Re-apply the small-context threshold floor for the NEW window,
|
||||
# starting from the originally-configured percent (not the possibly
|
||||
# floored live value) so a small -> large switch drops back to the
|
||||
# configured threshold and a large -> small switch gains the floor.
|
||||
# Guard with getattr: compressors unpickled/constructed before this
|
||||
# attribute existed fall back to the live value.
|
||||
_configured_pct = getattr(
|
||||
self, "_configured_threshold_percent", self.threshold_percent,
|
||||
# Re-resolve per-model threshold for the NEW model, then re-apply the
|
||||
# small-context threshold floor. Starting from _config_threshold_percent
|
||||
# (the raw config value) so a switch from a model with an override to
|
||||
# one without correctly falls back to the global threshold.
|
||||
_config_pct = getattr(
|
||||
self, "_config_threshold_percent", self.threshold_percent,
|
||||
)
|
||||
_new_base = resolve_model_threshold(
|
||||
model, self.model_thresholds, _config_pct,
|
||||
)
|
||||
self._base_threshold_percent = _new_base
|
||||
self.threshold_percent = self._effective_threshold_percent(
|
||||
context_length, _configured_pct,
|
||||
context_length, _new_base,
|
||||
)
|
||||
# max_tokens=None here means "caller didn't specify" → keep the existing
|
||||
# output reservation. A switch that genuinely changes the output budget
|
||||
@@ -1319,13 +1367,26 @@ class ContextCompressor(ContextEngine):
|
||||
api_mode: str = "",
|
||||
abort_on_summary_failure: bool = False,
|
||||
max_tokens: int | None = None,
|
||||
model_thresholds: dict[str, float] | None = None,
|
||||
):
|
||||
self.model = model
|
||||
self.base_url = base_url
|
||||
self.api_key = api_key
|
||||
self.provider = provider
|
||||
self.api_mode = api_mode
|
||||
self.threshold_percent = threshold_percent
|
||||
# Per-model threshold overrides (longest substring match wins).
|
||||
# Stored as a plain dict; resolved in _resolve_threshold(), then the
|
||||
# small-context floor is applied on top.
|
||||
self.model_thresholds = model_thresholds or {}
|
||||
# _config_threshold_percent is the raw config value (before per-model
|
||||
# override or small-context floor). Used as the fallback when switching
|
||||
# to a model with no matching override.
|
||||
self._config_threshold_percent = threshold_percent
|
||||
# Resolve per-model override first, then apply the small-context floor.
|
||||
self._base_threshold_percent = resolve_model_threshold(
|
||||
model, self.model_thresholds, threshold_percent,
|
||||
)
|
||||
self.threshold_percent = self._base_threshold_percent
|
||||
self.protect_first_n = protect_first_n
|
||||
self.protect_last_n = protect_last_n
|
||||
self.summary_target_ratio = max(0.10, min(summary_target_ratio, 0.80))
|
||||
@@ -1355,9 +1416,11 @@ class ContextCompressor(ContextEngine):
|
||||
# resolved and BEFORE threshold_tokens is derived. The pre-floor
|
||||
# value is kept so update_model() can re-derive for a new window
|
||||
# (switching small -> large must drop back to the configured value).
|
||||
# Note: _base_threshold_percent already has the per-model override
|
||||
# applied, so the floor stacks on top of any model-specific threshold.
|
||||
self._configured_threshold_percent = self.threshold_percent
|
||||
self.threshold_percent = self._effective_threshold_percent(
|
||||
self.context_length, self.threshold_percent,
|
||||
self.context_length, self._base_threshold_percent,
|
||||
)
|
||||
threshold_percent = self.threshold_percent
|
||||
# Floor: never compress below MINIMUM_CONTEXT_LENGTH tokens even if
|
||||
@@ -1403,6 +1466,10 @@ class ContextCompressor(ContextEngine):
|
||||
|
||||
# Stores the previous compaction summary for iterative updates
|
||||
self._previous_summary: Optional[str] = None
|
||||
# Provenance for the rolling summary. A compaction handoff can carry
|
||||
# role="user" solely to satisfy provider alternation, so role alone
|
||||
# cannot prove that a human-authored turn ever existed.
|
||||
self._summary_has_user_turn: Optional[bool] = None
|
||||
# Anti-thrashing: track whether last compression was effective
|
||||
self._last_compression_savings_pct: float = 100.0
|
||||
self._ineffective_compression_count: int = 0
|
||||
@@ -1990,6 +2057,9 @@ class ContextCompressor(ContextEngine):
|
||||
role = msg.get("role", "unknown")
|
||||
text = _compact_fallback_turn(msg.get("content"))
|
||||
_collect_path_mentions(text, relevant_files)
|
||||
synthetic_user = (
|
||||
role == "user" and self._is_synthetic_compression_user_turn(msg)
|
||||
)
|
||||
|
||||
turn_text = text
|
||||
turn_tool_names: list[str] = []
|
||||
@@ -2000,12 +2070,13 @@ class ContextCompressor(ContextEngine):
|
||||
if turn_tool_names:
|
||||
prefix = "tool calls: " + ", ".join(turn_tool_names[:6])
|
||||
turn_text = f"{prefix}; {turn_text}" if turn_text else prefix
|
||||
_remember_dropped_turn(str(role).upper(), turn_text)
|
||||
turn_label = "INTERNAL CONTEXT" if synthetic_user else str(role).upper()
|
||||
_remember_dropped_turn(turn_label, turn_text)
|
||||
|
||||
if len(text) > 600:
|
||||
text = text[:420].rstrip() + " ... " + text[-160:].lstrip()
|
||||
|
||||
if role == "user" and text:
|
||||
if role == "user" and text and not synthetic_user:
|
||||
user_asks.append(text)
|
||||
elif role == "assistant":
|
||||
tool_names: list[str] = []
|
||||
@@ -2051,7 +2122,7 @@ class ContextCompressor(ContextEngine):
|
||||
active_task = (
|
||||
f"User asked: {user_asks[-1]!r}"
|
||||
if user_asks
|
||||
else "Unknown from deterministic fallback."
|
||||
else _NO_USER_TASK_SENTINEL
|
||||
)
|
||||
previous_summary_note = ""
|
||||
if self._previous_summary:
|
||||
@@ -2194,6 +2265,9 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
|
||||
if _sanitized_memory_context
|
||||
else ""
|
||||
)
|
||||
has_user_turn = getattr(self, "_summary_has_user_turn", None)
|
||||
if has_user_turn is None:
|
||||
has_user_turn = self._transcript_has_real_user_turn(turns_to_summarize)
|
||||
|
||||
# Current date for temporal anchoring (see ## Temporal Anchoring below).
|
||||
# Date-only granularity matches system_prompt.py:337 (PR #20451) and the
|
||||
@@ -2211,18 +2285,86 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
|
||||
# Preamble shared by both first-compaction and iterative-update prompts.
|
||||
# Keep the wording deliberately plain: Azure/OpenAI-compatible content
|
||||
# filters have flagged stronger "injection" / "do not respond" framing.
|
||||
if has_user_turn:
|
||||
_language_and_provenance_rule = (
|
||||
"Write the summary in the same language the user was using in the "
|
||||
"conversation — do not translate or switch to English. "
|
||||
)
|
||||
_historical_task_instructions = """[THE SINGLE MOST IMPORTANT FIELD. Capture the user's most recent unfulfilled
|
||||
input verbatim — the exact words they used. This includes:
|
||||
- Explicit task assignments ("<specific user task>")
|
||||
- Questions awaiting an answer ("<specific user question>")
|
||||
- Decisions awaiting input ("<option A or B?>")
|
||||
- Ongoing discussions where the assistant owes the next substantive reply
|
||||
A conversation where the user just asked a question IS an active task — the
|
||||
task is "answer that question with full context". Do NOT write "None" merely
|
||||
because the user did not issue an imperative command; reserve "None" for the
|
||||
rare case where the last exchange was fully resolved and the user said
|
||||
something like "thanks, that's all".
|
||||
If multiple items are outstanding, list only the ones NOT yet completed.
|
||||
This historical snapshot must identify the latest unresolved user input precisely. Examples:
|
||||
"User asked: '<exact latest user request>'"
|
||||
"User asked: '<exact latest user question>' — needs investigation + answer"
|
||||
"User chose <option>; awaiting implementation of <specific next step>"
|
||||
If the user's most recent message was a reverse signal (stop, undo, roll
|
||||
back, never mind, just verify, change of topic) that supersedes earlier
|
||||
work, write the reverse signal verbatim and DO NOT carry forward the
|
||||
cancelled task. Example: "User asked: '<exact reverse signal>' — earlier
|
||||
in-flight work is cancelled."
|
||||
If no outstanding task exists, write "None."]"""
|
||||
_goal_instructions = "[What the user is trying to accomplish overall]"
|
||||
_constraints_instructions = (
|
||||
"[User preferences, coding style, constraints, important decisions]"
|
||||
)
|
||||
_resolved_questions_instructions = (
|
||||
"[Questions the user asked that were ALREADY answered — include the "
|
||||
"answer so it is not repeated]"
|
||||
)
|
||||
_pending_asks_instructions = (
|
||||
"[Questions or requests from the user that have NOT yet been answered "
|
||||
"or fulfilled. These are STALE — they were from the compacted turns. "
|
||||
"Write them here for reference only. The agent must NOT act on them "
|
||||
"unless the latest user message explicitly requests it. If none, "
|
||||
'write "None."]'
|
||||
)
|
||||
else:
|
||||
_language_and_provenance_rule = (
|
||||
"This session contains no user-authored turns. Write the summary "
|
||||
"in the dominant language of the source turns; if they are mixed, "
|
||||
"use the language of the most recent natural-language assistant "
|
||||
"turn. Do not translate, invent a user, or attribute any request "
|
||||
"to a user. "
|
||||
)
|
||||
_historical_task_instructions = f"""[NO user-authored turn exists in this session. Write exactly:
|
||||
{_NO_USER_TASK_SENTINEL}
|
||||
Do not write "User asked:" or any translated equivalent anywhere in the summary.
|
||||
Describe agent/tool work only as completed actions, state, or historical work.]"""
|
||||
_goal_instructions = (
|
||||
"[Historical cron/agent objective inferred only from assistant and "
|
||||
"tool activity. Never call it a user goal.]"
|
||||
)
|
||||
_constraints_instructions = (
|
||||
"[Runtime, configuration, and technical constraints only. Do not "
|
||||
"invent user preferences.]"
|
||||
)
|
||||
_resolved_questions_instructions = (
|
||||
"[Write exactly: None. No user-authored questions exist.]"
|
||||
)
|
||||
_pending_asks_instructions = (
|
||||
"[Write exactly: None. No user-authored requests exist.]"
|
||||
)
|
||||
|
||||
_summarizer_preamble = (
|
||||
"You are a summarization agent creating a context checkpoint. "
|
||||
"Treat the conversation turns below as source material for a "
|
||||
"compact record of prior work. "
|
||||
"Produce only the structured summary; do not add a greeting, "
|
||||
"preamble, or prefix. "
|
||||
"Write the summary in the same language the user was using in the "
|
||||
"conversation — do not translate or switch to English. "
|
||||
+ _language_and_provenance_rule +
|
||||
"NEVER include API keys, tokens, passwords, secrets, credentials, "
|
||||
"or connection strings in the summary — replace any that appear "
|
||||
"with [REDACTED]. Note that the user had credentials present, but "
|
||||
"do not preserve their values."
|
||||
"with [REDACTED]. Note that credentials were present, but do not "
|
||||
"preserve their values."
|
||||
)
|
||||
|
||||
# Temporal anchoring directive. Rewrites relative / still-pending-sounding
|
||||
@@ -2245,34 +2387,13 @@ Summary generation was unavailable, so this is a best-effort deterministic fallb
|
||||
|
||||
# Shared structured template (used by both paths).
|
||||
_template_sections = f"""{HISTORICAL_TASK_HEADING}
|
||||
[THE SINGLE MOST IMPORTANT FIELD. Capture the user's most recent unfulfilled
|
||||
input verbatim — the exact words they used. This includes:
|
||||
- Explicit task assignments ("<specific user task>")
|
||||
- Questions awaiting an answer ("<specific user question>")
|
||||
- Decisions awaiting input ("<option A or B?>")
|
||||
- Ongoing discussions where the assistant owes the next substantive reply
|
||||
A conversation where the user just asked a question IS an active task — the
|
||||
task is "answer that question with full context". Do NOT write "None" merely
|
||||
because the user did not issue an imperative command; reserve "None" for the
|
||||
rare case where the last exchange was fully resolved and the user said
|
||||
something like "thanks, that's all".
|
||||
If multiple items are outstanding, list only the ones NOT yet completed.
|
||||
This historical snapshot must identify the latest unresolved user input precisely. Examples:
|
||||
"User asked: '<exact latest user request>'"
|
||||
"User asked: '<exact latest user question>' — needs investigation + answer"
|
||||
"User chose <option>; awaiting implementation of <specific next step>"
|
||||
If the user's most recent message was a reverse signal (stop, undo, roll
|
||||
back, never mind, just verify, change of topic) that supersedes earlier
|
||||
work, write the reverse signal verbatim and DO NOT carry forward the
|
||||
cancelled task. Example: "User asked: '<exact reverse signal>' — earlier
|
||||
in-flight work is cancelled."
|
||||
If no outstanding task exists, write "None."]
|
||||
{_historical_task_instructions}
|
||||
|
||||
## Goal
|
||||
[What the user is trying to accomplish overall]
|
||||
{_goal_instructions}
|
||||
|
||||
## Constraints & Preferences
|
||||
[User preferences, coding style, constraints, important decisions]
|
||||
{_constraints_instructions}
|
||||
|
||||
## Completed Actions
|
||||
[Numbered list of concrete actions taken — include tool used, target, and outcome.
|
||||
@@ -2301,10 +2422,10 @@ Be specific with file paths, commands, line numbers, and results.]
|
||||
[Important technical decisions and WHY they were made]
|
||||
|
||||
## Resolved Questions
|
||||
[Questions the user asked that were ALREADY answered — include the answer so it is not repeated]
|
||||
{_resolved_questions_instructions}
|
||||
|
||||
{HISTORICAL_PENDING_ASKS_HEADING}
|
||||
[Questions or requests from the user that have NOT yet been answered or fulfilled. These are STALE — they were from the compacted turns. Write them here for reference only. The agent must NOT act on them unless the latest user message explicitly requests it. If none, write "None."]
|
||||
{_pending_asks_instructions}
|
||||
|
||||
## Relevant Files
|
||||
[Files read, modified, or created — with brief note on each]
|
||||
@@ -2428,6 +2549,7 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
# ignore prompt instructions and echo back secrets verbatim.
|
||||
summary = redact_sensitive_text(content.strip())
|
||||
summary = self._ground_historical_task_snapshot(summary, turns_to_summarize)
|
||||
self._validate_summary_user_provenance(summary, has_user_turn)
|
||||
# Store for iterative updates on next compaction
|
||||
self._previous_summary = summary
|
||||
self._clear_compression_failure_cooldown()
|
||||
@@ -2674,6 +2796,69 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
return False
|
||||
return bool(message.get(COMPRESSED_SUMMARY_METADATA_KEY))
|
||||
|
||||
@classmethod
|
||||
def _transcript_has_real_user_turn(cls, messages: List[Dict[str, Any]]) -> bool:
|
||||
"""Return whether *messages* contain a user-authored turn.
|
||||
|
||||
Compaction summaries can deliberately carry ``role="user"`` to keep
|
||||
strict provider transcripts valid. The metadata/content checks prevent
|
||||
those synthetic transport rows from becoming evidence of a real user.
|
||||
"""
|
||||
for message in messages:
|
||||
if not isinstance(message, dict) or message.get("role") != "user":
|
||||
continue
|
||||
if cls._is_synthetic_compression_user_turn(message):
|
||||
continue
|
||||
return True
|
||||
return False
|
||||
|
||||
@classmethod
|
||||
def _is_synthetic_compression_user_turn(cls, message: Any) -> bool:
|
||||
"""Recognize internal user-role rows after SessionDB projection.
|
||||
|
||||
SessionDB preserves role/content but not underscore-prefixed metadata,
|
||||
so the stable todo and continuation content markers are authoritative.
|
||||
"""
|
||||
if not isinstance(message, dict) or message.get("role") != "user":
|
||||
return False
|
||||
if cls._has_compressed_summary_metadata(message):
|
||||
return True
|
||||
content = message.get("content")
|
||||
if cls._is_context_summary_content(content):
|
||||
return True
|
||||
text = _content_text_for_contains(content).strip()
|
||||
return text in {
|
||||
COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
_LEGACY_COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
} or text.startswith(
|
||||
TODO_INJECTION_HEADER + "\n"
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _validate_summary_user_provenance(summary: str, has_user_turn: bool) -> None:
|
||||
"""Reject user attribution when the source transcript has no user."""
|
||||
if has_user_turn:
|
||||
return
|
||||
match = re.search(
|
||||
rf"(?ms)^{re.escape(HISTORICAL_TASK_HEADING)}\s*\n(.*?)(?=\n##\s|\Z)",
|
||||
summary,
|
||||
)
|
||||
task_snapshot = match.group(1).strip() if match else ""
|
||||
# NOTE: the "User asked:" scan covers the WHOLE summary, so tool output
|
||||
# quoted verbatim in e.g. Completed Actions can false-positive in a
|
||||
# zero-user session. That is acceptable: the RuntimeError only rides
|
||||
# the existing retry/deterministic-fallback path (which emits the
|
||||
# no-user sentinel itself), so a rare false positive costs one retry
|
||||
# rather than letting fabricated user attribution persist.
|
||||
if (
|
||||
task_snapshot != _NO_USER_TASK_SENTINEL
|
||||
or re.search(r"\bUser\s+asked\s*:", summary, re.IGNORECASE)
|
||||
):
|
||||
raise RuntimeError(
|
||||
"Context compression summary invented user attribution for a "
|
||||
"session with no user-authored turns"
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _derive_auto_focus_topic(
|
||||
cls,
|
||||
@@ -2685,9 +2870,9 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
msg = messages[idx]
|
||||
if msg.get("role") != "user":
|
||||
continue
|
||||
content = msg.get("content")
|
||||
if cls._is_context_summary_content(content):
|
||||
if cls._is_synthetic_compression_user_turn(msg):
|
||||
continue
|
||||
content = msg.get("content")
|
||||
text = redact_sensitive_text(_content_text_for_contains(content).strip())
|
||||
if not text:
|
||||
continue
|
||||
@@ -2968,8 +3153,9 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
"""
|
||||
for i in range(len(messages) - 1, head_end - 1, -1):
|
||||
msg = messages[i]
|
||||
if msg.get("role") == "user" and not self._is_context_summary_content(
|
||||
msg.get("content")
|
||||
if (
|
||||
msg.get("role") == "user"
|
||||
and not self._is_synthetic_compression_user_turn(msg)
|
||||
):
|
||||
return i
|
||||
return -1
|
||||
@@ -3444,9 +3630,24 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
summary_search_start,
|
||||
compress_end,
|
||||
)
|
||||
real_user_present = self._transcript_has_real_user_turn(messages)
|
||||
if summary_idx is not None:
|
||||
if summary_body and not self._previous_summary:
|
||||
self._previous_summary = summary_body
|
||||
provenance = messages[summary_idx].get(
|
||||
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY
|
||||
)
|
||||
if real_user_present:
|
||||
self._summary_has_user_turn = True
|
||||
elif isinstance(provenance, bool):
|
||||
self._summary_has_user_turn = provenance
|
||||
elif self._summary_has_user_turn is None:
|
||||
# Legacy handoffs predate provenance metadata. Preserve their
|
||||
# user context unless the exact no-user sentinel proves the
|
||||
# stronger invariant; guessing False could erase a real ask.
|
||||
self._summary_has_user_turn = not (
|
||||
summary_body and _NO_USER_TASK_SENTINEL in summary_body
|
||||
)
|
||||
turns_to_summarize = messages[max(compress_start, summary_idx + 1):compress_end]
|
||||
elif self._previous_summary:
|
||||
# No handoff summary found in the current messages, but
|
||||
@@ -3455,6 +3656,9 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
# it so _generate_summary() does not inject cross-session content
|
||||
# into the summarizer prompt via the iterative-update path.
|
||||
self._previous_summary = None
|
||||
self._summary_has_user_turn = real_user_present
|
||||
else:
|
||||
self._summary_has_user_turn = real_user_present
|
||||
|
||||
if not self.quiet_mode:
|
||||
logger.info(
|
||||
@@ -3548,6 +3752,19 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
# Phase 4: Assemble compressed message list
|
||||
compressed = []
|
||||
for i in range(compress_start):
|
||||
# If an earlier compaction handoff is in the protected head
|
||||
# (common after resume / in-place compaction), do not carry it
|
||||
# forward verbatim. It has already been rehydrated into
|
||||
# _previous_summary above and _generate_summary() will emit the
|
||||
# updated replacement below. Keeping both makes repeated
|
||||
# compactions accumulate old summaries and prevents the live prompt
|
||||
# from actually shrinking.
|
||||
if (
|
||||
summary_idx is not None
|
||||
and i == summary_idx
|
||||
and self._is_context_summary_content(messages[i].get("content"))
|
||||
):
|
||||
continue
|
||||
msg = _fresh_compaction_message_copy(messages[i])
|
||||
if i == 0 and msg.get("role") == "system":
|
||||
existing = msg.get("content")
|
||||
@@ -3574,7 +3791,7 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
)
|
||||
|
||||
_merge_summary_into_tail = False
|
||||
last_head_role = messages[compress_start - 1].get("role", "user") if compress_start > 0 else "user"
|
||||
last_head_role = compressed[-1].get("role", "user") if compressed else "user"
|
||||
first_tail_role = messages[compress_end].get("role", "user") if compress_end < n_messages else "user"
|
||||
# When the only protected head message is the system prompt, the
|
||||
# summary becomes the first *visible* message in the API request
|
||||
@@ -3645,6 +3862,9 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
"role": summary_role,
|
||||
"content": summary,
|
||||
COMPRESSED_SUMMARY_METADATA_KEY: True,
|
||||
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY: bool(
|
||||
self._summary_has_user_turn
|
||||
),
|
||||
})
|
||||
|
||||
for i in range(compress_end, n_messages):
|
||||
@@ -3674,6 +3894,9 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
# Mark the merged message so frontends can identify it as
|
||||
# containing a compression summary prefix.
|
||||
msg[COMPRESSED_SUMMARY_METADATA_KEY] = True
|
||||
msg[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] = bool(
|
||||
self._summary_has_user_turn
|
||||
)
|
||||
# Content rewritten → the api_content sidecar (exact bytes
|
||||
# previously sent) is stale; drop it so replay can't resend
|
||||
# the pre-merge bytes without the summary.
|
||||
@@ -3733,3 +3956,26 @@ This compaction should PRIORITISE preserving all information related to the focu
|
||||
self._last_compression_made_progress = True
|
||||
|
||||
return compressed
|
||||
|
||||
|
||||
def is_compaction_summary_message(message: Any) -> bool:
|
||||
"""Return True when *message* is a context-compaction handoff summary.
|
||||
|
||||
Public API for consumers outside the compressor (memory providers,
|
||||
frontends) that must not treat compaction summaries as real user or
|
||||
assistant turns — e.g. fact extraction harvesting the compactor's own
|
||||
output as user statements (#57682).
|
||||
|
||||
Prefers the in-process ``COMPRESSED_SUMMARY_METADATA_KEY`` marker and
|
||||
falls back to the content heuristics in ``_is_context_summary_content``
|
||||
(which cover the merged-into-tail and historical-prefix cases), because
|
||||
the metadata key is stripped by the wire sanitizers and does not survive
|
||||
all session-store round-trips.
|
||||
"""
|
||||
if isinstance(message, dict):
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY):
|
||||
return True
|
||||
content = message.get("content")
|
||||
else:
|
||||
content = message
|
||||
return ContextCompressor._is_context_summary_content(content)
|
||||
|
||||
@@ -260,4 +260,19 @@ class ContextEngine(ABC):
|
||||
(e.g. recalculate DAG budgets, switch summary models).
|
||||
"""
|
||||
self.context_length = context_length
|
||||
# Apply per-model threshold overrides if set (longest substring match).
|
||||
# Falls back to _config_threshold_percent (the raw config value) when
|
||||
# no override matches. Plugin engines that override update_model() can
|
||||
# call resolve_model_threshold() for the same logic.
|
||||
from agent.context_compressor import resolve_model_threshold
|
||||
if not hasattr(self, "_config_threshold_percent"):
|
||||
# Snapshot the pre-override percent ONCE so repeated model
|
||||
# switches fall back to the engine's configured value, not the
|
||||
# previous model's override.
|
||||
self._config_threshold_percent = self.threshold_percent
|
||||
self._base_threshold_percent = resolve_model_threshold(
|
||||
model, getattr(self, "model_thresholds", {}),
|
||||
self._config_threshold_percent,
|
||||
)
|
||||
self.threshold_percent = self._base_threshold_percent
|
||||
self.threshold_tokens = int(context_length * self.threshold_percent)
|
||||
|
||||
@@ -31,6 +31,7 @@ from __future__ import annotations
|
||||
import copy
|
||||
import inspect
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
@@ -231,6 +232,51 @@ def _supported_compression_kwargs(
|
||||
return {name: value for name, value in candidates.items() if name in parameters}
|
||||
|
||||
|
||||
class _CompressionActivityHeartbeat:
|
||||
"""Refresh the agent inactivity tracker while compression blocks in an aux call."""
|
||||
|
||||
def __init__(self, agent: Any, interval_seconds: float | None = None) -> None:
|
||||
self._agent = agent
|
||||
if interval_seconds is None:
|
||||
interval_seconds = getattr(agent, "_compression_activity_heartbeat_interval", 60.0)
|
||||
try:
|
||||
interval_seconds = float(interval_seconds or 60.0)
|
||||
except (TypeError, ValueError):
|
||||
interval_seconds = 60.0
|
||||
if not math.isfinite(interval_seconds):
|
||||
interval_seconds = 60.0
|
||||
self._interval_seconds = max(0.1, interval_seconds)
|
||||
self._stop = threading.Event()
|
||||
self._thread = threading.Thread(
|
||||
target=self._run,
|
||||
name="compression-activity-heartbeat",
|
||||
daemon=True,
|
||||
)
|
||||
|
||||
def start(self) -> "_CompressionActivityHeartbeat":
|
||||
self._touch("context compression started")
|
||||
self._thread.start()
|
||||
return self
|
||||
|
||||
def stop(self, desc: str = "context compression completed") -> None:
|
||||
self._stop.set()
|
||||
if self._thread.is_alive() and threading.current_thread() is not self._thread:
|
||||
self._thread.join(timeout=1.0)
|
||||
self._touch(desc)
|
||||
|
||||
def _touch(self, desc: str) -> None:
|
||||
try:
|
||||
touch = getattr(self._agent, "_touch_activity", None)
|
||||
if callable(touch):
|
||||
touch(desc)
|
||||
except Exception:
|
||||
logger.debug("compression activity heartbeat touch failed", exc_info=True)
|
||||
|
||||
def _run(self) -> None:
|
||||
while not self._stop.wait(self._interval_seconds):
|
||||
self._touch("context compression in progress")
|
||||
|
||||
|
||||
class _CompressionLockLeaseRefresher:
|
||||
def __init__(
|
||||
self,
|
||||
@@ -458,6 +504,29 @@ def check_compression_model_feasibility(agent: Any) -> None:
|
||||
new_threshold / main_ctx
|
||||
)
|
||||
safe_pct = int((aux_context / main_ctx) * 100) if main_ctx else 50
|
||||
# The "lower the threshold" suggestion must survive the built-in
|
||||
# trigger recomputation (#67422): _effective_threshold_percent()
|
||||
# raises sub-75% values back up for main windows under 512K, and
|
||||
# _compute_threshold_tokens() further applies the output-token
|
||||
# reservation, the 64K floor, and the degenerate-window guard.
|
||||
# Recommending a value those would override is silently ignored
|
||||
# and this warning would reappear every session — so mirror the
|
||||
# compressor's own math and only offer the option when the
|
||||
# recomputed trigger actually fits the auxiliary model's context.
|
||||
# External engines own compaction policy (#44439); the built-in
|
||||
# floor doesn't apply to them, so keep the plain suggestion.
|
||||
from agent.context_compressor import ContextCompressor as _CC
|
||||
|
||||
recomputed_threshold = None
|
||||
if main_ctx and isinstance(agent.context_compressor, _CC):
|
||||
recomputed_threshold = _CC._compute_threshold_tokens(
|
||||
main_ctx,
|
||||
_CC._effective_threshold_percent(main_ctx, safe_pct / 100),
|
||||
getattr(agent.context_compressor, "max_tokens", None),
|
||||
)
|
||||
threshold_suggestion_viable = (
|
||||
recomputed_threshold is None or recomputed_threshold <= aux_context
|
||||
)
|
||||
# Build human-readable "model (provider)" labels for both
|
||||
# the main model and the compression model so users can
|
||||
# tell at a glance which provider each side is actually
|
||||
@@ -491,15 +560,32 @@ def check_compression_model_feasibility(agent: Any) -> None:
|
||||
f"{old_threshold:,} tokens. "
|
||||
f"Auto-lowered this session's threshold to "
|
||||
f"{new_threshold:,} tokens so compression can run.\n"
|
||||
f" To make this permanent, edit config.yaml — either:\n"
|
||||
f" 1. Use a larger compression model:\n"
|
||||
f" auxiliary:\n"
|
||||
f" compression:\n"
|
||||
f" model: <model-with-{old_threshold:,}+-context>\n"
|
||||
f" 2. Lower the compression threshold:\n"
|
||||
f" compression:\n"
|
||||
f" threshold: 0.{safe_pct:02d}"
|
||||
)
|
||||
if threshold_suggestion_viable:
|
||||
msg += (
|
||||
f" To make this permanent, edit config.yaml — either:\n"
|
||||
f" 1. Use a larger compression model:\n"
|
||||
f" auxiliary:\n"
|
||||
f" compression:\n"
|
||||
f" model: <model-with-{old_threshold:,}+-context>\n"
|
||||
f" 2. Lower the compression threshold:\n"
|
||||
f" compression:\n"
|
||||
f" threshold: 0.{safe_pct:02d}"
|
||||
)
|
||||
else:
|
||||
msg += (
|
||||
f" To make this permanent, use a larger compression "
|
||||
f"model in config.yaml:\n"
|
||||
f" auxiliary:\n"
|
||||
f" compression:\n"
|
||||
f" model: <model-with-{old_threshold:,}+-context>\n"
|
||||
f" (Lowering compression.threshold cannot help here — "
|
||||
f"with {_main_label}'s {main_ctx:,}-token window, "
|
||||
f"Hermes's small-context floor and output reservation "
|
||||
f"would recompute the trigger to "
|
||||
f"{recomputed_threshold:,} tokens, still above the "
|
||||
f"compression model's {aux_context:,}.)"
|
||||
)
|
||||
agent._compression_warning = msg
|
||||
agent._emit_status(msg)
|
||||
logger.warning(
|
||||
@@ -613,7 +699,7 @@ def _is_real_user_message(message: Any) -> bool:
|
||||
return False
|
||||
from agent.context_compressor import ContextCompressor
|
||||
|
||||
return not ContextCompressor._is_context_summary_content(text)
|
||||
return not ContextCompressor._is_synthetic_compression_user_turn(message)
|
||||
|
||||
|
||||
def _merge_anchor_into_user_message(target: dict, anchor: dict) -> None:
|
||||
@@ -690,7 +776,10 @@ def _ensure_compressed_has_user_turn(original_messages: list, compressed: list)
|
||||
"""Preserve human intent, not merely a synthetic user-role placeholder."""
|
||||
if any(_is_real_user_message(message) for message in compressed):
|
||||
return
|
||||
from agent.context_compressor import _fresh_compaction_message_copy
|
||||
from agent.context_compressor import (
|
||||
COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
_fresh_compaction_message_copy,
|
||||
)
|
||||
|
||||
for message in reversed(original_messages):
|
||||
if _is_real_user_message(message):
|
||||
@@ -701,13 +790,77 @@ def _ensure_compressed_has_user_turn(original_messages: list, compressed: list)
|
||||
return
|
||||
compressed.append({
|
||||
"role": "user",
|
||||
"content": (
|
||||
"Continue from the compressed conversation context above. "
|
||||
"This marker exists because no human user turn was available."
|
||||
),
|
||||
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
})
|
||||
|
||||
|
||||
_PENDING_CONTEXT_ENGINE_NOTIFICATION = (
|
||||
"_pending_context_engine_compression_notification"
|
||||
)
|
||||
|
||||
|
||||
def _notify_context_engine_compression_complete(
|
||||
agent: Any,
|
||||
*,
|
||||
new_session_id: str,
|
||||
old_session_id: str,
|
||||
) -> bool:
|
||||
"""Notify the active context engine after a durable compression commit."""
|
||||
callback = getattr(agent.context_compressor, "on_session_start", None)
|
||||
if not callable(callback):
|
||||
return False
|
||||
try:
|
||||
callback(
|
||||
new_session_id,
|
||||
boundary_reason="compression",
|
||||
old_session_id=old_session_id,
|
||||
platform=getattr(agent, "platform", None) or "cli",
|
||||
conversation_id=getattr(agent, "_gateway_session_key", None),
|
||||
)
|
||||
except Exception:
|
||||
# Context-engine hooks are observers. A callback failure must not undo
|
||||
# history that the core or an outer host transaction already committed.
|
||||
logger.debug(
|
||||
"context engine on_session_start (compression) failed",
|
||||
exc_info=True,
|
||||
)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _queue_context_engine_compression_notification(
|
||||
agent: Any,
|
||||
*,
|
||||
new_session_id: str,
|
||||
old_session_id: str,
|
||||
) -> None:
|
||||
"""Stage exactly one existing hook call for an outer host transaction."""
|
||||
if callable(getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)):
|
||||
raise RuntimeError("a compression notification is already pending")
|
||||
|
||||
def _notify() -> bool:
|
||||
return _notify_context_engine_compression_complete(
|
||||
agent,
|
||||
new_session_id=new_session_id,
|
||||
old_session_id=old_session_id,
|
||||
)
|
||||
|
||||
setattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, _notify)
|
||||
|
||||
|
||||
def finalize_context_engine_compression_notification(
|
||||
agent: Any,
|
||||
*,
|
||||
committed: bool,
|
||||
) -> bool:
|
||||
"""Emit or discard a deferred notification; repeated calls are no-ops."""
|
||||
pending = getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)
|
||||
setattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None)
|
||||
if not committed or not callable(pending):
|
||||
return False
|
||||
return bool(pending())
|
||||
|
||||
|
||||
def compress_context(
|
||||
agent: Any,
|
||||
messages: list,
|
||||
@@ -717,6 +870,7 @@ def compress_context(
|
||||
task_id: str = "default",
|
||||
focus_topic: Optional[str] = None,
|
||||
force: bool = False,
|
||||
defer_context_engine_notification: bool = False,
|
||||
) -> Tuple[list, str]:
|
||||
"""Compress conversation context and split the session in SQLite.
|
||||
|
||||
@@ -734,6 +888,8 @@ def compress_context(
|
||||
by the manual ``/compress`` slash command so users can retry
|
||||
immediately after an auto-compress abort. Auto-compress
|
||||
callers use the default ``False``.
|
||||
defer_context_engine_notification: Delay the existing context-engine
|
||||
hook until a manual host commits its outer history transaction.
|
||||
|
||||
Returns:
|
||||
``(compressed_messages, new_system_prompt)`` tuple. When
|
||||
@@ -742,6 +898,12 @@ def compress_context(
|
||||
prompt — the session is NOT rotated. Callers should detect the
|
||||
no-op via ``len(returned) == len(input)`` and stop the retry loop.
|
||||
"""
|
||||
if (
|
||||
defer_context_engine_notification
|
||||
and callable(getattr(agent, _PENDING_CONTEXT_ENGINE_NOTIFICATION, None))
|
||||
):
|
||||
raise RuntimeError("a compression notification is already pending")
|
||||
|
||||
# Codex app-server sessions: the codex agent owns the real thread context;
|
||||
# Hermes' summarizer would only rewrite a local mirror without shrinking
|
||||
# the actual thread (#36801). Route compaction to the app server's own
|
||||
@@ -1020,6 +1182,7 @@ def compress_context(
|
||||
existing_prompt = agent._build_system_prompt(system_message)
|
||||
return messages, existing_prompt
|
||||
|
||||
_activity_heartbeat: Optional[_CompressionActivityHeartbeat] = None
|
||||
try:
|
||||
if _lock_holder is not None:
|
||||
_lock_refresher = _CompressionLockLeaseRefresher(
|
||||
@@ -1070,13 +1233,20 @@ def compress_context(
|
||||
)
|
||||
|
||||
messages_before_compression = copy.deepcopy(messages)
|
||||
_activity_heartbeat = _CompressionActivityHeartbeat(agent).start()
|
||||
compressed = compress_fn(messages, **compress_kwargs)
|
||||
except BaseException:
|
||||
# ANY exception after lock acquisition — memory hook, capability
|
||||
# inspection, engine lookup, or compress() — must release the lock so
|
||||
# the session isn't permanently blocked from future compression.
|
||||
if _activity_heartbeat is not None:
|
||||
_activity_heartbeat.stop("context compression failed")
|
||||
_activity_heartbeat = None
|
||||
_release_lock()
|
||||
raise
|
||||
finally:
|
||||
if _activity_heartbeat is not None:
|
||||
_activity_heartbeat.stop("context compression completed")
|
||||
|
||||
try:
|
||||
# Capture boundary quality before session-rotation callbacks run. Built-in
|
||||
@@ -1206,6 +1376,7 @@ def compress_context(
|
||||
new_system_prompt = agent._build_system_prompt(system_message)
|
||||
agent._cached_system_prompt = new_system_prompt
|
||||
|
||||
_session_commit_succeeded = False
|
||||
if agent._session_db:
|
||||
try:
|
||||
# Trigger memory extraction on the current session before the
|
||||
@@ -1386,6 +1557,7 @@ def compress_context(
|
||||
for message in compressed
|
||||
if isinstance(message, dict)
|
||||
}
|
||||
_session_commit_succeeded = True
|
||||
except Exception as e:
|
||||
# If the rotation rolled back to the parent (orphan-avoidance
|
||||
# above), agent.session_id is the still-indexed parent and
|
||||
@@ -1406,6 +1578,9 @@ def compress_context(
|
||||
# id on rotation, the (unchanged) current id in-place.
|
||||
_old_sid = locals().get("old_session_id")
|
||||
_is_boundary = bool(_old_sid) or in_place
|
||||
_context_engine_boundary_committed = _session_commit_succeeded and (
|
||||
bool(_old_sid) or compacted_in_place
|
||||
)
|
||||
_boundary_parent = _old_sid or agent.session_id or ""
|
||||
|
||||
# Notify the context engine that a compaction boundary occurred. Plugin
|
||||
@@ -1414,17 +1589,19 @@ def compress_context(
|
||||
# re-initializing fresh. See hermes-lcm#68. Built-in ContextCompressor
|
||||
# ignores kwargs. Fires in BOTH modes: rotation passes old→new ids; in-place
|
||||
# passes the SAME id (the boundary is real even though the id didn't move).
|
||||
try:
|
||||
if _is_boundary and hasattr(agent.context_compressor, "on_session_start"):
|
||||
agent.context_compressor.on_session_start(
|
||||
agent.session_id or "",
|
||||
boundary_reason="compression",
|
||||
if _context_engine_boundary_committed:
|
||||
if defer_context_engine_notification:
|
||||
_queue_context_engine_compression_notification(
|
||||
agent,
|
||||
new_session_id=agent.session_id or "",
|
||||
old_session_id=_boundary_parent,
|
||||
)
|
||||
else:
|
||||
_notify_context_engine_compression_complete(
|
||||
agent,
|
||||
new_session_id=agent.session_id or "",
|
||||
old_session_id=_boundary_parent,
|
||||
platform=getattr(agent, "platform", None) or "cli",
|
||||
conversation_id=getattr(agent, "_gateway_session_key", None),
|
||||
)
|
||||
except Exception as _ce_err:
|
||||
logger.debug("context engine on_session_start (compression): %s", _ce_err)
|
||||
|
||||
# Notify memory providers of the compaction boundary so provider-cached
|
||||
# per-session state (Hindsight's _document_id, accumulated turn buffers,
|
||||
@@ -1594,7 +1771,20 @@ def _compress_context_via_codex_app_server(
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
result = codex_session.compact_thread()
|
||||
_activity_heartbeat: Optional[_CompressionActivityHeartbeat] = None
|
||||
try:
|
||||
_activity_heartbeat = _CompressionActivityHeartbeat(agent).start()
|
||||
result = codex_session.compact_thread()
|
||||
except BaseException:
|
||||
if _activity_heartbeat is not None:
|
||||
_activity_heartbeat.stop("context compression failed")
|
||||
raise
|
||||
|
||||
if getattr(result, "interrupted", False) or getattr(result, "error", None):
|
||||
_activity_heartbeat.stop("context compression failed")
|
||||
else:
|
||||
_activity_heartbeat.stop("context compression completed")
|
||||
|
||||
if getattr(result, "should_retire", False):
|
||||
try:
|
||||
codex_session.close()
|
||||
|
||||
@@ -685,6 +685,13 @@ def run_conversation(
|
||||
truncated_tool_call_retries = 0
|
||||
truncated_response_parts: List[str] = []
|
||||
compression_attempts = 0
|
||||
# One resolved per-turn compression attempt cap, shared by every site that
|
||||
# consumes ``compression_attempts``: the pre-API pressure gate, the
|
||||
# overflow/413 retry handlers, and the post-tool compaction gate.
|
||||
# Config-driven via compression.max_attempts (parsed + validated in
|
||||
# agent_init); default 3 preserves the prior hardcoded behavior for
|
||||
# objects without the attribute (older pickles / minimal stubs).
|
||||
max_compression_attempts = getattr(agent, "max_compression_attempts", 3)
|
||||
_last_preflight_pressure: Optional[int] = None
|
||||
_preflight_compression_blocked = _ctx.preflight_compression_blocked
|
||||
_turn_exit_reason = "unknown" # Diagnostic: why the loop ended
|
||||
@@ -699,6 +706,10 @@ def run_conversation(
|
||||
# reused as the final response — not merely because any interim was
|
||||
# streamed. (#65919 review: response-loss blocker)
|
||||
_pending_verification_response_previewed = False
|
||||
# If pre-API compression fires after MoA advisors have produced guidance,
|
||||
# retain that ephemeral output and rebase it onto the compacted transcript
|
||||
# on the next loop iteration. This prevents a second advisor fan-out.
|
||||
pending_moa_prepared_request = None
|
||||
|
||||
# Per-turn tally of consecutive successful credential-pool token refreshes,
|
||||
# keyed by (provider, pool-entry-id). A persistent upstream 401 lets
|
||||
@@ -1082,6 +1093,29 @@ def run_conversation(
|
||||
# the OpenAI SDK. Sanitizing here prevents the 3-retry cycle.
|
||||
_sanitize_messages_surrogates(api_messages)
|
||||
|
||||
# Build a persistent-MoA request before measuring compression pressure.
|
||||
# MoA reference output is injected into the aggregator prompt, but it
|
||||
# is deliberately ephemeral and therefore absent from ``messages``.
|
||||
# Preparing here makes the pre-API guard measure the exact prompt the
|
||||
# aggregator will receive; ``create()`` consumes this private prepared
|
||||
# request later without running the advisors a second time.
|
||||
_moa_prepared_request = None
|
||||
if agent.provider == "moa":
|
||||
_moa_completions = getattr(getattr(agent.client, "chat", None), "completions", None)
|
||||
if pending_moa_prepared_request is not None:
|
||||
_rebase_moa_request = getattr(_moa_completions, "rebase_prepared_request", None)
|
||||
if callable(_rebase_moa_request):
|
||||
_moa_prepared_request = _rebase_moa_request(
|
||||
pending_moa_prepared_request, api_messages
|
||||
)
|
||||
pending_moa_prepared_request = None
|
||||
if _moa_prepared_request is None:
|
||||
_prepare_moa_request = getattr(_moa_completions, "prepare", None)
|
||||
if callable(_prepare_moa_request):
|
||||
_moa_prepared_request = _prepare_moa_request(api_messages)
|
||||
if _moa_prepared_request is not None:
|
||||
api_messages = _moa_prepared_request["messages"]
|
||||
|
||||
# One image-stripped message estimate feeds both figures. Was: a
|
||||
# str(msg) char walk (re-serialized base64 every call) + a second
|
||||
# messages walk inside estimate_request_tokens_rough. Tools added
|
||||
@@ -1168,21 +1202,24 @@ def run_conversation(
|
||||
if (
|
||||
agent.compression_enabled
|
||||
and len(messages) > 1
|
||||
and compression_attempts < 3
|
||||
and compression_attempts < max_compression_attempts
|
||||
and not _preflight_compression_blocked
|
||||
and not _defer_preflight(request_pressure_tokens)
|
||||
and not _compression_cooldown
|
||||
and _compressor.should_compress(request_pressure_tokens)
|
||||
):
|
||||
if _moa_prepared_request is not None:
|
||||
pending_moa_prepared_request = _moa_prepared_request
|
||||
compression_attempts += 1
|
||||
logger.info(
|
||||
"Pre-API compression: ~%s request tokens >= %s threshold "
|
||||
"(context=%s, attempt=%s/3)",
|
||||
"(context=%s, attempt=%s/%s)",
|
||||
f"{request_pressure_tokens:,}",
|
||||
f"{int(getattr(_compressor, 'threshold_tokens', 0) or 0):,}",
|
||||
f"{int(getattr(_compressor, 'context_length', 0) or 0):,}"
|
||||
if getattr(_compressor, "context_length", 0) else "unknown",
|
||||
compression_attempts,
|
||||
max_compression_attempts,
|
||||
)
|
||||
agent._emit_status(
|
||||
f"📦 Pre-API compression: ~{request_pressure_tokens:,} tokens "
|
||||
@@ -1252,7 +1289,6 @@ def run_conversation(
|
||||
retry_count = 0
|
||||
max_retries = agent._api_max_retries
|
||||
_retry = TurnRetryState()
|
||||
max_compression_attempts = 3
|
||||
|
||||
finish_reason = "stop"
|
||||
response = None # Guard against UnboundLocalError if all retries fail
|
||||
@@ -1421,6 +1457,12 @@ def run_conversation(
|
||||
if env_var_enabled("HERMES_DUMP_REQUESTS"):
|
||||
agent._dump_api_request_debug(api_kwargs, reason="preflight")
|
||||
|
||||
# This object is private to the in-process MoA facade. Add it
|
||||
# only after middleware, hooks, and debug dumps so none of them
|
||||
# attempts to serialize it as part of the provider payload.
|
||||
if _moa_prepared_request is not None and agent.provider == "moa":
|
||||
api_kwargs["_moa_prepared_request"] = _moa_prepared_request
|
||||
|
||||
# Always prefer the streaming path — even without stream
|
||||
# consumers. Streaming gives us fine-grained health
|
||||
# checking (90s stale-stream detection, 60s read timeout)
|
||||
@@ -5175,7 +5217,12 @@ def run_conversation(
|
||||
messages, tools=agent.tools or None
|
||||
)
|
||||
|
||||
if agent.compression_enabled and _compressor.should_compress(_real_tokens):
|
||||
if (
|
||||
agent.compression_enabled
|
||||
and compression_attempts < max_compression_attempts
|
||||
and _compressor.should_compress(_real_tokens)
|
||||
):
|
||||
compression_attempts += 1
|
||||
agent._safe_print(" ⟳ compacting context…")
|
||||
messages, active_system_prompt = agent._compress_context(
|
||||
messages, system_message,
|
||||
|
||||
+117
-63
@@ -905,7 +905,114 @@ class MoAChatCompletions:
|
||||
except Exception as exc: # pragma: no cover - display must never break the turn
|
||||
logger.debug("MoA reference_callback failed for %s: %s", event, exc)
|
||||
|
||||
def prepare(self, messages: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""Run the advisor fan-out and return the exact aggregator request.
|
||||
|
||||
The normal agent loop needs to measure this augmented prompt before its
|
||||
compression gate. ``create()`` also uses this method for direct callers;
|
||||
when the loop supplies the returned private object back to ``create()``,
|
||||
the advisor fan-out is not repeated.
|
||||
"""
|
||||
return self.create(messages=messages, _moa_prepare_only=True)
|
||||
|
||||
def rebase_prepared_request(
|
||||
self, prepared: dict[str, Any], messages: list[dict[str, Any]]
|
||||
) -> dict[str, Any]:
|
||||
"""Apply already-generated advisor guidance to a rebuilt API transcript.
|
||||
|
||||
Context compression changes the persisted transcript but not the
|
||||
ephemeral advisor result. Reusing the guidance avoids a second costly
|
||||
fan-out while keeping the aggregator request aligned with the compacted
|
||||
history.
|
||||
"""
|
||||
guidance = prepared.get("guidance")
|
||||
agg_messages = [dict(message) for message in messages]
|
||||
if guidance:
|
||||
_attach_reference_guidance(agg_messages, str(guidance))
|
||||
return {**prepared, "messages": agg_messages}
|
||||
|
||||
def _call_prepared_aggregator(
|
||||
self, prepared: dict[str, Any], api_kwargs: dict[str, Any]
|
||||
) -> Any:
|
||||
"""Send an already prepared MoA aggregator request exactly once."""
|
||||
agg_messages = prepared["messages"]
|
||||
aggregator = prepared["aggregator"]
|
||||
aggregator_temperature = prepared["aggregator_temperature"]
|
||||
if aggregator.get("provider") == "moa":
|
||||
raise RuntimeError("MoA aggregator cannot be another MoA preset")
|
||||
agg_kwargs = dict(api_kwargs)
|
||||
max_tokens: Any = agg_kwargs.get("max_tokens")
|
||||
tools: Any = agg_kwargs.get("tools")
|
||||
extra_body: Any = agg_kwargs.get("extra_body")
|
||||
# Record the exact aggregator INPUT (incl. the injected reference
|
||||
# context) into the pending trace so a trace captures what the
|
||||
# aggregator actually saw, not a reconstruction.
|
||||
if self._pending_trace is not None:
|
||||
self._pending_trace["aggregator_input_messages"] = agg_messages
|
||||
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
|
||||
# The aggregator is the acting model. Resolve its slot to the provider's
|
||||
# real runtime (base_url/api_key/api_mode) and call it through the same
|
||||
# request-building path any model uses — so per-model wire-format
|
||||
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
|
||||
# temperature) applies identically to it. MoA imposes no output cap:
|
||||
# max_tokens is passed through from the caller (normally None → omitted
|
||||
# → the model's real maximum). The preset's old hardcoded 4096 default
|
||||
# is gone — it truncated long syntheses.
|
||||
# When the agent's streaming consumer calls us with stream=True, run the
|
||||
# references first (above) and then return the aggregator's RAW token
|
||||
# stream so the acting model's output reaches the user live. The consumer
|
||||
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
|
||||
# back to a non-streaming retry on error. The non-streaming path
|
||||
# (stream=False) is unchanged — no stream/stream_options/timeout are
|
||||
# forwarded, so its behavior is byte-for-byte identical to before.
|
||||
stream = bool(api_kwargs.get("stream"))
|
||||
stream_kwargs: dict[str, Any] = {}
|
||||
if stream:
|
||||
stream_kwargs["stream"] = True
|
||||
stream_kwargs["stream_options"] = (
|
||||
api_kwargs.get("stream_options") or {"include_usage": True}
|
||||
)
|
||||
# Forward the consumer's per-request (stream read) timeout so it
|
||||
# actually governs the aggregator stream, not just call_llm's default.
|
||||
if api_kwargs.get("timeout") is not None:
|
||||
stream_kwargs["timeout"] = api_kwargs["timeout"]
|
||||
_agg_response = call_llm(
|
||||
task="moa_aggregator",
|
||||
messages=agg_messages,
|
||||
temperature=aggregator_temperature,
|
||||
max_tokens=max_tokens,
|
||||
tools=tools,
|
||||
extra_body=extra_body,
|
||||
# Prepared requests must retain the acting aggregator's reasoning
|
||||
# policy exactly as the direct create() path does (#64187).
|
||||
reasoning_config=_aggregator_reasoning_config(aggregator),
|
||||
**stream_kwargs,
|
||||
**_slot_runtime(aggregator),
|
||||
)
|
||||
# Non-streaming path (quiet mode / eval / subagents): the aggregator
|
||||
# output is available inline, so capture it into the pending trace now.
|
||||
# Streaming path: the aggregator's raw token stream is returned to the
|
||||
# consumer live and its acting output lands as the turn's assistant
|
||||
# message; the trace marks it streamed and points there.
|
||||
if self._pending_trace is not None:
|
||||
if stream:
|
||||
self._pending_trace["aggregator_streamed"] = True
|
||||
self._pending_trace["aggregator_output"] = None
|
||||
else:
|
||||
self._pending_trace["aggregator_streamed"] = False
|
||||
try:
|
||||
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
self._pending_trace["aggregator_output"] = None
|
||||
return _agg_response
|
||||
|
||||
def create(self, **api_kwargs: Any) -> Any:
|
||||
prepared_request = api_kwargs.pop("_moa_prepared_request", None)
|
||||
if prepared_request is not None:
|
||||
if not isinstance(prepared_request, dict):
|
||||
raise TypeError("_moa_prepared_request must be a dict")
|
||||
return self._call_prepared_aggregator(prepared_request, api_kwargs)
|
||||
|
||||
from hermes_cli.config import load_config
|
||||
from hermes_cli.moa_config import resolve_moa_preset
|
||||
|
||||
@@ -1065,6 +1172,7 @@ class MoAChatCompletions:
|
||||
ref_count=_ref_count,
|
||||
)
|
||||
|
||||
guidance: str | None = None
|
||||
agg_messages = [dict(m) for m in messages]
|
||||
if reference_outputs:
|
||||
joined = "\n\n".join(
|
||||
@@ -1082,69 +1190,15 @@ class MoAChatCompletions:
|
||||
)
|
||||
_attach_reference_guidance(agg_messages, guidance)
|
||||
|
||||
if aggregator.get("provider") == "moa":
|
||||
raise RuntimeError("MoA aggregator cannot be another MoA preset")
|
||||
agg_kwargs = dict(api_kwargs)
|
||||
agg_kwargs["messages"] = agg_messages
|
||||
# Record the exact aggregator INPUT (incl. the injected reference
|
||||
# context) into the pending trace so a trace captures what the
|
||||
# aggregator actually saw, not a reconstruction.
|
||||
if self._pending_trace is not None:
|
||||
self._pending_trace["aggregator_input_messages"] = agg_messages
|
||||
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
|
||||
# The aggregator is the acting model. Resolve its slot to the provider's
|
||||
# real runtime (base_url/api_key/api_mode) and call it through the same
|
||||
# request-building path any model uses — so per-model wire-format
|
||||
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
|
||||
# temperature) applies identically to it. MoA imposes no output cap:
|
||||
# max_tokens is passed through from the caller (normally None → omitted
|
||||
# → the model's real maximum). The preset's old hardcoded 4096 default
|
||||
# is gone — it truncated long syntheses.
|
||||
# When the agent's streaming consumer calls us with stream=True, run the
|
||||
# references first (above) and then return the aggregator's RAW token
|
||||
# stream so the acting model's output reaches the user live. The consumer
|
||||
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
|
||||
# back to a non-streaming retry on error. The non-streaming path
|
||||
# (stream=False) is unchanged — no stream/stream_options/timeout are
|
||||
# forwarded, so its behavior is byte-for-byte identical to before.
|
||||
stream = bool(api_kwargs.get("stream"))
|
||||
stream_kwargs: dict[str, Any] = {}
|
||||
if stream:
|
||||
stream_kwargs["stream"] = True
|
||||
stream_kwargs["stream_options"] = (
|
||||
api_kwargs.get("stream_options") or {"include_usage": True}
|
||||
)
|
||||
# Forward the consumer's per-request (stream read) timeout so it
|
||||
# actually governs the aggregator stream, not just call_llm's default.
|
||||
if api_kwargs.get("timeout") is not None:
|
||||
stream_kwargs["timeout"] = api_kwargs["timeout"]
|
||||
_agg_response = call_llm(
|
||||
task="moa_aggregator",
|
||||
messages=agg_messages,
|
||||
temperature=aggregator_temperature,
|
||||
max_tokens=agg_kwargs.get("max_tokens"),
|
||||
tools=agg_kwargs.get("tools"),
|
||||
extra_body=agg_kwargs.get("extra_body"),
|
||||
reasoning_config=_aggregator_reasoning_config(aggregator),
|
||||
**stream_kwargs,
|
||||
**_slot_runtime(aggregator),
|
||||
)
|
||||
# Non-streaming path (quiet mode / eval / subagents): the aggregator
|
||||
# output is available inline, so capture it into the pending trace now.
|
||||
# Streaming path: the aggregator's raw token stream is returned to the
|
||||
# consumer live and its acting output lands as the turn's assistant
|
||||
# message; the trace marks it streamed and points there.
|
||||
if self._pending_trace is not None:
|
||||
if stream:
|
||||
self._pending_trace["aggregator_streamed"] = True
|
||||
self._pending_trace["aggregator_output"] = None
|
||||
else:
|
||||
self._pending_trace["aggregator_streamed"] = False
|
||||
try:
|
||||
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
|
||||
except Exception: # pragma: no cover - defensive
|
||||
self._pending_trace["aggregator_output"] = None
|
||||
return _agg_response
|
||||
prepared_request = {
|
||||
"messages": agg_messages,
|
||||
"guidance": guidance,
|
||||
"aggregator": aggregator,
|
||||
"aggregator_temperature": aggregator_temperature,
|
||||
}
|
||||
if api_kwargs.pop("_moa_prepare_only", False):
|
||||
return prepared_request
|
||||
return self._call_prepared_aggregator(prepared_request, api_kwargs)
|
||||
|
||||
|
||||
class MoAClient:
|
||||
|
||||
+80
-6
@@ -2666,16 +2666,61 @@ async def get_model_context_length_async(
|
||||
)
|
||||
|
||||
|
||||
def _is_cjk_token_dense_char(ch: str) -> bool:
|
||||
code = ord(ch)
|
||||
return (
|
||||
0x1100 <= code <= 0x11FF # Hangul Jamo
|
||||
or 0x2E80 <= code <= 0x9FFF # CJK radicals/ideographs
|
||||
or 0xA960 <= code <= 0xA97F # Hangul Jamo Extended-A
|
||||
or 0xAC00 <= code <= 0xD7AF # Hangul Syllables
|
||||
or 0xF900 <= code <= 0xFAFF # CJK compatibility ideographs
|
||||
or 0xFF00 <= code <= 0xFFEF # Fullwidth forms / halfwidth kana
|
||||
)
|
||||
|
||||
|
||||
# Same codepoint ranges as _is_cjk_token_dense_char, as a compiled character
|
||||
# class so dense-char counting runs in C (``len(text) - len(re.sub(...))``)
|
||||
# instead of a per-char Python loop. MUST stay in sync with
|
||||
# _is_cjk_token_dense_char.
|
||||
_CJK_DENSE_RE = re.compile(
|
||||
"[\u1100-\u11ff" # Hangul Jamo
|
||||
"\u2e80-\u9fff" # CJK radicals/ideographs
|
||||
"\ua960-\ua97f" # Hangul Jamo Extended-A
|
||||
"\uac00-\ud7af" # Hangul Syllables
|
||||
"\uf900-\ufaff" # CJK compatibility ideographs
|
||||
"\uff00-\uffef]" # Fullwidth forms / halfwidth kana
|
||||
)
|
||||
|
||||
|
||||
def estimate_tokens_rough(text: str) -> int:
|
||||
"""Rough token estimate (~4 chars/token) for pre-flight checks.
|
||||
"""Rough token estimate for pre-flight checks.
|
||||
|
||||
Uses ceiling division so short texts (1-3 chars) never estimate as
|
||||
0 tokens, which would cause the compressor and pre-flight checks to
|
||||
systematically undercount when many short tool results are present.
|
||||
CJK/Hangul/Kana text is much denser than English under common LLM
|
||||
tokenizers, so count those codepoints as roughly one token each instead
|
||||
of applying the English-centric ~4 chars/token rule.
|
||||
|
||||
Perf: this runs on every message in every preflight/compaction walk,
|
||||
including MB-scale tool outputs, so the common all-ASCII case must stay
|
||||
O(1). ``str.isascii()`` is a flag check on CPython's compact unicode
|
||||
representation (no scan), and the CJK counting itself is a single
|
||||
C-level ``re.findall`` rather than a per-character Python loop.
|
||||
"""
|
||||
if not text:
|
||||
return 0
|
||||
return (len(text) + 3) // 4
|
||||
text = str(text)
|
||||
if text.isascii():
|
||||
# O(1) fast path — ASCII text cannot contain token-dense CJK chars.
|
||||
return (len(text) + 3) // 4
|
||||
dense = len(text) - len(_CJK_DENSE_RE.sub("", text))
|
||||
if not dense:
|
||||
# Non-ASCII but no CJK (accents, Cyrillic, emoji, ...): keep the
|
||||
# classic ~4 chars/token rule.
|
||||
return (len(text) + 3) // 4
|
||||
sparse = len(text) - dense
|
||||
return dense + ((sparse + 3) // 4)
|
||||
|
||||
|
||||
def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
|
||||
@@ -2687,12 +2732,12 @@ def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
|
||||
estimated at ~250K tokens and trigger premature context compression.
|
||||
"""
|
||||
_IMAGE_TOKEN_COST = 1500
|
||||
total_chars = 0
|
||||
text_tokens = 0
|
||||
image_tokens = 0
|
||||
for msg in messages:
|
||||
total_chars += _estimate_message_chars(msg)
|
||||
text_tokens += _estimate_message_tokens_without_images(msg)
|
||||
image_tokens += _count_image_tokens(msg, _IMAGE_TOKEN_COST)
|
||||
return ((total_chars + 3) // 4) + image_tokens
|
||||
return text_tokens + image_tokens
|
||||
|
||||
|
||||
def _count_image_tokens(msg: Dict[str, Any], cost_per_image: int) -> int:
|
||||
@@ -2754,6 +2799,35 @@ def _estimate_message_chars(msg: Dict[str, Any]) -> int:
|
||||
return len(str(shadow))
|
||||
|
||||
|
||||
def _estimate_message_tokens_without_images(msg: Dict[str, Any]) -> int:
|
||||
"""Token estimate for a message shadow with image payloads stripped."""
|
||||
if not isinstance(msg, dict):
|
||||
return estimate_tokens_rough(str(msg))
|
||||
shadow: Dict[str, Any] = {}
|
||||
for k, v in msg.items():
|
||||
if k == "_anthropic_content_blocks":
|
||||
continue
|
||||
if k == "content":
|
||||
if isinstance(v, list):
|
||||
cleaned = []
|
||||
for part in v:
|
||||
if isinstance(part, dict):
|
||||
if part.get("type") in {"image", "image_url", "input_image"}:
|
||||
cleaned.append({"type": part.get("type"), "image": "[stripped]"})
|
||||
else:
|
||||
cleaned.append(part)
|
||||
else:
|
||||
cleaned.append(part)
|
||||
shadow[k] = cleaned
|
||||
elif isinstance(v, dict) and v.get("_multimodal"):
|
||||
shadow[k] = v.get("text_summary", "")
|
||||
else:
|
||||
shadow[k] = v
|
||||
else:
|
||||
shadow[k] = v
|
||||
return estimate_tokens_rough(str(shadow))
|
||||
|
||||
|
||||
def estimate_request_tokens_rough(
|
||||
messages: List[Dict[str, Any]],
|
||||
*,
|
||||
@@ -2770,7 +2844,7 @@ def estimate_request_tokens_rough(
|
||||
"""
|
||||
total = 0
|
||||
if system_prompt:
|
||||
total += (len(system_prompt) + 3) // 4
|
||||
total += estimate_tokens_rough(system_prompt)
|
||||
if messages:
|
||||
total += estimate_messages_tokens_rough(messages)
|
||||
if tools:
|
||||
|
||||
@@ -92,6 +92,79 @@ class TurnResult:
|
||||
_TURN_ABORTED_MARKERS = ("<turn_aborted>", "<turn_aborted/>")
|
||||
|
||||
|
||||
def _notification_scope_ids(
|
||||
note: dict,
|
||||
) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Extract the thread/turn identity carried by a notification."""
|
||||
if not isinstance(note, dict):
|
||||
return None, None
|
||||
params = note.get("params") or {}
|
||||
if not isinstance(params, dict):
|
||||
return None, None
|
||||
|
||||
nested_turn = params.get("turn") or {}
|
||||
nested_item = params.get("item") or {}
|
||||
|
||||
observed_thread_id = params.get("threadId") or params.get("thread_id")
|
||||
if observed_thread_id is None and isinstance(nested_turn, dict):
|
||||
observed_thread_id = (
|
||||
nested_turn.get("threadId")
|
||||
or nested_turn.get("thread_id")
|
||||
)
|
||||
if observed_thread_id is None and isinstance(nested_item, dict):
|
||||
observed_thread_id = (
|
||||
nested_item.get("threadId")
|
||||
or nested_item.get("thread_id")
|
||||
)
|
||||
|
||||
observed_turn_id = params.get("turnId") or params.get("turn_id")
|
||||
if observed_turn_id is None and isinstance(nested_turn, dict):
|
||||
observed_turn_id = nested_turn.get("id") or nested_turn.get("turnId")
|
||||
if observed_turn_id is None and isinstance(nested_item, dict):
|
||||
observed_turn_id = (
|
||||
nested_item.get("turnId")
|
||||
or nested_item.get("turn_id")
|
||||
)
|
||||
|
||||
return observed_thread_id, observed_turn_id
|
||||
|
||||
|
||||
def _notification_belongs_to_turn(
|
||||
note: dict,
|
||||
*,
|
||||
thread_id: Optional[str],
|
||||
turn_id: Optional[str],
|
||||
) -> bool:
|
||||
"""Return whether a multiplexed notification belongs to this turn.
|
||||
|
||||
Codex app-server can carry parent and hosted subagent threads over one
|
||||
JSON-RPC connection. An explicitly foreign child or
|
||||
stale-turn event must not mutate the active parent's transcript or mark
|
||||
its turn complete. Unscoped notifications remain accepted for protocol
|
||||
compatibility.
|
||||
"""
|
||||
if not isinstance(note, dict):
|
||||
return False
|
||||
|
||||
observed_thread_id, observed_turn_id = _notification_scope_ids(note)
|
||||
|
||||
if (
|
||||
thread_id is not None
|
||||
and observed_thread_id is not None
|
||||
and str(observed_thread_id) != str(thread_id)
|
||||
):
|
||||
return False
|
||||
|
||||
if (
|
||||
turn_id is not None
|
||||
and observed_turn_id is not None
|
||||
and str(observed_turn_id) != str(turn_id)
|
||||
):
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def _coerce_turn_input_text(user_input: Any) -> str:
|
||||
"""Collapse Hermes/OpenAI rich content into app-server text input.
|
||||
|
||||
@@ -505,6 +578,17 @@ class CodexAppServerSession:
|
||||
pending = self._client.take_notification(timeout=0)
|
||||
if pending is None:
|
||||
break
|
||||
if not _notification_belongs_to_turn(
|
||||
pending,
|
||||
thread_id=self._thread_id,
|
||||
turn_id=result.turn_id,
|
||||
):
|
||||
logger.debug(
|
||||
"ignoring foreign codex notification while draining "
|
||||
"server request: method=%s",
|
||||
pending.get("method"),
|
||||
)
|
||||
continue
|
||||
# Mirror the main notification-handling block below so
|
||||
# display events surface and stay in step with projector
|
||||
# state. Without this, item/started / item/completed
|
||||
@@ -550,6 +634,16 @@ class CodexAppServerSession:
|
||||
continue
|
||||
|
||||
method = note.get("method", "")
|
||||
if not _notification_belongs_to_turn(
|
||||
note,
|
||||
thread_id=self._thread_id,
|
||||
turn_id=result.turn_id,
|
||||
):
|
||||
logger.debug(
|
||||
"ignoring foreign codex notification: method=%s", method
|
||||
)
|
||||
continue
|
||||
|
||||
if self._on_event is not None:
|
||||
try:
|
||||
self._on_event(note)
|
||||
@@ -737,6 +831,48 @@ class CodexAppServerSession:
|
||||
continue
|
||||
|
||||
method = note.get("method", "")
|
||||
observed_thread_id, observed_turn_id = _notification_scope_ids(note)
|
||||
if result.turn_id is None:
|
||||
if method == "turn/started":
|
||||
if (
|
||||
observed_thread_id is not None
|
||||
and str(observed_thread_id) != str(self._thread_id)
|
||||
):
|
||||
logger.debug(
|
||||
"ignoring foreign compact turn/started: thread=%s",
|
||||
observed_thread_id,
|
||||
)
|
||||
continue
|
||||
if observed_turn_id is None:
|
||||
logger.debug(
|
||||
"ignoring compact turn/started without a turn id"
|
||||
)
|
||||
continue
|
||||
result.turn_id = str(observed_turn_id)
|
||||
elif observed_turn_id is not None or method in {
|
||||
"item/completed",
|
||||
"turn/completed",
|
||||
}:
|
||||
# thread/compact/start does not return a turn id. Until the
|
||||
# new turn/started arrives, any terminal/projectable event
|
||||
# is stale or cannot be safely attributed to this compaction.
|
||||
logger.debug(
|
||||
"ignoring codex notification before compact turn start: "
|
||||
"method=%s",
|
||||
method,
|
||||
)
|
||||
continue
|
||||
|
||||
if not _notification_belongs_to_turn(
|
||||
note,
|
||||
thread_id=self._thread_id,
|
||||
turn_id=result.turn_id,
|
||||
):
|
||||
logger.debug(
|
||||
"ignoring foreign codex notification: method=%s", method
|
||||
)
|
||||
continue
|
||||
|
||||
if self._on_event is not None:
|
||||
try:
|
||||
self._on_event(note)
|
||||
|
||||
@@ -665,7 +665,13 @@ def build_turn_context(
|
||||
f">= {_compressor.threshold_tokens:,} threshold. "
|
||||
"This may take a moment."
|
||||
)
|
||||
for _pass in range(3):
|
||||
# Preflight passes honor the same configured per-turn cap
|
||||
# (compression.max_attempts) as the loop's compression sites;
|
||||
# default 3 preserves the prior hardcoded behavior.
|
||||
_max_preflight_passes = max(
|
||||
1, int(getattr(agent, "max_compression_attempts", 3) or 3)
|
||||
)
|
||||
for _pass in range(_max_preflight_passes):
|
||||
_orig_len = len(messages)
|
||||
_orig_tokens = _preflight_tokens
|
||||
messages, active_system_prompt = agent._compress_context(
|
||||
|
||||
@@ -31,7 +31,7 @@ describe('desktop slash command curation', () => {
|
||||
|
||||
it('hides terminal, messaging, and dedicated-UI commands from suggestions', () => {
|
||||
expect(isDesktopSlashSuggestion('/clear')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/compact')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/density')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/redraw')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/approve')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/model')).toBe(false)
|
||||
@@ -40,6 +40,14 @@ describe('desktop slash command curation', () => {
|
||||
expect(isDesktopSlashSuggestion('/curator')).toBe(false)
|
||||
})
|
||||
|
||||
it('routes /compact to /compress (context compression), not the TUI display toggle', () => {
|
||||
expect(resolveDesktopCommand('/compact')?.name).toBe('/compress')
|
||||
expect(isDesktopSlashCommand('/compact')).toBe(true)
|
||||
// Alias stays out of the popover so /compress is the single visible entry.
|
||||
expect(isDesktopSlashSuggestion('/compact')).toBe(false)
|
||||
expect(isDesktopSlashSuggestion('/compress')).toBe(true)
|
||||
})
|
||||
|
||||
it('surfaces /tools, /save, and /personality on the desktop', () => {
|
||||
expect(isDesktopSlashSuggestion('/tools')).toBe(true)
|
||||
expect(isDesktopSlashSuggestion('/save')).toBe(true)
|
||||
|
||||
@@ -148,7 +148,7 @@ const DESKTOP_COMMAND_SPECS: readonly DesktopCommandSpec[] = [
|
||||
surface: exec()
|
||||
},
|
||||
{ name: '/background', description: 'Run a prompt in the background', aliases: ['/bg', '/btw'], surface: exec() },
|
||||
{ name: '/compress', description: 'Compress this conversation context', surface: exec() },
|
||||
{ name: '/compress', description: 'Compress this conversation context', aliases: ['/compact'], surface: exec() },
|
||||
{ name: '/debug', description: 'Create a debug report', surface: exec() },
|
||||
{ name: '/goal', description: 'Manage the standing goal for this session', surface: exec() },
|
||||
{ name: '/personality', description: 'Switch personality for this session', surface: exec(), args: true },
|
||||
@@ -187,10 +187,10 @@ const NO_DESKTOP_SURFACE: Record<DesktopUnavailableReason, readonly string[]> =
|
||||
terminal: [
|
||||
'/busy',
|
||||
'/clear',
|
||||
'/compact',
|
||||
'/config',
|
||||
'/copy',
|
||||
'/cron',
|
||||
'/density',
|
||||
'/details',
|
||||
'/exit',
|
||||
'/footer',
|
||||
|
||||
@@ -414,6 +414,16 @@ compression:
|
||||
# compaction doesn't fire with half the window still free; set above 0.75 to override.
|
||||
threshold: 0.50
|
||||
|
||||
# Per-model threshold overrides: keys are substring-matched against the model
|
||||
# name (longest match wins). Useful when some models need different compaction
|
||||
# points — e.g. a 1M-context model can compress later (0.30) while a 128K
|
||||
# model needs to compress earlier (0.60). The small-context floor (75% for
|
||||
# <512K models) still applies on top of per-model overrides.
|
||||
# model_thresholds:
|
||||
# "glm-5.2": 0.40
|
||||
# "claude-sonnet": 0.35
|
||||
# "gpt-5": 0.30
|
||||
|
||||
# Existing Codex gpt-5.5 behavior: raise Hermes' compaction trigger to 85%
|
||||
# for the ChatGPT Codex OAuth route. Set false to opt back down to threshold.
|
||||
codex_gpt55_autoraise: true
|
||||
@@ -429,6 +439,12 @@ compression:
|
||||
# compression of older turns.
|
||||
protect_last_n: 20
|
||||
|
||||
# Compression retry rounds before a turn gives up with "max compression
|
||||
# attempts reached" (default: 3, same as the previous hardcoded value).
|
||||
# Raise (e.g. 6) for tool-schema-heavy sessions where 3 rounds cannot bring
|
||||
# the request estimate under the threshold. Validated >= 1, hard cap 10.
|
||||
max_attempts: 3
|
||||
|
||||
# Codex app-server (codex CLI runtime) thread-compaction mode. The codex
|
||||
# agent owns the real thread context on this runtime, so Hermes' summarizer
|
||||
# cannot shrink it — compaction goes through the app server instead.
|
||||
|
||||
@@ -9928,6 +9928,9 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
|
||||
split_history_for_partial_compress,
|
||||
summarize_compress_preview,
|
||||
)
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
# Args after the command word (e.g. "/compress here 3" -> "here 3").
|
||||
raw_args = ""
|
||||
@@ -10027,14 +10030,8 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
|
||||
approx_tokens=approx_tokens,
|
||||
focus_topic=focus_topic or None,
|
||||
force=True,
|
||||
defer_context_engine_notification=True,
|
||||
)
|
||||
# Re-append the verbatim tail after the compressed head.
|
||||
# The split guarantees `tail` begins on a user turn, so the
|
||||
# compressed-head -> tail boundary is normally valid
|
||||
# (the head's compressed output ends on assistant/tool).
|
||||
# rejoin_compressed_head_and_tail() additionally guards the
|
||||
# seam against any illegal user->user / assistant->assistant
|
||||
# adjacency, defending provider role-alternation rules.
|
||||
if partial and tail:
|
||||
compressed = rejoin_compressed_head_and_tail(compressed, tail)
|
||||
self.conversation_history = compressed
|
||||
@@ -10054,6 +10051,10 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
|
||||
# compressed handoff for the child session. Persist it from
|
||||
# offset 0 so resume can recover the continuation after exit.
|
||||
self.agent._flush_messages_to_session_db(self.conversation_history, None)
|
||||
finalize_context_engine_compression_notification(
|
||||
self.agent,
|
||||
committed=True,
|
||||
)
|
||||
new_tokens = estimate_request_tokens_rough(
|
||||
self.conversation_history,
|
||||
system_prompt=_sys_prompt,
|
||||
@@ -10078,6 +10079,10 @@ class HermesCLI(CLIAgentSetupMixin, CLICommandsMixin, CLIBillingMixin):
|
||||
print(f" {summary['note']}")
|
||||
|
||||
except Exception as e:
|
||||
finalize_context_engine_compression_notification(
|
||||
self.agent,
|
||||
committed=False,
|
||||
)
|
||||
print(f" ❌ Compression failed: {e}")
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
WeiYusc
|
||||
@@ -0,0 +1,2 @@
|
||||
bpross
|
||||
# PR #29541 salvage
|
||||
@@ -0,0 +1 @@
|
||||
bennybuoy
|
||||
@@ -0,0 +1 @@
|
||||
LiangYang666
|
||||
@@ -0,0 +1,2 @@
|
||||
sowork-skills
|
||||
# PR #33224 salvage (slack: truncate edit_message to prevent msg_too_long)
|
||||
@@ -0,0 +1 @@
|
||||
tandixit95
|
||||
@@ -0,0 +1 @@
|
||||
dombejar
|
||||
@@ -0,0 +1 @@
|
||||
dsitmilis
|
||||
@@ -0,0 +1,2 @@
|
||||
kamonspecial
|
||||
# PR #59621 salvage (slack: empty rich_text content guards)
|
||||
@@ -0,0 +1 @@
|
||||
LiangYang666
|
||||
@@ -0,0 +1 @@
|
||||
plainOldCode
|
||||
@@ -0,0 +1,2 @@
|
||||
tw0316
|
||||
# PR #56618 salvage (slack: table column_settings + blocks rejection fallback)
|
||||
@@ -0,0 +1,2 @@
|
||||
xcompass
|
||||
# PR #61261 salvage
|
||||
@@ -0,0 +1 @@
|
||||
Sora-bluesky
|
||||
@@ -0,0 +1 @@
|
||||
yitang
|
||||
@@ -0,0 +1 @@
|
||||
the3asic
|
||||
@@ -0,0 +1 @@
|
||||
wernerhp
|
||||
@@ -127,6 +127,8 @@ MAX_REQUEST_BYTES = 10_000_000 # 10 MB — accommodates long agent conversation
|
||||
CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS = 30.0
|
||||
MAX_NORMALIZED_TEXT_LENGTH = 65_536 # 64 KB cap for normalized content parts
|
||||
MAX_CONTENT_LIST_SIZE = 1_000 # Max items when content is an array
|
||||
RESPONSES_AUTO_TRUNCATION_HISTORY_LIMIT = 100
|
||||
_COMPRESSED_SUMMARY_METADATA_KEY = "_compressed_summary"
|
||||
|
||||
|
||||
def _coerce_port(value: Any, default: int = DEFAULT_PORT) -> int:
|
||||
@@ -166,6 +168,71 @@ def _coerce_request_bool(value: Any, default: bool = False) -> bool:
|
||||
return default
|
||||
|
||||
|
||||
def _message_text_prefix(content: Any) -> str:
|
||||
if isinstance(content, str):
|
||||
return content[:128]
|
||||
if not isinstance(content, list):
|
||||
return ""
|
||||
parts: List[str] = []
|
||||
for item in content[:4]:
|
||||
if isinstance(item, str):
|
||||
parts.append(item)
|
||||
elif isinstance(item, dict):
|
||||
text = item.get("text")
|
||||
if isinstance(text, str):
|
||||
parts.append(text)
|
||||
if sum(len(part) for part in parts) >= 128:
|
||||
break
|
||||
return "\n".join(parts)[:128]
|
||||
|
||||
|
||||
def _is_compressed_summary_message(message: Any) -> bool:
|
||||
if not isinstance(message, dict):
|
||||
return False
|
||||
if message.get(_COMPRESSED_SUMMARY_METADATA_KEY):
|
||||
return True
|
||||
prefix = _message_text_prefix(message.get("content"))
|
||||
return prefix.startswith("[CONTEXT COMPACTION") or prefix.startswith("[CONTEXT SUMMARY]:")
|
||||
|
||||
|
||||
def _auto_truncate_response_history(
|
||||
conversation_history: List[Dict[str, Any]],
|
||||
*,
|
||||
limit: int = RESPONSES_AUTO_TRUNCATION_HISTORY_LIMIT,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Keep recent Responses history without dropping the compaction handoff.
|
||||
|
||||
Compaction summaries are preserved wherever they sit in the history —
|
||||
the gateway /compress path can leave them after a retained system head
|
||||
(see ``context_compressor`` force-user-leading handling), so a
|
||||
leading-block-only scan would silently drop them.
|
||||
"""
|
||||
if limit <= 0 or len(conversation_history) <= limit:
|
||||
return conversation_history
|
||||
|
||||
summary_indices = [
|
||||
index
|
||||
for index, message in enumerate(conversation_history)
|
||||
if _is_compressed_summary_message(message)
|
||||
]
|
||||
if not summary_indices:
|
||||
return conversation_history[-limit:]
|
||||
|
||||
kept_indices = set(summary_indices[:limit])
|
||||
remaining = limit - len(kept_indices)
|
||||
if remaining > 0:
|
||||
summary_index_set = set(summary_indices)
|
||||
for index in range(len(conversation_history) - 1, -1, -1):
|
||||
if index in summary_index_set:
|
||||
continue
|
||||
kept_indices.add(index)
|
||||
remaining -= 1
|
||||
if remaining <= 0:
|
||||
break
|
||||
|
||||
return [conversation_history[index] for index in sorted(kept_indices)]
|
||||
|
||||
|
||||
def _normalize_chat_content(
|
||||
content: Any, *, _max_depth: int = 10, _depth: int = 0,
|
||||
) -> str:
|
||||
@@ -3310,6 +3377,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
response_env: Dict[str, Any],
|
||||
*,
|
||||
conversation_history_snapshot: Optional[List[Dict[str, Any]]] = None,
|
||||
session_id_snapshot: Optional[str] = None,
|
||||
) -> None:
|
||||
if not store:
|
||||
return
|
||||
@@ -3320,7 +3388,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"response": response_env,
|
||||
"conversation_history": conversation_history_snapshot,
|
||||
"instructions": instructions,
|
||||
"session_id": session_id,
|
||||
"session_id": session_id_snapshot or session_id,
|
||||
})
|
||||
if conversation:
|
||||
self._response_store.set_conversation(conversation, response_id)
|
||||
@@ -3723,9 +3791,15 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
result,
|
||||
final_response_text,
|
||||
)
|
||||
# Compression-aware transcript substitution happens inside
|
||||
# _build_response_conversation_history (result["_compressed"]);
|
||||
# here we only propagate a compression-rotated session_id so
|
||||
# previous_response_id chaining resumes the child session.
|
||||
_result_sid = result.get("session_id") if isinstance(result, dict) else None
|
||||
_persist_response_snapshot(
|
||||
completed_env,
|
||||
conversation_history_snapshot=full_history,
|
||||
session_id_snapshot=_result_sid if isinstance(_result_sid, str) and _result_sid else None,
|
||||
)
|
||||
terminal_snapshot_persisted = True
|
||||
await _write_event("response.completed", {
|
||||
@@ -3898,8 +3972,8 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
return web.json_response(_openai_error("No user message found in input"), status=400)
|
||||
|
||||
# Truncation support
|
||||
if body.get("truncation") == "auto" and len(conversation_history) > 100:
|
||||
conversation_history = conversation_history[-100:]
|
||||
if body.get("truncation") == "auto":
|
||||
conversation_history = _auto_truncate_response_history(conversation_history)
|
||||
|
||||
# Reuse session from previous_response_id chain so the dashboard
|
||||
# groups the entire conversation under one session entry.
|
||||
@@ -4038,6 +4112,16 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
final_response,
|
||||
)
|
||||
|
||||
# Persist the effective session ID surfaced by _run_agent so that
|
||||
# compression-triggered session rotations propagate to the stored
|
||||
# response and the X-Hermes-Session-Id header. Without this,
|
||||
# previous_response_id chaining keeps resuming the pre-rotation
|
||||
# session and re-triggers compression on every subsequent request.
|
||||
_effective_session_id = session_id
|
||||
_result_sid = result.get("session_id") if isinstance(result, dict) else None
|
||||
if isinstance(_result_sid, str) and _result_sid:
|
||||
_effective_session_id = _result_sid
|
||||
|
||||
# Build output items from the current turn only. AIAgent returns a
|
||||
# full transcript in result["messages"], while older/mocked paths may
|
||||
# return only the current turn suffix.
|
||||
@@ -4068,14 +4152,14 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"response": response_data,
|
||||
"conversation_history": full_history,
|
||||
"instructions": instructions,
|
||||
"session_id": session_id,
|
||||
"session_id": _effective_session_id,
|
||||
})
|
||||
# Update conversation mapping so the next request with the same
|
||||
# conversation name automatically chains to this response
|
||||
if conversation:
|
||||
self._response_store.set_conversation(conversation, response_id)
|
||||
|
||||
response_headers = {"X-Hermes-Session-Id": session_id}
|
||||
response_headers = {"X-Hermes-Session-Id": _effective_session_id}
|
||||
if gateway_session_key:
|
||||
response_headers["X-Hermes-Session-Key"] = gateway_session_key
|
||||
return web.json_response(response_data, headers=response_headers)
|
||||
@@ -4432,7 +4516,16 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
result: Dict[str, Any],
|
||||
final_response: Any,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Build the stored Responses transcript without duplicating history."""
|
||||
"""Build the stored Responses transcript without duplicating history.
|
||||
|
||||
When context compression occurs during a turn the agent returns a
|
||||
compressed full transcript in ``result["messages"]`` (starting with a
|
||||
summary) and sets ``result["_compressed"] = True``. Because the
|
||||
compressed transcript does not share the input ``conversation_history``
|
||||
prefix, the normal turn-start detection fails and old code would
|
||||
concatenate the uncompressed history on front, bloating the stored
|
||||
context and re-triggering compression on every subsequent request.
|
||||
"""
|
||||
prior = list(conversation_history)
|
||||
current_user = {"role": "user", "content": user_message}
|
||||
agent_messages = result.get("messages") if isinstance(result, dict) else None
|
||||
@@ -4446,6 +4539,16 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
if turn_start:
|
||||
return list(agent_messages)
|
||||
|
||||
# turn_start == 0: agent_messages does not start with prior.
|
||||
# This can happen because compression rewrote the transcript
|
||||
# (summary prefix replaces original history), OR because
|
||||
# agent_messages only carries the current turn without prior.
|
||||
# The ``_compressed`` flag (set by _run_agent after compaction)
|
||||
# distinguishes — skip the concatenation and use the compressed
|
||||
# transcript directly.
|
||||
if result.get("_compressed"):
|
||||
return list(agent_messages)
|
||||
|
||||
full_history = prior
|
||||
full_history.append(current_user)
|
||||
full_history.extend(agent_messages)
|
||||
@@ -4706,6 +4809,18 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
_eff_sid = getattr(agent, "session_id", session_id)
|
||||
if isinstance(_eff_sid, str) and _eff_sid:
|
||||
result["session_id"] = _eff_sid
|
||||
# Signal whether context compression occurred during this turn
|
||||
# so _build_response_conversation_history can skip the
|
||||
# prior-concatenation path and store the compressed transcript
|
||||
# directly. Rotation mode changes agent.session_id; in-place
|
||||
# mode sets _last_compaction_in_place (see #38763).
|
||||
_compacted_in_place = bool(getattr(agent, "_last_compaction_in_place", False))
|
||||
_session_rotated = (
|
||||
isinstance(_eff_sid, str) and isinstance(session_id, str)
|
||||
and _eff_sid != session_id
|
||||
)
|
||||
if _compacted_in_place or _session_rotated:
|
||||
result["_compressed"] = True
|
||||
return result, usage
|
||||
finally:
|
||||
clear_session_vars(tokens)
|
||||
|
||||
@@ -17643,6 +17643,7 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
|
||||
("model", "max_tokens"),
|
||||
("compression", "enabled"),
|
||||
("compression", "threshold"),
|
||||
("compression", "model_thresholds"),
|
||||
("compression", "codex_gpt55_autoraise"),
|
||||
("compression", "codex_app_server_auto"),
|
||||
("compression", "target_ratio"),
|
||||
@@ -19626,6 +19627,17 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
|
||||
if event_type not in {"tool.started",}:
|
||||
return
|
||||
|
||||
# Never render a progress bubble for the clarify tool. The
|
||||
# adapter's send_clarify IS the user-facing rendering (interactive
|
||||
# buttons or the numbered-text fallback), so a progress bubble is
|
||||
# pure duplication — and in verbose mode it dumps the raw
|
||||
# tool-call args JSON ({"question": ..., "choices": [...]}) into
|
||||
# the chat. Because the progress queue drains on a background
|
||||
# task, that raw JSON typically lands right underneath the
|
||||
# rendered prompt (#52374).
|
||||
if tool_name == "clarify":
|
||||
return
|
||||
|
||||
# Suppress tool-progress bubbles once the user has sent `stop`.
|
||||
# When the LLM response carries N parallel tool calls, the agent
|
||||
# fires N "tool.started" events back-to-back before checking for
|
||||
|
||||
@@ -3423,6 +3423,9 @@ class GatewaySlashCommandsMixin:
|
||||
split_history_for_partial_compress,
|
||||
summarize_compress_preview,
|
||||
)
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
_raw_args = (event.get_command_args() or "").strip()
|
||||
# Strip --preview/--dry-run/--aggressive before positional parsing
|
||||
# so the flags coexist with 'here [N]' / focus-topic forms.
|
||||
@@ -3551,11 +3554,15 @@ class GatewaySlashCommandsMixin:
|
||||
loop = asyncio.get_running_loop()
|
||||
compressed, _ = await loop.run_in_executor(
|
||||
None,
|
||||
lambda: tmp_agent._compress_context(head, "", approx_tokens=approx_tokens, focus_topic=focus_topic, force=True)
|
||||
lambda: tmp_agent._compress_context(
|
||||
head,
|
||||
"",
|
||||
approx_tokens=approx_tokens,
|
||||
focus_topic=focus_topic,
|
||||
force=True,
|
||||
defer_context_engine_notification=True,
|
||||
)
|
||||
)
|
||||
|
||||
# Re-append the verbatim tail after the compressed head,
|
||||
# guarding the seam against illegal role adjacency.
|
||||
if partial and tail:
|
||||
compressed = rejoin_compressed_head_and_tail(compressed, tail)
|
||||
|
||||
@@ -3625,6 +3632,10 @@ class GatewaySlashCommandsMixin:
|
||||
await self.async_session_store.update_session(
|
||||
session_entry.session_key, last_prompt_tokens=0
|
||||
)
|
||||
finalize_context_engine_compression_notification(
|
||||
tmp_agent,
|
||||
committed=True,
|
||||
)
|
||||
new_tokens = estimate_request_tokens_rough(
|
||||
compressed, system_prompt=_sys_prompt, tools=_tools
|
||||
)
|
||||
@@ -3655,6 +3666,10 @@ class GatewaySlashCommandsMixin:
|
||||
_aux_fail_model = getattr(compressor, "_last_aux_model_failure_model", None)
|
||||
_aux_fail_err = getattr(compressor, "_last_aux_model_failure_error", None)
|
||||
finally:
|
||||
finalize_context_engine_compression_notification(
|
||||
tmp_agent,
|
||||
committed=False,
|
||||
)
|
||||
# Evict cached agent so next turn rebuilds system prompt
|
||||
# from current files (SOUL.md, memory, etc.).
|
||||
self._evict_cached_agent(session_key)
|
||||
@@ -4151,6 +4166,7 @@ class GatewaySlashCommandsMixin:
|
||||
# replays the parent's exact wire bytes (warm provider
|
||||
# prompt cache) instead of a full cold prefill.
|
||||
api_content=extract_api_content_sidecar(msg),
|
||||
timestamp=msg.get("timestamp"),
|
||||
)
|
||||
except Exception:
|
||||
pass # Best-effort copy
|
||||
|
||||
@@ -968,6 +968,7 @@ class CLICommandsMixin:
|
||||
# replays the parent's exact wire bytes (warm provider
|
||||
# prompt cache) instead of a full cold prefill.
|
||||
api_content=extract_api_content_sidecar(msg),
|
||||
timestamp=msg.get("timestamp"),
|
||||
)
|
||||
except Exception:
|
||||
pass # Best-effort copy
|
||||
|
||||
@@ -1471,6 +1471,11 @@ DEFAULT_CONFIG = {
|
||||
# set this above 0.75 to override the floor.
|
||||
"target_ratio": 0.20, # fraction of threshold to preserve as recent tail
|
||||
"protect_last_n": 20, # minimum recent messages to keep uncompressed
|
||||
"max_attempts": 3, # compression retry rounds before a turn gives up
|
||||
# with "max compression attempts reached". Raise
|
||||
# (e.g. 6) for tool-schema-heavy sessions where 3
|
||||
# rounds cannot clear the request estimate.
|
||||
# Validated >= 1, hard-capped at 10.
|
||||
"hygiene_hard_message_limit": 5000, # gateway session-hygiene force-compress threshold by message count
|
||||
"protect_first_n": 3, # non-system head messages always preserved
|
||||
# verbatim, in ADDITION to the system prompt
|
||||
@@ -1529,6 +1534,17 @@ DEFAULT_CONFIG = {
|
||||
# session_search and recoverable, not deleted.
|
||||
# Default False during rollout; will flip on
|
||||
# after live validation.
|
||||
"model_thresholds": {}, # Per-model threshold overrides. Keys are
|
||||
# substring-matched against the model name
|
||||
# (longest match wins); values replace the
|
||||
# global `threshold` for that model, e.g.
|
||||
# model_thresholds:
|
||||
# "glm-5.2": 0.40
|
||||
# "claude-sonnet": 0.35
|
||||
# The small-context floor (0.75 for <512K
|
||||
# models) still applies on top of overrides
|
||||
# (raise-only: an override above the floor
|
||||
# wins; one below it is raised to the floor).
|
||||
},
|
||||
|
||||
# Kanban subsystem (orchestrator workers + dispatcher-driven child tasks).
|
||||
|
||||
+79
-3
@@ -2709,6 +2709,7 @@ class SessionDB:
|
||||
parent_session_id: str = None,
|
||||
cwd: str = None,
|
||||
profile_name: str = None,
|
||||
git_repo_root: str = None,
|
||||
) -> None:
|
||||
"""Insert a session row, enriching NULL metadata on conflict.
|
||||
|
||||
@@ -2727,14 +2728,30 @@ class SessionDB:
|
||||
a persisted, now-inactive row belongs to the caller's chat/thread before
|
||||
switching to it (IDOR scoping — without them the ``sessions`` table has
|
||||
no chat/thread to compare).
|
||||
|
||||
When ``parent_session_id`` is set (compression fork, delegate/subagent
|
||||
spawn, branch continuation) and this row's own ``cwd``/``git_repo_root``/
|
||||
``git_branch`` are still NULL after the insert, they are backfilled from
|
||||
the parent row. Callers of ``create_session`` for a child session
|
||||
historically didn't propagate these fields themselves (e.g. the
|
||||
compression-fork path), so a lineage could silently lose its working
|
||||
directory and drop out of the project sidebar every time it forked
|
||||
(#64709). This only fills NULLs — an explicit ``cwd``/``git_repo_root``
|
||||
on the child is never overwritten. For compression forks specifically
|
||||
(parent ended with ``end_reason='compression'``), the gateway origin
|
||||
columns (``user_id``/``session_key``/``chat_id``/``chat_type``/
|
||||
``thread_id``/``display_name``/``origin_json``) are inherited too, so a
|
||||
crash before the gateway re-records the peer can't strand the child
|
||||
without a recoverable routing mapping (#59527).
|
||||
"""
|
||||
def _do(conn):
|
||||
conn.execute(
|
||||
"""INSERT INTO sessions (
|
||||
id, source, user_id, session_key, chat_id, chat_type, thread_id,
|
||||
model, model_config, system_prompt, parent_session_id, cwd, profile_name, started_at
|
||||
model, model_config, system_prompt, parent_session_id, cwd,
|
||||
profile_name, git_repo_root, started_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
model = COALESCE(sessions.model, excluded.model),
|
||||
model_config = COALESCE(sessions.model_config, excluded.model_config),
|
||||
@@ -2745,7 +2762,8 @@ class SessionDB:
|
||||
thread_id = COALESCE(sessions.thread_id, excluded.thread_id),
|
||||
parent_session_id = COALESCE(sessions.parent_session_id, excluded.parent_session_id),
|
||||
cwd = COALESCE(sessions.cwd, excluded.cwd),
|
||||
profile_name = COALESCE(sessions.profile_name, excluded.profile_name)""",
|
||||
profile_name = COALESCE(sessions.profile_name, excluded.profile_name),
|
||||
git_repo_root = COALESCE(sessions.git_repo_root, excluded.git_repo_root)""",
|
||||
(
|
||||
session_id,
|
||||
source,
|
||||
@@ -2760,9 +2778,67 @@ class SessionDB:
|
||||
parent_session_id,
|
||||
cwd,
|
||||
profile_name,
|
||||
git_repo_root,
|
||||
time.time(),
|
||||
),
|
||||
)
|
||||
if parent_session_id:
|
||||
conn.execute(
|
||||
"""UPDATE sessions
|
||||
SET cwd = COALESCE(sessions.cwd,
|
||||
(SELECT p.cwd FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
git_repo_root = COALESCE(sessions.git_repo_root,
|
||||
(SELECT p.git_repo_root FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
git_branch = COALESCE(sessions.git_branch,
|
||||
(SELECT p.git_branch FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id))
|
||||
WHERE id = ? AND parent_session_id IS NOT NULL""",
|
||||
(session_id,),
|
||||
)
|
||||
# Belt-and-suspenders for gateway routing metadata (#59527):
|
||||
# the gateway re-records the peer on the child after rotation
|
||||
# (d5b4879d4), but a hard crash between child creation and that
|
||||
# write leaves the child row without origin columns, so
|
||||
# ``find_latest_gateway_session_for_peer`` can't recover the
|
||||
# mapping on restart. Inherit them from the parent at creation
|
||||
# time — but ONLY for compression forks (parent already ended
|
||||
# with end_reason='compression'). Delegate/subagent children
|
||||
# are spawned while the parent is still live and must NOT
|
||||
# inherit routing keys, or peer recovery could repoint gateway
|
||||
# traffic into a subagent's session.
|
||||
conn.execute(
|
||||
"""UPDATE sessions
|
||||
SET user_id = COALESCE(sessions.user_id,
|
||||
(SELECT p.user_id FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
session_key = COALESCE(sessions.session_key,
|
||||
(SELECT p.session_key FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
chat_id = COALESCE(sessions.chat_id,
|
||||
(SELECT p.chat_id FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
chat_type = COALESCE(sessions.chat_type,
|
||||
(SELECT p.chat_type FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
thread_id = COALESCE(sessions.thread_id,
|
||||
(SELECT p.thread_id FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
display_name = COALESCE(sessions.display_name,
|
||||
(SELECT p.display_name FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id)),
|
||||
origin_json = COALESCE(sessions.origin_json,
|
||||
(SELECT p.origin_json FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id))
|
||||
WHERE id = ? AND parent_session_id IS NOT NULL
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM sessions p
|
||||
WHERE p.id = sessions.parent_session_id
|
||||
AND p.end_reason = 'compression'
|
||||
)""",
|
||||
(session_id,),
|
||||
)
|
||||
self._execute_write(_do)
|
||||
|
||||
def create_session(self, session_id: str, source: str, **kwargs) -> str:
|
||||
|
||||
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
|
||||
|
||||
from agent.memory_provider import MemoryProvider
|
||||
from tools.registry import tool_error
|
||||
from utils import is_truthy_value
|
||||
from .store import MemoryStore
|
||||
from .retrieval import FactRetriever
|
||||
from hermes_cli.config import cfg_get
|
||||
@@ -235,7 +236,10 @@ class HolographicMemoryProvider(MemoryProvider):
|
||||
return tool_error(f"Unknown tool: {tool_name}")
|
||||
|
||||
def on_session_end(self, messages: List[Dict[str, Any]]) -> None:
|
||||
if not self._config.get("auto_extract", False):
|
||||
# is_truthy_value: the config schema declares auto_extract as a string
|
||||
# enum ("false"/"true"), and a plain truthiness check treats the string
|
||||
# "false" as enabled (#57682).
|
||||
if not is_truthy_value(self._config.get("auto_extract", False)):
|
||||
return
|
||||
if not self._store or not messages:
|
||||
return
|
||||
@@ -368,6 +372,35 @@ class HolographicMemoryProvider(MemoryProvider):
|
||||
# -- Auto-extraction (on_session_end) ------------------------------------
|
||||
|
||||
def _auto_extract_facts(self, messages: list) -> None:
|
||||
# Local import (pattern used in initialize()): the compressor module is
|
||||
# heavier than this plugin and is only needed when auto_extract is on.
|
||||
from agent.context_compressor import (
|
||||
_MERGED_PRIOR_CONTEXT_HEADER,
|
||||
_MERGED_SUMMARY_DELIMITER,
|
||||
is_compaction_summary_message,
|
||||
)
|
||||
|
||||
def _pre_delimiter_user_segment(msg: dict):
|
||||
"""Return the genuine user text preceding a merged-into-tail
|
||||
compaction summary, or None when the whole message is a summary.
|
||||
|
||||
Merge-into-tail messages (agent/context_compressor.py ~3163-3190)
|
||||
wrap real prior tail content BEFORE ``_MERGED_SUMMARY_DELIMITER``,
|
||||
prefixed with ``_MERGED_PRIOR_CONTEXT_HEADER``, then append the
|
||||
generated handoff summary AFTER the delimiter. Dropping the whole
|
||||
row (as ``is_compaction_summary_message`` alone would suggest)
|
||||
discards that genuine pre-delimiter content too (#57690 review).
|
||||
Only the summary suffix must be excluded from harvesting.
|
||||
"""
|
||||
content = msg.get("content", "")
|
||||
if not isinstance(content, str) or _MERGED_SUMMARY_DELIMITER not in content:
|
||||
return None
|
||||
pre = content.split(_MERGED_SUMMARY_DELIMITER, 1)[0]
|
||||
if pre.startswith(_MERGED_PRIOR_CONTEXT_HEADER):
|
||||
pre = pre[len(_MERGED_PRIOR_CONTEXT_HEADER):]
|
||||
pre = pre.strip()
|
||||
return pre or None
|
||||
|
||||
_PREF_PATTERNS = [
|
||||
re.compile(r'\bI\s+(?:prefer|like|love|use|want|need)\s+(.+)', re.IGNORECASE),
|
||||
re.compile(r'\bmy\s+(?:favorite|preferred|default)\s+\w+\s+is\s+(.+)', re.IGNORECASE),
|
||||
@@ -382,7 +415,20 @@ class HolographicMemoryProvider(MemoryProvider):
|
||||
for msg in messages:
|
||||
if msg.get("role") != "user":
|
||||
continue
|
||||
content = msg.get("content", "")
|
||||
# Compaction handoff summaries can be inserted as role="user"
|
||||
# messages; their prose reliably matches the decision patterns, so
|
||||
# without this guard the compactor's own output is stored as a
|
||||
# durable "fact" on every rollover (#57682). A merge-into-tail
|
||||
# summary also carries genuine pre-delimiter user content in the
|
||||
# SAME row; harvest that segment instead of dropping the whole
|
||||
# message (#57690 review).
|
||||
pre_delimiter_segment = _pre_delimiter_user_segment(msg)
|
||||
if pre_delimiter_segment is not None:
|
||||
content = pre_delimiter_segment
|
||||
elif is_compaction_summary_message(msg):
|
||||
continue
|
||||
else:
|
||||
content = msg.get("content", "")
|
||||
if not isinstance(content, str) or len(content) < 10:
|
||||
continue
|
||||
|
||||
|
||||
@@ -56,9 +56,9 @@ from gateway.platforms.base import (
|
||||
)
|
||||
|
||||
try: # sibling module; support both package and flat plugin-dir import
|
||||
from .block_kit import render_blocks
|
||||
from .block_kit import render_blocks, sanitize_blocks
|
||||
except ImportError: # pragma: no cover - plugin loaded outside package context
|
||||
from block_kit import render_blocks # type: ignore
|
||||
from block_kit import render_blocks, sanitize_blocks # type: ignore
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -113,6 +113,63 @@ def check_slack_requirements() -> bool:
|
||||
return ensure_and_bind("platform.slack", _import, globals(), prompt=False)
|
||||
|
||||
|
||||
def _collect_slack_block_mentions(blocks: list) -> list:
|
||||
"""Return ``<@UID>`` mention tokens authored in non-quoted Block Kit text.
|
||||
|
||||
Slack's flat top-level ``text`` field does NOT contain mentions that were
|
||||
authored only inside Block Kit ``blocks`` (e.g. a ``rich_text_section`` with
|
||||
a ``user`` element). This walker recovers those mentions so the gates can
|
||||
see Block-Kit-only mentions instead of silently dropping them (#52387).
|
||||
|
||||
Mentions nested inside ``rich_text_quote`` (quoted/forwarded content) are
|
||||
deliberately ignored, so quoted text cannot trick the bot into responding
|
||||
(matches the existing channel-routing contract).
|
||||
"""
|
||||
mentions: list = []
|
||||
|
||||
def _walk(node, in_quote: bool) -> None:
|
||||
if isinstance(node, list):
|
||||
for item in node:
|
||||
_walk(item, in_quote)
|
||||
return
|
||||
if not isinstance(node, dict):
|
||||
return
|
||||
node_type = node.get("type")
|
||||
quoted = in_quote or node_type == "rich_text_quote"
|
||||
if node_type == "user" and not quoted:
|
||||
uid = node.get("user_id", "")
|
||||
if uid:
|
||||
mentions.append(f"<@{uid}>")
|
||||
for key in ("elements", "element"):
|
||||
child = node.get(key)
|
||||
if child is not None:
|
||||
_walk(child, quoted)
|
||||
|
||||
try:
|
||||
_walk(blocks, False)
|
||||
except Exception: # pragma: no cover - defensive, never break gating
|
||||
return []
|
||||
return mentions
|
||||
|
||||
|
||||
def _slack_mention_detection_text(event: dict) -> str:
|
||||
"""Return the text used for @mention detection on a Slack message event.
|
||||
|
||||
Combines the flat top-level ``text`` with any ``<@UID>`` mentions recovered
|
||||
from non-quoted Block Kit blocks (#52387), so a genuine Block-Kit-only
|
||||
mention reaches the gates while quoted/forwarded mentions stay ignored.
|
||||
"""
|
||||
flat = event.get("text", "") or ""
|
||||
blocks = event.get("blocks")
|
||||
if not blocks:
|
||||
return flat
|
||||
mentions = _collect_slack_block_mentions(blocks)
|
||||
extra = [m for m in mentions if m not in flat]
|
||||
if not extra:
|
||||
return flat
|
||||
return (flat.strip() + "\n" + " ".join(extra)).strip()
|
||||
|
||||
|
||||
def _extract_text_from_slack_blocks(blocks: list) -> str:
|
||||
"""Extract readable text from Slack Block Kit blocks, including quoted/forwarded content.
|
||||
|
||||
@@ -208,6 +265,47 @@ def _extract_text_from_slack_blocks(blocks: list) -> str:
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def _extract_text_from_slack_attachments(attachments: list) -> str:
|
||||
"""Extract readable text from legacy Slack message ``attachments``.
|
||||
|
||||
Apps such as Alertmanager, Grafana, PagerDuty, and CI bots post messages
|
||||
with an empty top-level ``text`` and the real content inside ``attachments``
|
||||
(Slack's legacy secondary-content format) or nested Block Kit ``blocks``.
|
||||
Without this, such messages are invisible when the agent reads thread
|
||||
history — e.g. an alert that started the very thread the agent was asked to
|
||||
investigate would come through blank.
|
||||
|
||||
Prefers structured fields (``pretext``/``title``/``text``/``fields``) and
|
||||
only falls back to an attachment's ``fallback`` string when it carries
|
||||
nothing else.
|
||||
"""
|
||||
if not attachments:
|
||||
return ""
|
||||
|
||||
lines: list[str] = []
|
||||
for att in attachments:
|
||||
if not isinstance(att, dict):
|
||||
continue
|
||||
got: list[str] = [
|
||||
str(att[key]) for key in ("pretext", "title", "text") if att.get(key)
|
||||
]
|
||||
for field in att.get("fields", []) or []:
|
||||
if not isinstance(field, dict):
|
||||
continue
|
||||
got += [str(field[k]) for k in ("title", "value") if field.get(k)]
|
||||
nested = att.get("blocks")
|
||||
if nested:
|
||||
block_text = _extract_text_from_slack_blocks(nested)
|
||||
if block_text:
|
||||
got.append(block_text)
|
||||
# Only use the (often duplicative) fallback when nothing structured exists.
|
||||
if not got and att.get("fallback"):
|
||||
got.append(str(att["fallback"]))
|
||||
lines += got
|
||||
|
||||
return "\n".join(line for line in lines if line).strip()
|
||||
|
||||
|
||||
def _serialize_slack_blocks_for_agent(blocks: list, max_chars: int = 6000) -> str:
|
||||
"""Return a compact, redacted JSON view of the current message's Block Kit payload."""
|
||||
if not blocks:
|
||||
@@ -275,6 +373,47 @@ def _serialize_slack_blocks_for_agent(blocks: list, max_chars: int = 6000) -> st
|
||||
return f"[Slack Block Kit payload for this message]\n```json\n{payload}\n```"
|
||||
|
||||
|
||||
def _extract_urls_from_slack_blocks(blocks: list) -> list[str]:
|
||||
"""Walk a Block Kit ``blocks`` tree and return URLs found on any element.
|
||||
|
||||
Returns URLs preserving discovery order with duplicates removed. Used to
|
||||
surface the actionable links (``View graph``, ``View incident``, etc.)
|
||||
embedded in bot-posted alerts so an agent reading the thread can fetch
|
||||
or click them. The companion serializer
|
||||
:func:`_serialize_slack_blocks_for_agent` deliberately strips ``url`` to
|
||||
keep the JSON view compact and to avoid exposing arbitrary URLs through
|
||||
the generic payload dump; this helper is the targeted opt-in for
|
||||
use sites where URLs are the whole point of the message.
|
||||
"""
|
||||
if not blocks:
|
||||
return []
|
||||
|
||||
found: list[str] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def _maybe_add(value: Any) -> None:
|
||||
if isinstance(value, str) and value.startswith(("http://", "https://")):
|
||||
if value not in seen:
|
||||
seen.add(value)
|
||||
found.append(value)
|
||||
|
||||
def _walk(node: Any) -> None:
|
||||
if isinstance(node, dict):
|
||||
# The common URL-bearing keys across Block Kit (buttons, link
|
||||
# elements in rich_text, image accessories, etc.).
|
||||
for key in ("url", "image_url", "external_url"):
|
||||
if key in node:
|
||||
_maybe_add(node[key])
|
||||
for value in node.values():
|
||||
_walk(value)
|
||||
elif isinstance(node, list):
|
||||
for item in node:
|
||||
_walk(item)
|
||||
|
||||
_walk(blocks)
|
||||
return found
|
||||
|
||||
|
||||
def _apply_slack_proxy(client: Any, proxy_url: Optional[str]) -> None:
|
||||
"""Apply a resolved proxy to a Slack SDK client or clear it explicitly."""
|
||||
if hasattr(client, "proxy"):
|
||||
@@ -459,6 +598,9 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
# Track pending approval message_ts → resolved flag to prevent
|
||||
# double-clicks on approval buttons.
|
||||
self._approval_resolved: Dict[str, bool] = {}
|
||||
# Same guard for clarify prompts (interactive multiple-choice
|
||||
# buttons); mirrors _approval_resolved.
|
||||
self._clarify_resolved: Dict[str, bool] = {}
|
||||
# Track timestamps of messages sent by the bot so we can respond
|
||||
# to thread replies even without an explicit @mention.
|
||||
self._bot_message_ts: set = set()
|
||||
@@ -1221,6 +1363,15 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
|
||||
self._app.action("hermes_feedback")(self._handle_feedback_action)
|
||||
|
||||
# Register Block Kit action handlers for clarify buttons
|
||||
# (interactive multiple-choice prompts; see tools/clarify_gateway.py).
|
||||
# Choice buttons use indexed action IDs so each ID is unique within
|
||||
# its actions block, as required by Slack's Block Kit schema.
|
||||
self._app.action(
|
||||
_re.compile(r"^hermes_clarify_choice_\d+$")
|
||||
)(self._handle_clarify_action)
|
||||
self._app.action("hermes_clarify_other")(self._handle_clarify_action)
|
||||
|
||||
# Register plugin-provided Block Kit action handlers.
|
||||
#
|
||||
# Plugins call ``ctx.register_slack_action_handler(action_id, cb)``
|
||||
@@ -1427,6 +1578,11 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
# Convert standard markdown → Slack mrkdwn
|
||||
formatted = self.format_message(content)
|
||||
|
||||
# Guard against empty/whitespace-only messages — Slack API
|
||||
# returns ``no_text`` for chat.postMessage with blank text.
|
||||
if not formatted or not formatted.strip():
|
||||
return SendResult(success=True)
|
||||
|
||||
# Split long messages, preserving code block boundaries
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
|
||||
@@ -1458,9 +1614,23 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if broadcast and i == 0:
|
||||
kwargs["reply_broadcast"] = True
|
||||
|
||||
last_result = await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_postMessage(**kwargs)
|
||||
try:
|
||||
last_result = await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_postMessage(**kwargs)
|
||||
except Exception as e:
|
||||
if kwargs.get("blocks") and self._is_block_payload_rejection(e):
|
||||
retry_kwargs = dict(kwargs)
|
||||
retry_kwargs.pop("blocks", None)
|
||||
logger.info(
|
||||
"[Slack] Block Kit payload rejected; retrying send without blocks: %s",
|
||||
e,
|
||||
)
|
||||
last_result = await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_postMessage(**retry_kwargs)
|
||||
else:
|
||||
raise
|
||||
|
||||
# Clear Slack Assistant status as soon as the final message is posted.
|
||||
if thread_ts:
|
||||
@@ -1543,6 +1713,12 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
formatted = self.format_message(content)
|
||||
# Slack's chat.update has the same ~40k char limit as postMessage.
|
||||
# Unlike send() we can't split into multiple messages (we're
|
||||
# editing an existing one), so truncate to fit — an oversized
|
||||
# payload fails the whole edit with ``msg_too_long``.
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
formatted = chunks[0] if chunks else formatted
|
||||
update_kwargs: Dict[str, Any] = {
|
||||
"channel": chat_id,
|
||||
"ts": message_id,
|
||||
@@ -1556,9 +1732,26 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
blocks = self._maybe_blocks(content)
|
||||
if blocks:
|
||||
update_kwargs["blocks"] = blocks
|
||||
await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_update(**update_kwargs)
|
||||
try:
|
||||
await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_update(**update_kwargs)
|
||||
except Exception as e:
|
||||
if update_kwargs.get("blocks") and self._is_block_payload_rejection(e):
|
||||
retry_kwargs = dict(update_kwargs)
|
||||
# Explicitly clear any stale blocks when falling back to the
|
||||
# flat text update path; otherwise Slack can preserve the
|
||||
# prior block layout for an edited message.
|
||||
retry_kwargs["blocks"] = []
|
||||
logger.info(
|
||||
"[Slack] Block Kit payload rejected; retrying edit without blocks: %s",
|
||||
e,
|
||||
)
|
||||
await self._get_client(
|
||||
chat_id, team_id=self._metadata_team_id(metadata)
|
||||
).chat_update(**retry_kwargs)
|
||||
else:
|
||||
raise
|
||||
if finalize:
|
||||
await self.stop_typing(chat_id, metadata=metadata)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
@@ -2010,6 +2203,31 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
|
||||
# ----- Markdown → mrkdwn conversion -----
|
||||
|
||||
@staticmethod
|
||||
def _is_block_payload_rejection(error: BaseException) -> bool:
|
||||
"""Return True for Slack errors recoverable by removing ``blocks``.
|
||||
|
||||
Rich Block Kit output is a progressive enhancement over the plain
|
||||
``text`` fallback. If Slack rejects the structured payload as invalid
|
||||
or too large, retrying the same content without blocks is safe and
|
||||
prevents a formatting bug from dropping the whole response.
|
||||
"""
|
||||
recoverable_codes = {
|
||||
"invalid_blocks",
|
||||
"msg_too_long",
|
||||
"too_many_blocks",
|
||||
}
|
||||
response = getattr(error, "response", None)
|
||||
response_get = getattr(response, "get", None)
|
||||
if callable(response_get):
|
||||
try:
|
||||
if response_get("error") in recoverable_codes:
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
message = str(error)
|
||||
return any(code in message for code in recoverable_codes)
|
||||
|
||||
def _rich_blocks_enabled(self) -> bool:
|
||||
"""Whether to render outbound agent messages as Slack Block Kit blocks.
|
||||
|
||||
@@ -2078,7 +2296,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return None
|
||||
try:
|
||||
blocks = render_blocks(content, mrkdwn_fn=self.format_message)
|
||||
return self._append_feedback_block(blocks)
|
||||
return sanitize_blocks(self._append_feedback_block(blocks))
|
||||
except Exception: # pragma: no cover - renderer already guards itself
|
||||
logger.debug("[Slack] block render failed; using plain text", exc_info=True)
|
||||
return None
|
||||
@@ -3127,8 +3345,14 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if allow_bots == "none":
|
||||
return
|
||||
elif allow_bots == "mentions":
|
||||
text_check = event.get("text", "")
|
||||
# Include Block-Kit-only mentions, not just the flat text (#52387)
|
||||
text_check = _slack_mention_detection_text(event)
|
||||
if self._bot_user_id and f"<@{self._bot_user_id}>" not in text_check:
|
||||
logger.debug(
|
||||
"[Slack] Dropping bot message under allow_bots=mentions: "
|
||||
"no <@%s> mention in flat text or blocks",
|
||||
self._bot_user_id,
|
||||
)
|
||||
return
|
||||
# "all" falls through to process the message
|
||||
# Always ignore our own messages to prevent echo loops
|
||||
@@ -3348,7 +3572,8 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
# 3. The message is in a thread where the bot was previously @mentioned, OR
|
||||
# 4. There's an existing session for this thread (survives restarts)
|
||||
bot_uid = self._team_bot_user_ids.get(team_id, self._bot_user_id)
|
||||
routing_text = original_text or ""
|
||||
# Detect mentions authored only inside Block Kit blocks too (#52387)
|
||||
routing_text = _slack_mention_detection_text(event) or original_text or ""
|
||||
is_mentioned = bool(
|
||||
(bot_uid and f"<@{bot_uid}>" in routing_text)
|
||||
or self._slack_message_matches_mention_patterns(routing_text)
|
||||
@@ -3874,7 +4099,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
kwargs: Dict[str, Any] = {
|
||||
"channel": chat_id,
|
||||
"text": f"⚠️ Command approval required: {cmd_preview[:100]}",
|
||||
"blocks": blocks,
|
||||
"blocks": sanitize_blocks(blocks),
|
||||
}
|
||||
if thread_ts:
|
||||
kwargs["thread_ts"] = thread_ts
|
||||
@@ -3954,7 +4179,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
kwargs: Dict[str, Any] = {
|
||||
"channel": chat_id,
|
||||
"text": f"{title or 'Confirm'}: {body[:100]}",
|
||||
"blocks": blocks,
|
||||
"blocks": sanitize_blocks(blocks),
|
||||
}
|
||||
if thread_ts:
|
||||
kwargs["thread_ts"] = thread_ts
|
||||
@@ -3969,6 +4194,105 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
logger.error("[Slack] send_slash_confirm failed: %s", e, exc_info=True)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
async def send_clarify(
|
||||
self,
|
||||
chat_id: str,
|
||||
question: str,
|
||||
choices: Optional[list],
|
||||
clarify_id: str,
|
||||
session_key: str,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Render a clarify prompt as Block Kit interactive buttons.
|
||||
|
||||
Multi-choice mode (``choices`` non-empty): one button per option
|
||||
(unique ``hermes_clarify_choice_<idx>`` action_id, ``value`` packs
|
||||
``clarify_id|idx``) plus a final "✏️ Other…" button
|
||||
(``hermes_clarify_other``). A choice click resolves the clarify
|
||||
primitive directly; the "Other" button flips the entry into
|
||||
text-capture mode so the gateway's platform-agnostic text-intercept
|
||||
(:meth:`GatewayRunner._handle_message`) picks up the next typed
|
||||
message and resolves the clarify — no Slack-specific text machinery.
|
||||
|
||||
Open-ended mode (``choices`` empty): delegates to the base
|
||||
implementation, which renders the plain question and arms the same
|
||||
text-intercept.
|
||||
"""
|
||||
# Open-ended prompts have no buttons — the base implementation renders
|
||||
# the plain question and arms the gateway text-intercept for us.
|
||||
if not choices:
|
||||
return await super().send_clarify(
|
||||
chat_id=chat_id,
|
||||
question=question,
|
||||
choices=choices,
|
||||
clarify_id=clarify_id,
|
||||
session_key=session_key,
|
||||
metadata=metadata,
|
||||
)
|
||||
|
||||
if not self._app:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
|
||||
try:
|
||||
thread_ts = self._resolve_thread_ts(None, metadata)
|
||||
|
||||
# Escape the Slack mrkdwn control chars (&, <, >) so a question
|
||||
# containing them renders literally instead of as markup/mentions.
|
||||
# Section text caps at 3000 chars — budget the question so the
|
||||
# wrapper never pushes the block over the limit (overflow →
|
||||
# invalid_blocks → no buttons).
|
||||
q = (question or "").replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
body = f"❓ {q}"
|
||||
budget = 3000 - len("...")
|
||||
if len(body) > budget:
|
||||
body = body[:budget] + "..."
|
||||
|
||||
# One button per choice + a free-text "Other" button. Slack caps
|
||||
# an actions block at 5 elements; the clarify tool caps choices at
|
||||
# 4 (+ Other = 5) so this is normally one block, but chunk anyway
|
||||
# so a larger choice list degrades gracefully instead of 400ing.
|
||||
elements = []
|
||||
for idx, choice in enumerate(choices):
|
||||
label = str(choice).strip() or f"Option {idx + 1}"
|
||||
elements.append({
|
||||
"type": "button",
|
||||
"text": {"type": "plain_text", "text": label[:75], "emoji": True},
|
||||
"action_id": f"hermes_clarify_choice_{idx}",
|
||||
"value": f"{clarify_id}|{idx}",
|
||||
})
|
||||
elements.append({
|
||||
"type": "button",
|
||||
"text": {"type": "plain_text", "text": "✏️ Other…", "emoji": True},
|
||||
"action_id": "hermes_clarify_other",
|
||||
"value": f"{clarify_id}|other",
|
||||
})
|
||||
|
||||
blocks: list = [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": body}},
|
||||
]
|
||||
for start in range(0, len(elements), 5):
|
||||
blocks.append({"type": "actions", "elements": elements[start:start + 5]})
|
||||
|
||||
kwargs: Dict[str, Any] = {
|
||||
"channel": chat_id,
|
||||
"text": body,
|
||||
"blocks": blocks,
|
||||
}
|
||||
if thread_ts:
|
||||
kwargs["thread_ts"] = thread_ts
|
||||
|
||||
result = await self._get_client(chat_id).chat_postMessage(**kwargs)
|
||||
msg_ts = result.get("ts", "")
|
||||
if msg_ts:
|
||||
# Mark unresolved so the action handler's atomic-pop guard can
|
||||
# reject double-clicks (mirrors _approval_resolved).
|
||||
self._clarify_resolved[msg_ts] = False
|
||||
|
||||
return SendResult(success=True, message_id=msg_ts, raw_response=result)
|
||||
except Exception as e:
|
||||
logger.error("[Slack] send_clarify failed: %s", e, exc_info=True)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
def _is_interactive_user_authorized(
|
||||
self,
|
||||
user_id: str,
|
||||
@@ -4098,6 +4422,11 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
original_text = block.get("text", {}).get("text", "")
|
||||
break
|
||||
|
||||
# Slack re-escapes HTML entities in the interaction payload
|
||||
# (< → <, > → >, & → &), which can inflate the text
|
||||
# past the 3000-char section-block limit on chat.update.
|
||||
original_text = original_text[:3000]
|
||||
|
||||
updated_blocks = [
|
||||
{
|
||||
"type": "section",
|
||||
@@ -4119,7 +4448,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
channel=channel_id,
|
||||
ts=msg_ts,
|
||||
text=decision_text,
|
||||
blocks=updated_blocks,
|
||||
blocks=sanitize_blocks(updated_blocks),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("[Slack] Failed to update slash-confirm message: %s", e)
|
||||
@@ -4265,6 +4594,11 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
original_text = block.get("text", {}).get("text", "")
|
||||
break
|
||||
|
||||
# Slack re-escapes HTML entities in the interaction payload
|
||||
# (< → <, > → >, & → &), which can inflate the text
|
||||
# past the 3000-char section-block limit on chat.update.
|
||||
original_text = original_text[:3000]
|
||||
|
||||
updated_blocks = [
|
||||
{
|
||||
"type": "section",
|
||||
@@ -4286,15 +4620,197 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
channel=channel_id,
|
||||
ts=msg_ts,
|
||||
text=decision_text,
|
||||
blocks=updated_blocks,
|
||||
blocks=sanitize_blocks(updated_blocks),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("[Slack] Failed to update approval message: %s", e)
|
||||
|
||||
# (approval already resolved above; state consumed by atomic pop)
|
||||
|
||||
async def _update_clarify_message(
|
||||
self,
|
||||
channel_id: str,
|
||||
msg_ts: str,
|
||||
question_text: str,
|
||||
decision_text: str,
|
||||
) -> None:
|
||||
"""Rewrite a clarify message to show the outcome and drop the buttons."""
|
||||
updated_blocks = [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {"type": "mrkdwn", "text": question_text or "Clarification"},
|
||||
},
|
||||
{
|
||||
"type": "context",
|
||||
"elements": [{"type": "mrkdwn", "text": decision_text}],
|
||||
},
|
||||
]
|
||||
try:
|
||||
await self._get_client(channel_id).chat_update(
|
||||
channel=channel_id,
|
||||
ts=msg_ts,
|
||||
text=decision_text,
|
||||
blocks=updated_blocks,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("[Slack] Failed to update clarify message: %s", e)
|
||||
|
||||
async def _handle_clarify_action(self, ack, body, action) -> None:
|
||||
"""Handle a clarify button click (a choice or "Other") from Block Kit."""
|
||||
await ack()
|
||||
|
||||
action_id = action.get("action_id", "")
|
||||
value = action.get("value", "")
|
||||
message = body.get("message", {})
|
||||
msg_ts = message.get("ts", "")
|
||||
channel_id = body.get("channel", {}).get("id", "")
|
||||
user_name = body.get("user", {}).get("name", "unknown")
|
||||
user_id = body.get("user", {}).get("id", "")
|
||||
|
||||
if not self._is_interactive_user_authorized(
|
||||
user_id,
|
||||
channel_id=channel_id,
|
||||
user_name=user_name,
|
||||
):
|
||||
logger.warning(
|
||||
"[Slack] Unauthorized clarify click by %s (%s) - ignoring",
|
||||
user_name, user_id,
|
||||
)
|
||||
return
|
||||
|
||||
# value packs ``clarify_id|<idx|other>``.
|
||||
if "|" not in value:
|
||||
logger.warning("[Slack] Malformed clarify value: %s", value)
|
||||
return
|
||||
clarify_id, token = value.split("|", 1)
|
||||
|
||||
# Double-click guard — atomic pop; first caller gets False (proceed),
|
||||
# any later click gets the True default and bails (mirrors approval).
|
||||
if self._clarify_resolved.pop(msg_ts, True):
|
||||
return
|
||||
|
||||
# Preserve the original question so the resolved message keeps context.
|
||||
original_text = ""
|
||||
for block in message.get("blocks", []):
|
||||
if block.get("type") == "section":
|
||||
original_text = block.get("text", {}).get("text", "")
|
||||
break
|
||||
|
||||
from tools import clarify_gateway as _clarify_mod
|
||||
|
||||
# "Other" → enter text-capture mode. The gateway's text-intercept
|
||||
# resolves the clarify from the user's next typed message, so there is
|
||||
# no Slack-side text bookkeeping: mark_awaiting_text flips the entry and
|
||||
# GatewayRunner._handle_message does the rest.
|
||||
if action_id == "hermes_clarify_other" or token == "other":
|
||||
if not _clarify_mod.mark_awaiting_text(clarify_id):
|
||||
# Entry evicted (clarify_timeout) or gateway restarted between
|
||||
# ask and tap — a typed answer would go nowhere.
|
||||
await self._update_clarify_message(
|
||||
channel_id, msg_ts, original_text,
|
||||
f"⏳ This prompt expired — please send a new request. (by {user_name})",
|
||||
)
|
||||
return
|
||||
await self._update_clarify_message(
|
||||
channel_id, msg_ts, original_text,
|
||||
f"✏️ Awaiting typed answer from {user_name}…",
|
||||
)
|
||||
return
|
||||
|
||||
# Numeric choice → resolve immediately with the chosen option text.
|
||||
try:
|
||||
idx = int(token)
|
||||
except (ValueError, TypeError):
|
||||
logger.warning("[Slack] Invalid clarify choice token: %s", token)
|
||||
return
|
||||
|
||||
# Look up the canonical choice text from the registered entry (mirrors
|
||||
# the Telegram adapter); fall back to a positional label on a race with
|
||||
# timeout / session reset.
|
||||
resolved_text: Optional[str] = None
|
||||
try:
|
||||
entry = _clarify_mod._entries.get(clarify_id) # type: ignore[attr-defined]
|
||||
if entry and entry.choices and 0 <= idx < len(entry.choices):
|
||||
resolved_text = str(entry.choices[idx])
|
||||
except Exception:
|
||||
resolved_text = None
|
||||
if resolved_text is None:
|
||||
resolved_text = f"choice {idx + 1}"
|
||||
|
||||
if _clarify_mod.resolve_gateway_clarify(clarify_id, resolved_text):
|
||||
await self._update_clarify_message(
|
||||
channel_id, msg_ts, original_text,
|
||||
f"✅ {user_name}: {resolved_text}",
|
||||
)
|
||||
logger.info(
|
||||
"Slack button resolved clarify (id=%s, choice=%r, user=%s)",
|
||||
clarify_id, resolved_text, user_name,
|
||||
)
|
||||
else:
|
||||
# Entry evicted / gateway restarted — surface expiry instead of a
|
||||
# misleading ✓ on a button the agent will never receive.
|
||||
await self._update_clarify_message(
|
||||
channel_id, msg_ts, original_text,
|
||||
f"⏳ This prompt expired — please send a new request. (by {user_name})",
|
||||
)
|
||||
logger.warning(
|
||||
"[Slack] clarify resolve returned False (id=%s) — expired/reset",
|
||||
clarify_id,
|
||||
)
|
||||
|
||||
# ----- Thread context fetching -----
|
||||
|
||||
@staticmethod
|
||||
def _render_message_text(msg: dict, bot_uid: str = "") -> str:
|
||||
"""Return bounded display text for a Slack message, surfacing Block Kit content.
|
||||
|
||||
Starts with ``text``, strips bot mentions, then appends rich-text
|
||||
content and actionable URLs from ``blocks`` when present. Unlike
|
||||
:func:`_serialize_slack_blocks_for_agent` (which can emit up to
|
||||
6 000 chars of JSON per message), this helper produces only the
|
||||
readable text and URL list needed by thread-context and parent-
|
||||
text rendering — bounded by what the blocks actually contain,
|
||||
not a JSON dump.
|
||||
"""
|
||||
msg_text = (msg.get("text") or "").strip()
|
||||
if bot_uid:
|
||||
msg_text = msg_text.replace(f"<@{bot_uid}>", "").strip()
|
||||
|
||||
blocks = msg.get("blocks")
|
||||
extras: list[str] = []
|
||||
if blocks:
|
||||
rich_text = _extract_text_from_slack_blocks(blocks).strip()
|
||||
if rich_text and rich_text not in msg_text:
|
||||
extras.append(rich_text)
|
||||
for block in blocks:
|
||||
block_type = (block or {}).get("type", "")
|
||||
if block_type in ("section", "header", "context"):
|
||||
text_obj = block.get("text") or {}
|
||||
if isinstance(text_obj, dict):
|
||||
section_text = (text_obj.get("text") or "").strip()
|
||||
if section_text and section_text not in msg_text and all(section_text not in e for e in extras):
|
||||
extras.append(section_text)
|
||||
# Legacy ``attachments`` (Alertmanager, Grafana, PagerDuty, CI bots):
|
||||
# apps often post with an empty ``text`` and the real content in
|
||||
# attachment fields or attachment-nested blocks.
|
||||
attachments_text = _extract_text_from_slack_attachments(
|
||||
msg.get("attachments") or []
|
||||
).strip()
|
||||
if attachments_text and attachments_text not in msg_text and all(
|
||||
attachments_text not in e for e in extras
|
||||
):
|
||||
extras.append(attachments_text)
|
||||
if blocks:
|
||||
urls = _extract_urls_from_slack_blocks(blocks)
|
||||
new_urls = [u for u in urls if u not in msg_text and all(u not in e for e in extras)]
|
||||
if new_urls:
|
||||
extras.append("URLs: " + ", ".join(new_urls))
|
||||
if extras:
|
||||
addendum = "\n".join(extras)
|
||||
msg_text = (msg_text + "\n" + addendum).strip() if msg_text else addendum
|
||||
|
||||
return msg_text
|
||||
|
||||
async def _fetch_thread_context(
|
||||
self,
|
||||
channel_id: str,
|
||||
@@ -4397,14 +4913,10 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
):
|
||||
continue
|
||||
|
||||
msg_text = msg.get("text", "").strip()
|
||||
msg_text = self._render_message_text(msg, bot_uid=bot_uid)
|
||||
if not msg_text:
|
||||
continue
|
||||
|
||||
# Strip bot mentions from context messages
|
||||
if bot_uid:
|
||||
msg_text = msg_text.replace(f"<@{bot_uid}>", "").strip()
|
||||
|
||||
prefix = "[thread parent] " if is_parent else ""
|
||||
display_user = msg_user or "unknown"
|
||||
# Prefer the bot's own name when the message is a bot post.
|
||||
@@ -4502,9 +5014,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if parent.get("ts", "") != thread_ts:
|
||||
return ""
|
||||
bot_uid = self._team_bot_user_ids.get(team_id, self._bot_user_id)
|
||||
text = (parent.get("text") or "").strip()
|
||||
if bot_uid:
|
||||
text = text.replace(f"<@{bot_uid}>", "").strip()
|
||||
text = self._render_message_text(parent, bot_uid=bot_uid or "")
|
||||
return text
|
||||
except Exception as exc: # pragma: no cover - defensive
|
||||
logger.debug("[Slack] Failed to fetch thread parent text: %s", exc)
|
||||
@@ -4946,6 +5456,14 @@ async def _standalone_send(
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
if not formatted or not formatted.strip():
|
||||
logger.debug("[Slack] _standalone_send: skipping empty/whitespace message")
|
||||
return {
|
||||
"success": True,
|
||||
"platform": "slack",
|
||||
"skipped": "empty_text",
|
||||
}
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
except ImportError:
|
||||
|
||||
@@ -146,9 +146,31 @@ def _inline_elements(text: str) -> List[Dict[str, Any]]:
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _header_block(text: str) -> Block:
|
||||
def _nonempty_elements(elements: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""Make a rich_text child-element list safe for Slack.
|
||||
|
||||
Slack rejects any ``rich_text_section`` / ``rich_text_preformatted`` /
|
||||
``rich_text_quote`` whose ``elements`` list is empty or contains a ``text``
|
||||
element of zero length (``invalid_blocks``: "missing element" / "must be
|
||||
more than 0 characters"). Empty content is common — ragged table rows are
|
||||
padded with ``""``, agents emit empty code fences around empty tool output,
|
||||
blank quote lines and empty list items occur in the wild — so drop
|
||||
zero-length text elements and, if nothing remains, substitute a single
|
||||
space, which renders as blank yet stays schema-valid. Used by every
|
||||
rich_text builder so empty content can never poison the whole payload.
|
||||
"""
|
||||
els = [e for e in elements if not (e.get("type") == "text" and not e.get("text"))]
|
||||
return els or [{"type": "text", "text": " "}]
|
||||
|
||||
|
||||
def _header_block(text: str) -> Optional[Block]:
|
||||
# header blocks are plain_text only, 150 char cap.
|
||||
clean = re.sub(r"[*_~`]", "", text).strip()
|
||||
if not clean:
|
||||
# Emphasis-/whitespace-only header (e.g. "# ***" or "# ") reduces to
|
||||
# empty; Slack rejects an empty plain_text with invalid_blocks. Skip it
|
||||
# (caller drops None) rather than poison the whole payload.
|
||||
return None
|
||||
if len(clean) > MAX_HEADER_TEXT:
|
||||
clean = clean[: MAX_HEADER_TEXT - 1] + "…"
|
||||
return {"type": "header", "text": {"type": "plain_text", "text": clean, "emoji": True}}
|
||||
@@ -165,7 +187,7 @@ def _preformatted_block(text: str) -> Block:
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_preformatted",
|
||||
"elements": [{"type": "text", "text": text.rstrip("\n")}],
|
||||
"elements": _nonempty_elements([{"type": "text", "text": text.rstrip("\n")}]),
|
||||
}
|
||||
],
|
||||
}
|
||||
@@ -179,7 +201,7 @@ def _quote_block(lines: List[str]) -> Block:
|
||||
section_children.extend(_inline_elements(ln))
|
||||
return {
|
||||
"type": "rich_text",
|
||||
"elements": [{"type": "rich_text_quote", "elements": section_children}],
|
||||
"elements": [{"type": "rich_text_quote", "elements": _nonempty_elements(section_children)}],
|
||||
}
|
||||
|
||||
|
||||
@@ -205,9 +227,13 @@ def _list_block(items: List[Tuple[int, bool, str]]) -> Block:
|
||||
}
|
||||
elements.append(cur)
|
||||
cur_key = key
|
||||
assert cur is not None
|
||||
if cur is None:
|
||||
# Defensive: should never happen (first iteration always enters
|
||||
# the ``if key != cur_key`` block above), but guard explicitly
|
||||
# so ``python -O`` doesn't silently drop the check.
|
||||
continue
|
||||
cur["elements"].append(
|
||||
{"type": "rich_text_section", "elements": _inline_elements(text)}
|
||||
{"type": "rich_text_section", "elements": _nonempty_elements(_inline_elements(text))}
|
||||
)
|
||||
return {"type": "rich_text", "elements": elements}
|
||||
|
||||
@@ -252,11 +278,16 @@ def _split_row(row: str) -> List[str]:
|
||||
|
||||
|
||||
def _rich_text_cell(text: str) -> Dict[str, Any]:
|
||||
"""A ``rich_text`` table cell carrying inline-formatted content."""
|
||||
"""A ``rich_text`` table cell carrying inline-formatted content.
|
||||
|
||||
Empty cells are common (ragged rows are padded with ``""``); Slack rejects
|
||||
a cell whose section is empty or carries a zero-length text element, so the
|
||||
elements are routed through ``_nonempty_elements``.
|
||||
"""
|
||||
return {
|
||||
"type": "rich_text",
|
||||
"elements": [
|
||||
{"type": "rich_text_section", "elements": _inline_elements(text)}
|
||||
{"type": "rich_text_section", "elements": _nonempty_elements(_inline_elements(text))}
|
||||
],
|
||||
}
|
||||
|
||||
@@ -285,18 +316,25 @@ def _table_block(rows: List[str], sep_line: str) -> Optional[Block]:
|
||||
return None
|
||||
|
||||
aligns = _parse_alignment(sep_line)
|
||||
column_settings: List[Optional[Dict[str, Any]]] = []
|
||||
# Slack requires every provided ``column_settings`` entry to be an object.
|
||||
# Missing trailing entries inherit defaults, so only emit settings through
|
||||
# the last non-default alignment. Earlier default-left placeholders still
|
||||
# need explicit valid objects to preserve positional alignment.
|
||||
last_non_default = -1
|
||||
for c in range(min(ncols, MAX_TABLE_COLS)):
|
||||
align = aligns[c] if c < len(aligns) else "left"
|
||||
# Only emit a setting when it differs from the default (left, no wrap);
|
||||
# use null to skip a column, per the Slack schema.
|
||||
column_settings.append({"align": align} if align != "left" else None)
|
||||
if align != "left":
|
||||
last_non_default = c
|
||||
column_settings: List[Dict[str, Any]] = []
|
||||
for c in range(last_non_default + 1):
|
||||
align = aligns[c] if c < len(aligns) else "left"
|
||||
column_settings.append({"align": align})
|
||||
|
||||
block: Block = {
|
||||
"type": "table",
|
||||
"rows": [[_rich_text_cell(cell) for cell in row] for row in parsed],
|
||||
}
|
||||
if any(cs is not None for cs in column_settings):
|
||||
if column_settings:
|
||||
block["column_settings"] = column_settings
|
||||
return block
|
||||
|
||||
@@ -402,7 +440,9 @@ def render_blocks(
|
||||
hm = _HEADER_RE.match(line)
|
||||
if hm:
|
||||
flush_para()
|
||||
blocks.append(_header_block(hm.group(2)))
|
||||
header = _header_block(hm.group(2))
|
||||
if header is not None:
|
||||
blocks.append(header)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
@@ -510,3 +550,118 @@ def _split_text(text: str, limit: int) -> List[str]:
|
||||
if remaining:
|
||||
out.append(remaining)
|
||||
return out
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Outbound payload boundary — last-resort clamp before the Slack API
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _clamp_text_obj(text_obj: Dict[str, Any], limit: int) -> Dict[str, Any]:
|
||||
"""Return ``text_obj`` with its ``text`` clamped to ``limit`` chars."""
|
||||
txt = text_obj.get("text") or ""
|
||||
if len(txt) <= limit:
|
||||
return text_obj
|
||||
clamped = dict(text_obj)
|
||||
clamped["text"] = txt[: limit - 1].rstrip() + "…"
|
||||
return clamped
|
||||
|
||||
|
||||
def sanitize_blocks(blocks: Optional[List[Block]]) -> Optional[List[Block]]:
|
||||
"""Clamp an outbound ``blocks`` payload to Slack's hard limits.
|
||||
|
||||
Defensive boundary applied wherever the adapter attaches ``blocks`` to
|
||||
``chat.postMessage`` / ``chat.update``. One oversized or malformed block
|
||||
fails the WHOLE call with ``invalid_blocks`` — approval cards then never
|
||||
update and messages silently drop — so instead of trusting every builder,
|
||||
the payload is normalized just before the API call:
|
||||
|
||||
* ``section`` / ``context`` text objects are truncated to the 3000-char
|
||||
cap with an ellipsis (Slack HTML-escapes ``< > &`` on storage, so text
|
||||
echoed back through interaction payloads can exceed the limit that the
|
||||
send path originally budgeted for — see #53693 / #62054).
|
||||
* ``header`` text is truncated to its 150-char cap.
|
||||
* Empty blocks (no text / no elements / no rows) are dropped — Slack
|
||||
rejects zero-length text objects and empty element lists.
|
||||
* ``table.column_settings`` entries must all be objects; ``null`` entries
|
||||
(emitted by older renderers, per the "use null to skip" misreading of
|
||||
the schema) are replaced with ``{}`` and default trailing entries are
|
||||
trimmed (#56615).
|
||||
* The payload is capped at Slack's 50-block maximum.
|
||||
|
||||
Returns the sanitized list, or ``None`` when nothing valid remains — the
|
||||
caller then sends the plain ``text`` fallback alone. Never raises.
|
||||
"""
|
||||
if not blocks:
|
||||
return None
|
||||
try:
|
||||
out: List[Block] = []
|
||||
for block in blocks:
|
||||
if not isinstance(block, dict) or not block.get("type"):
|
||||
continue
|
||||
btype = block["type"]
|
||||
|
||||
if btype == "section":
|
||||
text_obj = block.get("text")
|
||||
has_body = bool(block.get("fields")) or bool(block.get("accessory"))
|
||||
if isinstance(text_obj, dict):
|
||||
if not (text_obj.get("text") or "").strip() and not has_body:
|
||||
continue
|
||||
clamped = _clamp_text_obj(text_obj, MAX_SECTION_TEXT)
|
||||
if clamped is not text_obj:
|
||||
block = dict(block)
|
||||
block["text"] = clamped
|
||||
elif not has_body:
|
||||
continue
|
||||
|
||||
elif btype == "header":
|
||||
text_obj = block.get("text")
|
||||
if not isinstance(text_obj, dict) or not (text_obj.get("text") or "").strip():
|
||||
continue
|
||||
clamped = _clamp_text_obj(text_obj, MAX_HEADER_TEXT)
|
||||
if clamped is not text_obj:
|
||||
block = dict(block)
|
||||
block["text"] = clamped
|
||||
|
||||
elif btype == "context":
|
||||
elements = block.get("elements") or []
|
||||
if not elements:
|
||||
continue
|
||||
clamped_els = [
|
||||
_clamp_text_obj(el, MAX_SECTION_TEXT)
|
||||
if isinstance(el, dict) and el.get("type") in ("mrkdwn", "plain_text")
|
||||
else el
|
||||
for el in elements
|
||||
]
|
||||
if any(c is not e for c, e in zip(clamped_els, elements)):
|
||||
block = dict(block)
|
||||
block["elements"] = clamped_els
|
||||
|
||||
elif btype in ("rich_text", "actions", "context_actions"):
|
||||
if not block.get("elements"):
|
||||
continue
|
||||
|
||||
elif btype == "table":
|
||||
if not block.get("rows"):
|
||||
continue
|
||||
settings = block.get("column_settings")
|
||||
if isinstance(settings, list) and any(
|
||||
not isinstance(cs, dict) for cs in settings
|
||||
):
|
||||
fixed = [cs if isinstance(cs, dict) else {} for cs in settings]
|
||||
while fixed and not fixed[-1]:
|
||||
fixed.pop()
|
||||
block = dict(block)
|
||||
if fixed:
|
||||
block["column_settings"] = fixed
|
||||
else:
|
||||
block.pop("column_settings", None)
|
||||
|
||||
out.append(block)
|
||||
|
||||
if not out:
|
||||
return None
|
||||
return out[:MAX_BLOCKS]
|
||||
except Exception:
|
||||
# A sanitizer bug must never take down the send path.
|
||||
return None
|
||||
|
||||
+12
-1
@@ -6162,7 +6162,17 @@ class AIAgent:
|
||||
"""
|
||||
return self.api_mode != "codex_responses"
|
||||
|
||||
def _compress_context(self, messages: list, system_message: str, *, approx_tokens: int = None, task_id: str = "default", focus_topic: str = None, force: bool = False) -> tuple:
|
||||
def _compress_context(
|
||||
self,
|
||||
messages: list,
|
||||
system_message: str,
|
||||
*,
|
||||
approx_tokens: int = None,
|
||||
task_id: str = "default",
|
||||
focus_topic: str = None,
|
||||
force: bool = False,
|
||||
defer_context_engine_notification: bool = False,
|
||||
) -> tuple:
|
||||
"""Forwarder — see ``agent.conversation_compression.compress_context``.
|
||||
|
||||
``force=True`` is passed by the manual ``/compress`` slash command
|
||||
@@ -6175,6 +6185,7 @@ class AIAgent:
|
||||
self, messages, system_message,
|
||||
approx_tokens=approx_tokens, task_id=task_id, focus_topic=focus_topic,
|
||||
force=force,
|
||||
defer_context_engine_notification=defer_context_engine_notification,
|
||||
)
|
||||
|
||||
def _set_tool_guardrail_halt(self, decision: ToolGuardrailDecision) -> None:
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Standalone structural validator for plugin-catalog entry files.
|
||||
|
||||
Validates ``plugin-catalog/*.yaml`` catalog entries and
|
||||
``plugin-catalog/removed.yaml`` against the catalog contract schema, using
|
||||
only stdlib + PyYAML so the admission CI (and third-party repos) can run it
|
||||
WITHOUT installing hermes-agent.
|
||||
|
||||
NOTE: this script intentionally duplicates the schema rules instead of
|
||||
importing ``hermes_cli`` — the whole point is the no-install requirement for
|
||||
cheap cross-repo CI use. The runtime twin of this schema lives in
|
||||
``hermes_cli/plugin_catalog.py``; if the contract changes there, update the
|
||||
rules here in lockstep.
|
||||
|
||||
Usage:
|
||||
python3 scripts/validate_plugin_catalog.py plugin-catalog/
|
||||
python3 scripts/validate_plugin_catalog.py entry.yaml removed.yaml
|
||||
python3 scripts/validate_plugin_catalog.py --json plugin-catalog/
|
||||
|
||||
Exit codes: 0 = all files valid (warnings allowed), 1 = at least one error.
|
||||
Human output is one ``<file>: ERROR: ...`` / ``<file>: warning: ...`` line
|
||||
per finding; ``--json`` emits a machine-readable report on stdout instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover - dependency guidance only
|
||||
print(
|
||||
"ERROR: PyYAML is required (pip install pyyaml)",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
NAME_RE = re.compile(r"^[a-z0-9_-]{1,64}$")
|
||||
SHA_RE = re.compile(r"^[0-9a-f]{40}$")
|
||||
TIERS = ("official", "community")
|
||||
PLATFORMS = ("linux", "macos", "windows")
|
||||
CAPABILITY_KEYS = (
|
||||
"provides_tools",
|
||||
"provides_hooks",
|
||||
"provides_middleware",
|
||||
"requires_env",
|
||||
)
|
||||
# Top-level keys the contract knows about. Unknown keys WARN (forward
|
||||
# compatibility: newer catalogs must stay valid under older validators).
|
||||
KNOWN_KEYS = {
|
||||
"name",
|
||||
"repo",
|
||||
"sha",
|
||||
"subdir",
|
||||
"description",
|
||||
"maintainer",
|
||||
"tier",
|
||||
"requires_hermes",
|
||||
"docs_url",
|
||||
"platforms",
|
||||
"capabilities",
|
||||
}
|
||||
REQUIRED_KEYS = ("name", "repo", "sha", "description", "maintainer")
|
||||
|
||||
# One comparator clause of a requires_hermes spec, e.g. ">=0.19" or "!=1.2.3".
|
||||
_COMPARATOR_RE = re.compile(r"^(>=|<=|==|!=|>|<)\s*\d+(\.\d+)*$")
|
||||
|
||||
|
||||
def _is_nonempty_str(value: object) -> bool:
|
||||
return isinstance(value, str) and value.strip() != ""
|
||||
|
||||
|
||||
def _check_requires_hermes(spec: object, errors: list[str]) -> None:
|
||||
if not isinstance(spec, str):
|
||||
errors.append(f"requires_hermes must be a string, got {type(spec).__name__}")
|
||||
return
|
||||
if spec.strip() == "":
|
||||
return # empty = no constraint
|
||||
for clause in spec.split(","):
|
||||
if not _COMPARATOR_RE.match(clause.strip()):
|
||||
errors.append(
|
||||
f"requires_hermes clause {clause.strip()!r} is not a valid "
|
||||
"comparator spec (expected e.g. '>=0.19')"
|
||||
)
|
||||
|
||||
|
||||
def validate_entry(data: object) -> tuple[list[str], list[str]]:
|
||||
"""Validate one catalog entry document. Returns (errors, warnings)."""
|
||||
errors: list[str] = []
|
||||
warnings: list[str] = []
|
||||
|
||||
if not isinstance(data, dict):
|
||||
return ["top-level document must be a YAML mapping"], warnings
|
||||
|
||||
for key in REQUIRED_KEYS:
|
||||
if key not in data:
|
||||
errors.append(f"missing required key: {key}")
|
||||
|
||||
for key in sorted(set(data) - KNOWN_KEYS):
|
||||
warnings.append(f"unknown top-level key {key!r} (ignored by this validator)")
|
||||
|
||||
name = data.get("name")
|
||||
if "name" in data and (not isinstance(name, str) or not NAME_RE.match(name)):
|
||||
errors.append(f"name {name!r} must match [a-z0-9_-]{{1,64}}")
|
||||
|
||||
repo = data.get("repo")
|
||||
if "repo" in data and (
|
||||
not isinstance(repo, str) or not repo.startswith("https://")
|
||||
):
|
||||
errors.append(f"repo {repo!r} must be an https:// URL")
|
||||
|
||||
sha = data.get("sha")
|
||||
if "sha" in data and (not isinstance(sha, str) or not SHA_RE.match(sha)):
|
||||
errors.append(f"sha {sha!r} must be exactly 40 lowercase hex characters")
|
||||
|
||||
for key in ("description", "maintainer"):
|
||||
if key in data and not _is_nonempty_str(data[key]):
|
||||
errors.append(f"{key} must be a non-empty string")
|
||||
|
||||
tier = data.get("tier", "community")
|
||||
if tier not in TIERS:
|
||||
errors.append(f"tier {tier!r} must be one of {list(TIERS)}")
|
||||
|
||||
if "requires_hermes" in data:
|
||||
_check_requires_hermes(data["requires_hermes"], errors)
|
||||
|
||||
platforms = data.get("platforms", [])
|
||||
if platforms is None:
|
||||
platforms = []
|
||||
if not isinstance(platforms, list):
|
||||
errors.append("platforms must be a list")
|
||||
else:
|
||||
bad = [p for p in platforms if p not in PLATFORMS]
|
||||
if bad:
|
||||
errors.append(f"platforms {bad!r} not in allowed set {list(PLATFORMS)}")
|
||||
|
||||
caps = data.get("capabilities", {})
|
||||
if caps is None:
|
||||
caps = {}
|
||||
if not isinstance(caps, dict):
|
||||
errors.append("capabilities must be a mapping")
|
||||
else:
|
||||
for key in sorted(set(caps) - set(CAPABILITY_KEYS)):
|
||||
warnings.append(f"unknown capabilities key {key!r}")
|
||||
for key in CAPABILITY_KEYS:
|
||||
if key not in caps:
|
||||
continue
|
||||
value = caps[key]
|
||||
if not isinstance(value, list) or not all(
|
||||
isinstance(item, str) for item in value
|
||||
):
|
||||
errors.append(f"capabilities.{key} must be a list of strings")
|
||||
|
||||
return errors, warnings
|
||||
|
||||
|
||||
def validate_removed(data: object) -> tuple[list[str], list[str]]:
|
||||
"""Validate the removed.yaml document. Returns (errors, warnings)."""
|
||||
errors: list[str] = []
|
||||
warnings: list[str] = []
|
||||
|
||||
if not isinstance(data, dict):
|
||||
return ["top-level document must be a YAML mapping"], warnings
|
||||
|
||||
removed = data.get("removed")
|
||||
if removed is None:
|
||||
errors.append("missing required key: removed")
|
||||
return errors, warnings
|
||||
if not isinstance(removed, list):
|
||||
errors.append("removed must be a list")
|
||||
return errors, warnings
|
||||
|
||||
for i, item in enumerate(removed):
|
||||
if not isinstance(item, dict):
|
||||
errors.append(f"removed[{i}] must be a mapping")
|
||||
continue
|
||||
if not _is_nonempty_str(item.get("name")):
|
||||
errors.append(f"removed[{i}] missing non-empty 'name'")
|
||||
for key in ("repo", "reason", "date"):
|
||||
if key in item and not isinstance(item[key], str):
|
||||
errors.append(f"removed[{i}].{key} must be a string")
|
||||
|
||||
return errors, warnings
|
||||
|
||||
|
||||
def validate_file(path: Path) -> tuple[list[str], list[str]]:
|
||||
"""Validate one YAML file (dispatching on filename). Returns (errors, warnings)."""
|
||||
try:
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
data = yaml.safe_load(fh)
|
||||
except OSError as exc:
|
||||
return [f"cannot read file: {exc}"], []
|
||||
except yaml.YAMLError as exc:
|
||||
return [f"invalid YAML: {exc}"], []
|
||||
|
||||
if path.name == "removed.yaml":
|
||||
return validate_removed(data)
|
||||
return validate_entry(data)
|
||||
|
||||
|
||||
def collect_paths(args: list[str]) -> list[Path]:
|
||||
paths: list[Path] = []
|
||||
for arg in args:
|
||||
p = Path(arg)
|
||||
if p.is_dir():
|
||||
paths.extend(sorted(p.glob("*.yaml")))
|
||||
paths.extend(sorted(p.glob("*.yml")))
|
||||
else:
|
||||
paths.append(p)
|
||||
return paths
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Standalone structural validator for plugin-catalog entry files."
|
||||
)
|
||||
parser.add_argument(
|
||||
"paths",
|
||||
nargs="+",
|
||||
help="catalog entry files, removed.yaml, or a directory of them",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--json",
|
||||
action="store_true",
|
||||
help="emit a machine-readable JSON report on stdout",
|
||||
)
|
||||
opts = parser.parse_args(argv)
|
||||
|
||||
files = collect_paths(opts.paths)
|
||||
if not files:
|
||||
print("ERROR: no YAML files found", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
report = []
|
||||
any_errors = False
|
||||
for path in files:
|
||||
errors, warnings = validate_file(path)
|
||||
any_errors = any_errors or bool(errors)
|
||||
report.append(
|
||||
{
|
||||
"path": str(path),
|
||||
"ok": not errors,
|
||||
"errors": errors,
|
||||
"warnings": warnings,
|
||||
}
|
||||
)
|
||||
|
||||
if opts.json:
|
||||
print(json.dumps({"ok": not any_errors, "files": report}, indent=2))
|
||||
else:
|
||||
for entry in report:
|
||||
for err in entry["errors"]:
|
||||
print(f"{entry['path']}: ERROR: {err}")
|
||||
for warn in entry["warnings"]:
|
||||
print(f"{entry['path']}: warning: {warn}")
|
||||
checked = len(report)
|
||||
bad = sum(1 for e in report if not e["ok"])
|
||||
if any_errors:
|
||||
print(f"FAIL: {bad}/{checked} file(s) invalid")
|
||||
else:
|
||||
print(f"OK: {checked} file(s) valid")
|
||||
|
||||
return 1 if any_errors else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -287,7 +287,7 @@ class TestSessionOps:
|
||||
"tools",
|
||||
"context",
|
||||
"reset",
|
||||
"compact",
|
||||
"compress",
|
||||
"steer",
|
||||
"queue",
|
||||
"version",
|
||||
@@ -1527,7 +1527,7 @@ class TestSlashCommands:
|
||||
|
||||
assert "Context usage: ~25,000 / 100,000 tokens (25.0%)" in result
|
||||
assert "Compression: ~55,000 tokens until threshold (~80,000, 80%)" in result
|
||||
assert "Tip: run /compact" in result
|
||||
assert "Tip: run /compress" in result
|
||||
|
||||
def test_context_says_compression_due_when_past_threshold(self, agent, mock_manager):
|
||||
state = self._make_state(mock_manager)
|
||||
@@ -1544,7 +1544,7 @@ class TestSlashCommands:
|
||||
result = agent._handle_slash_command("/context", state)
|
||||
|
||||
assert "Context usage: ~82,000 / 100,000 tokens (82.0%)" in result
|
||||
assert "Compression: due now (threshold ~80,000, 80%). Run /compact." in result
|
||||
assert "Compression: due now (threshold ~80,000, 80%). Run /compress." in result
|
||||
|
||||
def test_reset_clears_history(self, agent, mock_manager):
|
||||
state = self._make_state(mock_manager)
|
||||
@@ -1620,7 +1620,7 @@ class TestSlashCommands:
|
||||
side_effect=[40, 12],
|
||||
),
|
||||
):
|
||||
result = agent._handle_slash_command("/compact", state)
|
||||
result = agent._handle_slash_command("/compress", state)
|
||||
|
||||
assert "Context compressed: 4 -> 1 messages" in result
|
||||
assert "~40 -> ~12 tokens" in result
|
||||
|
||||
@@ -945,7 +945,7 @@ class TestConvertMessages:
|
||||
{"role": "tool", "tool_call_id": "tc_1", "content": "search results"},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
blocks = result[0]["content"]
|
||||
blocks = next(m for m in result if m["role"] == "assistant")["content"]
|
||||
assert blocks[0] == {"type": "text", "text": "Let me search."}
|
||||
assert blocks[1]["type"] == "tool_use"
|
||||
assert blocks[1]["id"] == "tc_1"
|
||||
@@ -963,8 +963,13 @@ class TestConvertMessages:
|
||||
{"role": "tool", "tool_call_id": "tc_1", "content": "result data"},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
# tool result is in the second message (user role)
|
||||
user_msg = [m for m in result if m["role"] == "user"][0]
|
||||
# tool result is in the user message following the assistant turn
|
||||
user_msg = next(
|
||||
m for m in result
|
||||
if m["role"] == "user"
|
||||
and isinstance(m["content"], list)
|
||||
and any(b.get("type") == "tool_result" for b in m["content"])
|
||||
)
|
||||
assert user_msg["content"][0]["type"] == "tool_result"
|
||||
assert user_msg["content"][0]["tool_use_id"] == "tc_1"
|
||||
|
||||
@@ -983,7 +988,12 @@ class TestConvertMessages:
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
# assistant + merged user (with 2 tool_results)
|
||||
user_msgs = [m for m in result if m["role"] == "user"]
|
||||
user_msgs = [
|
||||
m for m in result
|
||||
if m["role"] == "user"
|
||||
and isinstance(m["content"], list)
|
||||
and any(b.get("type") == "tool_result" for b in m["content"])
|
||||
]
|
||||
assert len(user_msgs) == 1
|
||||
assert len(user_msgs[0]["content"]) == 2
|
||||
|
||||
@@ -1041,7 +1051,12 @@ class TestConvertMessages:
|
||||
{"role": "tool", "tool_call_id": "tc_orphan", "content": "stale result"},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
user_msg = [m for m in result if m["role"] == "user"][0]
|
||||
user_msg = next(
|
||||
m for m in result
|
||||
if m["role"] == "user"
|
||||
and isinstance(m["content"], list)
|
||||
and any(b.get("type") == "tool_result" for b in m["content"])
|
||||
)
|
||||
tool_results = [
|
||||
b for b in user_msg["content"] if b.get("type") == "tool_result"
|
||||
]
|
||||
@@ -1096,7 +1111,12 @@ class TestConvertMessages:
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
asst = [m for m in result if m["role"] == "assistant"][0]
|
||||
assert any(b.get("type") == "tool_use" for b in asst["content"])
|
||||
user = [m for m in result if m["role"] == "user"][0]
|
||||
user = next(
|
||||
m for m in result
|
||||
if m["role"] == "user"
|
||||
and isinstance(m["content"], list)
|
||||
and any(b.get("type") == "tool_result" for b in m["content"])
|
||||
)
|
||||
assert any(b.get("type") == "tool_result" for b in user["content"])
|
||||
|
||||
def test_system_with_cache_control(self):
|
||||
@@ -1121,7 +1141,8 @@ class TestConvertMessages:
|
||||
])
|
||||
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
assistant_blocks = result[0]["content"]
|
||||
assistant_msg = next(m for m in result if m["role"] == "assistant")
|
||||
assistant_blocks = assistant_msg["content"]
|
||||
|
||||
assert assistant_blocks[0]["type"] == "text"
|
||||
assert assistant_blocks[0]["text"] == "Hello from assistant"
|
||||
@@ -1207,7 +1228,12 @@ class TestConvertMessages:
|
||||
], native_anthropic=True)
|
||||
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
user_msg = [m for m in result if m["role"] == "user"][0]
|
||||
user_msg = next(
|
||||
m for m in result
|
||||
if m["role"] == "user"
|
||||
and isinstance(m["content"], list)
|
||||
and any(b.get("type") == "tool_result" for b in m["content"])
|
||||
)
|
||||
tool_block = user_msg["content"][0]
|
||||
|
||||
assert tool_block["type"] == "tool_result"
|
||||
@@ -1356,6 +1382,90 @@ class TestConvertMessages:
|
||||
assert isinstance(result[0]["content"], list)
|
||||
assert result[0]["content"] == [{"type": "text", "text": "(empty message)"}]
|
||||
|
||||
def test_leading_assistant_after_compaction_gets_user_turn_prepended(self):
|
||||
"""The adapter backstops compactors that emit a leading assistant summary."""
|
||||
messages = [
|
||||
{"role": "system", "content": "You are helpful."},
|
||||
{"role": "assistant", "content": "[Context compaction summary] earlier work…"},
|
||||
{"role": "user", "content": "continue"},
|
||||
]
|
||||
|
||||
system, result = convert_messages_to_anthropic(messages)
|
||||
|
||||
assert system == "You are helpful."
|
||||
assert result[0]["role"] == "user"
|
||||
assert result[0]["content"] == [{"type": "text", "text": " "}]
|
||||
assert result[1]["role"] == "assistant"
|
||||
assert any(
|
||||
m["role"] == "assistant" and "Context compaction summary" in str(m["content"])
|
||||
for m in result
|
||||
)
|
||||
|
||||
def test_double_compaction_no_system_in_messages_leads_with_user(self):
|
||||
"""Exact post-double-compaction shape on the auto path (#52160).
|
||||
|
||||
On the auto path the system prompt is NOT inside messages[] and after
|
||||
the second compaction protect_head has decayed to 0, so the
|
||||
assistant-role summary is messages[0]. The converted payload must
|
||||
still lead with a user turn or Anthropic 400s.
|
||||
"""
|
||||
messages = [
|
||||
{"role": "assistant", "content": "[Context compaction summary] earlier work…"},
|
||||
{"role": "user", "content": "continue"},
|
||||
]
|
||||
|
||||
system, result = convert_messages_to_anthropic(messages)
|
||||
|
||||
assert system is None
|
||||
assert result[0]["role"] == "user"
|
||||
assert result[0]["content"] == [{"type": "text", "text": " "}]
|
||||
assert result[1]["role"] == "assistant"
|
||||
assert "Context compaction summary" in str(result[1]["content"])
|
||||
|
||||
def test_leading_user_message_is_not_modified(self):
|
||||
"""A normal transcript that already starts with user must be untouched."""
|
||||
messages = [
|
||||
{"role": "system", "content": "sys"},
|
||||
{"role": "user", "content": "hello"},
|
||||
{"role": "assistant", "content": "hi"},
|
||||
]
|
||||
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
|
||||
assert len(result) == 2
|
||||
assert result[0]["role"] == "user"
|
||||
assert result[0]["content"] == "hello"
|
||||
|
||||
def test_leading_assistant_with_tool_use_after_compaction_is_repaired(self):
|
||||
"""Repair the leading role without disturbing adjacent tool pairs."""
|
||||
messages = [
|
||||
{"role": "system", "content": "sys"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "running it",
|
||||
"tool_calls": [
|
||||
{"id": "toolu_1", "function": {"name": "terminal", "arguments": "{}"}},
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "toolu_1", "content": "ok"},
|
||||
{"role": "user", "content": "next"},
|
||||
]
|
||||
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
|
||||
assert result[0]["role"] == "user"
|
||||
asst_idx = next(
|
||||
i for i, m in enumerate(result)
|
||||
if m["role"] == "assistant"
|
||||
and any(b.get("type") == "tool_use" for b in m["content"] if isinstance(b, dict))
|
||||
)
|
||||
nxt = result[asst_idx + 1]
|
||||
assert nxt["role"] == "user"
|
||||
assert any(
|
||||
isinstance(b, dict) and b.get("type") == "tool_result" and b.get("tool_use_id") == "toolu_1"
|
||||
for b in nxt["content"]
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Build kwargs
|
||||
@@ -2076,7 +2186,7 @@ class TestThinkingBlockSignatureManagement:
|
||||
},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
blocks = result[0]["content"]
|
||||
blocks = next(m for m in result if m["role"] == "assistant")["content"]
|
||||
thinking = [b for b in blocks if b.get("type") == "thinking"]
|
||||
assert len(thinking) == 1
|
||||
assert thinking[0]["signature"] == "sig_valid"
|
||||
@@ -2094,7 +2204,7 @@ class TestThinkingBlockSignatureManagement:
|
||||
},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
blocks = result[0]["content"]
|
||||
blocks = next(m for m in result if m["role"] == "assistant")["content"]
|
||||
|
||||
# No thinking blocks should remain
|
||||
assert not any(b.get("type") == "thinking" for b in blocks)
|
||||
@@ -2114,7 +2224,7 @@ class TestThinkingBlockSignatureManagement:
|
||||
},
|
||||
]
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
blocks = result[0]["content"]
|
||||
blocks = next(m for m in result if m["role"] == "assistant")["content"]
|
||||
redacted = [b for b in blocks if b.get("type") == "redacted_thinking"]
|
||||
assert len(redacted) == 1
|
||||
assert redacted[0]["data"] == "opaque_signature_data"
|
||||
@@ -2208,7 +2318,7 @@ class TestThinkingBlockSignatureManagement:
|
||||
_, result = convert_messages_to_anthropic(messages)
|
||||
# First assistant is non-last, so thinking is stripped completely.
|
||||
# The original content was empty and thinking was unsigned → placeholder
|
||||
first_assistant = result[0]
|
||||
first_assistant = next(m for m in result if m["role"] == "assistant")
|
||||
assert first_assistant["role"] == "assistant"
|
||||
assert len(first_assistant["content"]) >= 1
|
||||
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent.context_compressor import ContextCompressor, _estimate_msg_budget_tokens
|
||||
from agent.model_metadata import (
|
||||
_is_cjk_token_dense_char,
|
||||
estimate_messages_tokens_rough,
|
||||
estimate_tokens_rough,
|
||||
)
|
||||
|
||||
|
||||
def test_cjk_text_is_not_estimated_as_four_chars_per_token():
|
||||
assert estimate_tokens_rough("a" * 400) == 100
|
||||
assert estimate_tokens_rough("가" * 400) >= 400
|
||||
|
||||
|
||||
def test_message_estimate_counts_korean_content_as_token_dense():
|
||||
messages = [{"role": "user", "content": "압축 테스트 " + ("가" * 1000)}]
|
||||
|
||||
assert estimate_messages_tokens_rough(messages) >= 1000
|
||||
|
||||
|
||||
def test_compressor_tail_budget_uses_cjk_aware_message_estimate():
|
||||
korean_msg = {"role": "assistant", "content": "가" * 2000}
|
||||
english_msg = {"role": "assistant", "content": "a" * 2000}
|
||||
|
||||
assert _estimate_msg_budget_tokens(korean_msg) > _estimate_msg_budget_tokens(english_msg)
|
||||
|
||||
|
||||
def test_cjk_tail_does_not_expand_to_english_char_budget():
|
||||
with patch("agent.context_compressor.get_model_context_length", return_value=65536):
|
||||
compressor = ContextCompressor(
|
||||
"test/model",
|
||||
protect_first_n=3,
|
||||
protect_last_n=20,
|
||||
summary_target_ratio=0.2,
|
||||
quiet_mode=True,
|
||||
)
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": "head 1"},
|
||||
{"role": "assistant", "content": "head 2"},
|
||||
{"role": "user", "content": "head 3"},
|
||||
]
|
||||
for idx in range(40):
|
||||
role = "assistant" if idx % 2 else "user"
|
||||
messages.append({"role": role, "content": "가" * 1200})
|
||||
|
||||
compress_start = compressor._align_boundary_forward(
|
||||
messages,
|
||||
compressor._protect_head_size(messages),
|
||||
)
|
||||
compress_end = compressor._find_tail_cut_by_tokens(messages, compress_start)
|
||||
|
||||
assert len(messages) - compress_end < 31
|
||||
|
||||
|
||||
def _reference_per_char_estimate(text: str) -> int:
|
||||
"""The pre-perf-gate per-character reference implementation."""
|
||||
dense = 0
|
||||
sparse = 0
|
||||
for ch in text:
|
||||
if _is_cjk_token_dense_char(ch):
|
||||
dense += 1
|
||||
else:
|
||||
sparse += 1
|
||||
return dense + ((sparse + 3) // 4)
|
||||
|
||||
|
||||
def test_perf_gated_estimator_matches_per_char_reference():
|
||||
samples = [
|
||||
"",
|
||||
"ab",
|
||||
"a" * 400,
|
||||
"가" * 400,
|
||||
"압축 테스트 " + ("가" * 1000),
|
||||
"café résumé naïve", # non-ASCII, no CJK
|
||||
"hello 안녕 world",
|
||||
"アイウエオ テスト", # halfwidth kana (fullwidth-forms block)
|
||||
"漢字とかな交じり文です。",
|
||||
"русский текст", # Cyrillic — non-ASCII, non-CJK
|
||||
]
|
||||
for text in samples:
|
||||
assert estimate_tokens_rough(text) == _reference_per_char_estimate(text), repr(text)
|
||||
|
||||
|
||||
def test_ascii_fast_path_keeps_classic_four_chars_per_token():
|
||||
# Pure ASCII must be bit-identical to the historical (len+3)//4 rule.
|
||||
for text in ("x", "xyz", "a" * 1000, "tool output\n" * 500):
|
||||
assert estimate_tokens_rough(text) == (len(text) + 3) // 4
|
||||
|
||||
|
||||
def test_non_ascii_non_cjk_keeps_classic_rule():
|
||||
text = "café résumé " * 40
|
||||
assert estimate_tokens_rough(text) == (len(text) + 3) // 4
|
||||
@@ -17,6 +17,7 @@ from unittest.mock import MagicMock, patch
|
||||
import pytest
|
||||
|
||||
from agent.context_compressor import (
|
||||
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY,
|
||||
COMPRESSED_SUMMARY_METADATA_KEY,
|
||||
ContextCompressor,
|
||||
)
|
||||
@@ -75,6 +76,7 @@ class TestMetadataFlagNeverReachesWire:
|
||||
with '_'. A bare key would reach strict gateways (Fireworks etc.)
|
||||
and 400 with 'Extra inputs are not permitted'."""
|
||||
assert COMPRESSED_SUMMARY_METADATA_KEY.startswith("_")
|
||||
assert COMPRESSED_SUMMARY_HAS_USER_TURN_KEY.startswith("_")
|
||||
|
||||
def test_chat_completions_transport_strips_flag(self):
|
||||
from agent.transports.chat_completions import ChatCompletionsTransport
|
||||
@@ -83,7 +85,11 @@ class TestMetadataFlagNeverReachesWire:
|
||||
out = _compress(cc, _make_messages())
|
||||
wire = ChatCompletionsTransport().convert_messages(out, model="some-model")
|
||||
assert not any(
|
||||
isinstance(m, dict) and COMPRESSED_SUMMARY_METADATA_KEY in m
|
||||
isinstance(m, dict)
|
||||
and (
|
||||
COMPRESSED_SUMMARY_METADATA_KEY in m
|
||||
or COMPRESSED_SUMMARY_HAS_USER_TURN_KEY in m
|
||||
)
|
||||
for m in wire
|
||||
)
|
||||
# Sanitization must not destroy the in-process flag on the originals.
|
||||
|
||||
@@ -94,6 +94,148 @@ def _count_children(db: SessionDB, parent_sid: str) -> int:
|
||||
return len(rows)
|
||||
|
||||
|
||||
def _wait_for_touch(touch_calls: list[str], value: str, timeout: float = 1.0) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if value in touch_calls:
|
||||
return
|
||||
time.sleep(0.01)
|
||||
pytest.fail(f"Timed out waiting for touch activity {value!r}; calls={touch_calls!r}")
|
||||
|
||||
|
||||
def test_compression_activity_heartbeat_touches_agent_during_long_compress(tmp_path: Path) -> None:
|
||||
"""Long compression must refresh agent activity so gateway watchdogs do not fire."""
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "HEARTBEAT_TEST"
|
||||
db.create_session(session_id, source="test")
|
||||
|
||||
agent = _build_agent_with_db(db, session_id)
|
||||
agent._compression_activity_heartbeat_interval = 0.1
|
||||
touch_calls: list[str] = []
|
||||
agent._touch_activity = lambda desc: touch_calls.append(desc)
|
||||
|
||||
def _slow_compress(*_a, **_kw):
|
||||
_wait_for_touch(touch_calls, "context compression in progress")
|
||||
return [
|
||||
{"role": "user", "content": "[CONTEXT COMPACTION] summary"},
|
||||
{"role": "user", "content": "tail"},
|
||||
]
|
||||
|
||||
agent.context_compressor.compress.side_effect = _slow_compress
|
||||
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
|
||||
|
||||
agent._compress_context(messages, "sys", approx_tokens=120_000)
|
||||
|
||||
assert touch_calls[0] == "context compression started"
|
||||
assert "context compression in progress" in touch_calls
|
||||
assert touch_calls[-1] == "context compression completed"
|
||||
assert db.get_compression_lock_holder(session_id) is None
|
||||
|
||||
|
||||
def test_compression_activity_heartbeat_stops_on_compress_exception(tmp_path: Path) -> None:
|
||||
"""Exception paths must stop the heartbeat and release the compression lock."""
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "HEARTBEAT_FAIL_TEST"
|
||||
db.create_session(session_id, source="test")
|
||||
|
||||
agent = _build_agent_with_db(db, session_id)
|
||||
agent._compression_activity_heartbeat_interval = 0.1
|
||||
touch_calls: list[str] = []
|
||||
agent._touch_activity = lambda desc: touch_calls.append(desc)
|
||||
|
||||
def _failing_compress(*_a, **_kw):
|
||||
_wait_for_touch(touch_calls, "context compression in progress")
|
||||
raise RuntimeError("compress boom")
|
||||
|
||||
agent.context_compressor.compress.side_effect = _failing_compress
|
||||
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
|
||||
|
||||
with pytest.raises(RuntimeError, match="compress boom"):
|
||||
agent._compress_context(messages, "sys", approx_tokens=120_000)
|
||||
|
||||
assert touch_calls[0] == "context compression started"
|
||||
assert "context compression in progress" in touch_calls
|
||||
assert touch_calls[-1] == "context compression failed"
|
||||
assert db.get_compression_lock_holder(session_id) is None
|
||||
|
||||
|
||||
def test_compression_activity_heartbeat_ignores_touch_errors(tmp_path: Path) -> None:
|
||||
"""Activity touch failures must not affect compression success semantics."""
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "HEARTBEAT_TOUCH_ERROR_TEST"
|
||||
db.create_session(session_id, source="test")
|
||||
|
||||
agent = _build_agent_with_db(db, session_id)
|
||||
agent._compression_activity_heartbeat_interval = 0.1
|
||||
agent._touch_activity = lambda _desc: (_ for _ in ()).throw(RuntimeError("touch boom"))
|
||||
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
|
||||
|
||||
compressed, _sp = agent._compress_context(messages, "sys", approx_tokens=120_000)
|
||||
|
||||
assert compressed[0]["content"] == "[CONTEXT COMPACTION] summary"
|
||||
assert db.get_compression_lock_holder(session_id) is None
|
||||
|
||||
|
||||
def test_compression_activity_heartbeat_strict_signature_fallback_releases_lock(tmp_path: Path) -> None:
|
||||
"""Strict compressor signatures still compress while heartbeat cleanup runs.
|
||||
|
||||
Main inspects the engine signature up front (_supported_compression_kwargs)
|
||||
instead of catching TypeError, so a strict-signature engine is invoked
|
||||
exactly once with only the kwargs it accepts. The heartbeat (with a
|
||||
non-numeric configured interval falling back to the default) must still
|
||||
wrap the call and stop cleanly.
|
||||
"""
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "HEARTBEAT_TYPEERROR_TEST"
|
||||
db.create_session(session_id, source="test")
|
||||
|
||||
agent = _build_agent_with_db(db, session_id)
|
||||
agent._compression_activity_heartbeat_interval = "not-a-number"
|
||||
touch_calls: list[str] = []
|
||||
agent._touch_activity = lambda desc: touch_calls.append(desc)
|
||||
messages = [{"role": "user", "content": f"m{i}"} for i in range(20)]
|
||||
|
||||
strict_calls: list[int | None] = []
|
||||
|
||||
def _strict_compress(messages, current_tokens=None):
|
||||
strict_calls.append(current_tokens)
|
||||
return [
|
||||
{"role": "user", "content": "[CONTEXT COMPACTION] strict summary"},
|
||||
{"role": "user", "content": "tail"},
|
||||
]
|
||||
|
||||
agent.context_compressor.compress = _strict_compress
|
||||
|
||||
compressed, _sp = agent._compress_context(messages, "sys", approx_tokens=120_000)
|
||||
|
||||
assert compressed[0]["content"] == "[CONTEXT COMPACTION] strict summary"
|
||||
assert touch_calls[0] == "context compression started"
|
||||
assert touch_calls[-1] == "context compression completed"
|
||||
assert db.get_compression_lock_holder(session_id) is None
|
||||
assert strict_calls == [120_000]
|
||||
|
||||
|
||||
def test_compression_activity_heartbeat_nonfinite_interval_falls_back(tmp_path: Path) -> None:
|
||||
"""Non-finite heartbeat intervals must not reach Event.wait()."""
|
||||
from agent.conversation_compression import _CompressionActivityHeartbeat
|
||||
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "HEARTBEAT_NONFINITE_INTERVAL_TEST"
|
||||
db.create_session(session_id, source="test")
|
||||
|
||||
agent = _build_agent_with_db(db, session_id)
|
||||
touch_calls: list[str] = []
|
||||
agent._touch_activity = lambda desc: touch_calls.append(desc)
|
||||
|
||||
heartbeat = _CompressionActivityHeartbeat(agent, interval_seconds=float("inf"))
|
||||
|
||||
assert heartbeat._interval_seconds == 60.0
|
||||
heartbeat.start()
|
||||
heartbeat.stop()
|
||||
assert touch_calls == ["context compression started", "context compression completed"]
|
||||
|
||||
|
||||
|
||||
def test_concurrent_compression_does_not_fork_session(tmp_path: Path) -> None:
|
||||
"""Two AIAgents that share a session_id MUST NOT both rotate it.
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""compression.max_attempts — config-driven compression retry cap.
|
||||
|
||||
The conversation loop's compression retry cap was hardcoded to 3, stranding
|
||||
sessions that legitimately need more rounds — e.g. a restart history reload
|
||||
whose incompressible tool schemas keep the request estimate above the
|
||||
threshold while the messages themselves compress fine (the #62605 failure
|
||||
class). The cap is now parsed from ``compression.max_attempts`` in
|
||||
``agent_init`` and read by the loop via
|
||||
``getattr(agent, "max_compression_attempts", 3)``.
|
||||
|
||||
These tests pin the parse/validate/attach seam: default preserved, custom
|
||||
value honored, floor and ceiling enforced, garbage tolerated.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
from pathlib import Path
|
||||
|
||||
from hermes_state import SessionDB
|
||||
from run_agent import AIAgent
|
||||
|
||||
|
||||
def _config(max_attempts=None) -> dict:
|
||||
compression = {
|
||||
"enabled": True,
|
||||
"threshold": 0.50,
|
||||
"target_ratio": 0.20,
|
||||
"protect_first_n": 3,
|
||||
"protect_last_n": 20,
|
||||
}
|
||||
if max_attempts is not None:
|
||||
compression["max_attempts"] = max_attempts
|
||||
return {
|
||||
"compression": compression,
|
||||
"prompt_caching": {"cache_ttl": "5m"},
|
||||
"sessions": {},
|
||||
"bedrock": {},
|
||||
}
|
||||
|
||||
|
||||
def _make_agent(monkeypatch, tmp_path: Path, *, max_attempts=None):
|
||||
from hermes_cli import config as config_mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
config_mod, "load_config", lambda: _config(max_attempts=max_attempts)
|
||||
)
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
agent = AIAgent(
|
||||
base_url="https://chatgpt.com/backend-api/codex",
|
||||
api_key="test-key",
|
||||
provider="openai-codex",
|
||||
model="gpt-5.5",
|
||||
enabled_toolsets=[],
|
||||
disabled_toolsets=[],
|
||||
quiet_mode=True,
|
||||
skip_memory=True,
|
||||
session_db=db,
|
||||
session_id="max-attempts-test",
|
||||
)
|
||||
return agent
|
||||
|
||||
|
||||
class TestCompressionMaxAttemptsConfig:
|
||||
def test_default_is_three_when_unset(self, monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path)
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
def test_custom_value_is_honored(self, monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=6)
|
||||
assert agent.max_compression_attempts == 6
|
||||
|
||||
def test_hard_capped_at_ten(self, monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=25)
|
||||
assert agent.max_compression_attempts == 10
|
||||
|
||||
def test_zero_and_negative_fall_back_to_default(self, monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=0)
|
||||
assert agent.max_compression_attempts == 3
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=-2)
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
def test_non_integer_falls_back_to_default(self, monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts="lots")
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
def test_boolean_is_rejected_not_coerced(self, monkeypatch, tmp_path):
|
||||
# bool subclasses int: int(True) == 1 would silently near-disable
|
||||
# compression retries. YAML `max_attempts: true` must fall back to 3.
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=True)
|
||||
assert agent.max_compression_attempts == 3
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=False)
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
def test_fractional_float_is_rejected_not_truncated(self, monkeypatch, tmp_path):
|
||||
# "4.7 attempts" is a config mistake, not a request for 4.
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=4.7)
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
def test_integral_float_and_numeric_string_are_accepted(
|
||||
self, monkeypatch, tmp_path
|
||||
):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=5.0)
|
||||
assert agent.max_compression_attempts == 5
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts="6")
|
||||
assert agent.max_compression_attempts == 6
|
||||
|
||||
def test_loop_pickup_degrades_to_default_when_attribute_missing(
|
||||
self, monkeypatch, tmp_path
|
||||
):
|
||||
# The loop reads getattr(agent, "max_compression_attempts", 3): a
|
||||
# configured agent exposes its value, and an object without the
|
||||
# attribute (older pickle / minimal stub) degrades to the prior
|
||||
# hardcoded behavior.
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=7)
|
||||
assert getattr(agent, "max_compression_attempts", 3) == 7
|
||||
assert getattr(object(), "max_compression_attempts", 3) == 3
|
||||
@@ -143,6 +143,46 @@ class TestOrphanRollbackOnCreateFailure:
|
||||
_ = real_create # silence unused
|
||||
|
||||
|
||||
class TestWorkspaceMetadataFollowsRotation:
|
||||
def test_child_row_inherits_cwd_repo_and_origin_on_rotation(self, tmp_path: Path):
|
||||
"""Behavioral #64709/#59527: drive the REAL compression rotation path
|
||||
and assert the child session row carries the parent's workspace and
|
||||
gateway-origin metadata, so the project sidebar entry and the peer
|
||||
routing mapping both survive the compaction boundary."""
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
parent = "PARENT_CWD_ROT"
|
||||
db.create_session(
|
||||
parent,
|
||||
source="telegram",
|
||||
user_id="u1",
|
||||
session_key="telegram:u1:c1",
|
||||
chat_id="c1",
|
||||
chat_type="private",
|
||||
)
|
||||
db.update_session_cwd(
|
||||
parent, "/work/repo", git_branch="main", git_repo_root="/work/repo"
|
||||
)
|
||||
agent = _build_agent_with_db(db, parent, platform="telegram")
|
||||
|
||||
agent._compress_context(_msgs(), "sys", approx_tokens=120_000)
|
||||
child = agent.session_id
|
||||
assert child != parent # rotation happened
|
||||
|
||||
row = db.get_session(child)
|
||||
assert row is not None
|
||||
assert row["parent_session_id"] == parent
|
||||
# Workspace metadata (#64709): sidebar grouping keys must survive.
|
||||
assert row["cwd"] == "/work/repo"
|
||||
assert row["git_repo_root"] == "/work/repo"
|
||||
assert row["git_branch"] == "main"
|
||||
# Gateway origin metadata (#59527): routing keys must survive even if
|
||||
# the gateway never gets to re-record the peer (crash window).
|
||||
assert row["session_key"] == "telegram:u1:c1"
|
||||
assert row["chat_id"] == "c1"
|
||||
assert row["chat_type"] == "private"
|
||||
assert row["user_id"] == "u1"
|
||||
|
||||
|
||||
class TestPlatformForwardedAtBoundary:
|
||||
def test_on_session_start_receives_platform(self, tmp_path: Path):
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
|
||||
@@ -70,8 +70,10 @@ class TestToolCallEnvelopeEstimate:
|
||||
|
||||
def test_non_dict_tool_calls_do_not_crash(self):
|
||||
msg = {"role": "assistant", "content": "hi", "tool_calls": ["weird", None]}
|
||||
# Non-dict entries are ignored (as before) without raising.
|
||||
assert _estimate_msg_budget_tokens(msg) == len("hi") // _CHARS_PER_TOKEN + 10
|
||||
# Non-dict entries are ignored (as before) without raising. String
|
||||
# content now goes through estimate_tokens_rough (ceiling division),
|
||||
# so 2 chars estimate as 1 token instead of flooring to 0.
|
||||
assert _estimate_msg_budget_tokens(msg) == (len("hi") + 3) // _CHARS_PER_TOKEN + 10
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
|
||||
@@ -70,12 +70,14 @@ def _make_compressor():
|
||||
c.last_rough_tokens_when_real_prompt_fit = 0
|
||||
c.awaiting_real_usage_after_compression = False
|
||||
c._previous_summary = None
|
||||
c._summary_has_user_turn = None
|
||||
return c
|
||||
|
||||
|
||||
def _simulate_cron_session_state(c):
|
||||
"""Simulate per-session state that a cron compaction would leave behind."""
|
||||
c._previous_summary = "Cron session summary that must not leak"
|
||||
c._summary_has_user_turn = False
|
||||
c._last_summary_error = "Cron session summary error"
|
||||
c._last_summary_dropped_count = 5
|
||||
c._last_summary_fallback_used = True
|
||||
@@ -166,6 +168,7 @@ def test_on_session_end_matches_on_session_reset_surface():
|
||||
|
||||
per_session_attrs = [
|
||||
"_previous_summary",
|
||||
"_summary_has_user_turn",
|
||||
"_last_summary_error",
|
||||
"_last_summary_dropped_count",
|
||||
"_last_summary_fallback_used",
|
||||
|
||||
@@ -89,3 +89,26 @@ def test_handoff_in_protected_head_populates_previous_summary_before_update():
|
||||
assert compressor._previous_summary == old_summary
|
||||
assert seen_turns
|
||||
assert all(old_summary not in str(msg.get("content", "")) for msg in seen_turns)
|
||||
|
||||
|
||||
def test_handoff_in_protected_head_is_replaced_not_duplicated():
|
||||
"""Re-compaction must replace a protected old handoff with the updated one."""
|
||||
compressor = _compressor()
|
||||
old_summary = "OLD-PROTECTED-HANDOFF unique old summary body"
|
||||
|
||||
with patch("agent.context_compressor.call_llm", return_value=_response("UPDATED summary body")):
|
||||
compressed = compressor.compress(_messages_with_handoff(old_summary))
|
||||
|
||||
# The summary may be emitted standalone or merged into the first tail
|
||||
# message (alternation corner case), so detect it the same way the
|
||||
# compressor does rather than via a startswith(SUMMARY_PREFIX) check.
|
||||
summary_messages = [
|
||||
msg
|
||||
for msg in compressed
|
||||
if isinstance(msg, dict)
|
||||
and ContextCompressor._is_context_summary_content(msg.get("content"))
|
||||
]
|
||||
assert len(summary_messages) == 1
|
||||
assert "UPDATED summary body" in str(summary_messages[0]["content"])
|
||||
assert old_summary not in str(summary_messages[0]["content"])
|
||||
assert old_summary not in "\n".join(str(msg.get("content") or "") for msg in compressed)
|
||||
|
||||
@@ -0,0 +1,376 @@
|
||||
"""Regression coverage for zero-user compaction integrity (#64539)."""
|
||||
|
||||
import os
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from agent.context_compressor import (
|
||||
COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
COMPRESSED_SUMMARY_HAS_USER_TURN_KEY,
|
||||
COMPRESSED_SUMMARY_METADATA_KEY,
|
||||
HISTORICAL_TASK_HEADING,
|
||||
SUMMARY_PREFIX,
|
||||
ContextCompressor,
|
||||
_NO_USER_TASK_SENTINEL,
|
||||
)
|
||||
from agent.conversation_compression import (
|
||||
_ensure_compressed_has_user_turn,
|
||||
compress_context,
|
||||
)
|
||||
from hermes_state import SessionDB
|
||||
from tools.todo_tool import TODO_INJECTION_HEADER
|
||||
|
||||
|
||||
def _response(content: str) -> SimpleNamespace:
|
||||
return SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(content=content))]
|
||||
)
|
||||
|
||||
|
||||
def _valid_zero_user_summary(label: str = "Checked artifacts.") -> str:
|
||||
return f"""{HISTORICAL_TASK_HEADING}
|
||||
{_NO_USER_TASK_SENTINEL}
|
||||
|
||||
## Goal
|
||||
Historical cron work only.
|
||||
|
||||
## Completed Actions
|
||||
1. {label}
|
||||
|
||||
## Resolved Questions
|
||||
None. No user-authored questions exist.
|
||||
|
||||
## Historical Pending User Asks
|
||||
None. No user-authored requests exist.
|
||||
"""
|
||||
|
||||
|
||||
def _assistant_tool_turns(start: int, count: int) -> list[dict]:
|
||||
turns: list[dict] = []
|
||||
for idx in range(start, start + count):
|
||||
turns.extend(
|
||||
[
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Continuing scheduled work in English.",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": f"call-{idx}",
|
||||
"function": {
|
||||
"name": "terminal",
|
||||
"arguments": '{"command":"pwd"}',
|
||||
},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": f"call-{idx}",
|
||||
"content": "/workspace/project\n" + ("x" * 300),
|
||||
},
|
||||
]
|
||||
)
|
||||
return turns
|
||||
|
||||
|
||||
def _assistant_turns(start: int, count: int) -> list[dict]:
|
||||
return [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": f"Scheduled step {idx} completed. " + ("x" * 500),
|
||||
}
|
||||
for idx in range(start, start + count)
|
||||
]
|
||||
|
||||
|
||||
def _lifecycle_agent(db: SessionDB, session_id: str):
|
||||
with patch.dict(os.environ, {"OPENROUTER_API_KEY": "test-key"}):
|
||||
from run_agent import AIAgent
|
||||
|
||||
agent = AIAgent(
|
||||
api_key="test-key",
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
model="test/model",
|
||||
quiet_mode=True,
|
||||
session_db=db,
|
||||
session_id=session_id,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
)
|
||||
agent.compression_in_place = True
|
||||
agent.context_compressor.protect_first_n = 0
|
||||
agent.context_compressor.protect_last_n = 2
|
||||
agent.context_compressor.tail_token_budget = 80
|
||||
agent._todo_store.write(
|
||||
[{"id": "inspect", "content": "Inspect artifacts", "status": "pending"}]
|
||||
)
|
||||
return agent
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def compressor() -> ContextCompressor:
|
||||
with patch(
|
||||
"agent.context_compressor.get_model_context_length",
|
||||
return_value=100_000,
|
||||
):
|
||||
instance = ContextCompressor(
|
||||
model="test/model",
|
||||
threshold_percent=0.50,
|
||||
protect_first_n=0,
|
||||
protect_last_n=2,
|
||||
quiet_mode=True,
|
||||
)
|
||||
instance.tail_token_budget = 80
|
||||
return instance
|
||||
|
||||
|
||||
def test_generate_summary_rejects_fabricated_user_ask(compressor):
|
||||
fabricated = f"""{HISTORICAL_TASK_HEADING}
|
||||
User asked: 'Waar zijn de bestanden gedownload?'
|
||||
|
||||
## Goal
|
||||
Vind de bestanden.
|
||||
"""
|
||||
|
||||
with patch(
|
||||
"agent.context_compressor.call_llm",
|
||||
return_value=_response(fabricated),
|
||||
):
|
||||
result = compressor._generate_summary(_assistant_tool_turns(0, 2))
|
||||
|
||||
assert result is None
|
||||
assert compressor._previous_summary is None
|
||||
assert "invented user attribution" in compressor._last_summary_error
|
||||
|
||||
|
||||
def test_zero_user_prompt_anchors_source_language_and_exact_sentinel(compressor):
|
||||
captured_prompt = ""
|
||||
|
||||
def fake_call_llm(**kwargs):
|
||||
nonlocal captured_prompt
|
||||
captured_prompt = kwargs["messages"][0]["content"]
|
||||
return _response(_valid_zero_user_summary())
|
||||
|
||||
with patch("agent.context_compressor.call_llm", side_effect=fake_call_llm):
|
||||
result = compressor._generate_summary(_assistant_tool_turns(0, 2))
|
||||
|
||||
assert result == f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary().strip()}"
|
||||
assert "dominant language of the source turns" in captured_prompt
|
||||
assert _NO_USER_TASK_SENTINEL in captured_prompt
|
||||
assert "Do not write \"User asked:\"" in captured_prompt
|
||||
|
||||
|
||||
def test_zero_user_provenance_survives_iterative_compaction(compressor):
|
||||
messages = _assistant_tool_turns(0, 12)
|
||||
first_summary = f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary('First pass').strip()}"
|
||||
|
||||
with patch.object(compressor, "_generate_summary", return_value=first_summary):
|
||||
first = compressor.compress(messages, current_tokens=90_000)
|
||||
|
||||
first_handoffs = [
|
||||
message
|
||||
for message in first
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
]
|
||||
assert len(first_handoffs) == 1
|
||||
assert first_handoffs[0][COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
|
||||
|
||||
with patch(
|
||||
"agent.context_compressor.get_model_context_length",
|
||||
return_value=100_000,
|
||||
):
|
||||
resumed = ContextCompressor(
|
||||
model="test/model",
|
||||
threshold_percent=0.50,
|
||||
protect_first_n=0,
|
||||
protect_last_n=2,
|
||||
quiet_mode=True,
|
||||
)
|
||||
resumed.tail_token_budget = 80
|
||||
# SessionDB persists the summary content/role but not arbitrary internal
|
||||
# message keys. Simulate that round trip: the exact sentinel must recover
|
||||
# false provenance even when both in-process metadata keys are absent.
|
||||
persisted_handoff = dict(first_handoffs[0])
|
||||
persisted_handoff.pop(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
persisted_handoff.pop(COMPRESSED_SUMMARY_HAS_USER_TURN_KEY)
|
||||
second_input = [persisted_handoff, *_assistant_tool_turns(20, 12)]
|
||||
|
||||
def assert_provenance_then_summarize(*_args, **_kwargs):
|
||||
assert resumed._summary_has_user_turn is False
|
||||
return f"{SUMMARY_PREFIX}\n{_valid_zero_user_summary('Second pass').strip()}"
|
||||
|
||||
with patch.object(
|
||||
resumed,
|
||||
"_generate_summary",
|
||||
side_effect=assert_provenance_then_summarize,
|
||||
):
|
||||
second = resumed.compress(second_input, current_tokens=90_000)
|
||||
|
||||
second_handoffs = [
|
||||
message
|
||||
for message in second
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
]
|
||||
assert len(second_handoffs) == 1
|
||||
assert second_handoffs[0][COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
|
||||
|
||||
|
||||
def test_compress_context_todo_snapshot_stays_synthetic_across_two_boundaries(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
hermes_home = tmp_path / "hermes-home"
|
||||
monkeypatch.setenv("HERMES_HOME", str(hermes_home))
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
session_id = "zero-user-todo-lifecycle"
|
||||
db.create_session(session_id, source="cron", model="test/model")
|
||||
|
||||
first_agent = _lifecycle_agent(db, session_id)
|
||||
with patch(
|
||||
"agent.context_compressor.call_llm",
|
||||
return_value=_response(_valid_zero_user_summary("First boundary")),
|
||||
):
|
||||
first, _ = compress_context(
|
||||
first_agent,
|
||||
_assistant_turns(0, 24),
|
||||
"system",
|
||||
approx_tokens=90_000,
|
||||
force=True,
|
||||
)
|
||||
|
||||
first_handoff = next(
|
||||
message
|
||||
for message in first
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
)
|
||||
assert first_handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
|
||||
assert "First boundary" in first_handoff["content"]
|
||||
assert any(
|
||||
message.get("role") == "user"
|
||||
and str(message.get("content") or "").startswith(TODO_INJECTION_HEADER)
|
||||
for message in first
|
||||
)
|
||||
projected = db.get_messages_as_conversation(session_id)
|
||||
assert projected
|
||||
assert all(
|
||||
COMPRESSED_SUMMARY_METADATA_KEY not in message
|
||||
and COMPRESSED_SUMMARY_HAS_USER_TURN_KEY not in message
|
||||
for message in projected
|
||||
)
|
||||
|
||||
second_agent = _lifecycle_agent(db, session_id)
|
||||
with patch(
|
||||
"agent.context_compressor.call_llm",
|
||||
return_value=_response(_valid_zero_user_summary("Second boundary")),
|
||||
):
|
||||
second, _ = compress_context(
|
||||
second_agent,
|
||||
[*projected, *_assistant_turns(30, 24)],
|
||||
"system",
|
||||
approx_tokens=90_000,
|
||||
force=True,
|
||||
)
|
||||
|
||||
handoff = next(
|
||||
message
|
||||
for message in second
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
)
|
||||
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
|
||||
assert "Second boundary" in handoff["content"]
|
||||
assert "User asked:" not in handoff["content"]
|
||||
db.close()
|
||||
|
||||
|
||||
def test_continuation_user_marker_is_not_reused_as_real_provenance():
|
||||
todo_snapshot = f"{TODO_INJECTION_HEADER}\n- [ ] inspect. Inspect artifacts (pending)"
|
||||
compressed = [{"role": "assistant", "content": "Scheduled work completed."}]
|
||||
|
||||
_ensure_compressed_has_user_turn(
|
||||
[{"role": "user", "content": todo_snapshot}],
|
||||
compressed,
|
||||
)
|
||||
|
||||
assert compressed[-1] == {
|
||||
"role": "user",
|
||||
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
}
|
||||
projected = [{"role": row["role"], "content": row["content"]} for row in compressed]
|
||||
assert ContextCompressor._transcript_has_real_user_turn(projected) is False
|
||||
|
||||
|
||||
def test_continuation_markers_are_not_human_anchors():
|
||||
from agent.conversation_compression import _is_real_user_message
|
||||
|
||||
legacy = (
|
||||
"Continue from the compressed conversation context above. "
|
||||
"This marker exists because the compacted transcript contained "
|
||||
"no preserved user turn."
|
||||
)
|
||||
assert not _is_real_user_message(
|
||||
{"role": "user", "content": COMPRESSION_CONTINUATION_USER_CONTENT}
|
||||
)
|
||||
assert not _is_real_user_message({"role": "user", "content": legacy})
|
||||
|
||||
|
||||
def test_static_fallback_does_not_attribute_synthetic_rows_to_user(compressor):
|
||||
todo_snapshot = f"{TODO_INJECTION_HEADER}\n- [ ] inspect. Inspect artifacts (pending)"
|
||||
fallback = compressor._build_static_fallback_summary(
|
||||
[
|
||||
{"role": "user", "content": todo_snapshot},
|
||||
{
|
||||
"role": "user",
|
||||
"content": COMPRESSION_CONTINUATION_USER_CONTENT,
|
||||
},
|
||||
*_assistant_tool_turns(0, 2),
|
||||
]
|
||||
)
|
||||
|
||||
assert _NO_USER_TASK_SENTINEL in fallback
|
||||
assert "User asked:" not in fallback
|
||||
assert "INTERNAL CONTEXT:" in fallback
|
||||
|
||||
|
||||
def test_zero_user_deterministic_fallback_uses_same_provenance(compressor):
|
||||
messages = _assistant_tool_turns(0, 12)
|
||||
|
||||
with patch.object(compressor, "_generate_summary", return_value=None):
|
||||
result = compressor.compress(messages, current_tokens=90_000)
|
||||
|
||||
handoff = next(
|
||||
message
|
||||
for message in result
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
)
|
||||
assert _NO_USER_TASK_SENTINEL in handoff["content"]
|
||||
assert "User asked:" not in handoff["content"]
|
||||
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is False
|
||||
|
||||
|
||||
def test_real_user_turn_sets_provenance_true(compressor):
|
||||
messages = [
|
||||
{"role": "user", "content": "Please inspect the build artifacts."},
|
||||
*_assistant_tool_turns(0, 12),
|
||||
]
|
||||
summary = f"{SUMMARY_PREFIX}\n{HISTORICAL_TASK_HEADING}\nUser asked: 'Please inspect the build artifacts.'"
|
||||
|
||||
with patch.object(compressor, "_generate_summary", return_value=summary):
|
||||
result = compressor.compress(messages, current_tokens=90_000)
|
||||
|
||||
handoff = next(
|
||||
message
|
||||
for message in result
|
||||
if message.get(COMPRESSED_SUMMARY_METADATA_KEY)
|
||||
)
|
||||
assert handoff[COMPRESSED_SUMMARY_HAS_USER_TURN_KEY] is True
|
||||
|
||||
|
||||
def test_session_boundaries_clear_summary_provenance(compressor):
|
||||
compressor._summary_has_user_turn = False
|
||||
compressor.on_session_reset()
|
||||
assert compressor._summary_has_user_turn is None
|
||||
|
||||
compressor._summary_has_user_turn = True
|
||||
compressor.on_session_end("cron-session", [])
|
||||
assert compressor._summary_has_user_turn is None
|
||||
@@ -58,9 +58,9 @@ class TestEstimateTokensRough:
|
||||
assert long > short
|
||||
|
||||
def test_unicode_multibyte(self):
|
||||
"""Unicode chars are still 1 Python char each — 4 chars/token holds."""
|
||||
"""CJK chars are token-dense: counted ~1 token each, not 4 chars/token."""
|
||||
text = "你好世界" # 4 CJK characters
|
||||
assert estimate_tokens_rough(text) == 1
|
||||
assert estimate_tokens_rough(text) == 4
|
||||
|
||||
|
||||
class TestEstimateMessagesTokensRough:
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Regression test for issue #65429.
|
||||
|
||||
An agent built with ``skip_memory=True`` AND ``enabled_toolsets=["memory"]``
|
||||
used to get a memory tool wired to ``store=None`` (the built-in
|
||||
``MemoryStore`` was skipped along with the external provider), so every
|
||||
memory call failed silently and the main auto-capture path was dead.
|
||||
|
||||
The fix creates the built-in store whenever memory is enabled in config OR the
|
||||
memory toolset is explicitly enabled, while the external-provider block stays
|
||||
gated on ``skip_memory``.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from run_agent import AIAgent
|
||||
|
||||
|
||||
class _FakeOpenAI:
|
||||
def __init__(self, **kw):
|
||||
self.api_key = kw.get("api_key", "test")
|
||||
self.base_url = kw.get("base_url", "http://test")
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
|
||||
def _make_agent(monkeypatch, enabled_toolsets=None, skip_memory=True):
|
||||
monkeypatch.setattr("run_agent.get_tool_definitions", lambda **kw: [])
|
||||
monkeypatch.setattr("run_agent.check_toolset_requirements", lambda: {})
|
||||
monkeypatch.setattr("run_agent.OpenAI", _FakeOpenAI)
|
||||
return AIAgent(
|
||||
api_key="test-key",
|
||||
base_url="http://test",
|
||||
provider="openrouter",
|
||||
api_mode="chat_completions",
|
||||
max_iterations=1,
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=skip_memory,
|
||||
enabled_toolsets=enabled_toolsets,
|
||||
)
|
||||
|
||||
|
||||
def test_skip_memory_with_memory_toolset_creates_store(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
|
||||
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=True)
|
||||
assert agent._memory_store is not None, (
|
||||
"memory toolset enabled despite skip_memory=True must still build "
|
||||
"the built-in store (#65429)"
|
||||
)
|
||||
|
||||
|
||||
def test_skip_memory_without_memory_toolset_has_no_store(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
|
||||
agent = _make_agent(monkeypatch, enabled_toolsets=None, skip_memory=True)
|
||||
assert agent._memory_store is None, (
|
||||
"flush/background agent with skip_memory and no memory toolset must "
|
||||
"have no built-in store"
|
||||
)
|
||||
|
||||
|
||||
def test_memory_toolset_without_skip_memory_creates_store(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
|
||||
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=False)
|
||||
assert agent._memory_store is not None
|
||||
|
||||
|
||||
def test_skip_memory_memory_tool_handler_works_and_provider_skipped(
|
||||
monkeypatch, tmp_path
|
||||
):
|
||||
"""End-to-end behavioral check for #65429.
|
||||
|
||||
The memory tool handler must actually WORK (not return the
|
||||
"Memory is not available" store=None error) on a skip_memory=True agent
|
||||
with the memory toolset enabled, while the external memory provider
|
||||
sync/prefetch stays skipped (no MemoryManager is created).
|
||||
"""
|
||||
import json
|
||||
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hm"))
|
||||
agent = _make_agent(monkeypatch, enabled_toolsets=["memory"], skip_memory=True)
|
||||
|
||||
# Provider sync/prefetch must remain skipped: skip_memory still gates the
|
||||
# external memory provider block.
|
||||
assert agent._memory_manager is None, (
|
||||
"skip_memory=True must still skip the external memory provider"
|
||||
)
|
||||
|
||||
# Dispatch through the same entry point the tool executor uses
|
||||
# (agent/tool_executor.py wires store=agent._memory_store).
|
||||
from tools.memory_tool import memory_tool
|
||||
|
||||
raw = memory_tool(
|
||||
action="add",
|
||||
target="memory",
|
||||
content="User prefers concise answers.",
|
||||
store=agent._memory_store,
|
||||
)
|
||||
result = json.loads(raw)
|
||||
assert result.get("success") is True, (
|
||||
f"memory tool handler must work with skip_memory=True + memory "
|
||||
f"toolset (#65429), got: {raw}"
|
||||
)
|
||||
assert "Memory is not available" not in raw
|
||||
|
||||
# The write must actually persist to the profile-scoped memories dir.
|
||||
memory_md = tmp_path / "hm" / "memories" / "MEMORY.md"
|
||||
assert memory_md.exists()
|
||||
assert "User prefers concise answers." in memory_md.read_text()
|
||||
@@ -95,6 +95,17 @@ class FakeClient:
|
||||
|
||||
# Test helpers
|
||||
def queue_notification(self, method: str, **params):
|
||||
# Keep legacy fixture shorthand aligned with the IDs returned by the
|
||||
# fake thread/start and turn/start responses.
|
||||
if params.get("threadId") in {"t", "th"}:
|
||||
params["threadId"] = "thread-fake-001"
|
||||
if params.get("turnId") == "tu1":
|
||||
params["turnId"] = "turn-fake-001"
|
||||
turn = params.get("turn")
|
||||
if isinstance(turn, dict) and turn.get("id") == "tu1":
|
||||
turn = dict(turn)
|
||||
turn["id"] = "turn-fake-001"
|
||||
params["turn"] = turn
|
||||
self._notifications.append({"method": method, "params": params})
|
||||
|
||||
def queue_server_request(self, method: str, request_id: Any = "srv-1", **params):
|
||||
@@ -196,6 +207,173 @@ class TestRunTurn:
|
||||
# turn_id propagated for downstream session-DB linkage
|
||||
assert r.turn_id == "turn-fake-001"
|
||||
|
||||
def test_subagent_completion_does_not_end_parent_turn(self):
|
||||
"""A child completion must not become the parent's response."""
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-child-001",
|
||||
turnId="turn-child-001",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "child-message-1",
|
||||
"text": "child summary",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-child-001",
|
||||
turn={
|
||||
"id": "turn-child-001",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="turn-fake-001",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "parent-message-1",
|
||||
"text": "parent synthesis",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "turn-fake-001",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
result = make_session(client).run_turn("delegate this", turn_timeout=2.0)
|
||||
|
||||
assert result.final_text == "parent synthesis"
|
||||
assert result.interrupted is False
|
||||
assert result.error is None
|
||||
assert result.projected_messages == [
|
||||
{"role": "assistant", "content": "parent synthesis"}
|
||||
]
|
||||
|
||||
def test_stale_completion_on_parent_thread_is_ignored(self):
|
||||
"""A late completion from the previous parent turn is not terminal."""
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="turn-previous",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "stale-message",
|
||||
"text": "stale previous answer",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "turn-previous",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="turn-fake-001",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "current-message",
|
||||
"text": "current parent answer",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "turn-fake-001",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
result = make_session(client).run_turn("new prompt", turn_timeout=2.0)
|
||||
|
||||
assert result.final_text == "current parent answer"
|
||||
assert result.projected_messages == [
|
||||
{"role": "assistant", "content": "current parent answer"}
|
||||
]
|
||||
|
||||
def test_foreign_completion_in_server_request_drain_is_ignored(self):
|
||||
"""Approval draining must not project a child result into the parent."""
|
||||
client = FakeClient()
|
||||
client.queue_server_request(
|
||||
"item/commandExecution/requestApproval",
|
||||
request_id="approval-1",
|
||||
command="pwd",
|
||||
cwd="/tmp",
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-child-001",
|
||||
turnId="turn-child-001",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "child-message",
|
||||
"text": "child drain summary",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-child-001",
|
||||
turn={
|
||||
"id": "turn-child-001",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
original_respond = client.respond
|
||||
|
||||
def respond_and_release_parent(request_id, response):
|
||||
original_respond(request_id, response)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="turn-fake-001",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "parent-message",
|
||||
"text": "parent after approval",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "turn-fake-001",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
client.respond = respond_and_release_parent
|
||||
session = make_session(
|
||||
client,
|
||||
request_routing=_ServerRequestRouting(auto_approve_exec=True),
|
||||
)
|
||||
|
||||
result = session.run_turn("delegate then continue", turn_timeout=2.0)
|
||||
|
||||
assert client.responses == [("approval-1", {"decision": "accept"})]
|
||||
assert result.final_text == "parent after approval"
|
||||
assert result.projected_messages == [
|
||||
{"role": "assistant", "content": "parent after approval"}
|
||||
]
|
||||
|
||||
def test_token_usage_notification_is_captured(self):
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
@@ -528,6 +706,121 @@ class TestCompactThread:
|
||||
assert r.token_usage_last["totalTokens"] == 12
|
||||
assert r.model_context_window == 200000
|
||||
|
||||
def test_compact_thread_ignores_foreign_child_completion(self):
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
"turn/started",
|
||||
threadId="thread-child-001",
|
||||
turn={"id": "child-compact-turn"},
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-child-001",
|
||||
turnId="child-compact-turn",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "child-compact-message",
|
||||
"text": "child compact summary",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-child-001",
|
||||
turn={
|
||||
"id": "child-compact-turn",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/started",
|
||||
threadId="thread-fake-001",
|
||||
turn={"id": "compact-turn-1"},
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="compact-turn-1",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "parent-compact-message",
|
||||
"text": "parent compacted",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "compact-turn-1",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
result = make_session(client).compact_thread(turn_timeout=2.0)
|
||||
|
||||
assert result.error is None
|
||||
assert result.turn_id == "compact-turn-1"
|
||||
assert result.final_text == "parent compacted"
|
||||
assert result.projected_messages == [
|
||||
{"role": "assistant", "content": "parent compacted"}
|
||||
]
|
||||
|
||||
def test_compact_thread_ignores_stale_same_thread_completion_before_start(self):
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="previous-compact-turn",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "stale-compact-message",
|
||||
"text": "stale compact result",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "previous-compact-turn",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/started",
|
||||
threadId="thread-fake-001",
|
||||
turn={"id": "compact-turn-1"},
|
||||
)
|
||||
client.queue_notification(
|
||||
"item/completed",
|
||||
threadId="thread-fake-001",
|
||||
turnId="compact-turn-1",
|
||||
item={
|
||||
"type": "agentMessage",
|
||||
"id": "current-compact-message",
|
||||
"text": "current compact result",
|
||||
},
|
||||
)
|
||||
client.queue_notification(
|
||||
"turn/completed",
|
||||
threadId="thread-fake-001",
|
||||
turn={
|
||||
"id": "compact-turn-1",
|
||||
"status": "completed",
|
||||
"error": None,
|
||||
},
|
||||
)
|
||||
|
||||
result = make_session(client).compact_thread(turn_timeout=2.0)
|
||||
|
||||
assert result.error is None
|
||||
assert result.turn_id == "compact-turn-1"
|
||||
assert result.final_text == "current compact result"
|
||||
assert result.projected_messages == [
|
||||
{"role": "assistant", "content": "current compact result"}
|
||||
]
|
||||
|
||||
def test_compact_thread_interrupted_returns_non_success(self):
|
||||
client = FakeClient()
|
||||
client.queue_notification(
|
||||
|
||||
@@ -2102,6 +2102,273 @@ class TestResponsesEndpoint:
|
||||
assert stored_history.count(first_history[0]) == 1
|
||||
assert stored_history.count({"role": "user", "content": "Now add 1 more"}) == 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_previous_response_id_stores_compressed_transcript_directly(self, adapter):
|
||||
"""After compression, stored history is the compressed transcript, not prior + compressed."""
|
||||
prior_history = [
|
||||
{"role": "user", "content": "What is 1+1?"},
|
||||
{"role": "assistant", "content": "2"},
|
||||
] * 10 # 20 messages — enough to simulate a long conversation
|
||||
adapter._response_store.put(
|
||||
"resp_prev",
|
||||
{
|
||||
"response": {"id": "resp_prev", "status": "completed"},
|
||||
"conversation_history": list(prior_history),
|
||||
"session_id": "api-test-session",
|
||||
},
|
||||
)
|
||||
|
||||
compressed_history = [
|
||||
# Compressed transcript starts with summary, NOT with prior[0]
|
||||
{"role": "user", "content": "[Compressed summary of earlier conversation]"},
|
||||
{"role": "user", "content": "Now add 1 more"},
|
||||
{"role": "assistant", "content": "3"},
|
||||
]
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
|
||||
mock_run.return_value = (
|
||||
{
|
||||
"final_response": "3",
|
||||
"messages": list(compressed_history),
|
||||
"_compressed": True,
|
||||
"api_calls": 1,
|
||||
},
|
||||
{"input_tokens": 0, "output_tokens": 0, "total_tokens": 0},
|
||||
)
|
||||
resp = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "Now add 1 more",
|
||||
"previous_response_id": "resp_prev",
|
||||
},
|
||||
)
|
||||
assert resp.status == 200
|
||||
data = await resp.json()
|
||||
|
||||
stored = adapter._response_store.get(data["id"])
|
||||
stored_history = stored["conversation_history"]
|
||||
# Must NOT contain the original prior_history messages
|
||||
for msg in prior_history:
|
||||
assert msg not in stored_history, (
|
||||
f"Prior history message leaked into stored compressed transcript: {msg}"
|
||||
)
|
||||
# Must contain the compressed transcript
|
||||
assert stored_history == compressed_history
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rotation_compression_exercises_detection_and_persists_rotated_session_id(
|
||||
self, adapter
|
||||
):
|
||||
"""Fake-agent rotation: _run_agent detects session_id change, sets
|
||||
_compressed, and the Responses handler persists the rotated session_id
|
||||
so subsequent previous_response_id chaining loads the compressed child."""
|
||||
prior_history = [
|
||||
{"role": "user", "content": "What is 1+1?"},
|
||||
{"role": "assistant", "content": "2"},
|
||||
] * 10
|
||||
|
||||
adapter._response_store.put(
|
||||
"resp_prev_rot",
|
||||
{
|
||||
"response": {"id": "resp_prev_rot", "status": "completed"},
|
||||
"conversation_history": list(prior_history),
|
||||
"session_id": "api-test-session",
|
||||
},
|
||||
)
|
||||
|
||||
compressed_history = [
|
||||
{"role": "user", "content": "[Compressed summary]"},
|
||||
{"role": "user", "content": "Now add 1 more"},
|
||||
{"role": "assistant", "content": "3"},
|
||||
]
|
||||
|
||||
# Fake agent whose session_id was rotated by compression
|
||||
mock_agent = MagicMock()
|
||||
mock_agent.session_id = "rotated-child-session"
|
||||
mock_agent._last_compaction_in_place = False
|
||||
mock_agent.session_prompt_tokens = 0
|
||||
mock_agent.session_completion_tokens = 0
|
||||
mock_agent.session_total_tokens = 0
|
||||
mock_agent.run_conversation.return_value = {
|
||||
"final_response": "3",
|
||||
"messages": list(compressed_history),
|
||||
"api_calls": 1,
|
||||
}
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_create_agent", return_value=mock_agent):
|
||||
resp = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "Now add 1 more",
|
||||
"previous_response_id": "resp_prev_rot",
|
||||
},
|
||||
)
|
||||
assert resp.status == 200
|
||||
data = await resp.json()
|
||||
|
||||
# The detection path in _run_agent must have set _compressed
|
||||
# _run_agent mutates the result dict in place -- verify via stored data
|
||||
stored = adapter._response_store.get(data["id"])
|
||||
assert stored is not None
|
||||
|
||||
# Stored history must be the compressed transcript, not prior + compressed
|
||||
stored_history = stored["conversation_history"]
|
||||
for msg in prior_history:
|
||||
assert msg not in stored_history
|
||||
assert stored_history == compressed_history
|
||||
|
||||
# Stored session_id must be the rotated child, not the original
|
||||
assert stored["session_id"] == "rotated-child-session"
|
||||
|
||||
# Response header must also reflect the rotated session
|
||||
assert resp.headers.get("X-Hermes-Session-Id") == "rotated-child-session"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_inplace_compression_exercises_detection_and_persists_compressed_history(
|
||||
self, adapter
|
||||
):
|
||||
"""Fake-agent in-place: _run_agent detects _last_compaction_in_place,
|
||||
sets _compressed, and the Responses handler persists compressed history
|
||||
WITHOUT rotating the session_id."""
|
||||
prior_history = [
|
||||
{"role": "user", "content": "What is 1+1?"},
|
||||
{"role": "assistant", "content": "2"},
|
||||
] * 10
|
||||
|
||||
adapter._response_store.put(
|
||||
"resp_prev_inplace",
|
||||
{
|
||||
"response": {"id": "resp_prev_inplace", "status": "completed"},
|
||||
"conversation_history": list(prior_history),
|
||||
"session_id": "api-test-session",
|
||||
},
|
||||
)
|
||||
|
||||
compressed_history = [
|
||||
{"role": "user", "content": "[Compressed summary in-place]"},
|
||||
{"role": "user", "content": "Continue"},
|
||||
{"role": "assistant", "content": "42"},
|
||||
]
|
||||
|
||||
# Fake agent: in-place compaction, session_id unchanged
|
||||
mock_agent = MagicMock()
|
||||
mock_agent.session_id = "api-test-session" # same as input -- no rotation
|
||||
mock_agent._last_compaction_in_place = True
|
||||
mock_agent.session_prompt_tokens = 0
|
||||
mock_agent.session_completion_tokens = 0
|
||||
mock_agent.session_total_tokens = 0
|
||||
mock_agent.run_conversation.return_value = {
|
||||
"final_response": "42",
|
||||
"messages": list(compressed_history),
|
||||
"api_calls": 1,
|
||||
}
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_create_agent", return_value=mock_agent):
|
||||
resp = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "Continue",
|
||||
"previous_response_id": "resp_prev_inplace",
|
||||
},
|
||||
)
|
||||
assert resp.status == 200
|
||||
data = await resp.json()
|
||||
|
||||
stored = adapter._response_store.get(data["id"])
|
||||
assert stored is not None
|
||||
|
||||
# Stored history must be the compressed transcript
|
||||
stored_history = stored["conversation_history"]
|
||||
for msg in prior_history:
|
||||
assert msg not in stored_history
|
||||
assert stored_history == compressed_history
|
||||
|
||||
# Session_id must NOT change for in-place compaction
|
||||
assert stored["session_id"] == "api-test-session"
|
||||
assert resp.headers.get("X-Hermes-Session-Id") == "api-test-session"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_chained_rotation_propagates_effective_session_id(self, adapter):
|
||||
"""Two-request chain: first request triggers rotation, second request
|
||||
loads history using the rotated session_id stored by the first."""
|
||||
# First request -- no previous_response_id, establishes the conversation
|
||||
mock_agent_1 = MagicMock()
|
||||
mock_agent_1.session_id = "child-session-after-rotation"
|
||||
mock_agent_1._last_compaction_in_place = False
|
||||
mock_agent_1.session_prompt_tokens = 0
|
||||
mock_agent_1.session_completion_tokens = 0
|
||||
mock_agent_1.session_total_tokens = 0
|
||||
compressed_msg_1 = [
|
||||
{"role": "user", "content": "[Summary of turn 1]"},
|
||||
{"role": "assistant", "content": "Hello back"},
|
||||
]
|
||||
mock_agent_1.run_conversation.return_value = {
|
||||
"final_response": "Hello back",
|
||||
"messages": list(compressed_msg_1),
|
||||
"api_calls": 1,
|
||||
}
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_create_agent", return_value=mock_agent_1):
|
||||
resp1 = await cli.post(
|
||||
"/v1/responses",
|
||||
json={"model": "hermes-agent", "input": "Hello"},
|
||||
)
|
||||
assert resp1.status == 200
|
||||
data1 = await resp1.json()
|
||||
response_id_1 = data1["id"]
|
||||
|
||||
# Verify the rotated session_id was persisted
|
||||
stored_1 = adapter._response_store.get(response_id_1)
|
||||
assert stored_1["session_id"] == "child-session-after-rotation"
|
||||
assert stored_1["conversation_history"] == compressed_msg_1
|
||||
|
||||
# Second request -- chains via previous_response_id
|
||||
# _run_agent is called with the session_id from the stored response
|
||||
mock_agent_2 = MagicMock()
|
||||
mock_agent_2.session_id = "child-session-after-rotation"
|
||||
mock_agent_2._last_compaction_in_place = False
|
||||
mock_agent_2.session_prompt_tokens = 0
|
||||
mock_agent_2.session_completion_tokens = 0
|
||||
mock_agent_2.session_total_tokens = 0
|
||||
mock_agent_2.run_conversation.return_value = {
|
||||
"final_response": "Goodbye",
|
||||
"messages": [
|
||||
{"role": "user", "content": "[Summary of turn 1]"},
|
||||
{"role": "assistant", "content": "Hello back"},
|
||||
{"role": "user", "content": "Goodbye"},
|
||||
{"role": "assistant", "content": "See you!"},
|
||||
],
|
||||
"api_calls": 1,
|
||||
}
|
||||
|
||||
with patch.object(adapter, "_create_agent", return_value=mock_agent_2):
|
||||
resp2 = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "Goodbye",
|
||||
"previous_response_id": response_id_1,
|
||||
},
|
||||
)
|
||||
assert resp2.status == 200
|
||||
|
||||
# The second _run_agent call must receive the rotated session_id
|
||||
# (from the stored response), not the original request session_id
|
||||
call_kwargs = mock_agent_2.run_conversation.call_args.kwargs
|
||||
# conversation_history loaded from store should be the compressed transcript
|
||||
assert call_kwargs["conversation_history"] == compressed_msg_1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_previous_response_id_outputs_only_current_turn_items(self, adapter):
|
||||
"""Response output must not replay previous tool artifacts."""
|
||||
@@ -3270,6 +3537,95 @@ class TestTruncation:
|
||||
call_kwargs = mock_run.call_args.kwargs
|
||||
# History should be truncated to 100
|
||||
assert len(call_kwargs["conversation_history"]) <= 100
|
||||
assert call_kwargs["conversation_history"][0]["content"] == "msg 50"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_truncation_auto_preserves_leading_compaction_summary(self, adapter):
|
||||
"""truncation=auto must not discard the compacted-history handoff."""
|
||||
mock_result = {"final_response": "OK", "messages": [], "api_calls": 1}
|
||||
|
||||
summary = {
|
||||
"role": "user",
|
||||
"content": "[CONTEXT COMPACTION — REFERENCE ONLY]\nEarlier work.",
|
||||
"_compressed_summary": True,
|
||||
}
|
||||
long_history = [summary] + [
|
||||
{"role": "user", "content": f"msg {i}"}
|
||||
for i in range(149)
|
||||
]
|
||||
adapter._response_store.put("resp_summary", {
|
||||
"response": {"id": "resp_summary", "object": "response"},
|
||||
"conversation_history": long_history,
|
||||
"instructions": None,
|
||||
})
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
|
||||
mock_run.return_value = (mock_result, {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0})
|
||||
resp = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "follow up",
|
||||
"previous_response_id": "resp_summary",
|
||||
"truncation": "auto",
|
||||
},
|
||||
)
|
||||
|
||||
assert resp.status == 200
|
||||
history = mock_run.call_args.kwargs["conversation_history"]
|
||||
assert len(history) == 100
|
||||
assert history[0] == summary
|
||||
assert history[1]["content"] == "msg 50"
|
||||
assert history[-1]["content"] == "msg 148"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_truncation_auto_preserves_non_leading_compaction_summary(self, adapter):
|
||||
"""A summary sitting after a retained system head must survive too.
|
||||
|
||||
The gateway /compress path can force a user-leading layout that
|
||||
leaves the compaction summary after a kept system message, so the
|
||||
preservation predicate must not assume the summary is at index 0.
|
||||
"""
|
||||
mock_result = {"final_response": "OK", "messages": [], "api_calls": 1}
|
||||
|
||||
system_head = {"role": "system", "content": "You are a helpful agent."}
|
||||
summary = {
|
||||
"role": "user",
|
||||
"content": "[CONTEXT COMPACTION — REFERENCE ONLY]\nEarlier work.",
|
||||
"_compressed_summary": True,
|
||||
}
|
||||
long_history = [system_head, summary] + [
|
||||
{"role": "user", "content": f"msg {i}"}
|
||||
for i in range(148)
|
||||
]
|
||||
adapter._response_store.put("resp_summary_mid", {
|
||||
"response": {"id": "resp_summary_mid", "object": "response"},
|
||||
"conversation_history": long_history,
|
||||
"instructions": None,
|
||||
})
|
||||
|
||||
app = _create_app(adapter)
|
||||
async with TestClient(TestServer(app)) as cli:
|
||||
with patch.object(adapter, "_run_agent", new_callable=AsyncMock) as mock_run:
|
||||
mock_run.return_value = (mock_result, {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0})
|
||||
resp = await cli.post(
|
||||
"/v1/responses",
|
||||
json={
|
||||
"model": "hermes-agent",
|
||||
"input": "follow up",
|
||||
"previous_response_id": "resp_summary_mid",
|
||||
"truncation": "auto",
|
||||
},
|
||||
)
|
||||
|
||||
assert resp.status == 200
|
||||
history = mock_run.call_args.kwargs["conversation_history"]
|
||||
assert len(history) == 100
|
||||
assert history[0] == summary
|
||||
assert history[1]["content"] == "msg 49"
|
||||
assert history[-1]["content"] == "msg 147"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_truncation_keeps_full_history(self, adapter):
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Regression tests for #52374 — raw clarify tool-call JSON must never leak
|
||||
into the chat as a tool-progress bubble.
|
||||
|
||||
The adapter's ``send_clarify`` is the user-facing rendering of a clarify
|
||||
prompt (interactive buttons, or the numbered-text fallback). The gateway's
|
||||
tool-progress callback used to also render a progress bubble for the
|
||||
``clarify`` tool.started event — in verbose mode that bubble contains the raw
|
||||
tool-call args JSON (``{"question": ..., "choices": [...]}``), and because the
|
||||
progress queue drains on a background task the JSON landed right underneath
|
||||
the rendered interactive prompt on Slack.
|
||||
"""
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import time
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import BasePlatformAdapter, SendResult
|
||||
from gateway.session import SessionSource
|
||||
|
||||
|
||||
class ProgressCaptureAdapter(BasePlatformAdapter):
|
||||
"""Records every send so the test can assert nothing leaked."""
|
||||
|
||||
def __init__(self, platform=Platform.SLACK):
|
||||
super().__init__(PlatformConfig(enabled=True, token="***"), platform)
|
||||
self.sent = []
|
||||
self.edits = []
|
||||
|
||||
async def connect(self, *, is_reconnect: bool = False) -> bool:
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
return None
|
||||
|
||||
async def send(self, chat_id, content, reply_to=None, metadata=None) -> SendResult:
|
||||
self.sent.append({"chat_id": chat_id, "content": content})
|
||||
return SendResult(success=True, message_id="m-1")
|
||||
|
||||
async def edit_message(self, chat_id, message_id, content) -> SendResult:
|
||||
self.edits.append({"chat_id": chat_id, "message_id": message_id, "content": content})
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
|
||||
async def send_typing(self, chat_id, metadata=None) -> None:
|
||||
return None
|
||||
|
||||
async def stop_typing(self, chat_id) -> None:
|
||||
return None
|
||||
|
||||
async def get_chat_info(self, chat_id: str):
|
||||
return {"id": chat_id}
|
||||
|
||||
|
||||
class ClarifyThenToolAgent:
|
||||
"""Emits a clarify tool.started (with raw args) then a normal tool."""
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
self.tool_progress_callback = kwargs.get("tool_progress_callback")
|
||||
self.tools = []
|
||||
|
||||
def run_conversation(self, message, conversation_history=None, task_id=None):
|
||||
cb = self.tool_progress_callback
|
||||
if cb is not None:
|
||||
cb(
|
||||
"tool.started",
|
||||
"clarify",
|
||||
"Which environment?",
|
||||
{"question": "Which environment?", "choices": ["staging", "production"]},
|
||||
)
|
||||
time.sleep(0.35)
|
||||
cb("tool.started", "terminal", "pwd", {})
|
||||
time.sleep(0.35)
|
||||
return {"final_response": "done", "messages": [], "api_calls": 1}
|
||||
|
||||
|
||||
def _make_runner(adapter):
|
||||
gateway_run = importlib.import_module("gateway.run")
|
||||
GatewayRunner = gateway_run.GatewayRunner
|
||||
runner = object.__new__(GatewayRunner)
|
||||
runner.adapters = {adapter.platform: adapter}
|
||||
runner._voice_mode = {}
|
||||
runner._prefill_messages = []
|
||||
runner._ephemeral_system_prompt = ""
|
||||
runner._reasoning_config = None
|
||||
runner._provider_routing = {}
|
||||
runner._fallback_model = None
|
||||
runner._session_db = None
|
||||
runner._running_agents = {}
|
||||
runner._session_run_generation = {}
|
||||
runner.hooks = types.SimpleNamespace(loaded_hooks=False)
|
||||
runner.config = types.SimpleNamespace(
|
||||
thread_sessions_per_user=False,
|
||||
group_sessions_per_user=False,
|
||||
stt_enabled=False,
|
||||
)
|
||||
return runner
|
||||
|
||||
|
||||
def _install_fakes(monkeypatch, mode):
|
||||
monkeypatch.setenv("HERMES_TOOL_PROGRESS_MODE", mode)
|
||||
|
||||
fake_dotenv = types.ModuleType("dotenv")
|
||||
fake_dotenv.load_dotenv = lambda *a, **k: None
|
||||
monkeypatch.setitem(sys.modules, "dotenv", fake_dotenv)
|
||||
|
||||
fake_run_agent = types.ModuleType("run_agent")
|
||||
fake_run_agent.AIAgent = ClarifyThenToolAgent
|
||||
monkeypatch.setitem(sys.modules, "run_agent", fake_run_agent)
|
||||
import tools.terminal_tool # noqa: F401 — register terminal emoji
|
||||
|
||||
gateway_run = importlib.import_module("gateway.run")
|
||||
monkeypatch.setattr(gateway_run, "_resolve_runtime_agent_kwargs", lambda: {"api_key": "***"})
|
||||
return gateway_run
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", ["verbose", "all"])
|
||||
@pytest.mark.asyncio
|
||||
async def test_clarify_tool_never_renders_progress_bubble(monkeypatch, tmp_path, mode):
|
||||
"""No progress bubble for clarify — in any mode, especially verbose.
|
||||
|
||||
Verbose mode used to dump the raw args JSON
|
||||
(``{"question": ..., "choices": [...]}``) into the chat right under the
|
||||
interactive prompt (#52374).
|
||||
"""
|
||||
adapter = ProgressCaptureAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
gateway_run = _install_fakes(monkeypatch, mode)
|
||||
monkeypatch.setattr(gateway_run, "_hermes_home", tmp_path)
|
||||
|
||||
source = SessionSource(platform=Platform.SLACK, chat_id="C1", chat_type="dm")
|
||||
|
||||
result = await runner._run_agent(
|
||||
message="hello",
|
||||
context_prompt="",
|
||||
history=[],
|
||||
source=source,
|
||||
session_id="sess-clarify-leak",
|
||||
session_key="agent:main:slack:dm:C1",
|
||||
)
|
||||
|
||||
assert result["final_response"] == "done"
|
||||
all_content = "\n".join(
|
||||
[m["content"] for m in adapter.sent] + [e["content"] for e in adapter.edits]
|
||||
)
|
||||
# Raw clarify args JSON must not leak anywhere.
|
||||
assert '"question"' not in all_content
|
||||
assert '"choices"' not in all_content
|
||||
assert "Which environment?" not in all_content
|
||||
# No clarify progress line at all (verb "Asking" / tool name).
|
||||
assert "clarify" not in all_content
|
||||
assert "Asking" not in all_content
|
||||
# The unrelated terminal tool still renders progress normally.
|
||||
assert "pwd" in all_content
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Regression tests for #62034 — pending multi-choice clarify prompts must not
|
||||
swallow unrelated thread follow-up messages.
|
||||
|
||||
When a NATIVE interactive multi-choice clarify (buttons rendered,
|
||||
``awaiting_text=False``) is pending, the gateway text-intercept used to
|
||||
consume ANY non-command message in the session as the clarify answer —
|
||||
arbitrary prose vanished into clarify resolution and the agent appeared to
|
||||
ignore the user's thread messages.
|
||||
|
||||
After the fix (``tools/clarify_gateway._coerce_text_response`` rejects
|
||||
arbitrary prose for native multi-choice prompts):
|
||||
|
||||
* numeric selections ("2") and exact choice labels still resolve, and
|
||||
* arbitrary prose falls through the intercept and continues as a normal
|
||||
message-handling turn.
|
||||
|
||||
Open-ended clarifies, explicit "Other" text-capture mode, and the base
|
||||
adapter's numbered-text fallback (which flips ``awaiting_text`` at send time)
|
||||
keep accepting free text.
|
||||
"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
)
|
||||
from gateway.session import SessionSource
|
||||
|
||||
|
||||
SESSION_KEY = "agent:main:slack:dm:D123:1111.2222"
|
||||
|
||||
|
||||
class _StubAdapter(BasePlatformAdapter):
|
||||
def __init__(self):
|
||||
super().__init__(PlatformConfig(enabled=True, token="test"), Platform.SLACK)
|
||||
|
||||
async def connect(self, *, is_reconnect: bool = False):
|
||||
return True
|
||||
|
||||
async def disconnect(self):
|
||||
pass
|
||||
|
||||
async def send(self, chat_id, content, reply_to=None, metadata=None):
|
||||
return SendResult(success=True, message_id="m1")
|
||||
|
||||
async def get_chat_info(self, chat_id):
|
||||
return {"id": chat_id, "type": "im"}
|
||||
|
||||
|
||||
class _FellThroughIntercept(Exception):
|
||||
"""Sentinel: _handle_message got PAST the clarify text-intercept."""
|
||||
|
||||
|
||||
def _event(text):
|
||||
return MessageEvent(
|
||||
text=text,
|
||||
message_type=MessageType.TEXT,
|
||||
source=SessionSource(
|
||||
platform=Platform.SLACK,
|
||||
chat_id="D123",
|
||||
chat_type="dm",
|
||||
user_id="U1",
|
||||
thread_id="1111.2222",
|
||||
),
|
||||
message_id="msg1",
|
||||
)
|
||||
|
||||
|
||||
def _clear_clarify_state():
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
with cm._lock:
|
||||
cm._entries.clear()
|
||||
cm._session_index.clear()
|
||||
cm._notify_cbs.clear()
|
||||
|
||||
|
||||
def _make_runner(adapter):
|
||||
from gateway.run import GatewayRunner
|
||||
|
||||
runner = GatewayRunner.__new__(GatewayRunner)
|
||||
runner._startup_restore_in_progress = False
|
||||
runner._scale_to_zero_note_real_inbound = lambda: None
|
||||
runner._is_user_authorized = lambda source: True
|
||||
runner._session_key_for_source = lambda source: SESSION_KEY
|
||||
runner._adapter_for_source = lambda source: adapter
|
||||
runner._update_prompt_pending = {}
|
||||
return runner
|
||||
|
||||
|
||||
async def _dispatch(runner, event):
|
||||
"""Run _handle_message with a tripwire installed AFTER the clarify
|
||||
intercept (the slash-confirm pending lookup is the next statement), so a
|
||||
raised ``_FellThroughIntercept`` proves the message was NOT swallowed."""
|
||||
import tools.slash_confirm as slash_confirm_mod
|
||||
|
||||
def _tripwire(_key):
|
||||
raise _FellThroughIntercept()
|
||||
|
||||
with patch("hermes_cli.plugins.invoke_hook", return_value=[]), \
|
||||
patch.object(slash_confirm_mod, "get_pending", _tripwire):
|
||||
return await runner._handle_message(event)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_thread_prose_not_swallowed_by_native_multi_choice_clarify():
|
||||
"""Arbitrary prose during a pending button-clarify continues as a normal turn."""
|
||||
_clear_clarify_state()
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _StubAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
# Native interactive multi-choice prompt: awaiting_text stays False.
|
||||
entry = cm.register("cl-native", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
|
||||
assert entry.awaiting_text is False
|
||||
|
||||
with pytest.raises(_FellThroughIntercept):
|
||||
await _dispatch(runner, _event("just checking the visual UI, no need to pass any data"))
|
||||
|
||||
# The clarify entry must still be pending and unresolved.
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cl-native")
|
||||
assert entry is not None
|
||||
assert not entry.event.is_set()
|
||||
_clear_clarify_state()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_numeric_reply_still_resolves_native_multi_choice_clarify():
|
||||
"""Typed "2" keeps resolving the button prompt through the same intercept."""
|
||||
_clear_clarify_state()
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _StubAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
cm.register("cl-num", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
|
||||
|
||||
result = await _dispatch(runner, _event("2"))
|
||||
|
||||
assert result == "" # intercepted + acknowledged silently
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cl-num")
|
||||
assert entry is not None
|
||||
assert entry.event.is_set()
|
||||
assert entry.response == "dropdown"
|
||||
_clear_clarify_state()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_exact_label_reply_still_resolves_native_multi_choice_clarify():
|
||||
_clear_clarify_state()
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _StubAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
cm.register("cl-label", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
|
||||
|
||||
result = await _dispatch(runner, _event("Buttons"))
|
||||
|
||||
assert result == ""
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cl-label")
|
||||
assert entry is not None
|
||||
assert entry.event.is_set()
|
||||
assert entry.response == "buttons"
|
||||
_clear_clarify_state()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_prose_still_accepted_after_other_flips_text_capture():
|
||||
"""After the user taps 'Other', free text IS the answer — must resolve."""
|
||||
_clear_clarify_state()
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _StubAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
cm.register("cl-other", SESSION_KEY, "Pick a UI variant", ["buttons", "dropdown"])
|
||||
assert cm.mark_awaiting_text("cl-other") is True
|
||||
|
||||
result = await _dispatch(runner, _event("a carousel actually"))
|
||||
|
||||
assert result == ""
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cl-other")
|
||||
assert entry is not None
|
||||
assert entry.event.is_set()
|
||||
assert entry.response == "a carousel actually"
|
||||
_clear_clarify_state()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_prose_still_accepted_for_open_ended_clarify():
|
||||
_clear_clarify_state()
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _StubAdapter()
|
||||
runner = _make_runner(adapter)
|
||||
cm.register("cl-open", SESSION_KEY, "What should I name it?", None)
|
||||
|
||||
result = await _dispatch(runner, _event("call it hermes-ux"))
|
||||
|
||||
assert result == ""
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cl-open")
|
||||
assert entry is not None
|
||||
assert entry.event.is_set()
|
||||
assert entry.response == "call it hermes-ux"
|
||||
_clear_clarify_state()
|
||||
@@ -939,6 +939,20 @@ class TestSlackProxyBehavior:
|
||||
assert adapter._handler.proxy == "http://proxy.example.com:3128"
|
||||
assert adapter._handler.client.proxy == "http://proxy.example.com:3128"
|
||||
assert "hermes_feedback" in created_apps[0].registered_actions
|
||||
assert "hermes_clarify_other" in created_apps[0].registered_actions
|
||||
clarify_choice_patterns = [
|
||||
action_id
|
||||
for action_id in created_apps[0].registered_actions
|
||||
if hasattr(action_id, "fullmatch")
|
||||
]
|
||||
assert any(
|
||||
pattern.fullmatch("hermes_clarify_choice_0")
|
||||
for pattern in clarify_choice_patterns
|
||||
)
|
||||
assert not any(
|
||||
pattern.fullmatch("hermes_clarify_choice")
|
||||
for pattern in clarify_choice_patterns
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_connect_clears_proxy_when_no_proxy_matches_slack(self):
|
||||
@@ -2912,6 +2926,15 @@ class TestEditMessage:
|
||||
kwargs = adapter._app.client.chat_update.call_args.kwargs
|
||||
assert kwargs["text"] == "AT&T < 5 > 3"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_message_truncates_oversized_content(self, adapter):
|
||||
"""Oversized edits are truncated instead of failing with msg_too_long."""
|
||||
adapter._app.client.chat_update = AsyncMock(return_value={"ok": True})
|
||||
result = await adapter.edit_message("C123", "1234.5678", "x" * 45000)
|
||||
assert result.success
|
||||
kwargs = adapter._app.client.chat_update.call_args.kwargs
|
||||
assert len(kwargs["text"]) <= adapter.MAX_MESSAGE_LENGTH
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TestEditMessageStreamingPipeline
|
||||
@@ -4071,6 +4094,69 @@ class TestMessageSplitting:
|
||||
assert "<https://en.wikipedia.org/wiki/Foo_(bar)|Foo>" in kwargs["text"]
|
||||
|
||||
|
||||
class TestEmptyTextGuard:
|
||||
"""Guard against Slack ``no_text`` errors when content is empty/whitespace."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_skips_empty_string(self, adapter):
|
||||
"""Empty content must not call chat_postMessage."""
|
||||
adapter._app.client.chat_postMessage = AsyncMock(return_value={"ts": "ts1"})
|
||||
result = await adapter.send("C123", "")
|
||||
assert result.success is True
|
||||
adapter._app.client.chat_postMessage.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_skips_whitespace_only(self, adapter):
|
||||
"""Whitespace-only content must not call chat_postMessage."""
|
||||
adapter._app.client.chat_postMessage = AsyncMock(return_value={"ts": "ts1"})
|
||||
result = await adapter.send("C123", " \n\t ")
|
||||
assert result.success is True
|
||||
adapter._app.client.chat_postMessage.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_standalone_send_skips_empty(self, monkeypatch):
|
||||
"""_standalone_send returns success without HTTP call on empty text."""
|
||||
from plugins.platforms.slack.adapter import _standalone_send
|
||||
from types import SimpleNamespace
|
||||
|
||||
pconfig = SimpleNamespace(token="xoxb-test", extra={})
|
||||
|
||||
# Patch aiohttp so the import succeeds, but it should never be used.
|
||||
mock_session = MagicMock()
|
||||
mock_session.__aenter__ = AsyncMock(return_value=mock_session)
|
||||
mock_session.__aexit__ = AsyncMock(return_value=False)
|
||||
monkeypatch.setattr(
|
||||
"plugins.platforms.slack.adapter.aiohttp",
|
||||
MagicMock(ClientSession=MagicMock(return_value=mock_session)),
|
||||
)
|
||||
|
||||
result = await _standalone_send(pconfig, "C123", "")
|
||||
assert result.get("success") is True
|
||||
assert result.get("skipped") == "empty_text"
|
||||
mock_session.post.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_standalone_send_skips_whitespace(self, monkeypatch):
|
||||
"""_standalone_send returns success without HTTP call on whitespace."""
|
||||
from plugins.platforms.slack.adapter import _standalone_send
|
||||
from types import SimpleNamespace
|
||||
|
||||
pconfig = SimpleNamespace(token="xoxb-test", extra={})
|
||||
|
||||
mock_session = MagicMock()
|
||||
mock_session.__aenter__ = AsyncMock(return_value=mock_session)
|
||||
mock_session.__aexit__ = AsyncMock(return_value=False)
|
||||
monkeypatch.setattr(
|
||||
"plugins.platforms.slack.adapter.aiohttp",
|
||||
MagicMock(ClientSession=MagicMock(return_value=mock_session)),
|
||||
)
|
||||
|
||||
result = await _standalone_send(pconfig, "C123", " \n ")
|
||||
assert result.get("success") is True
|
||||
assert result.get("skipped") == "empty_text"
|
||||
mock_session.post.assert_not_called()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TestReplyBroadcast
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -4995,3 +5081,114 @@ class TestThreadContextUnverifiedTagging:
|
||||
# Renders successfully without trust tag (exception → unknown trust).
|
||||
assert "U_X: hello" in content
|
||||
assert "[unverified]" not in content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TestThreadContextAppMessages
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestThreadContextAppMessages:
|
||||
"""App-posted messages (Alertmanager, Grafana, CI bots) frequently carry
|
||||
their content in ``attachments``/``blocks`` with an empty top-level
|
||||
``text``. Thread-context must fall back to those so, e.g., an alert that
|
||||
started the thread the bot was asked to investigate is not dropped."""
|
||||
|
||||
@staticmethod
|
||||
def _make_replies(messages):
|
||||
return AsyncMock(return_value={"messages": messages})
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_attachment_only_parent_is_included(self, adapter):
|
||||
"""Alertmanager-style parent: empty text, content in a legacy attachment."""
|
||||
adapter._thread_context_cache.clear()
|
||||
messages = [
|
||||
{ # parent posted by the Alertmanager app: text="" , content in attachment
|
||||
"ts": "100.0",
|
||||
"bot_id": "B_ALERTMGR",
|
||||
"subtype": "bot_message",
|
||||
"username": "Alertmanager",
|
||||
"text": "",
|
||||
"attachments": [
|
||||
{
|
||||
"fallback": "[FIRING:1] KubeJobFailed cluster-01 "
|
||||
"batch-job-123456",
|
||||
"color": "danger",
|
||||
}
|
||||
],
|
||||
},
|
||||
{"ts": "101.0", "user": "U_BOB", "text": "<@U_BOT> investigate"},
|
||||
]
|
||||
adapter._app.client.conversations_replies = self._make_replies(messages)
|
||||
|
||||
with patch.object(
|
||||
adapter, "_resolve_user_name",
|
||||
new=AsyncMock(side_effect=lambda uid, **_: uid),
|
||||
):
|
||||
content = await adapter._fetch_thread_context(
|
||||
channel_id="C1", thread_ts="100.0", current_ts="999.0",
|
||||
)
|
||||
|
||||
# The alert text (previously dropped) is now present in the context.
|
||||
assert "KubeJobFailed" in content
|
||||
assert "batch-job-123456" in content
|
||||
assert "[thread parent]" in content
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_blocks_only_message_is_included(self, adapter):
|
||||
"""Block Kit message with empty text falls back to block text."""
|
||||
adapter._thread_context_cache.clear()
|
||||
messages = [
|
||||
{"ts": "100.0", "user": "U_BOB", "text": "kickoff"},
|
||||
{
|
||||
"ts": "101.0",
|
||||
"bot_id": "B_CI",
|
||||
"subtype": "bot_message",
|
||||
"username": "CI",
|
||||
"text": "",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "rich_text",
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_section",
|
||||
"elements": [
|
||||
{"type": "text", "text": "deploy #42 succeeded"}
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
},
|
||||
]
|
||||
adapter._app.client.conversations_replies = self._make_replies(messages)
|
||||
|
||||
with patch.object(
|
||||
adapter, "_resolve_user_name",
|
||||
new=AsyncMock(side_effect=lambda uid, **_: uid),
|
||||
):
|
||||
content = await adapter._fetch_thread_context(
|
||||
channel_id="C1", thread_ts="100.0", current_ts="999.0",
|
||||
)
|
||||
|
||||
assert "deploy #42 succeeded" in content
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_message_without_any_text_is_skipped(self, adapter):
|
||||
"""A message with no text/blocks/attachments is still skipped (no crash)."""
|
||||
adapter._thread_context_cache.clear()
|
||||
messages = [
|
||||
{"ts": "100.0", "user": "U_BOB", "text": "hello"},
|
||||
{"ts": "101.0", "bot_id": "B_X", "subtype": "bot_message", "text": ""},
|
||||
]
|
||||
adapter._app.client.conversations_replies = self._make_replies(messages)
|
||||
|
||||
with patch.object(
|
||||
adapter, "_resolve_user_name",
|
||||
new=AsyncMock(side_effect=lambda uid, **_: uid),
|
||||
):
|
||||
content = await adapter._fetch_thread_context(
|
||||
channel_id="C1", thread_ts="100.0", current_ts="999.0",
|
||||
)
|
||||
|
||||
assert "hello" in content # the real message survives; empty bot msg dropped
|
||||
|
||||
@@ -275,6 +275,37 @@ class TestSlackApprovalAction:
|
||||
update_kwargs = mock_client.chat_update.call_args[1]
|
||||
assert "Denied by alice" in update_kwargs["text"]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_truncates_inflated_original_text(self):
|
||||
"""Interaction payload re-escapes HTML entities; text must be capped."""
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
adapter._approval_resolved["1.2"] = False
|
||||
|
||||
# Simulate Slack re-escaping: original was ~2990 chars, but & → &
|
||||
# etc. inflates it past 3000.
|
||||
inflated_text = "a" * 2990 + "&" * 10 # 2990 + 50 = 3040 chars
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "1.2", "blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": inflated_text}},
|
||||
]},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "alice", "id": "U_ALICE"},
|
||||
}
|
||||
action = {"action_id": "hermes_approve_once", "value": "session-key"}
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
with patch("tools.approval.resolve_gateway_approval", return_value=1):
|
||||
await adapter._handle_approval_action(ack, body, action)
|
||||
|
||||
update_kwargs = mock_client.chat_update.call_args[1]
|
||||
section_text = update_kwargs["blocks"][0]["text"]["text"]
|
||||
assert len(section_text) <= 3000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_global_allowlist_blocks_unauthorized_click(self, monkeypatch):
|
||||
adapter = _make_adapter()
|
||||
@@ -410,6 +441,40 @@ class TestSlackSlashConfirmAction:
|
||||
secondary_client.chat_postMessage.assert_awaited_once()
|
||||
adapter._team_clients["T1"].chat_update.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_truncates_inflated_original_text(self):
|
||||
"""Interaction payload re-escapes HTML entities; text must be capped."""
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
adapter._approval_resolved["2222.3333"] = False
|
||||
|
||||
# Simulate Slack re-escaping inflating text past 3000 chars.
|
||||
inflated_text = "b" * 2990 + "<" * 10 # 2990 + 40 = 3030 chars
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "2222.3333", "blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": inflated_text}},
|
||||
]},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "owner", "id": "U_OWNER"},
|
||||
}
|
||||
action = {
|
||||
"action_id": "hermes_confirm_once",
|
||||
"value": "agent:main:slack:group:C1:1111|confirm-1",
|
||||
}
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
mock_client.chat_postMessage = AsyncMock()
|
||||
|
||||
with patch("tools.slash_confirm.resolve", new=AsyncMock(return_value="ok")):
|
||||
await adapter._handle_slash_confirm_action(ack, body, action)
|
||||
|
||||
update_kwargs = mock_client.chat_update.call_args[1]
|
||||
section_text = update_kwargs["blocks"][0]["text"]["text"]
|
||||
assert len(section_text) <= 3000
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# _fetch_thread_context
|
||||
@@ -547,7 +612,145 @@ class TestSlackThreadContext:
|
||||
assert "メール要約: 本日の新着3件" in context
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_thread_context_excludes_self_bot_replies(self):
|
||||
async def test_fetch_thread_context_extracts_block_kit_parent(self):
|
||||
"""Bot-posted parents that put their content in ``blocks`` (Honeycomb,
|
||||
PagerDuty, Datadog, GitHub bot, etc.) used to be reduced to just the
|
||||
``text`` field — typically only the alert title — which dropped the
|
||||
URL/button payload that makes the alert useful to an agent replying
|
||||
in the thread. The fetched context must now include bounded display
|
||||
text and actionable URLs so section text and button URLs survive."""
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.conversations_replies = AsyncMock(return_value={
|
||||
"messages": [
|
||||
# Bot-posted alert: title in `text`, URL only in `blocks`.
|
||||
# Mirrors what Honeycomb, PagerDuty, etc. actually send.
|
||||
{
|
||||
"ts": "1000.0",
|
||||
"bot_id": "B_ALERT",
|
||||
"subtype": "bot_message",
|
||||
"username": "alertbot",
|
||||
"text": "low_alerts (checkout)",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "*Trigger fired:* low_alerts",
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "actions",
|
||||
"elements": [
|
||||
{
|
||||
"type": "button",
|
||||
"text": {"type": "plain_text", "text": "View graph"},
|
||||
"url": "https://example.example/view/abc123",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
# User reply that triggered the fetch.
|
||||
{"ts": "1000.1", "user": "U1", "text": "what's going on?"},
|
||||
]
|
||||
})
|
||||
adapter._user_name_cache = {"U1": "Alice"}
|
||||
|
||||
context = await adapter._fetch_thread_context(
|
||||
channel_id="C1",
|
||||
thread_ts="1000.0",
|
||||
current_ts="1000.1",
|
||||
team_id="T1",
|
||||
)
|
||||
|
||||
# Title still present.
|
||||
assert "low_alerts (checkout)" in context
|
||||
# URL from the action button must now surface.
|
||||
assert "https://example.example/view/abc123" in context
|
||||
# Marked as the thread parent.
|
||||
assert "[thread parent]" in context
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_thread_context_includes_blocks_only_parent(self):
|
||||
"""A parent message with empty ``text`` but non-empty ``blocks`` must
|
||||
still be included — without this, alerts that put *everything* in
|
||||
``blocks`` (some webhook integrations do this) are silently dropped
|
||||
because the ``if not msg_text: continue`` guard fires."""
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.conversations_replies = AsyncMock(return_value={
|
||||
"messages": [
|
||||
{
|
||||
"ts": "1000.0",
|
||||
"bot_id": "B_ALERT",
|
||||
"subtype": "bot_message",
|
||||
"username": "alertbot",
|
||||
"text": "",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "Build failed: <https://example.example/build/9|#9>",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{"ts": "1000.1", "user": "U1", "text": "looking"},
|
||||
]
|
||||
})
|
||||
adapter._user_name_cache = {"U1": "Alice"}
|
||||
|
||||
context = await adapter._fetch_thread_context(
|
||||
channel_id="C1",
|
||||
thread_ts="1000.0",
|
||||
current_ts="1000.1",
|
||||
team_id="T1",
|
||||
)
|
||||
|
||||
assert "[thread parent]" in context
|
||||
assert "https://example.example/build/9" in context
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_thread_parent_text_surfaces_block_urls(self):
|
||||
"""Cold-cache _fetch_thread_parent_text must use the same renderer as
|
||||
_fetch_thread_context so a bot-posted parent with a URL only in
|
||||
``blocks`` surfaces it in reply_to_text, not just in thread context."""
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.conversations_replies = AsyncMock(return_value={
|
||||
"messages": [
|
||||
{
|
||||
"ts": "1000.0",
|
||||
"bot_id": "B_ALERT",
|
||||
"subtype": "bot_message",
|
||||
"username": "alertbot",
|
||||
"text": "Incident triggered",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "actions",
|
||||
"elements": [
|
||||
{
|
||||
"type": "button",
|
||||
"text": {"type": "plain_text", "text": "View incident"},
|
||||
"url": "https://example.example/incident/42",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
})
|
||||
|
||||
text = await adapter._fetch_thread_parent_text(
|
||||
channel_id="C1",
|
||||
thread_ts="1000.0",
|
||||
team_id="T1",
|
||||
)
|
||||
|
||||
assert "Incident triggered" in text
|
||||
assert "https://example.example/incident/42" in text
|
||||
"""Parent (non-self bot) is kept, self-bot child replies are dropped,
|
||||
user replies are kept."""
|
||||
adapter = _make_adapter()
|
||||
|
||||
@@ -5,6 +5,7 @@ from plugins.platforms.slack.block_kit import (
|
||||
MAX_HEADER_TEXT,
|
||||
MAX_SECTION_TEXT,
|
||||
render_blocks,
|
||||
sanitize_blocks,
|
||||
)
|
||||
|
||||
|
||||
@@ -159,11 +160,34 @@ class TestTables:
|
||||
)
|
||||
blocks = render_blocks(md)
|
||||
cs = blocks[0]["column_settings"]
|
||||
# left is default -> null; center/right emitted
|
||||
assert cs[0] is None
|
||||
# Every provided entry must be a valid Slack column-settings object.
|
||||
# Left placeholders are explicit only when needed to preserve position.
|
||||
assert cs[0] == {"align": "left"}
|
||||
assert cs[1] == {"align": "center"}
|
||||
assert cs[2] == {"align": "right"}
|
||||
|
||||
def test_default_trailing_column_settings_are_omitted(self):
|
||||
md = (
|
||||
"| L | R | L2 |\n"
|
||||
"|---|---:|---|\n"
|
||||
"| 1 | 2 | 3 |"
|
||||
)
|
||||
blocks = render_blocks(md)
|
||||
assert blocks is not None
|
||||
cs = blocks[0]["column_settings"]
|
||||
assert cs == [{"align": "left"}, {"align": "right"}]
|
||||
assert all(isinstance(item, dict) for item in cs)
|
||||
|
||||
def test_all_default_table_omits_column_settings(self):
|
||||
md = (
|
||||
"| A | B |\n"
|
||||
"|---|---|\n"
|
||||
"| 1 | 2 |"
|
||||
)
|
||||
blocks = render_blocks(md)
|
||||
assert blocks is not None
|
||||
assert "column_settings" not in blocks[0]
|
||||
|
||||
def test_inline_formatting_inside_cells(self):
|
||||
md = (
|
||||
"| Item | Link |\n"
|
||||
@@ -228,3 +252,180 @@ class TestLimits:
|
||||
for junk in ["```unterminated\ncode", "| broken | table", "> ", "#" * 10]:
|
||||
# must not raise; either blocks or None
|
||||
render_blocks(junk)
|
||||
|
||||
|
||||
class TestEmptyContentGuards:
|
||||
"""Empty content must never produce a Slack-rejected (invalid_blocks) payload.
|
||||
|
||||
Slack rejects a rich_text_section / rich_text_preformatted /
|
||||
rich_text_quote whose ``elements`` is empty or contains a zero-length
|
||||
``text`` element, and a ``header`` whose plain_text is empty. Each guard
|
||||
below corresponds to a real chat.postMessage rejection observed in
|
||||
production ("missing element" / "must be more than 0 characters").
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _assert_schema_valid(blocks):
|
||||
def walk(o):
|
||||
if isinstance(o, dict):
|
||||
if o.get("type") in (
|
||||
"rich_text_section", "rich_text_preformatted", "rich_text_quote"
|
||||
):
|
||||
assert o.get("elements"), f"empty {o['type']} elements"
|
||||
if o.get("type") == "text":
|
||||
assert len(o.get("text", "")) > 0, "zero-length text element"
|
||||
if o.get("type") == "header":
|
||||
assert o["text"]["text"], "empty plain_text header"
|
||||
for v in o.values():
|
||||
walk(v)
|
||||
elif isinstance(o, list):
|
||||
for v in o:
|
||||
walk(v)
|
||||
|
||||
walk(blocks)
|
||||
|
||||
def test_ragged_and_empty_table_cells_are_schema_valid(self):
|
||||
# Blank middle cell + ragged short row (padded with "") must not emit
|
||||
# an empty section or a 0-char text element.
|
||||
md = (
|
||||
"| x | y | z |\n"
|
||||
"| --- | --- | --- |\n"
|
||||
"| 1 | | 3 |\n" # blank middle cell
|
||||
"| 4 |" # ragged row -> padded with empty cells
|
||||
)
|
||||
blocks = render_blocks(md)
|
||||
assert blocks[0]["type"] == "table"
|
||||
self._assert_schema_valid(blocks)
|
||||
|
||||
def test_empty_code_fence_quote_and_list_item_are_schema_valid(self):
|
||||
# Empty fenced code block (common around empty tool output), blank
|
||||
# quote line, and empty list item must all stay schema-valid.
|
||||
md = "```\n```\n\n> \n\n- \n- real item"
|
||||
blocks = render_blocks(md)
|
||||
assert blocks is not None
|
||||
self._assert_schema_valid(blocks)
|
||||
|
||||
def test_multiline_quote_preserves_newline_separators(self):
|
||||
# _quote_block separates lines with length-1 "\n" text elements; the
|
||||
# guard must KEEP them so a multi-line blockquote stays multi-line.
|
||||
blocks = render_blocks("> alpha\n> bravo")
|
||||
quote = None
|
||||
for b in blocks:
|
||||
for el in b.get("elements", []):
|
||||
if isinstance(el, dict) and el.get("type") == "rich_text_quote":
|
||||
quote = el
|
||||
assert quote is not None, "no rich_text_quote produced"
|
||||
texts = [e.get("text") for e in quote["elements"] if e.get("type") == "text"]
|
||||
assert "\n" in texts, "newline separator dropped from multi-line quote"
|
||||
assert any("alpha" in (t or "") for t in texts)
|
||||
assert any("bravo" in (t or "") for t in texts)
|
||||
|
||||
def test_emphasis_only_header_is_dropped_not_empty(self):
|
||||
# "# ***" reduces to "" after marker-strip; an empty plain_text header
|
||||
# is rejected by Slack, so the header is skipped entirely.
|
||||
blocks = render_blocks("# ***\n\nreal body")
|
||||
assert not any(b.get("type") == "header" for b in blocks)
|
||||
self._assert_schema_valid(blocks)
|
||||
|
||||
def test_normal_content_unaffected(self):
|
||||
# Guard must not alter well-formed content.
|
||||
md = "# Title\n\n| a | b |\n| --- | --- |\n| 1 | 2 |\n\n> quoted\n\n- item"
|
||||
blocks = render_blocks(md)
|
||||
assert any(b.get("type") == "header" for b in blocks)
|
||||
assert any(b.get("type") == "table" for b in blocks)
|
||||
self._assert_schema_valid(blocks)
|
||||
|
||||
|
||||
class TestSanitizeBlocks:
|
||||
"""Outbound boundary clamp: one bad block must never fail the whole call.
|
||||
|
||||
Regression coverage for the invalid_blocks / msg_too_long bug class
|
||||
(#56615 null column_settings, #62054 / #53693 >3000-char sections on
|
||||
approval chat.update after HTML-escaping inflation).
|
||||
"""
|
||||
|
||||
def test_none_and_empty_return_none(self):
|
||||
assert sanitize_blocks(None) is None
|
||||
assert sanitize_blocks([]) is None
|
||||
|
||||
def test_valid_payload_passes_through(self):
|
||||
blocks = render_blocks("# Title\n\nbody text\n\n- item")
|
||||
assert sanitize_blocks(blocks) == blocks
|
||||
|
||||
def test_oversized_section_text_is_clamped(self):
|
||||
blocks = [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "x" * 3500}},
|
||||
]
|
||||
out = sanitize_blocks(blocks)
|
||||
assert len(out[0]["text"]["text"]) <= MAX_SECTION_TEXT
|
||||
assert out[0]["text"]["text"].endswith("…")
|
||||
|
||||
def test_html_escape_inflated_approval_update_is_clamped(self):
|
||||
# #53693 / #62054: send path budgeted the RAW text to <=3000, but the
|
||||
# interaction payload echoes it back HTML-escaped (& -> &) so the
|
||||
# chat.update section exceeds the cap.
|
||||
inflated = "a" * 2990 + "&" * 10 # 3040 chars
|
||||
blocks = [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": inflated}},
|
||||
{"type": "context", "elements": [{"type": "mrkdwn", "text": "✅ ok"}]},
|
||||
]
|
||||
out = sanitize_blocks(blocks)
|
||||
assert len(out[0]["text"]["text"]) <= MAX_SECTION_TEXT
|
||||
# context block untouched
|
||||
assert out[1] == blocks[1]
|
||||
|
||||
def test_null_column_settings_entries_are_fixed(self):
|
||||
# #56615: Slack rejects null entries in table column_settings.
|
||||
table = {
|
||||
"type": "table",
|
||||
"rows": [[{"type": "rich_text", "elements": []}]],
|
||||
"column_settings": [None, {"align": "center"}, None],
|
||||
}
|
||||
out = sanitize_blocks([table])
|
||||
cs = out[0]["column_settings"]
|
||||
assert cs == [{}, {"align": "center"}]
|
||||
assert all(isinstance(c, dict) for c in cs)
|
||||
|
||||
def test_all_null_column_settings_are_dropped(self):
|
||||
table = {
|
||||
"type": "table",
|
||||
"rows": [[{"type": "rich_text", "elements": []}]],
|
||||
"column_settings": [None, None],
|
||||
}
|
||||
out = sanitize_blocks([table])
|
||||
assert "column_settings" not in out[0]
|
||||
|
||||
def test_empty_blocks_are_dropped(self):
|
||||
blocks = [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": " "}},
|
||||
{"type": "rich_text", "elements": []},
|
||||
{"type": "actions", "elements": []},
|
||||
{"type": "context", "elements": []},
|
||||
{"type": "header", "text": {"type": "plain_text", "text": ""}},
|
||||
{"type": "table", "rows": []},
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "keep me"}},
|
||||
]
|
||||
out = sanitize_blocks(blocks)
|
||||
assert out == [blocks[-1]]
|
||||
|
||||
def test_all_invalid_returns_none_for_plain_text_fallback(self):
|
||||
blocks = [{"type": "section", "text": {"type": "mrkdwn", "text": ""}}]
|
||||
assert sanitize_blocks(blocks) is None
|
||||
|
||||
def test_oversized_header_is_clamped(self):
|
||||
blocks = [
|
||||
{"type": "header", "text": {"type": "plain_text", "text": "h" * 200}},
|
||||
]
|
||||
out = sanitize_blocks(blocks)
|
||||
assert len(out[0]["text"]["text"]) <= MAX_HEADER_TEXT
|
||||
|
||||
def test_payload_capped_at_50_blocks(self):
|
||||
blocks = [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": f"b{i}"}}
|
||||
for i in range(60)
|
||||
]
|
||||
out = sanitize_blocks(blocks)
|
||||
assert len(out) == MAX_BLOCKS
|
||||
|
||||
def test_never_raises_on_garbage(self):
|
||||
assert sanitize_blocks([{"no_type": True}, "not-a-dict", 42]) is None
|
||||
|
||||
@@ -7,7 +7,7 @@ Verifies the opt-in behaviour contract:
|
||||
* multi-chunk (>39k) messages fall back to plain text
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
from unittest.mock import AsyncMock, MagicMock, call
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -29,6 +29,17 @@ def _make_adapter(extra=None):
|
||||
|
||||
|
||||
RICH_MD = "# Title\n\n- a\n - nested\n\n---\n\nbody text"
|
||||
RICH_TABLE_MD = (
|
||||
"| Item | Status | Note |\n"
|
||||
"|---|---:|---|\n"
|
||||
"| Hermes | ok | table |"
|
||||
)
|
||||
|
||||
|
||||
class SlackRejectedBlocks(Exception):
|
||||
def __init__(self, error="invalid_blocks"):
|
||||
super().__init__(f"Slack API rejected blocks: {error}")
|
||||
self.response = {"error": error}
|
||||
|
||||
|
||||
class TestSendMessageBlocks:
|
||||
@@ -98,6 +109,29 @@ class TestSendMessageBlocks:
|
||||
|
||||
assert "blocks" not in client.chat_postMessage.await_args.kwargs
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_block_rejection_retries_send_without_blocks_using_workspace_client(self):
|
||||
adapter, client = _make_adapter({"rich_blocks": True})
|
||||
client.chat_postMessage = AsyncMock(
|
||||
side_effect=[SlackRejectedBlocks("invalid_blocks"), {"ts": "111.333"}]
|
||||
)
|
||||
|
||||
result = await adapter.send(
|
||||
"C1", RICH_TABLE_MD, metadata={"team_id": "T_SECONDARY"}
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert adapter._get_client.call_args_list == [
|
||||
call("C1", team_id="T_SECONDARY"),
|
||||
call("C1", team_id="T_SECONDARY"),
|
||||
]
|
||||
assert client.chat_postMessage.await_count == 2
|
||||
first = client.chat_postMessage.await_args_list[0].kwargs
|
||||
second = client.chat_postMessage.await_args_list[1].kwargs
|
||||
assert "blocks" in first and first["blocks"]
|
||||
assert "blocks" not in second
|
||||
assert second["text"]
|
||||
|
||||
|
||||
class TestEditMessageBlocks:
|
||||
@pytest.mark.asyncio
|
||||
@@ -128,3 +162,30 @@ class TestEditMessageBlocks:
|
||||
adapter, client = _make_adapter() # rich_blocks off
|
||||
await adapter.edit_message("C1", "111.222", RICH_MD, finalize=True)
|
||||
assert "blocks" not in client.chat_update.await_args.kwargs
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_block_rejection_retries_edit_without_blocks_using_workspace_client(self):
|
||||
adapter, client = _make_adapter({"rich_blocks": True})
|
||||
client.chat_update = AsyncMock(
|
||||
side_effect=[SlackRejectedBlocks("invalid_blocks"), {"ts": "111.222"}]
|
||||
)
|
||||
|
||||
result = await adapter.edit_message(
|
||||
"C1",
|
||||
"111.222",
|
||||
RICH_TABLE_MD,
|
||||
finalize=True,
|
||||
metadata={"team_id": "T_SECONDARY"},
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert adapter._get_client.call_args_list == [
|
||||
call("C1", team_id="T_SECONDARY"),
|
||||
call("C1", team_id="T_SECONDARY"),
|
||||
]
|
||||
assert client.chat_update.await_count == 2
|
||||
first = client.chat_update.await_args_list[0].kwargs
|
||||
second = client.chat_update.await_args_list[1].kwargs
|
||||
assert "blocks" in first and first["blocks"]
|
||||
assert second["blocks"] == []
|
||||
assert second["text"]
|
||||
|
||||
@@ -0,0 +1,475 @@
|
||||
"""Tests for Slack Block Kit interactive clarify buttons.
|
||||
|
||||
Mirrors test_slack_approval_buttons.py (harness) and
|
||||
test_telegram_clarify_buttons.py (semantics) for the ``send_clarify`` override
|
||||
and the indexed ``hermes_clarify_choice_<idx>`` /
|
||||
``hermes_clarify_other`` action dispatch.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ensure the repo root is importable
|
||||
# ---------------------------------------------------------------------------
|
||||
_repo = str(Path(__file__).resolve().parents[2])
|
||||
if _repo not in sys.path:
|
||||
sys.path.insert(0, _repo)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Minimal Slack SDK mock so SlackAdapter can be imported (mirrors
|
||||
# test_slack_approval_buttons.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
def _ensure_slack_mock():
|
||||
if "slack_bolt" in sys.modules:
|
||||
return
|
||||
slack_bolt = MagicMock()
|
||||
slack_bolt.async_app.AsyncApp = MagicMock
|
||||
sys.modules["slack_bolt"] = slack_bolt
|
||||
sys.modules["slack_bolt.async_app"] = slack_bolt.async_app
|
||||
handler_mod = MagicMock()
|
||||
handler_mod.AsyncSocketModeHandler = MagicMock
|
||||
sys.modules["slack_bolt.adapter"] = MagicMock()
|
||||
sys.modules["slack_bolt.adapter.socket_mode"] = MagicMock()
|
||||
sys.modules["slack_bolt.adapter.socket_mode.async_handler"] = handler_mod
|
||||
sdk_mod = MagicMock()
|
||||
sdk_mod.web = MagicMock()
|
||||
sdk_mod.web.async_client = MagicMock()
|
||||
sdk_mod.web.async_client.AsyncWebClient = MagicMock
|
||||
sys.modules["slack_sdk"] = sdk_mod
|
||||
sys.modules["slack_sdk.web"] = sdk_mod.web
|
||||
sys.modules["slack_sdk.web.async_client"] = sdk_mod.web.async_client
|
||||
|
||||
|
||||
_ensure_slack_mock()
|
||||
|
||||
from plugins.platforms.slack.adapter import SlackAdapter
|
||||
from gateway.config import PlatformConfig
|
||||
|
||||
|
||||
def _make_adapter():
|
||||
config = PlatformConfig(enabled=True, token="xoxb-test-token")
|
||||
adapter = SlackAdapter(config)
|
||||
adapter._app = MagicMock()
|
||||
adapter._bot_user_id = "U_BOT"
|
||||
adapter._team_clients = {"T1": AsyncMock()}
|
||||
adapter._team_bot_user_ids = {"T1": "U_BOT"}
|
||||
adapter._channel_team = {"C1": "T1"}
|
||||
return adapter
|
||||
|
||||
|
||||
class _AuthRunner:
|
||||
def __init__(self, auth_fn=None):
|
||||
self._auth_fn = auth_fn or (lambda _source: True)
|
||||
|
||||
async def handle(self, event):
|
||||
return None
|
||||
|
||||
def _is_user_authorized(self, source):
|
||||
return self._auth_fn(source)
|
||||
|
||||
|
||||
def _attach_auth_runner(adapter, auth_fn=None):
|
||||
adapter.set_message_handler(_AuthRunner(auth_fn=auth_fn).handle)
|
||||
|
||||
|
||||
def _clear_clarify_state():
|
||||
from tools import clarify_gateway as cm
|
||||
with cm._lock:
|
||||
cm._entries.clear()
|
||||
cm._session_index.clear()
|
||||
cm._notify_cbs.clear()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# send_clarify — Block Kit render (a)
|
||||
# ===========================================================================
|
||||
|
||||
class TestSlackSendClarify:
|
||||
def setup_method(self):
|
||||
_clear_clarify_state()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multi_choice_renders_buttons_and_other(self):
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1234.5678"})
|
||||
|
||||
result = await adapter.send_clarify(
|
||||
chat_id="C1",
|
||||
question="Which environment?",
|
||||
choices=["staging", "production"],
|
||||
clarify_id="cid1",
|
||||
session_key="sk1",
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert result.message_id == "1234.5678"
|
||||
# ts recorded for the double-click guard
|
||||
assert adapter._clarify_resolved.get("1234.5678") is False
|
||||
|
||||
kwargs = mock_client.chat_postMessage.call_args[1]
|
||||
blocks = kwargs["blocks"]
|
||||
assert blocks[0]["type"] == "section"
|
||||
assert "Which environment?" in blocks[0]["text"]["text"]
|
||||
assert blocks[1]["type"] == "actions"
|
||||
elements = blocks[1]["elements"]
|
||||
# 2 choices + Other
|
||||
assert len(elements) == 3
|
||||
assert elements[0]["action_id"] == "hermes_clarify_choice_0"
|
||||
assert elements[0]["value"] == "cid1|0"
|
||||
assert elements[1]["action_id"] == "hermes_clarify_choice_1"
|
||||
assert elements[1]["value"] == "cid1|1"
|
||||
assert elements[0]["text"]["text"] == "staging"
|
||||
# Final button is the free-text "Other"
|
||||
assert elements[2]["action_id"] == "hermes_clarify_other"
|
||||
assert elements[2]["value"] == "cid1|other"
|
||||
for block in blocks:
|
||||
if block["type"] == "actions":
|
||||
action_ids = [element["action_id"] for element in block["elements"]]
|
||||
assert len(action_ids) == len(set(action_ids))
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_open_ended_no_buttons(self):
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "9.9"})
|
||||
|
||||
result = await adapter.send_clarify(
|
||||
chat_id="C1",
|
||||
question="What should I name the branch?",
|
||||
choices=None,
|
||||
clarify_id="cid-open",
|
||||
session_key="sk-open",
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
kwargs = mock_client.chat_postMessage.call_args[1]
|
||||
# Open-ended delegates to the base plain-text path — no action blocks.
|
||||
assert "blocks" not in kwargs or all(
|
||||
b.get("type") != "actions" for b in (kwargs.get("blocks") or [])
|
||||
)
|
||||
assert "What should I name the branch?" in kwargs["text"]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mrkdwn_escapes_question(self):
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.1"})
|
||||
|
||||
await adapter.send_clarify(
|
||||
chat_id="C1",
|
||||
question="Use <A> & <B>?",
|
||||
choices=["yes"],
|
||||
clarify_id="cid2",
|
||||
session_key="sk2",
|
||||
)
|
||||
section_text = mock_client.chat_postMessage.call_args[1]["blocks"][0]["text"]["text"]
|
||||
assert "<A>" not in section_text
|
||||
assert "<A>" in section_text
|
||||
assert "&" in section_text
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sends_in_thread(self):
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.2"})
|
||||
|
||||
await adapter.send_clarify(
|
||||
chat_id="C1",
|
||||
question="?",
|
||||
choices=["a"],
|
||||
clarify_id="cid3",
|
||||
session_key="sk3",
|
||||
metadata={"thread_id": "8888.0000"},
|
||||
)
|
||||
assert mock_client.chat_postMessage.call_args[1].get("thread_ts") == "8888.0000"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_not_connected(self):
|
||||
adapter = _make_adapter()
|
||||
adapter._app = None
|
||||
result = await adapter.send_clarify(
|
||||
chat_id="C1", question="?", choices=["a"], clarify_id="c", session_key="s"
|
||||
)
|
||||
assert result.success is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_five_choices_chunk_across_actions_blocks(self):
|
||||
"""Slack caps 5 elements per actions block; 5 choices + Other = 6
|
||||
buttons must spill into a second block instead of 400ing."""
|
||||
adapter = _make_adapter()
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_postMessage = AsyncMock(return_value={"ts": "1.3"})
|
||||
|
||||
await adapter.send_clarify(
|
||||
chat_id="C1",
|
||||
question="?",
|
||||
choices=["a", "b", "c", "d", "e"],
|
||||
clarify_id="cid5",
|
||||
session_key="sk5",
|
||||
)
|
||||
blocks = mock_client.chat_postMessage.call_args[1]["blocks"]
|
||||
action_blocks = [b for b in blocks if b["type"] == "actions"]
|
||||
assert len(action_blocks) == 2
|
||||
for b in action_blocks:
|
||||
assert len(b["elements"]) <= 5
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# _handle_clarify_action — choice click resolves (b)
|
||||
# ===========================================================================
|
||||
|
||||
class TestSlackClarifyChoiceAction:
|
||||
def setup_method(self):
|
||||
_clear_clarify_state()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_choice_resolves_with_choice_text(self):
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
cm.register("cidA", "sk-cb", "Pick", ["red", "green", "blue"])
|
||||
adapter._clarify_resolved["1234.5678"] = False
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {
|
||||
"ts": "1234.5678",
|
||||
"blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
|
||||
{"type": "actions", "elements": []},
|
||||
],
|
||||
},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "norbert", "id": "U_NORBERT"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_choice_1", "value": "cidA|1"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
|
||||
ack.assert_called_once()
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cidA")
|
||||
assert entry is not None
|
||||
assert entry.response == "green"
|
||||
assert entry.event.is_set()
|
||||
# Message updated with the answer, buttons dropped.
|
||||
update_kwargs = mock_client.chat_update.call_args[1]
|
||||
assert "green" in update_kwargs["text"]
|
||||
assert all(b["type"] != "actions" for b in update_kwargs["blocks"])
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_prevents_double_click(self):
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
cm.register("cidDup", "sk-dup", "Pick", ["x"])
|
||||
adapter._clarify_resolved["1.1"] = True # already resolved
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "1.1", "blocks": []},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "n", "id": "U1"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_choice", "value": "cidDup|0"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
|
||||
ack.assert_called_once()
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cidDup")
|
||||
assert entry is not None
|
||||
assert not entry.event.is_set()
|
||||
mock_client.chat_update.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unauthorized_click_ignored(self):
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter, auth_fn=lambda _s: False)
|
||||
cm.register("cidAuth", "sk-auth", "Pick", ["a", "b"])
|
||||
adapter._clarify_resolved["2.2"] = False
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "2.2", "blocks": []},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "mallory", "id": "U_BAD"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_choice", "value": "cidAuth|0"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cidAuth")
|
||||
assert entry is not None
|
||||
assert not entry.event.is_set()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_expired_choice_shows_notice(self):
|
||||
"""Late tap after the entry was evicted must surface expiry, not a ✓."""
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
# No entry registered → resolve returns False.
|
||||
adapter._clarify_resolved["3.3"] = False
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "3.3", "blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
|
||||
]},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "t", "id": "U_T"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_choice", "value": "cidGone|0"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
|
||||
assert "expired" in mock_client.chat_update.call_args[1]["text"].lower()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# _handle_clarify_action — "Other" → text-capture → typed reply (c)
|
||||
# ===========================================================================
|
||||
|
||||
class TestSlackClarifyOtherFlow:
|
||||
def setup_method(self):
|
||||
_clear_clarify_state()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_other_flips_to_text_mode_then_typed_reply_resolves(self):
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
cm.register("cidO", "sk-other", "Pick", ["x", "y"])
|
||||
adapter._clarify_resolved["4.4"] = False
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "4.4", "blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
|
||||
{"type": "actions", "elements": []},
|
||||
]},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "norbert", "id": "U_N"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_other", "value": "cidO|other"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
|
||||
# Entry flipped to text-capture; NOT yet resolved.
|
||||
pending = cm.get_pending_for_session("sk-other")
|
||||
assert pending is not None and pending.clarify_id == "cidO"
|
||||
assert pending.awaiting_text is True
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cidO")
|
||||
assert not entry.event.is_set()
|
||||
assert "awaiting" in mock_client.chat_update.call_args[1]["text"].lower()
|
||||
|
||||
# Now the gateway text-intercept (platform-agnostic) resolves from the
|
||||
# user's next typed message. We exercise that leveraged path directly.
|
||||
assert cm.resolve_text_response_for_session("sk-other", "my custom answer") is True
|
||||
with cm._lock:
|
||||
entry = cm._entries.get("cidO")
|
||||
assert entry.response == "my custom answer"
|
||||
assert entry.event.is_set()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_other_expired_shows_notice(self):
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
# No entry → mark_awaiting_text returns False.
|
||||
adapter._clarify_resolved["5.5"] = False
|
||||
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "5.5", "blocks": [
|
||||
{"type": "section", "text": {"type": "mrkdwn", "text": "❓ Pick"}},
|
||||
]},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "t", "id": "U_T"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_other", "value": "cidOtherGone|other"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
assert "expired" in mock_client.chat_update.call_args[1]["text"].lower()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_malformed_value_ignored(self):
|
||||
adapter = _make_adapter()
|
||||
_attach_auth_runner(adapter)
|
||||
adapter._clarify_resolved["6.6"] = False
|
||||
mock_client = adapter._team_clients["T1"]
|
||||
mock_client.chat_update = AsyncMock()
|
||||
|
||||
ack = AsyncMock()
|
||||
body = {
|
||||
"message": {"ts": "6.6", "blocks": []},
|
||||
"channel": {"id": "C1"},
|
||||
"user": {"name": "t", "id": "U_T"},
|
||||
}
|
||||
action = {"action_id": "hermes_clarify_choice", "value": "no-delimiter"}
|
||||
|
||||
await adapter._handle_clarify_action(ack, body, action)
|
||||
mock_client.chat_update.assert_not_called()
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# Base text-fallback unchanged for platforms without an override (e)
|
||||
# ===========================================================================
|
||||
|
||||
class TestBaseAdapterClarifyFallbackUnchanged:
|
||||
@pytest.mark.asyncio
|
||||
async def test_base_numbered_text_fallback(self):
|
||||
from gateway.platforms.base import BasePlatformAdapter, SendResult
|
||||
|
||||
class _Stub(BasePlatformAdapter):
|
||||
name = "stub"
|
||||
|
||||
def __init__(self):
|
||||
self.sent: list = []
|
||||
|
||||
async def connect(self, *, is_reconnect: bool = False): pass
|
||||
async def disconnect(self): pass
|
||||
async def send(self, chat_id, content, **kw):
|
||||
self.sent.append(content)
|
||||
return SendResult(success=True, message_id="1")
|
||||
async def edit(self, *a, **k): return SendResult(success=False)
|
||||
async def get_history(self, *a, **k): return []
|
||||
async def get_chat_info(self, *a, **k): return {}
|
||||
|
||||
adapter = _Stub()
|
||||
result = await adapter.send_clarify(
|
||||
chat_id="c", question="Pick a fruit",
|
||||
choices=["apple", "banana"], clarify_id="x", session_key="s",
|
||||
)
|
||||
assert result.success is True
|
||||
text = adapter.sent[0]
|
||||
assert "Pick a fruit" in text
|
||||
assert "1." in text and "apple" in text
|
||||
assert "2." in text and "banana" in text
|
||||
@@ -903,3 +903,92 @@ def test_mention_patterns_trigger_in_channel_without_literal_mention():
|
||||
assert _would_process(adapter, text="hey hermes what's the status") is True
|
||||
# Unrelated channel chatter is still ignored.
|
||||
assert _would_process(adapter, text="lunch anyone?") is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests: Block-Kit-only mention detection (#52387)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
from plugins.platforms.slack.adapter import _slack_mention_detection_text # noqa: E402
|
||||
|
||||
|
||||
def _blockkit_mention_event(bot_user_id=BOT_USER_ID, flat_text="Release notification"):
|
||||
"""A Slack event whose @mention lives ONLY inside Block Kit blocks."""
|
||||
return {
|
||||
"text": flat_text,
|
||||
"blocks": [
|
||||
{
|
||||
"type": "rich_text",
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_section",
|
||||
"elements": [
|
||||
{"type": "text", "text": "Hey "},
|
||||
{"type": "user", "user_id": bot_user_id},
|
||||
{"type": "text", "text": "! I will do a release"},
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def test_mention_detection_text_recovers_blockkit_mention():
|
||||
event = _blockkit_mention_event()
|
||||
merged = _slack_mention_detection_text(event)
|
||||
# The flat text alone never contains the mention...
|
||||
assert f"<@{BOT_USER_ID}>" not in event.get("text", "")
|
||||
# ...but the merged detection text does.
|
||||
assert f"<@{BOT_USER_ID}>" in merged
|
||||
|
||||
|
||||
def test_mention_detection_text_no_blocks_returns_flat_text():
|
||||
event = {"text": f"<@{BOT_USER_ID}> hello"}
|
||||
assert _slack_mention_detection_text(event) == f"<@{BOT_USER_ID}> hello"
|
||||
|
||||
|
||||
def test_mention_detection_text_no_mention_anywhere():
|
||||
event = {
|
||||
"text": "lunch anyone?",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "rich_text",
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_section",
|
||||
"elements": [{"type": "text", "text": "lunch anyone?"}],
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
assert f"<@{BOT_USER_ID}>" not in _slack_mention_detection_text(event)
|
||||
|
||||
|
||||
def test_mention_detection_text_ignores_quoted_blockkit_mention():
|
||||
"""A mention inside rich_text_quote (forwarded content) must NOT count."""
|
||||
event = {
|
||||
"text": "please review",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "rich_text",
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_quote",
|
||||
"elements": [
|
||||
{
|
||||
"type": "rich_text_section",
|
||||
"elements": [
|
||||
{"type": "text", "text": "Contains "},
|
||||
{"type": "user", "user_id": BOT_USER_ID},
|
||||
{"type": "text", "text": " in quoted text"},
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
assert f"<@{BOT_USER_ID}>" not in _slack_mention_detection_text(event)
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
"""Regression tests for #57682 — holographic auto_extract harvested
|
||||
context-compaction handoff summaries into fact_store, and ran even when
|
||||
configured off.
|
||||
|
||||
Two compounding defects:
|
||||
|
||||
1. Gate: the plugin's config schema declares ``auto_extract`` as a string enum
|
||||
(``"false"``/``"true"``), and the ``on_session_end`` gate used plain
|
||||
truthiness — ``not "false"`` is ``False`` — so extraction ran despite being
|
||||
explicitly configured off.
|
||||
|
||||
2. Eligibility: ``_auto_extract_facts`` scanned every ``role == "user"``
|
||||
message. Compaction handoff summaries can be inserted as ``role="user"``
|
||||
messages, and their prose reliably matches the decision patterns
|
||||
(``we decided/agreed/chose``, ``the project uses/needs/requires``), so the
|
||||
compactor's own output was stored as a durable ``project`` fact on every
|
||||
session rollover that followed a compaction.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from agent.context_compressor import (
|
||||
COMPRESSED_SUMMARY_METADATA_KEY,
|
||||
SUMMARY_PREFIX,
|
||||
_MERGED_PRIOR_CONTEXT_HEADER,
|
||||
_MERGED_SUMMARY_DELIMITER,
|
||||
is_compaction_summary_message,
|
||||
)
|
||||
from plugins.memory.holographic import HolographicMemoryProvider
|
||||
|
||||
|
||||
def _make_provider(tmp_path, **config):
|
||||
base = {"db_path": str(tmp_path / "memory_store.db"), "hrr_dim": 64}
|
||||
base.update(config)
|
||||
provider = HolographicMemoryProvider(config=base)
|
||||
provider.initialize(session_id="test-session")
|
||||
return provider
|
||||
|
||||
|
||||
def _fact_contents(provider):
|
||||
return [f["content"] for f in provider._store.list_facts(limit=100)]
|
||||
|
||||
|
||||
def _user(content, **extra):
|
||||
msg = {"role": "user", "content": content}
|
||||
msg.update(extra)
|
||||
return msg
|
||||
|
||||
|
||||
DECISION_MSG = "we decided to use PostgreSQL for the persistence layer"
|
||||
SUMMARY_MSG = (
|
||||
f"{SUMMARY_PREFIX}\n## Historical Task Snapshot\n"
|
||||
"The project uses a kanban board for all dispatch. "
|
||||
"We agreed to route reviews through the fan-in consumer."
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Defect 1 — string-boolean gate
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("off_value", [False, "false", "False", "no", "off", "0", None, ""])
|
||||
def test_auto_extract_off_values_disable_extraction(tmp_path, off_value):
|
||||
provider = _make_provider(tmp_path, auto_extract=off_value)
|
||||
provider.on_session_end([_user(DECISION_MSG)])
|
||||
assert _fact_contents(provider) == []
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("on_value", [True, "true", "True", "1", "yes", "on"])
|
||||
def test_auto_extract_on_values_enable_extraction(tmp_path, on_value):
|
||||
provider = _make_provider(tmp_path, auto_extract=on_value)
|
||||
provider.on_session_end([_user(DECISION_MSG)])
|
||||
facts = _fact_contents(provider)
|
||||
assert len(facts) == 1
|
||||
assert "PostgreSQL" in facts[0]
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Defect 2 — compaction summaries harvested as facts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_compaction_summary_not_harvested(tmp_path):
|
||||
"""The exact failure mode from #57682: summary prose matches the decision
|
||||
patterns but must not become a fact."""
|
||||
provider = _make_provider(tmp_path, auto_extract=True)
|
||||
provider.on_session_end([_user(SUMMARY_MSG)])
|
||||
assert _fact_contents(provider) == []
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_metadata_marked_summary_not_harvested(tmp_path):
|
||||
"""In-process summaries carry COMPRESSED_SUMMARY_METADATA_KEY even if a
|
||||
future prefix rewrite changes the content sentinel."""
|
||||
provider = _make_provider(tmp_path, auto_extract=True)
|
||||
marked = _user(DECISION_MSG, **{COMPRESSED_SUMMARY_METADATA_KEY: True})
|
||||
provider.on_session_end([marked])
|
||||
assert _fact_contents(provider) == []
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_merged_into_tail_summary_suffix_not_harvested_prefix_content_ignored(tmp_path):
|
||||
"""Merge-into-tail summaries embed the handoff prefix after the delimiter,
|
||||
not at the start of the message. The wrapped pre-delimiter segment here has
|
||||
no fact-pattern match, so nothing is harvested from either side."""
|
||||
provider = _make_provider(tmp_path, auto_extract=True)
|
||||
merged = _user(
|
||||
f"{_MERGED_PRIOR_CONTEXT_HEADER}\nplease fix the login bug\n"
|
||||
f"{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}"
|
||||
)
|
||||
provider.on_session_end([merged])
|
||||
assert _fact_contents(provider) == []
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_merged_into_tail_preserves_genuine_pre_delimiter_preference(tmp_path):
|
||||
"""#57690 review: teknium1 noted the ENTIRE merged row was being skipped,
|
||||
discarding genuine pre-delimiter user content (context_compressor.py
|
||||
~3163-3190 retains real prior tail text before the summary). The fix must
|
||||
extract and harvest that segment while still excluding the summary
|
||||
suffix."""
|
||||
provider = _make_provider(tmp_path, auto_extract=True)
|
||||
merged = _user(
|
||||
f"{_MERGED_PRIOR_CONTEXT_HEADER}\n"
|
||||
"I prefer tabs over spaces for indentation\n"
|
||||
f"{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}"
|
||||
)
|
||||
provider.on_session_end([merged])
|
||||
facts = _fact_contents(provider)
|
||||
assert len(facts) == 1
|
||||
assert "tabs over spaces" in facts[0]
|
||||
assert "kanban board" not in facts[0]
|
||||
assert "PostgreSQL" not in facts[0]
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
def test_real_user_messages_still_extracted_alongside_summary(tmp_path):
|
||||
"""The guard must skip only the summary, not suppress extraction for the
|
||||
genuine user turns around it."""
|
||||
provider = _make_provider(tmp_path, auto_extract=True)
|
||||
provider.on_session_end(
|
||||
[
|
||||
_user(SUMMARY_MSG),
|
||||
_user("I prefer tabs over spaces for indentation"),
|
||||
]
|
||||
)
|
||||
facts = _fact_contents(provider)
|
||||
assert len(facts) == 1
|
||||
assert "tabs over spaces" in facts[0]
|
||||
provider.shutdown()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# is_compaction_summary_message — public helper contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_helper_detects_prefix_metadata_and_merged_forms():
|
||||
assert is_compaction_summary_message(_user(SUMMARY_MSG))
|
||||
assert is_compaction_summary_message(
|
||||
_user("anything", **{COMPRESSED_SUMMARY_METADATA_KEY: True})
|
||||
)
|
||||
assert is_compaction_summary_message(
|
||||
_user(f"prior tail\n{_MERGED_SUMMARY_DELIMITER}\n{SUMMARY_MSG}")
|
||||
)
|
||||
assert not is_compaction_summary_message(_user(DECISION_MSG))
|
||||
assert not is_compaction_summary_message(_user(""))
|
||||
@@ -1,5 +1,8 @@
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from agent.codex_runtime import _record_codex_app_server_compaction
|
||||
from agent.conversation_compression import COMPACTION_STATUS, compress_context
|
||||
from agent.transports.codex_app_server_session import TurnResult
|
||||
@@ -19,6 +22,26 @@ class FakeCodexSession:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class SlowCodexSession(FakeCodexSession):
|
||||
def __init__(self, result, touch_calls):
|
||||
super().__init__(result)
|
||||
self.touch_calls = touch_calls
|
||||
|
||||
def compact_thread(self):
|
||||
self.calls += 1
|
||||
_wait_for_touch(self.touch_calls, "context compression in progress")
|
||||
return self.result
|
||||
|
||||
|
||||
def _wait_for_touch(touch_calls, desc, timeout=1.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if desc in touch_calls:
|
||||
return
|
||||
time.sleep(0.01)
|
||||
pytest.fail(f"timed out waiting for touch {desc!r}; saw {touch_calls!r}")
|
||||
|
||||
|
||||
class DummyAgent:
|
||||
def __init__(
|
||||
self,
|
||||
@@ -43,6 +66,11 @@ class DummyAgent:
|
||||
self.warnings = []
|
||||
self.events = []
|
||||
self.built_prompts = []
|
||||
self.touch_calls = []
|
||||
self._compression_activity_heartbeat_interval = 0.1
|
||||
|
||||
def _touch_activity(self, desc):
|
||||
self.touch_calls.append(desc)
|
||||
|
||||
def _emit_status(self, message):
|
||||
self.statuses.append(message)
|
||||
@@ -79,6 +107,33 @@ def test_codex_app_server_native_auto_mode_leaves_thread_compaction_to_codex():
|
||||
assert agent.events == []
|
||||
|
||||
|
||||
def test_codex_app_server_compaction_heartbeat_refreshes_activity_while_waiting():
|
||||
agent = DummyAgent(
|
||||
TurnResult(thread_id="thread-1", turn_id="compact-turn-1")
|
||||
)
|
||||
agent._codex_session = SlowCodexSession(
|
||||
agent._codex_session.result,
|
||||
agent.touch_calls,
|
||||
)
|
||||
messages = [{"role": "user", "content": "hi"}]
|
||||
|
||||
returned, prompt = compress_context(
|
||||
agent,
|
||||
messages,
|
||||
"system",
|
||||
approx_tokens=100000,
|
||||
task_id="test",
|
||||
force=True,
|
||||
)
|
||||
|
||||
assert returned is messages
|
||||
assert prompt == "cached prompt"
|
||||
assert agent._codex_session.calls == 1
|
||||
assert "context compression started" in agent.touch_calls
|
||||
assert "context compression in progress" in agent.touch_calls
|
||||
assert agent.touch_calls[-1] == "context compression completed"
|
||||
|
||||
|
||||
def test_codex_app_server_manual_compression_routes_to_codex_thread():
|
||||
agent = DummyAgent(
|
||||
TurnResult(thread_id="thread-1", turn_id="compact-turn-1")
|
||||
@@ -162,6 +217,8 @@ def test_codex_app_server_compression_failure_preserves_bookkeeping():
|
||||
assert agent.context_compressor.compression_count == 0
|
||||
assert agent.context_compressor.last_prompt_tokens == 123
|
||||
assert agent.warnings
|
||||
assert agent.touch_calls[0] == "context compression started"
|
||||
assert agent.touch_calls[-1] == "context compression failed"
|
||||
|
||||
|
||||
def test_codex_app_server_native_compaction_notice_emits_status_and_event():
|
||||
|
||||
@@ -16,7 +16,11 @@ import tempfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
class TestCompressionBoundaryHook:
|
||||
def _make_agent(self, session_db):
|
||||
@@ -92,6 +96,159 @@ class TestCompressionBoundaryHook:
|
||||
f"Expected new session_id as first positional arg, got {call!r}"
|
||||
assert call.kwargs.get("old_session_id") == original_sid, \
|
||||
f"Expected old_session_id={original_sid!r}, got {call.kwargs!r}"
|
||||
assert len(comp_calls) == 1
|
||||
|
||||
def test_automatic_notification_follows_core_persistence(self):
|
||||
from hermes_state import SessionDB
|
||||
|
||||
events = []
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db = SessionDB(db_path=Path(tmpdir) / "test.db")
|
||||
agent = self._make_agent(db)
|
||||
compressor = MagicMock()
|
||||
compressor.compress.return_value = [
|
||||
{"role": "user", "content": "summary"}
|
||||
]
|
||||
compressor.compression_count = 1
|
||||
compressor.last_prompt_tokens = 0
|
||||
compressor.last_completion_tokens = 0
|
||||
compressor._last_summary_error = None
|
||||
compressor._last_compress_aborted = False
|
||||
compressor.on_session_start.side_effect = (
|
||||
lambda *_args, **kwargs: events.append(
|
||||
kwargs.get("boundary_reason")
|
||||
)
|
||||
)
|
||||
agent.context_compressor = compressor
|
||||
original_update = db.update_system_prompt
|
||||
|
||||
def _record_update(*args, **kwargs):
|
||||
result = original_update(*args, **kwargs)
|
||||
events.append("persist")
|
||||
return result
|
||||
|
||||
with patch.object(db, "update_system_prompt", side_effect=_record_update):
|
||||
agent._compress_context(
|
||||
[{"role": "user", "content": "request"}],
|
||||
"sys",
|
||||
approx_tokens=100,
|
||||
)
|
||||
|
||||
assert events == ["persist", "compression"]
|
||||
|
||||
def test_failure_before_persistence_does_not_notify(self):
|
||||
from hermes_state import SessionDB
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db = SessionDB(db_path=Path(tmpdir) / "test.db")
|
||||
agent = self._make_agent(db)
|
||||
compressor = MagicMock()
|
||||
compressor.compress.side_effect = RuntimeError("synthetic compression failure")
|
||||
agent.context_compressor = compressor
|
||||
|
||||
with pytest.raises(RuntimeError, match="synthetic compression failure"):
|
||||
agent._compress_context(
|
||||
[{"role": "user", "content": "request"}],
|
||||
"sys",
|
||||
approx_tokens=100,
|
||||
)
|
||||
|
||||
compressor.on_session_start.assert_not_called()
|
||||
|
||||
def test_failure_during_persistence_does_not_notify(self):
|
||||
from hermes_state import SessionDB
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db = SessionDB(db_path=Path(tmpdir) / "test.db")
|
||||
agent = self._make_agent(db)
|
||||
compressor = MagicMock()
|
||||
compressor.compress.return_value = [
|
||||
{"role": "user", "content": "summary"}
|
||||
]
|
||||
compressor.compression_count = 1
|
||||
compressor.last_prompt_tokens = 0
|
||||
compressor.last_completion_tokens = 0
|
||||
compressor._last_summary_error = None
|
||||
compressor._last_compress_aborted = False
|
||||
agent.context_compressor = compressor
|
||||
|
||||
with patch.object(
|
||||
db,
|
||||
"update_system_prompt",
|
||||
side_effect=RuntimeError("synthetic commit failure"),
|
||||
):
|
||||
agent._compress_context(
|
||||
[{"role": "user", "content": "request"}],
|
||||
"sys",
|
||||
approx_tokens=100,
|
||||
)
|
||||
|
||||
boundary_calls = [
|
||||
call
|
||||
for call in compressor.on_session_start.call_args_list
|
||||
if call.kwargs.get("boundary_reason") == "compression"
|
||||
]
|
||||
assert boundary_calls == []
|
||||
|
||||
def test_no_progress_does_not_notify(self):
|
||||
from hermes_state import SessionDB
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db = SessionDB(db_path=Path(tmpdir) / "test.db")
|
||||
agent = self._make_agent(db)
|
||||
compressor = MagicMock()
|
||||
compressor.compress.side_effect = lambda messages, **_kwargs: messages
|
||||
compressor._last_compress_aborted = False
|
||||
agent.context_compressor = compressor
|
||||
messages = [{"role": "user", "content": "request"}]
|
||||
|
||||
returned, _ = agent._compress_context(
|
||||
messages,
|
||||
"sys",
|
||||
approx_tokens=100,
|
||||
)
|
||||
|
||||
assert returned is messages
|
||||
compressor.on_session_start.assert_not_called()
|
||||
|
||||
@pytest.mark.parametrize("committed", [True, False])
|
||||
def test_deferred_notification_finishes_exactly_once(self, committed):
|
||||
from hermes_state import SessionDB
|
||||
|
||||
events = []
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db = SessionDB(db_path=Path(tmpdir) / "test.db")
|
||||
agent = self._make_agent(db)
|
||||
compressor = MagicMock()
|
||||
compressor.compress.return_value = [
|
||||
{"role": "user", "content": "summary"}
|
||||
]
|
||||
compressor.compression_count = 1
|
||||
compressor.last_prompt_tokens = 0
|
||||
compressor.last_completion_tokens = 0
|
||||
compressor._last_summary_error = None
|
||||
compressor._last_compress_aborted = False
|
||||
compressor.on_session_start.side_effect = (
|
||||
lambda *_args, **_kwargs: events.append("notify")
|
||||
)
|
||||
agent.context_compressor = compressor
|
||||
|
||||
agent._compress_context(
|
||||
[{"role": "user", "content": "request"}],
|
||||
"sys",
|
||||
approx_tokens=100,
|
||||
force=True,
|
||||
defer_context_engine_notification=True,
|
||||
)
|
||||
|
||||
assert events == []
|
||||
assert finalize_context_engine_compression_notification(
|
||||
agent, committed=committed
|
||||
) is committed
|
||||
assert finalize_context_engine_compression_notification(
|
||||
agent, committed=True
|
||||
) is False
|
||||
assert events == (["notify"] if committed else [])
|
||||
|
||||
def test_no_hook_when_no_session_db(self):
|
||||
"""Without session_db, session_id does not rotate and the hook is not fired."""
|
||||
|
||||
@@ -95,7 +95,12 @@ def test_auto_corrects_threshold_when_aux_context_below_threshold(mock_get_clien
|
||||
assert "config.yaml" in messages[0]
|
||||
assert "auxiliary:" in messages[0]
|
||||
assert "compression:" in messages[0]
|
||||
assert "threshold:" in messages[0]
|
||||
# 200K main is under the 512K small-context limit and 80K/200K = 40% sits
|
||||
# below the 75% floor — a `threshold:` suggestion would be raised back to
|
||||
# 75% and ignored (#67422), so the message must not offer one and must
|
||||
# explain the recomputed trigger instead (0.75 * 200K = 150K).
|
||||
assert "threshold:" not in messages[0]
|
||||
assert "150,000" in messages[0]
|
||||
# Warning stored for gateway replay
|
||||
assert agent._compression_warning is not None
|
||||
# Threshold on the live compressor was actually lowered to aux_context.
|
||||
@@ -514,3 +519,97 @@ def test_run_conversation_clears_warning_after_replay(mock_get_client, mock_ctx_
|
||||
agent._compression_warning = None
|
||||
|
||||
assert len(callback_events) == 0
|
||||
|
||||
|
||||
# ── #67422: threshold suggestion must survive the small-context floor ────────
|
||||
|
||||
|
||||
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
|
||||
@patch("agent.auxiliary_client.get_text_auxiliary_client")
|
||||
def test_threshold_suggestion_kept_when_at_or_above_floor(mock_get_client, mock_ctx_len):
|
||||
"""Small-context main, but aux/main = 80% >= the 75% floor — the
|
||||
`threshold:` suggestion is still viable and must be offered."""
|
||||
agent = _make_agent(main_context=100_000, threshold_percent=0.85)
|
||||
# threshold = 85,000 — aux has 80,000 (above 64K floor, below threshold)
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://openrouter.ai/api/v1"
|
||||
mock_client.api_key = "sk-aux"
|
||||
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
|
||||
|
||||
messages = []
|
||||
agent._emit_status = lambda msg: messages.append(msg)
|
||||
|
||||
agent._check_compression_model_feasibility()
|
||||
|
||||
assert len(messages) == 1
|
||||
assert "threshold: 0.80" in messages[0]
|
||||
|
||||
|
||||
@patch("agent.model_metadata.get_model_context_length", return_value=300_000)
|
||||
@patch("agent.auxiliary_client.get_text_auxiliary_client")
|
||||
def test_threshold_suggestion_kept_for_large_context_main(mock_get_client, mock_ctx_len):
|
||||
"""Main window >= 512K has no floor — any suggestion is honored, so the
|
||||
`threshold:` option stays even below 75%."""
|
||||
agent = _make_agent(main_context=1_000_000, threshold_percent=0.50)
|
||||
# threshold = 500,000 — aux has 300,000
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://openrouter.ai/api/v1"
|
||||
mock_client.api_key = "sk-aux"
|
||||
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
|
||||
|
||||
messages = []
|
||||
agent._emit_status = lambda msg: messages.append(msg)
|
||||
|
||||
agent._check_compression_model_feasibility()
|
||||
|
||||
assert len(messages) == 1
|
||||
assert "threshold: 0.30" in messages[0]
|
||||
|
||||
|
||||
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
|
||||
@patch("agent.auxiliary_client.get_text_auxiliary_client")
|
||||
def test_threshold_suggestion_kept_when_reservation_shrinks_trigger(mock_get_client, mock_ctx_len):
|
||||
"""Output-token reservation can make a floored suggestion viable again:
|
||||
with max_tokens=120K on a 200K window, the recomputed trigger is
|
||||
max(0.75 * 80K, 64K) = 64K, which fits the 80K aux — so the suggestion
|
||||
must NOT be suppressed by the raw-window percentage check (sweeper
|
||||
regression for #67422)."""
|
||||
agent = _make_agent(main_context=200_000, threshold_percent=0.50)
|
||||
agent.context_compressor.max_tokens = 120_000
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://openrouter.ai/api/v1"
|
||||
mock_client.api_key = "sk-aux"
|
||||
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
|
||||
|
||||
messages = []
|
||||
agent._emit_status = lambda msg: messages.append(msg)
|
||||
|
||||
agent._check_compression_model_feasibility()
|
||||
|
||||
assert len(messages) == 1
|
||||
assert "threshold: 0.40" in messages[0]
|
||||
|
||||
|
||||
@patch("agent.model_metadata.get_model_context_length", return_value=80_000)
|
||||
@patch("agent.auxiliary_client.get_text_auxiliary_client")
|
||||
def test_plugin_engine_keeps_plain_suggestion(mock_get_client, mock_ctx_len):
|
||||
"""External context engines own compaction policy (#44439) — the built-in
|
||||
small-context floor must not suppress the threshold suggestion when the
|
||||
active compressor is not the built-in ContextCompressor."""
|
||||
agent = _make_agent(main_context=200_000, threshold_percent=0.50)
|
||||
plugin_engine = MagicMock() # not spec'd — fails isinstance(ContextCompressor)
|
||||
plugin_engine.context_length = 200_000
|
||||
plugin_engine.threshold_tokens = 100_000
|
||||
agent.context_compressor = plugin_engine
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://openrouter.ai/api/v1"
|
||||
mock_client.api_key = "sk-aux"
|
||||
mock_get_client.return_value = (mock_client, "google/gemini-3-flash-preview")
|
||||
|
||||
messages = []
|
||||
agent._emit_status = lambda msg: messages.append(msg)
|
||||
|
||||
agent._check_compression_model_feasibility()
|
||||
|
||||
assert len(messages) == 1
|
||||
assert "threshold: 0.40" in messages[0]
|
||||
|
||||
@@ -1265,3 +1265,117 @@ def test_reference_messages_drops_whitespace_only_string_user_turn():
|
||||
assert view[0] == {"role": "assistant", "content": "a"}
|
||||
assert view[-1] == {"role": "user", "content": "real"}
|
||||
assert all(str(m["content"]).strip() for m in view)
|
||||
|
||||
def test_moa_pre_api_compression_includes_reference_guidance(monkeypatch, tmp_path):
|
||||
"""The aggregator must not receive guidance that pushes it past compression.
|
||||
|
||||
The normal pre-API check sees only the persisted conversation. MoA adds
|
||||
reference guidance later, inside ``MoAChatCompletions.create()``, so this
|
||||
regression drives a raw request just below the threshold and makes the
|
||||
injected guidance cross it. Compression must occur before the aggregator
|
||||
request and leave the rebuilt request below the threshold.
|
||||
"""
|
||||
home = tmp_path / ".hermes"
|
||||
home.mkdir()
|
||||
(home / "config.yaml").write_text(
|
||||
"""
|
||||
moa:
|
||||
default_preset: review
|
||||
presets:
|
||||
review:
|
||||
reference_models:
|
||||
- provider: openrouter
|
||||
model: advisor
|
||||
aggregator:
|
||||
provider: openrouter
|
||||
model: aggregator
|
||||
""".strip(),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.setenv("HERMES_HOME", str(home))
|
||||
|
||||
events = []
|
||||
compression_inputs = []
|
||||
aggregator_request_tokens = []
|
||||
|
||||
def fake_estimate(messages, *args, **kwargs):
|
||||
rendered = str(messages)
|
||||
raw_tokens = 80 if "PRE_COMPACTION_HISTORY" in rendered else 20
|
||||
guidance_tokens = 40 if "Mixture of Agents reference context" in rendered else 0
|
||||
return raw_tokens + guidance_tokens
|
||||
|
||||
def fake_call_llm(**kwargs):
|
||||
if kwargs["task"] == "moa_reference":
|
||||
events.append("reference")
|
||||
return _response("advisor guidance")
|
||||
events.append("aggregator")
|
||||
aggregator_request_tokens.append(fake_estimate(kwargs["messages"]))
|
||||
return _response("aggregator acted")
|
||||
|
||||
monkeypatch.setattr("agent.moa_loop.call_llm", fake_call_llm)
|
||||
monkeypatch.setattr("agent.turn_context.estimate_request_tokens_rough", fake_estimate)
|
||||
monkeypatch.setattr("agent.conversation_loop.estimate_request_tokens_rough", fake_estimate)
|
||||
|
||||
agent = AIAgent(
|
||||
api_key="moa-virtual-provider",
|
||||
base_url="moa://local",
|
||||
model="review",
|
||||
provider="moa",
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
enabled_toolsets=["file"],
|
||||
max_iterations=3,
|
||||
)
|
||||
compressor = getattr(agent, "context_compressor")
|
||||
compressor.threshold_tokens = 100
|
||||
|
||||
def fake_compress(messages, *_args, **_kwargs):
|
||||
events.append("compress")
|
||||
compression_inputs.append(messages)
|
||||
return ([{"role": "user", "content": "SUMMARY"}], "system")
|
||||
|
||||
monkeypatch.setattr(agent, "_compress_context", fake_compress)
|
||||
|
||||
result = agent.run_conversation(
|
||||
"PRE_COMPACTION_HISTORY",
|
||||
conversation_history=[{"role": "assistant", "content": "prior response"}],
|
||||
)
|
||||
|
||||
assert result["final_response"] == "aggregator acted"
|
||||
assert events.index("compress") < events.index("aggregator")
|
||||
assert events.count("reference") == 1
|
||||
assert all("Mixture of Agents reference context" not in str(item) for item in compression_inputs)
|
||||
assert aggregator_request_tokens == [60]
|
||||
|
||||
|
||||
def test_prepared_aggregator_preserves_reasoning_config(monkeypatch):
|
||||
"""Prepared MoA requests retain the acting aggregator reasoning policy."""
|
||||
from agent import moa_loop
|
||||
|
||||
captured = {}
|
||||
expected_reasoning = {"enabled": True, "effort": "high"}
|
||||
|
||||
def fake_call_llm(**kwargs):
|
||||
captured.update(kwargs)
|
||||
return _response("aggregator acted")
|
||||
|
||||
monkeypatch.setattr(moa_loop, "call_llm", fake_call_llm)
|
||||
monkeypatch.setattr(moa_loop, "_aggregator_reasoning_config", lambda _slot: expected_reasoning)
|
||||
monkeypatch.setattr(
|
||||
moa_loop,
|
||||
"_slot_runtime",
|
||||
lambda slot: {"provider": slot["provider"], "model": slot["model"]},
|
||||
)
|
||||
|
||||
facade = moa_loop.MoAChatCompletions("review")
|
||||
facade._call_prepared_aggregator(
|
||||
{
|
||||
"messages": [{"role": "user", "content": "question"}],
|
||||
"aggregator": {"provider": "openrouter", "model": "aggregator"},
|
||||
"aggregator_temperature": None,
|
||||
},
|
||||
{},
|
||||
)
|
||||
|
||||
assert captured["reasoning_config"] == expected_reasoning
|
||||
|
||||
@@ -0,0 +1,233 @@
|
||||
"""Tests for per-model compression threshold overrides.
|
||||
|
||||
Users who swap between models with very different context windows (e.g. a
|
||||
256K model and a 1M model) need different compaction trigger points.
|
||||
``compression.model_thresholds`` in config.yaml lets them set per-model
|
||||
overrides that are resolved by longest substring match. The small-context
|
||||
floor (75% for <512K models) still applies on top of per-model overrides.
|
||||
"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent.context_compressor import ContextCompressor, resolve_model_threshold
|
||||
from agent.context_engine import ContextEngine
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# resolve_model_threshold helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestResolveModelThreshold:
|
||||
def test_no_overrides_returns_default(self):
|
||||
assert resolve_model_threshold("glm-5.2", None, 0.50) == 0.50
|
||||
assert resolve_model_threshold("glm-5.2", {}, 0.50) == 0.50
|
||||
|
||||
def test_empty_model_returns_default(self):
|
||||
assert resolve_model_threshold("", {"glm": 0.70}, 0.50) == 0.50
|
||||
|
||||
def test_exact_match(self):
|
||||
overrides = {"glm-5.2": 0.70}
|
||||
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.70
|
||||
|
||||
def test_substring_match(self):
|
||||
overrides = {"glm-5.2": 0.70}
|
||||
assert resolve_model_threshold("openai/glm-5.2", overrides, 0.50) == 0.70
|
||||
|
||||
def test_longest_match_wins(self):
|
||||
overrides = {"glm-5.2": 0.70, "glm-5.2-1M": 0.25}
|
||||
# "glm-5.2-1M" is a longer match than "glm-5.2"
|
||||
assert resolve_model_threshold("glm-5.2-1M", overrides, 0.50) == 0.25
|
||||
# "glm-5.2" alone still matches at 0.70
|
||||
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.70
|
||||
|
||||
def test_no_match_returns_default(self):
|
||||
overrides = {"claude-sonnet-4": 0.60}
|
||||
assert resolve_model_threshold("glm-5.2", overrides, 0.50) == 0.50
|
||||
|
||||
def test_override_can_lower_threshold(self):
|
||||
"""Per-model overrides work in both directions (raise and lower)."""
|
||||
overrides = {"small-model": 0.30}
|
||||
assert resolve_model_threshold("small-model", overrides, 0.50) == 0.30
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ContextCompressor integration
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestContextCompressorModelThresholds:
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=1_000_000)
|
||||
def test_init_large_context_with_override(self, _mock):
|
||||
"""Large context (>=512K) + per-model override: override applies directly."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2": 0.40},
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 1M context >= 512K, so no small-context floor — override wins
|
||||
assert cc.threshold_percent == 0.40
|
||||
assert cc.threshold_tokens == int(1_000_000 * 0.40)
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=1_000_000)
|
||||
def test_init_large_context_no_match(self, _mock):
|
||||
"""Large context + no matching override: global threshold used."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"claude-sonnet-4": 0.60},
|
||||
quiet_mode=True,
|
||||
)
|
||||
assert cc.threshold_percent == 0.50
|
||||
assert cc.threshold_tokens == int(1_000_000 * 0.50)
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
|
||||
def test_init_small_context_override_below_floor(self, _mock):
|
||||
"""Small context (<512K) + override below 75%: floor wins (raise-only)."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2": 0.40},
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 256K < 512K → floor at 0.75; override 0.40 < 0.75, so floor wins
|
||||
assert cc.threshold_percent == 0.75
|
||||
assert cc.threshold_tokens == int(256_000 * 0.75)
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
|
||||
def test_init_small_context_override_above_floor(self, _mock):
|
||||
"""Small context + override above 75%: override wins."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2": 0.80},
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 256K < 512K → floor at 0.75; override 0.80 > 0.75, so override wins
|
||||
assert cc.threshold_percent == 0.80
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
|
||||
def test_init_no_model_thresholds_dict(self, _mock):
|
||||
"""Empty model_thresholds dict = backward compatible."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 256K < 512K → floored at 0.75
|
||||
assert cc.threshold_percent == 0.75
|
||||
assert cc.model_thresholds == {}
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length", return_value=256_000)
|
||||
def test_init_none_model_thresholds(self, _mock):
|
||||
"""Passing None for model_thresholds is safe."""
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds=None,
|
||||
quiet_mode=True,
|
||||
)
|
||||
assert cc.model_thresholds == {}
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length")
|
||||
def test_update_model_re_resolves_threshold(self, mock_ctx):
|
||||
"""Switching models re-resolves the per-model threshold + re-applies floor."""
|
||||
mock_ctx.return_value = 256_000
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2": 0.80, "glm-5.2-1M": 0.25},
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 256K < 512K → floor at 0.75; override 0.80 > 0.75, so 0.80 wins
|
||||
assert cc.threshold_percent == 0.80
|
||||
|
||||
# Switch to the 1M model (large context, no floor)
|
||||
mock_ctx.return_value = 1_000_000
|
||||
cc.update_model(
|
||||
model="glm-5.2-1M",
|
||||
context_length=1_000_000,
|
||||
)
|
||||
# 1M >= 512K → no floor; override 0.25 applies directly
|
||||
assert cc.threshold_percent == 0.25
|
||||
assert cc.threshold_tokens == int(1_000_000 * 0.25)
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length")
|
||||
def test_update_model_falls_back_to_global(self, mock_ctx):
|
||||
"""Switching to a model with no override uses the global threshold."""
|
||||
mock_ctx.return_value = 1_000_000
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2": 0.40},
|
||||
quiet_mode=True,
|
||||
)
|
||||
# 1M context, override 0.40
|
||||
assert cc.threshold_percent == 0.40
|
||||
|
||||
# Switch to a model with no override (still large context)
|
||||
mock_ctx.return_value = 1_000_000
|
||||
cc.update_model(
|
||||
model="some-other-model",
|
||||
context_length=1_000_000,
|
||||
)
|
||||
# No override match → falls back to global 0.50; 1M >= 512K → no floor
|
||||
assert cc.threshold_percent == 0.50
|
||||
assert cc.threshold_tokens == int(1_000_000 * 0.50)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ContextEngine base class
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestContextEngineModelThresholds:
|
||||
def test_base_class_update_model_applies_overrides(self):
|
||||
"""The base-class update_model() applies model_thresholds if set."""
|
||||
class TestEngine(ContextEngine):
|
||||
@property
|
||||
def name(self):
|
||||
return "test"
|
||||
|
||||
def update_from_response(self, usage):
|
||||
pass
|
||||
|
||||
def should_compress(self, prompt_tokens=None):
|
||||
return False
|
||||
|
||||
def compress(self, messages, current_tokens=None, focus_topic=None):
|
||||
return messages
|
||||
|
||||
engine = TestEngine()
|
||||
engine.threshold_percent = 0.50
|
||||
engine._config_threshold_percent = 0.50
|
||||
engine.context_length = 0
|
||||
engine.model_thresholds = {"glm-5.2-1M": 0.25}
|
||||
|
||||
engine.update_model(model="glm-5.2-1M", context_length=1_000_000)
|
||||
assert engine.threshold_percent == 0.25
|
||||
assert engine.threshold_tokens == int(1_000_000 * 0.25)
|
||||
|
||||
def test_base_class_update_model_no_overrides(self):
|
||||
"""Without model_thresholds, the base class behaves as before."""
|
||||
class TestEngine(ContextEngine):
|
||||
@property
|
||||
def name(self):
|
||||
return "test"
|
||||
|
||||
def update_from_response(self, usage):
|
||||
pass
|
||||
|
||||
def should_compress(self, prompt_tokens=None):
|
||||
return False
|
||||
|
||||
def compress(self, messages, current_tokens=None, focus_topic=None):
|
||||
return messages
|
||||
|
||||
engine = TestEngine()
|
||||
engine.threshold_percent = 0.50
|
||||
engine._base_threshold_percent = 0.50
|
||||
engine.context_length = 0
|
||||
engine.model_thresholds = {}
|
||||
|
||||
engine.update_model(model="glm-5.2", context_length=256_000)
|
||||
assert engine.threshold_percent == 0.50
|
||||
assert engine.threshold_tokens == int(256_000 * 0.50)
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Follow-up regression tests for per-model compression threshold overrides.
|
||||
|
||||
Covers the gaps flagged in the review of PR #63020:
|
||||
|
||||
1. Plugin-engine init ordering — ``compression.model_thresholds`` must be
|
||||
assigned to a selected plugin context engine BEFORE the initial
|
||||
``update_model()`` call in agent init, so the initial model already gets
|
||||
its override (previously the override only took effect after the first
|
||||
``/model`` switch).
|
||||
2. ``compression.model_thresholds`` is a public key in ``DEFAULT_CONFIG``.
|
||||
3. Floor interaction on the ``update_model()`` (model-switch) path:
|
||||
an override below the small-context floor is raised to the floor
|
||||
(raise-only); an override above the floor wins.
|
||||
4. Base-class ``update_model()`` snapshots the pre-override percent once,
|
||||
so repeated switches fall back to the engine's configured threshold
|
||||
rather than a previous model's override.
|
||||
"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent.context_compressor import ContextCompressor
|
||||
from agent.context_engine import ContextEngine
|
||||
|
||||
|
||||
class _StubEngine(ContextEngine):
|
||||
"""Minimal concrete context engine for init-ordering tests."""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "stub"
|
||||
|
||||
def update_from_response(self, usage):
|
||||
pass
|
||||
|
||||
def should_compress(self, prompt_tokens=None):
|
||||
return False
|
||||
|
||||
def compress(self, messages, current_tokens=None):
|
||||
return messages
|
||||
|
||||
|
||||
def test_plugin_engine_gets_model_thresholds_before_initial_update_model():
|
||||
"""The initial model's override must apply during AIAgent init.
|
||||
|
||||
Regression test for the PR #63020 review finding: the plugin engine was
|
||||
initialized through update_model() before model_thresholds was assigned,
|
||||
so the initial model kept the global threshold until a /model switch.
|
||||
"""
|
||||
engine = _StubEngine()
|
||||
engine.threshold_percent = 0.50
|
||||
|
||||
cfg = {
|
||||
"context": {"engine": "stub"},
|
||||
"agent": {},
|
||||
"compression": {
|
||||
"threshold": 0.50,
|
||||
"model_thresholds": {"glm-5.2": 0.25},
|
||||
},
|
||||
}
|
||||
|
||||
with (
|
||||
patch("hermes_cli.config.load_config", return_value=cfg),
|
||||
patch("plugins.context_engine.load_context_engine", return_value=engine),
|
||||
patch("agent.model_metadata.get_model_context_length", return_value=1_000_000),
|
||||
patch("run_agent.get_tool_definitions", return_value=[]),
|
||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
||||
patch("run_agent.OpenAI"),
|
||||
):
|
||||
from run_agent import AIAgent
|
||||
|
||||
agent = AIAgent(
|
||||
model="glm-5.2",
|
||||
api_key="test-key-1234567890",
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
)
|
||||
|
||||
assert agent.context_compressor is engine
|
||||
# The override map arrived before the initial update_model() call, so the
|
||||
# very first resolution already used it.
|
||||
assert engine.model_thresholds == {"glm-5.2": 0.25}
|
||||
assert engine.threshold_percent == 0.25
|
||||
assert engine.threshold_tokens == int(1_000_000 * 0.25)
|
||||
|
||||
|
||||
def test_plugin_engine_without_overrides_keeps_global_threshold():
|
||||
"""Empty model_thresholds leaves plugin-engine init behavior unchanged."""
|
||||
engine = _StubEngine()
|
||||
engine.threshold_percent = 0.50
|
||||
|
||||
cfg = {
|
||||
"context": {"engine": "stub"},
|
||||
"agent": {},
|
||||
"compression": {"threshold": 0.50},
|
||||
}
|
||||
|
||||
with (
|
||||
patch("hermes_cli.config.load_config", return_value=cfg),
|
||||
patch("plugins.context_engine.load_context_engine", return_value=engine),
|
||||
patch("agent.model_metadata.get_model_context_length", return_value=1_000_000),
|
||||
patch("run_agent.get_tool_definitions", return_value=[]),
|
||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
||||
patch("run_agent.OpenAI"),
|
||||
):
|
||||
from run_agent import AIAgent
|
||||
|
||||
agent = AIAgent(
|
||||
model="glm-5.2",
|
||||
api_key="test-key-1234567890",
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
)
|
||||
|
||||
assert agent.context_compressor is engine
|
||||
assert getattr(engine, "model_thresholds", {}) == {}
|
||||
assert engine.threshold_percent == 0.50
|
||||
assert engine.threshold_tokens == int(1_000_000 * 0.50)
|
||||
|
||||
|
||||
def test_model_thresholds_key_in_default_config():
|
||||
"""compression.model_thresholds is a public DEFAULT_CONFIG key."""
|
||||
from hermes_cli.config import DEFAULT_CONFIG
|
||||
|
||||
assert "model_thresholds" in DEFAULT_CONFIG["compression"]
|
||||
assert DEFAULT_CONFIG["compression"]["model_thresholds"] == {}
|
||||
|
||||
|
||||
class TestFloorInteractionOnModelSwitch:
|
||||
"""The small-context floor stacks on per-model overrides at switch time."""
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length")
|
||||
def test_switch_override_below_floor_is_raised_to_floor(self, mock_ctx):
|
||||
"""Switching to a small-context model with a sub-floor override → floor."""
|
||||
mock_ctx.return_value = 1_000_000
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2-1M",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2-1M": 0.25, "small-model": 0.40},
|
||||
quiet_mode=True,
|
||||
)
|
||||
assert cc.threshold_percent == 0.25 # large context: override direct
|
||||
|
||||
# Switch to a <512K model whose override (0.40) is below the 0.75 floor.
|
||||
mock_ctx.return_value = 128_000
|
||||
cc.update_model(model="small-model", context_length=128_000)
|
||||
assert cc.threshold_percent == 0.75 # raise-only floor wins
|
||||
assert cc.threshold_tokens == int(128_000 * 0.75)
|
||||
|
||||
@patch("agent.context_compressor.get_model_context_length")
|
||||
def test_switch_override_above_floor_wins(self, mock_ctx):
|
||||
"""Switching to a small-context model with an above-floor override → override."""
|
||||
mock_ctx.return_value = 1_000_000
|
||||
cc = ContextCompressor(
|
||||
model="glm-5.2-1M",
|
||||
threshold_percent=0.50,
|
||||
model_thresholds={"glm-5.2-1M": 0.25, "small-model": 0.85},
|
||||
quiet_mode=True,
|
||||
)
|
||||
mock_ctx.return_value = 128_000
|
||||
cc.update_model(model="small-model", context_length=128_000)
|
||||
assert cc.threshold_percent == 0.85 # above the 0.75 floor: override wins
|
||||
assert cc.threshold_tokens == int(128_000 * 0.85)
|
||||
|
||||
|
||||
class TestBaseEngineConfigSnapshot:
|
||||
"""Base-class update_model() must not compound a previous override."""
|
||||
|
||||
def test_repeated_switches_fall_back_to_original_threshold(self):
|
||||
engine = _StubEngine()
|
||||
engine.threshold_percent = 0.50
|
||||
engine.context_length = 0
|
||||
engine.model_thresholds = {"glm-5.2-1M": 0.25}
|
||||
# NOTE: _config_threshold_percent deliberately NOT pre-set — the base
|
||||
# class must snapshot the original 0.50 on the first call, so the
|
||||
# second switch (no matching override) falls back to 0.50, not 0.25.
|
||||
|
||||
engine.update_model(model="glm-5.2-1M", context_length=1_000_000)
|
||||
assert engine.threshold_percent == 0.25
|
||||
|
||||
engine.update_model(model="some-other-model", context_length=1_000_000)
|
||||
assert engine.threshold_percent == 0.50
|
||||
assert engine.threshold_tokens == int(1_000_000 * 0.50)
|
||||
@@ -0,0 +1,206 @@
|
||||
"""Behavioral regression tests for the post-tool compression attempt cap.
|
||||
|
||||
The pre-API pressure gate, the overflow/413 error handlers, and the post-tool
|
||||
compaction gate all share ``compression_attempts`` as a per-turn backstop,
|
||||
bounded by the resolved ``compression.max_attempts`` cap (default 3). Before
|
||||
the fix the post-tool path neither checked nor incremented the counter, so a
|
||||
long tool loop could compact after every tool response for the lifetime of
|
||||
the turn.
|
||||
|
||||
These tests drive ``run_conversation()`` through real tool iterations with a
|
||||
compressor that always demands compression and assert ``_compress_context``
|
||||
fires at most ``max_compression_attempts`` times per turn — no source
|
||||
inspection, only observable behavior.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from run_agent import AIAgent
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _tool_call(i: int):
|
||||
return SimpleNamespace(
|
||||
id=f"call_{i}",
|
||||
type="function",
|
||||
function=SimpleNamespace(name="web_search", arguments='{"query": "x"}'),
|
||||
)
|
||||
|
||||
|
||||
def _tool_response(i: int):
|
||||
msg = SimpleNamespace(
|
||||
content=None,
|
||||
reasoning_content=None,
|
||||
reasoning=None,
|
||||
tool_calls=[_tool_call(i)],
|
||||
)
|
||||
choice = SimpleNamespace(message=msg, finish_reason="tool_calls")
|
||||
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
|
||||
|
||||
|
||||
def _stop_response():
|
||||
msg = SimpleNamespace(
|
||||
content="done",
|
||||
reasoning_content=None,
|
||||
reasoning=None,
|
||||
tool_calls=None,
|
||||
)
|
||||
choice = SimpleNamespace(message=msg, finish_reason="stop")
|
||||
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
|
||||
|
||||
|
||||
def _make_tool_defs(*names: str) -> list:
|
||||
return [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": n,
|
||||
"description": f"{n} tool",
|
||||
"parameters": {"type": "object", "properties": {}},
|
||||
},
|
||||
}
|
||||
for n in names
|
||||
]
|
||||
|
||||
|
||||
def _pressured_compressor() -> MagicMock:
|
||||
"""A compressor that always reports context pressure after tools run.
|
||||
|
||||
``should_defer_preflight_to_real_usage`` returns True so the turn-start
|
||||
preflight and the pre-API pressure gate stand down — isolating the
|
||||
post-tool gate as the only compression site under test.
|
||||
"""
|
||||
compressor = MagicMock()
|
||||
compressor.protect_first_n = 3
|
||||
compressor.protect_last_n = 20
|
||||
compressor.threshold_tokens = 10_000
|
||||
compressor.context_length = 200_000
|
||||
compressor.last_prompt_tokens = 150_000
|
||||
compressor.should_compress.return_value = True
|
||||
compressor.should_defer_preflight_to_real_usage.return_value = True
|
||||
compressor.get_active_compression_failure_cooldown.return_value = None
|
||||
return compressor
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def agent():
|
||||
with (
|
||||
patch("run_agent.get_tool_definitions", return_value=_make_tool_defs("web_search")),
|
||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
||||
patch("run_agent.OpenAI"),
|
||||
):
|
||||
a = AIAgent(
|
||||
api_key="test-key-1234567890",
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
max_iterations=10,
|
||||
)
|
||||
a.client = MagicMock()
|
||||
a._cached_system_prompt = "You are helpful."
|
||||
a._use_prompt_caching = False
|
||||
a._disable_streaming = True
|
||||
a.tool_delay = 0
|
||||
a.save_trajectories = False
|
||||
a.compression_enabled = True
|
||||
a.context_compressor = _pressured_compressor()
|
||||
return a
|
||||
|
||||
|
||||
def _run_tool_loop(agent, n_tool_iterations: int):
|
||||
"""Drive one turn: ``n_tool_iterations`` tool calls, then a stop."""
|
||||
responses = [_tool_response(i) for i in range(n_tool_iterations)]
|
||||
responses.append(_stop_response())
|
||||
agent.client.chat.completions.create.side_effect = responses
|
||||
|
||||
compress_calls = []
|
||||
|
||||
def _fake_compress(messages, system_message, **_kwargs):
|
||||
compress_calls.append(len(messages))
|
||||
return messages, "compressed prompt"
|
||||
|
||||
with (
|
||||
patch.object(agent, "_compress_context", side_effect=_fake_compress),
|
||||
patch.object(agent, "_persist_session"),
|
||||
patch.object(agent, "_save_trajectory"),
|
||||
patch.object(agent, "_cleanup_task_resources"),
|
||||
patch(
|
||||
"run_agent.handle_function_call",
|
||||
lambda name, args, task_id=None, **kwargs: json.dumps({"ok": True}),
|
||||
),
|
||||
):
|
||||
result = agent.run_conversation("do a lot of tool work")
|
||||
|
||||
return result, compress_calls
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPostToolCompressionAttemptCap:
|
||||
def test_post_tool_compression_capped_at_default_three(self, agent):
|
||||
"""7 tool iterations under constant pressure → exactly 3 compactions.
|
||||
|
||||
Before the fix the post-tool gate re-fired after every tool response
|
||||
(7 compactions here); the shared per-turn counter caps it at the
|
||||
resolved default of 3.
|
||||
"""
|
||||
assert agent.max_compression_attempts == 3 # config default
|
||||
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=7)
|
||||
|
||||
assert result["completed"] is True
|
||||
assert len(compress_calls) == 3, (
|
||||
f"post-tool compression must stop at the per-turn cap (3), "
|
||||
f"got {len(compress_calls)} compactions"
|
||||
)
|
||||
|
||||
def test_post_tool_compression_honors_configured_cap(self, agent):
|
||||
"""A raised compression.max_attempts cap lets more rounds run."""
|
||||
agent.max_compression_attempts = 5
|
||||
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=8)
|
||||
|
||||
assert result["completed"] is True
|
||||
assert len(compress_calls) == 5
|
||||
|
||||
def test_post_tool_compression_shares_counter_with_pre_api_gate(self, agent):
|
||||
"""Pre-API compactions consume the same per-turn budget.
|
||||
|
||||
Let the pre-API pressure gate fire once (defer disabled for the first
|
||||
check), then keep the pressure on through tool iterations: the
|
||||
combined total must still respect the cap.
|
||||
"""
|
||||
# First pre-API check does not defer → pre-API gate fires once;
|
||||
# afterwards defer again so only the post-tool gate keeps firing.
|
||||
defers = iter([False])
|
||||
agent.context_compressor.should_defer_preflight_to_real_usage.side_effect = (
|
||||
lambda _t: next(defers, True)
|
||||
)
|
||||
result, compress_calls = _run_tool_loop(agent, n_tool_iterations=7)
|
||||
|
||||
assert result["completed"] is True
|
||||
assert len(compress_calls) == 3, (
|
||||
"pre-API and post-tool compactions must share one per-turn "
|
||||
f"attempt budget, got {len(compress_calls)} total compactions"
|
||||
)
|
||||
|
||||
def test_cap_is_per_turn_not_per_session(self, agent):
|
||||
"""A fresh turn gets a fresh attempt budget."""
|
||||
_result, first = _run_tool_loop(agent, n_tool_iterations=5)
|
||||
agent.client.chat.completions.create.side_effect = None
|
||||
_result, second = _run_tool_loop(agent, n_tool_iterations=5)
|
||||
|
||||
assert len(first) == 3
|
||||
assert len(second) == 3
|
||||
@@ -0,0 +1,186 @@
|
||||
"""E2E: compression.max_attempts=6 drives a 4th+ preflight compaction pass.
|
||||
|
||||
The turn-start preflight loop in ``agent/turn_context.py`` was hardcoded to
|
||||
``range(3)``: even when every pass made real progress and the request stayed
|
||||
over threshold, the 4th pass never ran, regardless of configuration. The
|
||||
loop now sizes itself from the same resolved ``compression.max_attempts`` cap
|
||||
as the conversation loop's compression sites.
|
||||
|
||||
This test builds a real ``AIAgent`` from a config with
|
||||
``compression.max_attempts: 6`` (the config-driven path through
|
||||
``agent_init``), then drives a full ``run_conversation()`` turn in which the
|
||||
estimated request size keeps shrinking ~10% per compaction but stays above
|
||||
threshold — the exact "progress, but not enough yet" shape that legitimately
|
||||
needs more than three rounds. With cap=6 the preflight must run a 4th pass
|
||||
(and ultimately all six).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from hermes_state import SessionDB
|
||||
from run_agent import AIAgent
|
||||
|
||||
|
||||
def _config(max_attempts) -> dict:
|
||||
return {
|
||||
"compression": {
|
||||
"enabled": True,
|
||||
"threshold": 0.50,
|
||||
"target_ratio": 0.20,
|
||||
"protect_first_n": 3,
|
||||
"protect_last_n": 20,
|
||||
"max_attempts": max_attempts,
|
||||
},
|
||||
"prompt_caching": {"cache_ttl": "5m"},
|
||||
"sessions": {},
|
||||
"bedrock": {},
|
||||
}
|
||||
|
||||
|
||||
def _stop_response():
|
||||
msg = SimpleNamespace(
|
||||
content="done",
|
||||
reasoning_content=None,
|
||||
reasoning=None,
|
||||
tool_calls=None,
|
||||
)
|
||||
choice = SimpleNamespace(message=msg, finish_reason="stop")
|
||||
return SimpleNamespace(choices=[choice], model="test/model", usage=None)
|
||||
|
||||
|
||||
def _make_agent(monkeypatch, tmp_path: Path, *, max_attempts) -> AIAgent:
|
||||
from hermes_cli import config as config_mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
config_mod, "load_config", lambda: _config(max_attempts)
|
||||
)
|
||||
db = SessionDB(db_path=tmp_path / "state.db")
|
||||
with (
|
||||
contextlib.redirect_stdout(io.StringIO()),
|
||||
patch("run_agent.get_tool_definitions", return_value=[]),
|
||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
||||
patch("run_agent.OpenAI"),
|
||||
):
|
||||
agent = AIAgent(
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
api_key="test-key",
|
||||
model="test/model",
|
||||
enabled_toolsets=[],
|
||||
disabled_toolsets=[],
|
||||
quiet_mode=True,
|
||||
skip_memory=True,
|
||||
skip_context_files=True,
|
||||
session_db=db,
|
||||
session_id="preflight-cap-e2e",
|
||||
)
|
||||
agent.client = MagicMock()
|
||||
agent._cached_system_prompt = "You are helpful."
|
||||
agent._use_prompt_caching = False
|
||||
agent._disable_streaming = True
|
||||
agent.tool_delay = 0
|
||||
agent.save_trajectories = False
|
||||
return agent
|
||||
|
||||
|
||||
def test_preflight_runs_fourth_compaction_pass_at_cap_six(monkeypatch, tmp_path):
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=6)
|
||||
# Config-driven attach seam (agent_init) resolved the raised cap.
|
||||
assert agent.max_compression_attempts == 6
|
||||
|
||||
# Keep the request permanently over threshold while every compaction
|
||||
# makes material (~10% > the 5% progress floor) headway.
|
||||
compressor = agent.context_compressor
|
||||
compressor.threshold_tokens = 50_000
|
||||
|
||||
estimate_state = {"tokens": 1_000_000.0, "calls": 0}
|
||||
|
||||
def _shrinking_estimate(*_args, **_kwargs):
|
||||
if estimate_state["calls"]:
|
||||
estimate_state["tokens"] *= 0.9
|
||||
estimate_state["calls"] += 1
|
||||
return int(estimate_state["tokens"])
|
||||
|
||||
compress_calls = []
|
||||
|
||||
def _fake_compress(messages, system_message, **_kwargs):
|
||||
compress_calls.append(len(messages))
|
||||
return messages, "compressed prompt"
|
||||
|
||||
# 60 messages > protect_first_n + protect_last_n + 1, so the cheap
|
||||
# preflight count gate opens without patching internals.
|
||||
history = [
|
||||
{"role": "user" if i % 2 == 0 else "assistant", "content": f"msg {i}"}
|
||||
for i in range(60)
|
||||
]
|
||||
agent.client.chat.completions.create.return_value = _stop_response()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"agent.turn_context.estimate_request_tokens_rough",
|
||||
side_effect=_shrinking_estimate,
|
||||
),
|
||||
patch.object(agent, "_compress_context", side_effect=_fake_compress),
|
||||
patch.object(agent, "_persist_session"),
|
||||
patch.object(agent, "_save_trajectory"),
|
||||
patch.object(agent, "_cleanup_task_resources"),
|
||||
):
|
||||
result = agent.run_conversation("hello", conversation_history=history)
|
||||
|
||||
assert result["completed"] is True
|
||||
# The old hardcoded range(3) made a 4th pass impossible; cap=6 must
|
||||
# deliver it (and, with steady progress over threshold, all six).
|
||||
assert len(compress_calls) >= 4, (
|
||||
f"expected a 4th preflight compaction pass at cap=6, "
|
||||
f"got {len(compress_calls)} passes"
|
||||
)
|
||||
assert len(compress_calls) == 6
|
||||
|
||||
|
||||
def test_preflight_still_stops_at_default_three(monkeypatch, tmp_path):
|
||||
"""Unset compression.max_attempts keeps the historical 3-pass behavior."""
|
||||
agent = _make_agent(monkeypatch, tmp_path, max_attempts=None)
|
||||
assert agent.max_compression_attempts == 3
|
||||
|
||||
compressor = agent.context_compressor
|
||||
compressor.threshold_tokens = 50_000
|
||||
|
||||
estimate_state = {"tokens": 1_000_000.0, "calls": 0}
|
||||
|
||||
def _shrinking_estimate(*_args, **_kwargs):
|
||||
if estimate_state["calls"]:
|
||||
estimate_state["tokens"] *= 0.9
|
||||
estimate_state["calls"] += 1
|
||||
return int(estimate_state["tokens"])
|
||||
|
||||
compress_calls = []
|
||||
|
||||
def _fake_compress(messages, system_message, **_kwargs):
|
||||
compress_calls.append(len(messages))
|
||||
return messages, "compressed prompt"
|
||||
|
||||
history = [
|
||||
{"role": "user" if i % 2 == 0 else "assistant", "content": f"msg {i}"}
|
||||
for i in range(60)
|
||||
]
|
||||
agent.client.chat.completions.create.return_value = _stop_response()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"agent.turn_context.estimate_request_tokens_rough",
|
||||
side_effect=_shrinking_estimate,
|
||||
),
|
||||
patch.object(agent, "_compress_context", side_effect=_fake_compress),
|
||||
patch.object(agent, "_persist_session"),
|
||||
patch.object(agent, "_save_trajectory"),
|
||||
patch.object(agent, "_cleanup_task_resources"),
|
||||
):
|
||||
result = agent.run_conversation("hello", conversation_history=history)
|
||||
|
||||
assert result["completed"] is True
|
||||
assert len(compress_calls) == 3
|
||||
@@ -0,0 +1,273 @@
|
||||
"""Behavior tests for scripts/validate_plugin_catalog.py.
|
||||
|
||||
The script is the no-install structural validator used by the plugin-catalog
|
||||
admission CI: it must run with only stdlib + pyyaml, take file paths or a
|
||||
directory, exit 0/1, and support --json machine output. These tests exercise
|
||||
the CLI contract via subprocess (the same way CI invokes it).
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
SCRIPT = REPO_ROOT / "scripts" / "validate_plugin_catalog.py"
|
||||
|
||||
VALID_ENTRY = {
|
||||
"name": "example-plugin",
|
||||
"repo": "https://github.com/NousResearch/hermes-example-plugins",
|
||||
"sha": "38fe0fb53eff98d477f807432e965429e665ca33",
|
||||
"subdir": "",
|
||||
"description": "One-line description.",
|
||||
"maintainer": "NousResearch",
|
||||
"tier": "official",
|
||||
"requires_hermes": ">=0.19",
|
||||
"docs_url": "",
|
||||
"platforms": [],
|
||||
"capabilities": {
|
||||
"provides_tools": ["example_tool"],
|
||||
"provides_hooks": [],
|
||||
"provides_middleware": [],
|
||||
"requires_env": [],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def write_entry(tmp_path: Path, data: dict, filename: str | None = None) -> Path:
|
||||
name = filename or f"{data.get('name', 'entry')}.yaml"
|
||||
path = tmp_path / name
|
||||
path.write_text(yaml.safe_dump(data), encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def run_validator(*args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(SCRIPT), *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
# ── valid input ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_valid_entry_passes(tmp_path):
|
||||
path = write_entry(tmp_path, VALID_ENTRY)
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
|
||||
|
||||
def test_valid_entry_without_optional_fields_passes(tmp_path):
|
||||
entry = {
|
||||
"name": "minimal-plugin",
|
||||
"repo": "https://github.com/example/minimal",
|
||||
"sha": "a" * 40,
|
||||
"description": "Minimal.",
|
||||
"maintainer": "someone",
|
||||
}
|
||||
path = write_entry(tmp_path, entry)
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
|
||||
|
||||
# ── each malformed field fails with a pointed error ────────────────────
|
||||
|
||||
|
||||
def _expect_error(tmp_path, mutation: dict, expected_substring: str, drop: str = ""):
|
||||
entry = {**VALID_ENTRY, **mutation}
|
||||
if drop:
|
||||
entry.pop(drop, None)
|
||||
path = write_entry(tmp_path, entry, filename="entry.yaml")
|
||||
result = run_validator(str(path))
|
||||
combined = result.stdout + result.stderr
|
||||
assert result.returncode == 1, combined
|
||||
assert expected_substring in combined, combined
|
||||
assert "entry.yaml" in combined, combined
|
||||
|
||||
|
||||
def test_bad_name_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"name": "Bad Name!"}, "name")
|
||||
|
||||
|
||||
def test_name_too_long_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"name": "x" * 65}, "name")
|
||||
|
||||
|
||||
def test_non_https_repo_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"repo": "git@github.com:evil/x.git"}, "repo")
|
||||
|
||||
|
||||
def test_short_sha_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"sha": "abc123"}, "sha")
|
||||
|
||||
|
||||
def test_non_hex_sha_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"sha": "z" * 40}, "sha")
|
||||
|
||||
|
||||
def test_bad_tier_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"tier": "platinum"}, "tier")
|
||||
|
||||
|
||||
def test_empty_description_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"description": ""}, "description")
|
||||
|
||||
|
||||
def test_empty_maintainer_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"maintainer": ""}, "maintainer")
|
||||
|
||||
|
||||
def test_missing_required_field_fails(tmp_path):
|
||||
_expect_error(tmp_path, {}, "sha", drop="sha")
|
||||
|
||||
|
||||
def test_capabilities_value_not_a_list_fails(tmp_path):
|
||||
_expect_error(
|
||||
tmp_path,
|
||||
{"capabilities": {"provides_tools": "not-a-list"}},
|
||||
"provides_tools",
|
||||
)
|
||||
|
||||
|
||||
def test_capabilities_list_of_non_strings_fails(tmp_path):
|
||||
_expect_error(
|
||||
tmp_path,
|
||||
{"capabilities": {"requires_env": [1, 2]}},
|
||||
"requires_env",
|
||||
)
|
||||
|
||||
|
||||
def test_bad_requires_hermes_spec_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"requires_hermes": "banana"}, "requires_hermes")
|
||||
|
||||
|
||||
def test_comma_separated_requires_hermes_passes(tmp_path):
|
||||
entry = {**VALID_ENTRY, "requires_hermes": ">=0.19, <2.0"}
|
||||
path = write_entry(tmp_path, entry)
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
|
||||
|
||||
def test_unknown_platform_fails(tmp_path):
|
||||
_expect_error(tmp_path, {"platforms": ["linux", "amiga"]}, "platforms")
|
||||
|
||||
|
||||
def test_entry_not_a_mapping_fails(tmp_path):
|
||||
path = tmp_path / "entry.yaml"
|
||||
path.write_text("- just\n- a\n- list\n", encoding="utf-8")
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 1
|
||||
assert "mapping" in (result.stdout + result.stderr)
|
||||
|
||||
|
||||
# ── unknown top-level keys warn but do not fail ────────────────────────
|
||||
|
||||
|
||||
def test_unknown_key_warns_but_passes(tmp_path):
|
||||
entry = {**VALID_ENTRY, "future_field": "hello"}
|
||||
path = write_entry(tmp_path, entry)
|
||||
result = run_validator(str(path))
|
||||
combined = result.stdout + result.stderr
|
||||
assert result.returncode == 0, combined
|
||||
assert "future_field" in combined
|
||||
assert "warning" in combined.lower()
|
||||
|
||||
|
||||
# ── removed.yaml shape ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_valid_removed_yaml_passes(tmp_path):
|
||||
path = tmp_path / "removed.yaml"
|
||||
path.write_text(
|
||||
yaml.safe_dump(
|
||||
{
|
||||
"removed": [
|
||||
{
|
||||
"name": "some-plugin",
|
||||
"repo": "https://github.com/evil/some-plugin",
|
||||
"reason": "Exfiltrated env vars",
|
||||
"date": "2026-07-02",
|
||||
}
|
||||
]
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
|
||||
|
||||
def test_removed_yaml_not_a_list_fails(tmp_path):
|
||||
path = tmp_path / "removed.yaml"
|
||||
path.write_text(yaml.safe_dump({"removed": "nope"}), encoding="utf-8")
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 1
|
||||
assert "removed" in (result.stdout + result.stderr)
|
||||
|
||||
|
||||
def test_removed_item_missing_name_fails(tmp_path):
|
||||
path = tmp_path / "removed.yaml"
|
||||
path.write_text(
|
||||
yaml.safe_dump({"removed": [{"reason": "bad", "date": "2026-01-01"}]}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result = run_validator(str(path))
|
||||
assert result.returncode == 1
|
||||
assert "name" in (result.stdout + result.stderr)
|
||||
|
||||
|
||||
# ── --json machine output ──────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_json_output_shape_on_failure(tmp_path):
|
||||
bad = write_entry(tmp_path, {**VALID_ENTRY, "sha": "short"}, filename="bad.yaml")
|
||||
result = run_validator("--json", str(bad))
|
||||
assert result.returncode == 1
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["ok"] is False
|
||||
assert isinstance(payload["files"], list)
|
||||
entry = next(f for f in payload["files"] if f["path"].endswith("bad.yaml"))
|
||||
assert entry["ok"] is False
|
||||
assert any("sha" in e for e in entry["errors"])
|
||||
|
||||
|
||||
def test_json_output_shape_on_success_with_warning(tmp_path):
|
||||
good = write_entry(tmp_path, {**VALID_ENTRY, "future_field": 1})
|
||||
result = run_validator("--json", str(good))
|
||||
assert result.returncode == 0
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["ok"] is True
|
||||
(entry,) = payload["files"]
|
||||
assert entry["ok"] is True
|
||||
assert entry["errors"] == []
|
||||
assert any("future_field" in w for w in entry["warnings"])
|
||||
|
||||
|
||||
# ── directory mode ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_directory_mode_validates_all_entries_and_removed(tmp_path):
|
||||
write_entry(tmp_path, VALID_ENTRY)
|
||||
write_entry(tmp_path, {**VALID_ENTRY, "name": "bad-one", "sha": "nope"})
|
||||
(tmp_path / "removed.yaml").write_text(
|
||||
yaml.safe_dump({"removed": [{"name": "gone", "reason": "test"}]}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result = run_validator(str(tmp_path))
|
||||
combined = result.stdout + result.stderr
|
||||
assert result.returncode == 1
|
||||
assert "bad-one.yaml" in combined
|
||||
# the valid entry and removed.yaml must not produce errors
|
||||
assert combined.count("ERROR") == combined.count("bad-one.yaml: ERROR")
|
||||
|
||||
|
||||
def test_directory_mode_all_valid_exits_zero(tmp_path):
|
||||
write_entry(tmp_path, VALID_ENTRY)
|
||||
(tmp_path / "removed.yaml").write_text(
|
||||
yaml.safe_dump({"removed": []}), encoding="utf-8"
|
||||
)
|
||||
result = run_validator(str(tmp_path))
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
@@ -1,5 +1,8 @@
|
||||
from contextlib import nullcontext
|
||||
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
)
|
||||
from cli import HermesCLI
|
||||
|
||||
|
||||
@@ -9,8 +12,33 @@ class DummyAgent:
|
||||
self._cached_system_prompt = "FULL CACHED SYSTEM PROMPT SHOULD NOT BE NESTED"
|
||||
self.session_id = "new-session"
|
||||
self.calls = []
|
||||
self.flush_calls = []
|
||||
self.flush_error = None
|
||||
self.host_events = []
|
||||
self.boundary_calls = []
|
||||
self.context_compressor = type("ContextEngineStub", (), {})()
|
||||
self.context_compressor.on_session_start = self._record_boundary
|
||||
|
||||
def _compress_context(self, messages, system_message, *, approx_tokens=None, focus_topic=None, force=False):
|
||||
def _record_boundary(self, session_id, **kwargs):
|
||||
self.host_events.append("notify")
|
||||
self.boundary_calls.append((session_id, kwargs))
|
||||
|
||||
def _flush_messages_to_session_db(self, messages, _session_id=None):
|
||||
self.host_events.append("persist")
|
||||
self.flush_calls.append((list(messages), _session_id))
|
||||
if self.flush_error is not None:
|
||||
raise self.flush_error
|
||||
|
||||
def _compress_context(
|
||||
self,
|
||||
messages,
|
||||
system_message,
|
||||
*,
|
||||
approx_tokens=None,
|
||||
focus_topic=None,
|
||||
force=False,
|
||||
defer_context_engine_notification=False,
|
||||
):
|
||||
self.calls.append(
|
||||
{
|
||||
"messages": messages,
|
||||
@@ -18,8 +46,17 @@ class DummyAgent:
|
||||
"approx_tokens": approx_tokens,
|
||||
"focus_topic": focus_topic,
|
||||
"force": force,
|
||||
"defer_context_engine_notification": (
|
||||
defer_context_engine_notification
|
||||
),
|
||||
}
|
||||
)
|
||||
if defer_context_engine_notification:
|
||||
_queue_context_engine_compression_notification(
|
||||
self,
|
||||
new_session_id=self.session_id,
|
||||
old_session_id="old-session",
|
||||
)
|
||||
return ([{"role": "user", "content": "[CONTEXT SUMMARY]: compacted"}], "new system prompt")
|
||||
|
||||
|
||||
@@ -56,3 +93,27 @@ def test_manual_compress_does_not_pass_cached_system_prompt(monkeypatch):
|
||||
assert call["focus_topic"] == "database schema"
|
||||
assert cli.session_id == "new-session"
|
||||
assert cli._pending_title is None
|
||||
assert len(cli.agent.flush_calls) == 1
|
||||
assert cli.agent.host_events == ["persist", "notify"]
|
||||
assert len(cli.agent.boundary_calls) == 1
|
||||
|
||||
|
||||
def test_manual_compress_flush_failure_discards_notification(monkeypatch):
|
||||
cli = HermesCLI.__new__(HermesCLI)
|
||||
cli.conversation_history = [
|
||||
{"role": "user", "content": "one"},
|
||||
{"role": "assistant", "content": "two"},
|
||||
{"role": "user", "content": "three"},
|
||||
{"role": "assistant", "content": "four"},
|
||||
]
|
||||
cli.agent = DummyAgent()
|
||||
cli.agent.flush_error = RuntimeError("synthetic child flush failure")
|
||||
cli.session_id = "old-session"
|
||||
cli._pending_title = "old title"
|
||||
cli._busy_command = lambda _message: nullcontext()
|
||||
|
||||
cli._manual_compress("/compress")
|
||||
|
||||
assert len(cli.agent.flush_calls) == 1
|
||||
assert cli.agent.host_events == ["persist"]
|
||||
assert cli.agent.boundary_calls == []
|
||||
|
||||
@@ -139,6 +139,117 @@ class TestSessionLifecycle:
|
||||
assert session["cwd"] == "/work/repo"
|
||||
assert session["git_branch"] == "pets-feature"
|
||||
|
||||
def test_child_session_inherits_cwd_and_git_repo_root_from_parent(self, db):
|
||||
"""A parent_session_id child born without cwd/git_repo_root (e.g. the
|
||||
compression-fork path) must inherit both from its parent, so it
|
||||
doesn't silently drop out of the project sidebar (#64709)."""
|
||||
db.create_session(session_id="parent", source="cli")
|
||||
db.update_session_cwd("parent", "/work/repo", git_repo_root="/work/repo")
|
||||
|
||||
db.create_session(session_id="child", source="cli", parent_session_id="parent")
|
||||
|
||||
child = db.get_session("child")
|
||||
assert child["cwd"] == "/work/repo"
|
||||
assert child["git_repo_root"] == "/work/repo"
|
||||
|
||||
def test_child_session_explicit_cwd_is_not_overwritten_by_parent(self, db):
|
||||
"""A child that explicitly sets its own cwd/git_repo_root keeps it —
|
||||
parent inheritance only fills in NULLs, never clobbers."""
|
||||
db.create_session(session_id="parent", source="cli")
|
||||
db.update_session_cwd("parent", "/work/repo-a", git_repo_root="/work/repo-a")
|
||||
|
||||
db.create_session(
|
||||
session_id="child", source="cli", parent_session_id="parent",
|
||||
cwd="/work/repo-b", git_repo_root="/work/repo-b",
|
||||
)
|
||||
|
||||
child = db.get_session("child")
|
||||
assert child["cwd"] == "/work/repo-b"
|
||||
assert child["git_repo_root"] == "/work/repo-b"
|
||||
|
||||
def test_multi_generation_lineage_inherits_cwd(self, db):
|
||||
"""cwd/git_repo_root propagate through a multi-hop compression chain
|
||||
(root -> gen1 -> gen2), mirroring the multi-generation lineage from
|
||||
the reported issue where a single conversation forked repeatedly."""
|
||||
db.create_session(session_id="root", source="cli")
|
||||
db.update_session_cwd("root", "/work/repo", git_repo_root="/work/repo")
|
||||
|
||||
db.create_session(session_id="gen1", source="cli", parent_session_id="root")
|
||||
db.create_session(session_id="gen2", source="cli", parent_session_id="gen1")
|
||||
|
||||
assert db.get_session("gen1")["cwd"] == "/work/repo"
|
||||
assert db.get_session("gen2")["cwd"] == "/work/repo"
|
||||
|
||||
def test_child_session_inherits_git_branch_from_parent(self, db):
|
||||
"""git_branch travels with cwd/git_repo_root — the Desktop sidebar
|
||||
shows the branch chip per session, so a compression child born
|
||||
without it loses the chip even though the workspace didn't change."""
|
||||
db.create_session(session_id="parent", source="cli")
|
||||
db.update_session_cwd(
|
||||
"parent", "/work/repo", git_branch="feature-x", git_repo_root="/work/repo"
|
||||
)
|
||||
|
||||
db.create_session(session_id="child", source="cli", parent_session_id="parent")
|
||||
|
||||
child = db.get_session("child")
|
||||
assert child["git_branch"] == "feature-x"
|
||||
|
||||
def test_compression_child_inherits_gateway_origin_columns(self, db):
|
||||
"""A compression fork's child inherits gateway routing metadata
|
||||
(session_key/chat_id/...) from the ended parent, so a crash before
|
||||
the gateway re-records the peer can't strand it (#59527)."""
|
||||
db.create_session(
|
||||
session_id="parent", source="telegram",
|
||||
user_id="u1", session_key="telegram:u1:c1",
|
||||
chat_id="c1", chat_type="private", thread_id="t1",
|
||||
)
|
||||
db.record_gateway_session_peer(
|
||||
"parent", source="telegram", user_id="u1",
|
||||
session_key="telegram:u1:c1", chat_id="c1", chat_type="private",
|
||||
thread_id="t1", display_name="Chat One", origin_json='{"p":"telegram"}',
|
||||
)
|
||||
# Rotation path: parent is ended with 'compression' BEFORE the child
|
||||
# row is created (agent/conversation_compression.py).
|
||||
db.end_session("parent", "compression")
|
||||
|
||||
db.create_session(
|
||||
session_id="child", source="telegram", parent_session_id="parent"
|
||||
)
|
||||
|
||||
child = db.get_session("child")
|
||||
assert child["user_id"] == "u1"
|
||||
assert child["session_key"] == "telegram:u1:c1"
|
||||
assert child["chat_id"] == "c1"
|
||||
assert child["chat_type"] == "private"
|
||||
assert child["thread_id"] == "t1"
|
||||
assert child["display_name"] == "Chat One"
|
||||
assert child["origin_json"] == '{"p":"telegram"}'
|
||||
|
||||
def test_live_parent_child_does_not_inherit_gateway_origin(self, db):
|
||||
"""Delegate/subagent children (parent still live) must NOT inherit
|
||||
routing keys — peer recovery could otherwise repoint gateway traffic
|
||||
into a subagent's session."""
|
||||
db.create_session(
|
||||
session_id="parent", source="telegram",
|
||||
user_id="u1", session_key="telegram:u1:c1",
|
||||
chat_id="c1", chat_type="private",
|
||||
)
|
||||
|
||||
db.create_session(
|
||||
session_id="sub", source="telegram", parent_session_id="parent"
|
||||
)
|
||||
|
||||
sub = db.get_session("sub")
|
||||
assert sub["session_key"] is None
|
||||
assert sub["chat_id"] is None
|
||||
assert sub["user_id"] is None
|
||||
# Workspace metadata still inherits — that part is safe for any child.
|
||||
db.update_session_cwd("parent", "/work/repo", git_repo_root="/work/repo")
|
||||
db.create_session(
|
||||
session_id="sub2", source="telegram", parent_session_id="parent"
|
||||
)
|
||||
assert db.get_session("sub2")["cwd"] == "/work/repo"
|
||||
|
||||
def test_update_session_cwd_empty_branch_does_not_clobber(self, db):
|
||||
"""A failed branch probe (empty string) must not wipe a branch we
|
||||
already captured — only the cwd updates."""
|
||||
@@ -1639,6 +1750,236 @@ class TestMessageStorage:
|
||||
assert conv[0]["codex_reasoning_items"][0]["encrypted_content"] == "enc_blob_123"
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Timestamp preservation
|
||||
# =========================================================================
|
||||
|
||||
|
||||
class TestTimestampPreservation:
|
||||
"""Tests for the timestamp preservation feature.
|
||||
|
||||
``append_message()`` and ``replace_messages()`` now accept/forward an
|
||||
optional ``timestamp`` parameter. These tests verify custom timestamps
|
||||
survive the round trip through the DB and fall back to ``time.time()``
|
||||
when omitted.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _build_messages(ts_list, contents=None, roles=None):
|
||||
"""Build message dicts with explicit timestamps for testing."""
|
||||
if contents is None:
|
||||
contents = [f"msg-{i}" for i in range(len(ts_list))]
|
||||
if roles is None:
|
||||
roles = ["user", "assistant"] * (len(ts_list) // 2 + 1)
|
||||
return [
|
||||
{"role": roles[i], "content": contents[i], "timestamp": ts}
|
||||
for i, ts in enumerate(ts_list)
|
||||
]
|
||||
|
||||
def _raw_timestamps(self, db, session_id):
|
||||
"""Query timestamp column directly from SQLite for verification."""
|
||||
rows = db._conn.execute(
|
||||
"SELECT timestamp FROM messages WHERE session_id = ? ORDER BY id",
|
||||
(session_id,),
|
||||
).fetchall()
|
||||
return [r[0] for r in rows]
|
||||
|
||||
def test_append_message_with_explicit_timestamp(self, db):
|
||||
"""A caller-supplied timestamp is stored and round-tripped."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
ts = 1_234_567.0
|
||||
mid = db.append_message("s1", role="user", content="hello",
|
||||
timestamp=ts)
|
||||
msgs = db.get_messages("s1")
|
||||
assert len(msgs) == 1
|
||||
assert msgs[0]["timestamp"] == ts
|
||||
assert msgs[0]["id"] == mid
|
||||
raw = self._raw_timestamps(db, "s1")
|
||||
assert raw == [ts]
|
||||
|
||||
def test_append_message_multiple_timestamps(self, db):
|
||||
"""Multiple messages with different explicit timestamps."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
timestamps = [1_000_000.0, 2_000_000.0, 3_000_000.0]
|
||||
for i, ts in enumerate(timestamps, 1):
|
||||
db.append_message("s1", role="user", content=f"msg {i}",
|
||||
timestamp=ts)
|
||||
msgs = db.get_messages("s1")
|
||||
assert [m["timestamp"] for m in msgs] == timestamps
|
||||
assert self._raw_timestamps(db, "s1") == timestamps
|
||||
|
||||
def test_append_message_without_timestamp_defaults(self, db):
|
||||
"""Omitting timestamp stores a recent time.time() value."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
before = time.time()
|
||||
db.append_message("s1", role="user", content="hello")
|
||||
after = time.time()
|
||||
msgs = db.get_messages("s1")
|
||||
stored = msgs[0]["timestamp"]
|
||||
assert before <= stored <= after, (
|
||||
f"Expected timestamp between {before} and {after}, got {stored}"
|
||||
)
|
||||
raw_stored = self._raw_timestamps(db, "s1")[0]
|
||||
assert before <= raw_stored <= after
|
||||
|
||||
def test_append_message_mixed_timestamps(self, db):
|
||||
"""Messages with and without explicit timestamps — those without
|
||||
get a current time, those with keep their value."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
explicit_ts = 500_000.0
|
||||
db.append_message("s1", role="user", content="explicit",
|
||||
timestamp=explicit_ts)
|
||||
before = time.time()
|
||||
db.append_message("s1", role="user", content="default")
|
||||
after = time.time()
|
||||
msgs = db.get_messages("s1")
|
||||
assert msgs[0]["timestamp"] == explicit_ts
|
||||
assert before <= msgs[1]["timestamp"] <= after
|
||||
raw = self._raw_timestamps(db, "s1")
|
||||
assert raw[0] == explicit_ts
|
||||
assert before <= raw[1] <= after
|
||||
|
||||
def test_replace_messages_preserves_timestamps(self, db):
|
||||
"""Message dicts with ``timestamp`` passed to ``replace_messages``
|
||||
retain those timestamps after the rewrite."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
msgs_in = [
|
||||
{"role": "user", "content": "first", "timestamp": 100.0},
|
||||
{"role": "assistant", "content": "second", "timestamp": 200.0},
|
||||
{"role": "user", "content": "third", "timestamp": 300.0},
|
||||
]
|
||||
db.replace_messages("s1", msgs_in)
|
||||
msgs_out = db.get_messages("s1")
|
||||
assert [m["timestamp"] for m in msgs_out] == [100.0, 200.0, 300.0]
|
||||
assert self._raw_timestamps(db, "s1") == [100.0, 200.0, 300.0]
|
||||
|
||||
def test_replace_messages_fallback_when_no_timestamp(self, db):
|
||||
"""Message dicts without ``timestamp`` get auto-incrementing
|
||||
fallback values (starting from ~time.time())."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
db.replace_messages("s1", [
|
||||
{"role": "user", "content": "a"},
|
||||
{"role": "user", "content": "b"},
|
||||
])
|
||||
msgs = db.get_messages("s1")
|
||||
assert len(msgs) == 2
|
||||
t0, t1 = msgs[0]["timestamp"], msgs[1]["timestamp"]
|
||||
assert t1 > t0
|
||||
raw = self._raw_timestamps(db, "s1")
|
||||
assert len(raw) == 2
|
||||
assert raw[1] > raw[0]
|
||||
|
||||
def test_replace_messages_mixed_timestamps(self, db):
|
||||
"""Some messages with timestamp, some without — a message without
|
||||
timestamp uses the fallback clock, which is larger than any
|
||||
explicit historical timestamp."""
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
old_ts = 1_000.0
|
||||
db.replace_messages("s1", [
|
||||
{"role": "user", "content": "old", "timestamp": old_ts},
|
||||
{"role": "user", "content": "new"},
|
||||
])
|
||||
msgs = db.get_messages("s1")
|
||||
assert msgs[0]["timestamp"] == old_ts
|
||||
assert msgs[1]["timestamp"] > old_ts
|
||||
raw = self._raw_timestamps(db, "s1")
|
||||
assert raw[0] == old_ts
|
||||
assert raw[1] > old_ts
|
||||
|
||||
def test_fork_chain_preserves_timestamps(self, db):
|
||||
"""Simulate a /branch fork: copy messages from parent to child,
|
||||
verify timestamps are identical in both via raw SQL."""
|
||||
base_ts = 1_700_000_000.0
|
||||
timestamps = [base_ts + i * 20 for i in range(5)]
|
||||
contents = [
|
||||
"how do I fix a TypeError?",
|
||||
"show me the traceback",
|
||||
"TypeError at line 42",
|
||||
"issue in utils.py",
|
||||
"try int(...)",
|
||||
]
|
||||
roles = ["user", "assistant", "tool", "user", "assistant"]
|
||||
parent_msgs = self._build_messages(timestamps, contents, roles)
|
||||
|
||||
db.create_session(session_id="parent", source="cli")
|
||||
for msg in parent_msgs:
|
||||
db.append_message("parent", role=msg["role"],
|
||||
content=msg["content"],
|
||||
timestamp=msg["timestamp"])
|
||||
|
||||
db.create_session(session_id="child", source="cli",
|
||||
parent_session_id="parent")
|
||||
for msg in parent_msgs:
|
||||
db.append_message("child", role=msg["role"],
|
||||
content=msg["content"],
|
||||
timestamp=msg["timestamp"])
|
||||
|
||||
parent_raw = self._raw_timestamps(db, "parent")
|
||||
child_raw = self._raw_timestamps(db, "child")
|
||||
assert parent_raw == timestamps
|
||||
assert child_raw == timestamps
|
||||
assert parent_raw == child_raw
|
||||
|
||||
def test_branch_copy_roundtrip_preserves_timestamps(self, db):
|
||||
"""End-to-end branch copy: load the parent transcript via
|
||||
get_messages_as_conversation (the dict shape the CLI/gateway/TUI
|
||||
branch loops iterate) and re-append into a child forwarding
|
||||
``msg.get("timestamp")`` — the copies must keep the originals
|
||||
instead of being restamped with time.time() (#28841).
|
||||
"""
|
||||
timestamps = [1_600_000_000.0, 1_600_000_060.0, 1_600_000_120.0]
|
||||
db.create_session(session_id="parent", source="cli")
|
||||
for i, ts in enumerate(timestamps):
|
||||
db.append_message(
|
||||
"parent",
|
||||
role="user" if i % 2 == 0 else "assistant",
|
||||
content=f"msg-{i}",
|
||||
timestamp=ts,
|
||||
)
|
||||
|
||||
history = db.get_messages_as_conversation("parent")
|
||||
assert [m.get("timestamp") for m in history] == timestamps
|
||||
|
||||
db.create_session(session_id="child", source="cli",
|
||||
parent_session_id="parent")
|
||||
# Mirrors the branch copy loops in gateway/slash_commands.py,
|
||||
# hermes_cli/cli_commands_mixin.py and tui_gateway/server.py.
|
||||
for msg in history:
|
||||
db.append_message(
|
||||
"child",
|
||||
role=msg.get("role", "user"),
|
||||
content=msg.get("content"),
|
||||
timestamp=msg.get("timestamp"),
|
||||
)
|
||||
|
||||
assert self._raw_timestamps(db, "child") == timestamps
|
||||
|
||||
def test_compression_replace_roundtrip_preserves_timestamps(self, db):
|
||||
"""Compression-style rewrite: replace_messages with dicts loaded from
|
||||
get_messages_as_conversation must keep the surviving messages'
|
||||
original timestamps (#28841)."""
|
||||
timestamps = [1_500_000_000.0, 1_500_000_100.0, 1_500_000_200.0]
|
||||
db.create_session(session_id="s1", source="cli")
|
||||
for i, ts in enumerate(timestamps):
|
||||
db.append_message(
|
||||
"s1",
|
||||
role="user" if i % 2 == 0 else "assistant",
|
||||
content=f"msg-{i}",
|
||||
timestamp=ts,
|
||||
)
|
||||
|
||||
history = db.get_messages_as_conversation("s1")
|
||||
# Simulate a compression that keeps the last two turns verbatim and
|
||||
# prepends a fresh summary message (no timestamp — falls back to now).
|
||||
compressed = [{"role": "user", "content": "[summary]"}] + history[-2:]
|
||||
db.replace_messages("s1", compressed)
|
||||
|
||||
raw = self._raw_timestamps(db, "s1")
|
||||
assert len(raw) == 3
|
||||
assert raw[1:] == timestamps[-2:]
|
||||
assert raw[0] > timestamps[-1] # summary stamped with a current time
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# FTS5 search
|
||||
# =========================================================================
|
||||
|
||||
@@ -5926,26 +5926,37 @@ def test_session_compress_reports_aborted_summary_without_success(monkeypatch):
|
||||
|
||||
|
||||
def test_session_compress_syncs_session_key_after_rotation(monkeypatch):
|
||||
"""When AIAgent._compress_context rotates session_id (compression split),
|
||||
the gateway session_key must follow so subsequent approval routing,
|
||||
DB title/history lookups, and slash worker resume target the new
|
||||
continuation session — mirrors HermesCLI._manual_compress's
|
||||
session_id sync (cli.py).
|
||||
"""
|
||||
agent = types.SimpleNamespace(session_id="rotated-id")
|
||||
"""LCM notification follows the TUI's final session-key transition."""
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
events = []
|
||||
agent = types.SimpleNamespace(
|
||||
session_id="rotated-id",
|
||||
context_compressor=types.SimpleNamespace(
|
||||
on_session_start=lambda *_args, **_kwargs: events.append("notify")
|
||||
),
|
||||
)
|
||||
server._sessions["sid"] = _session(agent=agent)
|
||||
server._sessions["sid"]["session_key"] = "old-key"
|
||||
server._sessions["sid"]["pending_title"] = "stale title"
|
||||
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_compress_session_history",
|
||||
lambda session, focus_topic=None, **_kw: (2, {"total": 42}),
|
||||
)
|
||||
def _compress(session, focus_topic=None, **_kw):
|
||||
_queue_context_engine_compression_notification(
|
||||
session["agent"],
|
||||
new_session_id="rotated-id",
|
||||
old_session_id="old-key",
|
||||
)
|
||||
return 2, {"total": 42}
|
||||
|
||||
monkeypatch.setattr(server, "_compress_session_history", _compress)
|
||||
monkeypatch.setattr(server, "_session_info", lambda _agent, *a: {"model": "x"})
|
||||
restart_calls = []
|
||||
monkeypatch.setattr(
|
||||
server, "_restart_slash_worker", lambda sid, s: restart_calls.append(s)
|
||||
server,
|
||||
"_restart_slash_worker",
|
||||
lambda sid, s: (restart_calls.append(s), events.append("sync")),
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -5961,6 +5972,52 @@ def test_session_compress_syncs_session_key_after_rotation(monkeypatch):
|
||||
assert server._sessions["sid"]["session_key"] == "rotated-id"
|
||||
assert server._sessions["sid"]["pending_title"] is None
|
||||
assert len(restart_calls) == 1
|
||||
assert events == ["sync", "notify"]
|
||||
finally:
|
||||
server._sessions.pop("sid", None)
|
||||
|
||||
|
||||
def test_session_compress_sync_failure_discards_lcm_notification(monkeypatch):
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
events = []
|
||||
agent = types.SimpleNamespace(
|
||||
session_id="rotated-id",
|
||||
context_compressor=types.SimpleNamespace(
|
||||
on_session_start=lambda *_args, **_kwargs: events.append("notify")
|
||||
),
|
||||
)
|
||||
server._sessions["sid"] = _session(agent=agent)
|
||||
server._sessions["sid"]["session_key"] = "old-key"
|
||||
|
||||
def _compress(session, focus_topic=None, **_kw):
|
||||
_queue_context_engine_compression_notification(
|
||||
session["agent"],
|
||||
new_session_id="rotated-id",
|
||||
old_session_id="old-key",
|
||||
)
|
||||
return 2, {"total": 42}
|
||||
|
||||
monkeypatch.setattr(server, "_compress_session_history", _compress)
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_session_info",
|
||||
lambda *_args: (_ for _ in ()).throw(RuntimeError("finalization failed")),
|
||||
)
|
||||
|
||||
try:
|
||||
with patch("tui_gateway.server._emit"):
|
||||
resp = server.handle_request(
|
||||
{
|
||||
"id": "1",
|
||||
"method": "session.compress",
|
||||
"params": {"session_id": "sid"},
|
||||
}
|
||||
)
|
||||
assert resp["error"]["code"] == 5005
|
||||
assert events == []
|
||||
finally:
|
||||
server._sessions.pop("sid", None)
|
||||
|
||||
@@ -6418,6 +6475,30 @@ def test_commands_catalog_includes_tui_mouse_command():
|
||||
assert "/mouse" in tui_pairs
|
||||
|
||||
|
||||
def test_commands_catalog_has_no_duplicate_or_alias_colliding_names():
|
||||
"""No command may be advertised twice, and no advertised command may
|
||||
shadow an alias of a different command (e.g. the historical /compact
|
||||
collision where the registry aliased compact -> compress while the TUI
|
||||
also registered its own /compact display toggle; see #57133)."""
|
||||
resp = server.handle_request(
|
||||
{"id": "1", "method": "commands.catalog", "params": {}}
|
||||
)
|
||||
|
||||
names = [name for name, _ in resp["result"]["pairs"]]
|
||||
dupes = {n for n in names if names.count(n) > 1}
|
||||
assert not dupes, f"duplicate commands advertised in catalog: {sorted(dupes)}"
|
||||
|
||||
canon = resp["result"]["canon"]
|
||||
colliding = {
|
||||
name
|
||||
for name in names
|
||||
if canon.get(name.lower(), name) != name
|
||||
}
|
||||
assert not colliding, (
|
||||
f"catalog commands shadow aliases of other commands: {sorted(colliding)}"
|
||||
)
|
||||
|
||||
|
||||
def test_commands_catalog_filters_gateway_only_commands_and_keeps_status_visible():
|
||||
resp = server.handle_request(
|
||||
{"id": "1", "method": "commands.catalog", "params": {}}
|
||||
|
||||
@@ -82,14 +82,58 @@ class TestClarifyPrimitive:
|
||||
assert cm.wait_for_response("id3c", timeout=0.1) == "Y"
|
||||
|
||||
def test_resolve_text_response_accepts_custom_other_text(self):
|
||||
"""Arbitrary typed text should resolve as a custom Other answer."""
|
||||
"""Arbitrary typed text should resolve as a custom Other answer when awaiting_text is True."""
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
cm.register("id3d", "sk3d", "Pick", ["X", "Y"])
|
||||
# Flip to text-capture mode (user picked "Other")
|
||||
cm.mark_awaiting_text("id3d")
|
||||
custom = "None of those are valid options"
|
||||
assert cm.resolve_text_response_for_session("sk3d", custom) is True
|
||||
assert cm.wait_for_response("id3d", timeout=0.1) == custom
|
||||
|
||||
def test_resolve_text_rejects_arbitrary_prose_for_native_multi_choice(self):
|
||||
"""Native interactive multi-choice clarifies reject arbitrary prose unless awaiting_text is True."""
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
# Native multi-choice (buttons, not awaiting text)
|
||||
cm.register("id-strict", "sk-strict", "Pick one", ["A", "B", "C"])
|
||||
|
||||
# Arbitrary prose should be rejected
|
||||
assert cm.resolve_text_response_for_session("sk-strict", "just checking the visual UI") is False
|
||||
assert cm.resolve_text_response_for_session("sk-strict", "present 3 buttons") is False
|
||||
|
||||
# Numeric choices should still work
|
||||
assert cm.resolve_text_response_for_session("sk-strict", "2") is True
|
||||
assert cm.wait_for_response("id-strict", timeout=0.1) == "B"
|
||||
|
||||
# Exact label match should still work
|
||||
cm.register("id-strict2", "sk-strict2", "Pick", ["Option Alpha", "Option Beta"])
|
||||
assert cm.resolve_text_response_for_session("sk-strict2", "Option Alpha") is True
|
||||
assert cm.wait_for_response("id-strict2", timeout=0.1) == "Option Alpha"
|
||||
|
||||
def test_text_fallback_mode_allows_any_text(self):
|
||||
"""Text fallback mode (after base send_clarify calls mark_awaiting_text) accepts any text."""
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
entry = cm.register("id-tf", "sk-tf", "Pick one", ["A", "B", "C"])
|
||||
assert entry.awaiting_text is False
|
||||
|
||||
# Simulate base send_clarify calling mark_awaiting_text
|
||||
cm.mark_awaiting_text("id-tf")
|
||||
assert entry.awaiting_text is True
|
||||
|
||||
# Now arbitrary text is accepted
|
||||
custom = "I choose a custom answer"
|
||||
assert cm.resolve_text_response_for_session("sk-tf", custom) is True
|
||||
assert cm.wait_for_response("id-tf", timeout=0.1) == custom
|
||||
|
||||
# Numeric choices also work
|
||||
cm.register("id-tf2", "sk-tf2", "Pick", ["X", "Y"])
|
||||
cm.mark_awaiting_text("id-tf2")
|
||||
assert cm.resolve_text_response_for_session("sk-tf2", "1") is True
|
||||
assert cm.wait_for_response("id-tf2", timeout=0.1) == "X"
|
||||
|
||||
def test_other_button_flips_to_text_mode(self):
|
||||
"""mark_awaiting_text makes get_pending_for_session find the entry."""
|
||||
from tools import clarify_gateway as cm
|
||||
|
||||
@@ -159,7 +159,7 @@ class TestChildSystemPrompt(unittest.TestCase):
|
||||
class TestStripBlockedTools(unittest.TestCase):
|
||||
def test_removes_blocked_toolsets(self):
|
||||
result = _strip_blocked_tools(["terminal", "file", "delegation", "clarify", "memory", "code_execution"])
|
||||
self.assertEqual(sorted(result), ["file", "terminal"])
|
||||
self.assertEqual(sorted(result), ["code_execution", "file", "terminal"])
|
||||
|
||||
def test_preserves_allowed_toolsets(self):
|
||||
result = _strip_blocked_tools(["terminal", "file", "web", "browser"])
|
||||
@@ -235,10 +235,12 @@ class TestStripBlockedTools(unittest.TestCase):
|
||||
"clarify",
|
||||
"cronjob",
|
||||
"delegation",
|
||||
"code_execution",
|
||||
"memory",
|
||||
):
|
||||
self.assertIn(toolset_name, disabled)
|
||||
# code_execution is deliberately NOT denied — children keep
|
||||
# execute_code for programmatic tool calling (Teknium, Jul 2026).
|
||||
self.assertNotIn("code_execution", disabled)
|
||||
|
||||
definitions = model_tools.get_tool_definitions(
|
||||
enabled_toolsets=kwargs["enabled_toolsets"],
|
||||
@@ -1152,9 +1154,15 @@ class TestSubagentCostRollup(unittest.TestCase):
|
||||
|
||||
class TestBlockedTools(unittest.TestCase):
|
||||
def test_blocked_tools_constant(self):
|
||||
for tool in ["delegate_task", "clarify", "memory", "send_message", "execute_code"]:
|
||||
for tool in ["delegate_task", "clarify", "memory", "send_message", "cronjob"]:
|
||||
self.assertIn(tool, DELEGATE_BLOCKED_TOOLS)
|
||||
|
||||
def test_execute_code_not_blocked(self):
|
||||
"""Children retain execute_code (programmatic tool calling) so they
|
||||
can batch mechanical work instead of burning reasoning iterations
|
||||
(Teknium, Jul 2026)."""
|
||||
self.assertNotIn("execute_code", DELEGATE_BLOCKED_TOOLS)
|
||||
|
||||
def test_constants(self):
|
||||
from tools.delegate_tool import (
|
||||
_get_max_spawn_depth, _get_orchestrator_enabled,
|
||||
|
||||
@@ -638,3 +638,137 @@ class TestCronDemotion:
|
||||
# Interactive rows first, in original relative order; cron last, in
|
||||
# original relative order.
|
||||
assert [r["id"] for r in ordered] == [2, 4, 5, 1, 3]
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Compaction summary filtering (#43175)
|
||||
# =========================================================================
|
||||
|
||||
class TestCompactionSummaryFiltering:
|
||||
"""session_search discovery must exclude compaction handoffs from bookends."""
|
||||
|
||||
def test_is_compaction_summary_detects_prefix(self):
|
||||
from tools.session_search_tool import _is_compaction_summary
|
||||
assert _is_compaction_summary("[CONTEXT COMPACTION — REFERENCE ONLY] foo")
|
||||
assert _is_compaction_summary("[CONTEXT SUMMARY]: old summary")
|
||||
assert not _is_compaction_summary("Hello, how can I help?")
|
||||
assert not _is_compaction_summary("")
|
||||
assert not _is_compaction_summary(None)
|
||||
|
||||
def test_compaction_summary_excluded_from_bookend_start(self, db):
|
||||
"""Compaction handoff in bookend_start position must be filtered out."""
|
||||
db.create_session("s_compact", source="cli")
|
||||
# First message: a compaction handoff (should be filtered)
|
||||
db.append_message("s_compact", role="user",
|
||||
content="[CONTEXT COMPACTION — REFERENCE ONLY] "
|
||||
"Earlier turns were compacted into the summary below. " + "x" * 50000)
|
||||
# Second message: normal user message
|
||||
db.append_message("s_compact", role="user", content="Fix the zorgblat rendering bug")
|
||||
# Padding messages to push window away from session start (so bookend has room)
|
||||
for i in range(10):
|
||||
db.append_message("s_compact", role="user", content=f"setup step {i}")
|
||||
db.append_message("s_compact", role="assistant", content=f"setup done {i}")
|
||||
# Match target: uses a unique term so FTS5 anchors here, not at the start
|
||||
db.append_message("s_compact", role="user", content="investigate the frobnitz mob spawning in KubeJS")
|
||||
db.append_message("s_compact", role="assistant", content="I'll look into the frobnitz mob spawning issue.")
|
||||
# Tail messages
|
||||
for i in range(5):
|
||||
db.append_message("s_compact", role="user", content=f"tail {i}")
|
||||
db.append_message("s_compact", role="assistant", content=f"done tail {i}")
|
||||
db._conn.commit()
|
||||
|
||||
result = json.loads(session_search(query="frobnitz mob spawning", db=db, limit=1))
|
||||
assert result["success"] is True
|
||||
assert len(result["results"]) >= 1
|
||||
entry = result["results"][0]
|
||||
# bookend_start must NOT contain the compaction handoff
|
||||
for msg in entry.get("bookend_start", []):
|
||||
assert "[CONTEXT COMPACTION" not in (msg.get("content") or "")
|
||||
# The normal message should still be present in bookend_start
|
||||
bookend_contents = [m.get("content", "") for m in entry.get("bookend_start", [])]
|
||||
assert any("zorgblat" in c for c in bookend_contents)
|
||||
|
||||
def test_compaction_summary_excluded_from_bookend_end(self, db):
|
||||
"""Compaction handoff in bookend_end position must be filtered out."""
|
||||
db.create_session("s_compact_end", source="cli")
|
||||
# Normal opening
|
||||
db.append_message("s_compact_end", role="user", content="Build a website")
|
||||
db.append_message("s_compact_end", role="assistant", content="Sure, let me scaffold it.")
|
||||
# Match target (early in session so bookend_end has room)
|
||||
db.append_message("s_compact_end", role="user", content="fix the zorgblat rendering bug")
|
||||
db.append_message("s_compact_end", role="assistant", content="Investigating the zorgblat rendering issue.")
|
||||
# Many messages to create distance from the end
|
||||
for i in range(10):
|
||||
db.append_message("s_compact_end", role="user", content=f"feature {i}")
|
||||
db.append_message("s_compact_end", role="assistant", content=f"implemented {i}")
|
||||
# Last message: compaction handoff (should be filtered from bookend_end)
|
||||
db.append_message("s_compact_end", role="assistant",
|
||||
content="[CONTEXT COMPACTION — REFERENCE ONLY] "
|
||||
"Summary of all work done. " + "y" * 50000)
|
||||
db._conn.commit()
|
||||
|
||||
result = json.loads(session_search(query="zorgblat rendering", db=db, limit=1))
|
||||
assert result["success"] is True
|
||||
assert len(result["results"]) >= 1
|
||||
entry = result["results"][0]
|
||||
# bookend_end must NOT contain the compaction handoff
|
||||
for msg in entry.get("bookend_end", []):
|
||||
assert "[CONTEXT COMPACTION" not in (msg.get("content") or "")
|
||||
|
||||
def test_bookend_content_is_capped(self, db):
|
||||
"""Bookend messages must have content capped at 1200 chars."""
|
||||
db.create_session("s_long_bookend", source="cli")
|
||||
# First message: very long normal content
|
||||
db.append_message("s_long_bookend", role="user",
|
||||
content="Start the project. " + "z" * 5000)
|
||||
# Match target
|
||||
db.append_message("s_long_bookend", role="user", content="deploy to production")
|
||||
db.append_message("s_long_bookend", role="assistant", content="Deploying now.")
|
||||
for i in range(10):
|
||||
db.append_message("s_long_bookend", role="user", content=f"step {i}")
|
||||
db.append_message("s_long_bookend", role="assistant", content=f"done {i}")
|
||||
db._conn.commit()
|
||||
|
||||
result = json.loads(session_search(query="deploy production", db=db, limit=1))
|
||||
assert result["success"] is True
|
||||
entry = result["results"][0]
|
||||
for msg in entry.get("bookend_start", []):
|
||||
content = msg.get("content", "")
|
||||
# Content should be capped (1200 chars + "…" ellipsis)
|
||||
assert len(content) <= 1210 # 1200 + ellipsis + margin
|
||||
if msg.get("content_truncated"):
|
||||
assert msg["original_content_chars"] > 1200
|
||||
|
||||
def test_window_content_is_capped(self, db):
|
||||
"""Window messages must have content capped at 4000 chars."""
|
||||
db.create_session("s_long_window", source="cli")
|
||||
db.append_message("s_long_window", role="user", content="search keyword here")
|
||||
# Very long assistant reply containing the keyword
|
||||
db.append_message("s_long_window", role="assistant",
|
||||
content="Found it! keyword " + "a" * 10000)
|
||||
db._conn.commit()
|
||||
|
||||
result = json.loads(session_search(query="keyword", db=db, limit=1))
|
||||
assert result["success"] is True
|
||||
entry = result["results"][0]
|
||||
for msg in entry.get("messages", []):
|
||||
content = msg.get("content", "")
|
||||
assert len(content) <= 4010 # 4000 + ellipsis + margin
|
||||
|
||||
def test_legacy_context_summary_filtered(self, db):
|
||||
"""Legacy [CONTEXT SUMMARY]: prefix must also be filtered."""
|
||||
db.create_session("s_legacy", source="cli")
|
||||
db.append_message("s_legacy", role="user",
|
||||
content="[CONTEXT SUMMARY]: old compacted summary here")
|
||||
db.append_message("s_legacy", role="user", content="new task: build API")
|
||||
db.append_message("s_legacy", role="assistant", content="Building REST API now.")
|
||||
for i in range(10):
|
||||
db.append_message("s_legacy", role="user", content=f"step {i}")
|
||||
db.append_message("s_legacy", role="assistant", content=f"done {i}")
|
||||
db._conn.commit()
|
||||
|
||||
result = json.loads(session_search(query="build API", db=db, limit=1))
|
||||
assert result["success"] is True
|
||||
entry = result["results"][0]
|
||||
for msg in entry.get("bookend_start", []):
|
||||
assert "[CONTEXT SUMMARY]" not in (msg.get("content") or "")
|
||||
|
||||
@@ -334,3 +334,232 @@ for raw in sys.stdin:
|
||||
assert supervisor.is_running()
|
||||
finally:
|
||||
supervisor.shutdown()
|
||||
|
||||
|
||||
def _make_compress_host_session(events: list) -> dict:
|
||||
class _Agent:
|
||||
model = "host-model"
|
||||
provider = "host-provider"
|
||||
tools = []
|
||||
_cached_system_prompt = ""
|
||||
session_input_tokens = 1
|
||||
session_output_tokens = 1
|
||||
session_prompt_tokens = 1
|
||||
session_completion_tokens = 1
|
||||
session_total_tokens = 2
|
||||
session_api_calls = 1
|
||||
session_id = "rotated-id"
|
||||
|
||||
agent = _Agent()
|
||||
agent.context_compressor = type("ContextEngineStub", (), {})()
|
||||
agent.context_compressor.on_session_start = (
|
||||
lambda *_args, **_kwargs: events.append("notify")
|
||||
)
|
||||
return {
|
||||
"agent": agent,
|
||||
"session_key": "before-key",
|
||||
"history": [
|
||||
{"role": "user", "content": "before"},
|
||||
{"role": "assistant", "content": "before"},
|
||||
],
|
||||
"history_lock": threading.Lock(),
|
||||
"history_version": 2,
|
||||
"running": False,
|
||||
"manual_compression_lock": threading.Lock(),
|
||||
}
|
||||
|
||||
|
||||
def test_compute_host_compress_control_notifies_engine_after_commit(monkeypatch):
|
||||
"""The compute-host slash.compress route must fire the context-engine
|
||||
boundary hook exactly once, and only AFTER the host commits the compressed
|
||||
history + session-key sync (salvaged #65670, extended to this route)."""
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
from tui_gateway import server
|
||||
|
||||
out = io.StringIO()
|
||||
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
|
||||
events: list[str] = []
|
||||
session = _make_compress_host_session(events)
|
||||
|
||||
def _compress(sess, focus_topic=None, **_kwargs):
|
||||
# Simulate agent._compress_context(defer_context_engine_notification=True)
|
||||
_queue_context_engine_compression_notification(
|
||||
sess["agent"],
|
||||
new_session_id="rotated-id",
|
||||
old_session_id="before-key",
|
||||
)
|
||||
with sess["history_lock"]:
|
||||
sess["history"] = [{"role": "summary", "content": "compressed"}]
|
||||
sess["history_version"] = 3
|
||||
|
||||
def _sync(sid, sess):
|
||||
events.append("sync")
|
||||
sess["session_key"] = "after-key"
|
||||
|
||||
server._sessions["sid"] = session
|
||||
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
|
||||
monkeypatch.setattr(server, "_compress_session_history", _compress)
|
||||
monkeypatch.setattr(server, "_sync_session_key_after_compress", _sync)
|
||||
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_session_info",
|
||||
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
|
||||
)
|
||||
|
||||
try:
|
||||
host.handle_frame(
|
||||
{
|
||||
"type": "control",
|
||||
"sid": "sid",
|
||||
"request_id": "compress-1",
|
||||
"route_name": "slash.compress",
|
||||
"command": "/compress",
|
||||
}
|
||||
)
|
||||
ack = _wait_for_frame(
|
||||
out,
|
||||
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compress-1",
|
||||
)
|
||||
finally:
|
||||
server._sessions.pop("sid", None)
|
||||
host.close()
|
||||
|
||||
# Exactly one notification, after the session-key commit.
|
||||
assert events == ["sync", "notify"]
|
||||
assert ack["session_key"] == "after-key"
|
||||
# Nothing pending leaks onto the agent for a later compress to misfire.
|
||||
assert (
|
||||
finalize_context_engine_compression_notification(
|
||||
session["agent"], committed=True
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
|
||||
def test_compute_host_compress_control_failure_discards_notification(monkeypatch):
|
||||
"""When the host-side compress mirror fails after compression queued the
|
||||
boundary notification, the pending hook must be discarded — never left to
|
||||
fire against a boundary the host rejected."""
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
from tui_gateway import server
|
||||
|
||||
out = io.StringIO()
|
||||
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
|
||||
events: list[str] = []
|
||||
session = _make_compress_host_session(events)
|
||||
|
||||
def _compress(sess, focus_topic=None, **_kwargs):
|
||||
_queue_context_engine_compression_notification(
|
||||
sess["agent"],
|
||||
new_session_id="rotated-id",
|
||||
old_session_id="before-key",
|
||||
)
|
||||
|
||||
def _boom(*_args, **_kwargs):
|
||||
raise RuntimeError("synthetic host commit failure")
|
||||
|
||||
server._sessions["sid"] = session
|
||||
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
|
||||
monkeypatch.setattr(server, "_compress_session_history", _compress)
|
||||
monkeypatch.setattr(server, "_sync_session_key_after_compress", _boom)
|
||||
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_session_info",
|
||||
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
|
||||
)
|
||||
|
||||
try:
|
||||
host.handle_frame(
|
||||
{
|
||||
"type": "control",
|
||||
"sid": "sid",
|
||||
"request_id": "compress-2",
|
||||
"route_name": "slash.compress",
|
||||
"command": "/compress",
|
||||
}
|
||||
)
|
||||
ack = _wait_for_frame(
|
||||
out,
|
||||
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compress-2",
|
||||
)
|
||||
finally:
|
||||
server._sessions.pop("sid", None)
|
||||
host.close()
|
||||
|
||||
assert events == []
|
||||
assert "live session sync failed" in str(ack.get("output") or "")
|
||||
# The pending notification was discarded, not left on the agent.
|
||||
assert (
|
||||
finalize_context_engine_compression_notification(
|
||||
session["agent"], committed=True
|
||||
)
|
||||
is False
|
||||
)
|
||||
assert events == []
|
||||
|
||||
|
||||
def test_compute_host_compact_alias_routes_to_compress_mirror(monkeypatch):
|
||||
"""slash.compress control frames forward the user's raw alias verbatim;
|
||||
/compact must reach the compress mirror (and its deferred-notification
|
||||
finalize wiring), not silently no-op."""
|
||||
from agent.conversation_compression import (
|
||||
_queue_context_engine_compression_notification,
|
||||
)
|
||||
from tui_gateway import server
|
||||
|
||||
out = io.StringIO()
|
||||
host = ComputeHost(stdout=out, max_workers=1, heartbeat_secs=0)
|
||||
events: list[str] = []
|
||||
session = _make_compress_host_session(events)
|
||||
calls: dict[str, object] = {}
|
||||
|
||||
def _compress(sess, focus_topic=None, **_kwargs):
|
||||
calls["focus"] = focus_topic
|
||||
_queue_context_engine_compression_notification(
|
||||
sess["agent"],
|
||||
new_session_id="rotated-id",
|
||||
old_session_id="before-key",
|
||||
)
|
||||
|
||||
server._sessions["sid"] = session
|
||||
monkeypatch.setenv("HERMES_COMPUTE_HOST_CHILD", "1")
|
||||
monkeypatch.setattr(server, "_compress_session_history", _compress)
|
||||
monkeypatch.setattr(
|
||||
server, "_sync_session_key_after_compress", lambda *_a: events.append("sync")
|
||||
)
|
||||
monkeypatch.setattr(server, "_emit", lambda *_args, **_kwargs: None)
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_session_info",
|
||||
lambda _agent, _session=None: {"model": "host-model", "usage": {"total": 2}},
|
||||
)
|
||||
|
||||
try:
|
||||
host.handle_frame(
|
||||
{
|
||||
"type": "control",
|
||||
"sid": "sid",
|
||||
"request_id": "compact-1",
|
||||
"route_name": "slash.compress",
|
||||
"command": "/compact focus topic",
|
||||
}
|
||||
)
|
||||
ack = _wait_for_frame(
|
||||
out,
|
||||
lambda f: f.get("type") == "control.ack" and f.get("request_id") == "compact-1",
|
||||
)
|
||||
finally:
|
||||
server._sessions.pop("sid", None)
|
||||
host.close()
|
||||
|
||||
assert calls == {"focus": "focus topic"}
|
||||
assert events == ["sync", "notify"]
|
||||
assert ack["route_name"] == "slash.compress"
|
||||
|
||||
@@ -1289,6 +1289,101 @@ def test_session_branch_persists_branched_from_marker(server, monkeypatch):
|
||||
assert kwargs["model_config"] == {"_branched_from": parent_key}
|
||||
|
||||
|
||||
def test_session_branch_forwards_original_timestamps(server, monkeypatch):
|
||||
"""TUI /branch must copy the parent's messages WITH their original
|
||||
timestamps — append_message otherwise stamps time.time() at INSERT and
|
||||
the branch's whole history silently appears authored "now" (#28841).
|
||||
"""
|
||||
append_calls = []
|
||||
|
||||
class _DB:
|
||||
def get_session_title(self, _key):
|
||||
return "parent-title"
|
||||
|
||||
def get_next_title_in_lineage(self, base):
|
||||
return f"{base} 2"
|
||||
|
||||
def create_session(self, new_key, **kwargs):
|
||||
return new_key
|
||||
|
||||
def append_message(self, **kwargs):
|
||||
append_calls.append(kwargs)
|
||||
return None
|
||||
|
||||
def set_session_title(self, _key, _title):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(server, "_get_db", lambda: _DB())
|
||||
monkeypatch.setattr(server, "_resolve_model", lambda: "test/model")
|
||||
monkeypatch.setattr(server, "_new_session_key", lambda: "20260101_000001_child0")
|
||||
monkeypatch.setattr(
|
||||
server,
|
||||
"_make_agent",
|
||||
lambda _sid, key, session_id=None, session_db=None, **_kwargs: types.SimpleNamespace(
|
||||
model="test/model", session_id=session_id or key
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(server, "_init_session", lambda *_a, **_k: None)
|
||||
monkeypatch.setattr(server, "_set_session_context", lambda *_a, **_k: [])
|
||||
monkeypatch.setattr(server, "_clear_session_context", lambda *_a, **_k: None)
|
||||
monkeypatch.setattr(server, "_session_cwd", lambda _s: "/tmp/branch-cwd")
|
||||
|
||||
original_ts = [1_700_000_000.0, 1_700_000_020.0]
|
||||
parent_sid = "parent02"
|
||||
server._sessions[parent_sid] = {
|
||||
"session_key": "20260101_000000_parent",
|
||||
"history": [
|
||||
{"role": "user", "content": "hello", "timestamp": original_ts[0]},
|
||||
{"role": "assistant", "content": "hi!", "timestamp": original_ts[1]},
|
||||
],
|
||||
"history_lock": threading.Lock(),
|
||||
"cols": 80,
|
||||
}
|
||||
|
||||
resp = server.handle_request(
|
||||
{"id": "b2", "method": "session.branch", "params": {"session_id": parent_sid}}
|
||||
)
|
||||
|
||||
assert "error" not in resp, resp
|
||||
assert len(append_calls) == 2
|
||||
assert [c.get("timestamp") for c in append_calls] == original_ts
|
||||
|
||||
|
||||
def test_persist_branch_seed_forwards_original_timestamps(server, monkeypatch):
|
||||
"""First-turn branch seed persist must carry each copied message's
|
||||
original timestamp through to append_message (#28841)."""
|
||||
import contextlib
|
||||
|
||||
append_calls = []
|
||||
|
||||
class _DB:
|
||||
def append_message(self, **kwargs):
|
||||
append_calls.append(kwargs)
|
||||
return None
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _fake_session_db(_session):
|
||||
yield _DB()
|
||||
|
||||
monkeypatch.setattr(server, "_session_db", _fake_session_db)
|
||||
|
||||
original_ts = [100.0, 200.0]
|
||||
session = {
|
||||
"session_key": "20260101_000002_seed00",
|
||||
"parent_session_id": "20260101_000000_parent",
|
||||
"history": [
|
||||
{"role": "user", "content": "a", "timestamp": original_ts[0]},
|
||||
{"role": "assistant", "content": "b", "timestamp": original_ts[1]},
|
||||
],
|
||||
"history_lock": threading.Lock(),
|
||||
}
|
||||
|
||||
server._persist_branch_seed(session)
|
||||
|
||||
assert session.get("_branch_seed_persisted") is True
|
||||
assert [c.get("timestamp") for c in append_calls] == original_ts
|
||||
|
||||
|
||||
def test_make_agent_accepts_list_system_prompt(server, monkeypatch):
|
||||
captured = {}
|
||||
|
||||
|
||||
+53
-15
@@ -187,30 +187,68 @@ def get_pending_for_session(
|
||||
return None
|
||||
|
||||
|
||||
def _coerce_text_response(entry: _ClarifyEntry, response: str) -> str:
|
||||
"""Map typed choice replies to canonical choice text, otherwise keep custom text."""
|
||||
def _coerce_text_response(entry: _ClarifyEntry, response: str) -> Optional[str]:
|
||||
"""Map typed choice replies to canonical choice text, otherwise keep or reject custom text.
|
||||
|
||||
For native interactive multi-choice clarifies (button UI, awaiting_text=False):
|
||||
- Accept numeric selections ("2" → choice[1])
|
||||
- Accept exact choice label matches (case-insensitive)
|
||||
- Reject arbitrary prose (return None) so the message continues as a normal turn
|
||||
|
||||
For text fallback or awaiting_text mode:
|
||||
- Accept any text (numeric/label/custom) after passing through coercion
|
||||
|
||||
For open-ended clarifies (no choices):
|
||||
- Accept any text
|
||||
|
||||
Returns None when the response should be rejected (arbitrary prose for native multi-choice).
|
||||
"""
|
||||
text = str(response).strip()
|
||||
if entry.choices:
|
||||
try:
|
||||
idx = int(text) - 1
|
||||
except ValueError:
|
||||
idx = -1
|
||||
if 0 <= idx < len(entry.choices):
|
||||
return entry.choices[idx]
|
||||
for choice in entry.choices:
|
||||
if text.casefold() == str(choice).strip().casefold():
|
||||
return str(choice).strip()
|
||||
return text
|
||||
|
||||
if not entry.choices:
|
||||
# Open-ended: accept any text
|
||||
return text
|
||||
|
||||
# Try numeric selection first (always valid for multi-choice)
|
||||
try:
|
||||
idx = int(text) - 1
|
||||
except ValueError:
|
||||
idx = -1
|
||||
|
||||
if 0 <= idx < len(entry.choices):
|
||||
return entry.choices[idx]
|
||||
|
||||
# Try exact choice label match (always valid for multi-choice)
|
||||
for choice in entry.choices:
|
||||
if text.casefold() == str(choice).strip().casefold():
|
||||
return str(choice).strip()
|
||||
|
||||
# For text fallback or awaiting_text mode, accept custom text
|
||||
# For native interactive multi-choice mode, reject arbitrary prose
|
||||
if entry.awaiting_text:
|
||||
return text
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def resolve_text_response_for_session(session_key: str, response: str) -> bool:
|
||||
"""Resolve the oldest pending clarify in ``session_key`` from typed text."""
|
||||
"""Resolve the oldest pending clarify in ``session_key`` from typed text.
|
||||
|
||||
Returns False if no pending clarify exists or if the response was rejected
|
||||
(arbitrary prose for native interactive multi-choice clarifies).
|
||||
"""
|
||||
entry = get_pending_for_session(session_key, include_choice_prompts=True)
|
||||
if entry is None:
|
||||
return False
|
||||
|
||||
coerced = _coerce_text_response(entry, response)
|
||||
if coerced is None:
|
||||
# Response rejected: message should continue as a normal turn
|
||||
return False
|
||||
|
||||
return resolve_gateway_clarify(
|
||||
entry.clarify_id,
|
||||
_coerce_text_response(entry, response),
|
||||
coerced,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -49,7 +49,6 @@ DELEGATE_BLOCKED_TOOLS = frozenset(
|
||||
"clarify", # no user interaction
|
||||
"memory", # no writes to shared MEMORY.md
|
||||
"send_message", # no cross-platform side effects
|
||||
"execute_code", # children should reason step-by-step, not write scripts
|
||||
"cronjob", # no scheduling more work in the parent's name
|
||||
]
|
||||
)
|
||||
@@ -774,7 +773,7 @@ def _strip_blocked_tools(toolsets: List[str]) -> List[str]:
|
||||
"""
|
||||
# Composite toolsets that should never pass through to children, even
|
||||
# though their individual tools aren't all in DELEGATE_BLOCKED_TOOLS.
|
||||
_COMPOSITE_BLOCKED_TOOLSETS = frozenset({"delegation", "code_execution"})
|
||||
_COMPOSITE_BLOCKED_TOOLSETS = frozenset({"delegation"})
|
||||
blocked_toolset_names = {
|
||||
name
|
||||
for name, defn in TOOLSETS.items()
|
||||
@@ -3427,10 +3426,10 @@ def _build_top_level_description() -> str:
|
||||
"status) and verify it yourself — fetch the URL, stat the file, read "
|
||||
"back the content — before telling the user the operation succeeded.\n"
|
||||
"- Leaf subagents (role='leaf', the default) CANNOT call: "
|
||||
"delegate_task, clarify, memory, send_message, execute_code.\n"
|
||||
"delegate_task, clarify, memory, send_message.\n"
|
||||
"- Orchestrator subagents (role='orchestrator') retain "
|
||||
"delegate_task so they can spawn their own workers, but still "
|
||||
"cannot use clarify, memory, send_message, or execute_code. "
|
||||
"cannot use clarify, memory, or send_message. "
|
||||
f"Orchestrators are bounded by max_spawn_depth={max_depth} for this "
|
||||
f"user and can be disabled globally via "
|
||||
"delegation.orchestrator_enabled=false.\n"
|
||||
|
||||
@@ -55,6 +55,16 @@ _DEMOTED_SESSION_SOURCES = ("cron",)
|
||||
# the handful of distinct sessions a typical query returns.
|
||||
_DISCOVER_SCAN_LIMIT = 300
|
||||
|
||||
# Prefixes that identify generated context-compaction handoff summaries.
|
||||
# These are inserted by agent/context_compressor.py as normal user/assistant
|
||||
# messages but contain machine-generated summary metadata — not user content.
|
||||
# They must be excluded from discovery bookends to avoid re-introducing huge
|
||||
# compaction payloads into fresh sessions via session_search. (#43175)
|
||||
_COMPACTION_PREFIXES = (
|
||||
"[CONTEXT COMPACTION",
|
||||
"[CONTEXT SUMMARY]:",
|
||||
)
|
||||
|
||||
|
||||
def _format_timestamp(ts: Union[int, float, str, None]) -> str:
|
||||
"""Convert a Unix timestamp (float/int) or ISO string to a human-readable date.
|
||||
@@ -81,6 +91,15 @@ def _format_timestamp(ts: Union[int, float, str, None]) -> str:
|
||||
return str(ts)
|
||||
|
||||
|
||||
def _is_compaction_summary(content: str) -> bool:
|
||||
"""Return True if *content* looks like a generated compaction handoff."""
|
||||
if not content:
|
||||
return False
|
||||
stripped = content.lstrip()
|
||||
return any(stripped.startswith(p) for p in _COMPACTION_PREFIXES)
|
||||
|
||||
|
||||
|
||||
def _resolve_to_parent(db, session_id: str) -> str:
|
||||
"""Walk parent_session_id chain to the lineage root. Falls back to input on errors."""
|
||||
if not session_id:
|
||||
@@ -142,12 +161,30 @@ def _order_for_recall(raw_results: List[Dict[str, Any]]) -> List[Dict[str, Any]]
|
||||
)
|
||||
|
||||
|
||||
def _shape_message(m: Dict[str, Any], anchor_id: Optional[int] = None) -> Dict[str, Any]:
|
||||
"""Slim a message row for the tool response. Keeps content even if empty."""
|
||||
def _shape_message(
|
||||
m: Dict[str, Any],
|
||||
anchor_id: Optional[int] = None,
|
||||
max_content_len: Optional[int] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Slim a message row for the tool response. Keeps content even if empty.
|
||||
|
||||
When *max_content_len* is set, ``content`` is truncated to that many
|
||||
characters and ``content_truncated`` / ``original_content_chars`` metadata
|
||||
is added so callers know the payload was bounded.
|
||||
"""
|
||||
raw_content = m.get("content")
|
||||
if max_content_len and raw_content and len(raw_content) > max_content_len:
|
||||
content = raw_content[:max_content_len] + "…"
|
||||
truncated = True
|
||||
original_chars = len(raw_content)
|
||||
else:
|
||||
content = raw_content
|
||||
truncated = False
|
||||
original_chars = None
|
||||
entry = {
|
||||
"id": m.get("id"),
|
||||
"role": m.get("role"),
|
||||
"content": m.get("content"),
|
||||
"content": content,
|
||||
"timestamp": m.get("timestamp"),
|
||||
}
|
||||
if m.get("tool_name"):
|
||||
@@ -158,6 +195,9 @@ def _shape_message(m: Dict[str, Any], anchor_id: Optional[int] = None) -> Dict[s
|
||||
entry["tool_call_id"] = m.get("tool_call_id")
|
||||
if anchor_id is not None and m.get("id") == anchor_id:
|
||||
entry["anchor"] = True
|
||||
if truncated:
|
||||
entry["content_truncated"] = True
|
||||
entry["original_content_chars"] = original_chars
|
||||
# Strip None values to keep payload tight, but always keep content
|
||||
# (absent content is meaningful — tool-call-only assistant turns).
|
||||
return {k: v for k, v in entry.items() if v is not None or k in ("content",)}
|
||||
@@ -620,9 +660,17 @@ def _discover(
|
||||
"matched_role": match_info.get("role"),
|
||||
"match_message_id": msg_id,
|
||||
"snippet": match_info.get("snippet") or "",
|
||||
"bookend_start": [_shape_message(m) for m in (view.get("bookend_start") or [])],
|
||||
"messages": [_shape_message(m, anchor_id=msg_id) for m in (view.get("window") or [])],
|
||||
"bookend_end": [_shape_message(m) for m in (view.get("bookend_end") or [])],
|
||||
"bookend_start": [
|
||||
_shape_message(m, max_content_len=1200)
|
||||
for m in (view.get("bookend_start") or [])
|
||||
if not _is_compaction_summary(m.get("content", ""))
|
||||
],
|
||||
"messages": [_shape_message(m, anchor_id=msg_id, max_content_len=4000) for m in (view.get("window") or [])],
|
||||
"bookend_end": [
|
||||
_shape_message(m, max_content_len=1200)
|
||||
for m in (view.get("bookend_end") or [])
|
||||
if not _is_compaction_summary(m.get("content", ""))
|
||||
],
|
||||
"messages_before": view.get("messages_before", 0),
|
||||
"messages_after": view.get("messages_after", 0),
|
||||
}
|
||||
|
||||
+6
-1
@@ -36,6 +36,11 @@ MAX_TODO_ITEMS = 256
|
||||
# before it is parsed and re-injected (see AIAgent._hydrate_todo_store).
|
||||
MAX_TODO_RESULT_CHARS = 512_000
|
||||
_TRUNCATION_MARKER = "… [truncated]"
|
||||
# Persisted as ordinary message content. ContextCompressor uses this stable
|
||||
# header to distinguish the synthetic post-compaction row from a real user.
|
||||
TODO_INJECTION_HEADER = (
|
||||
"[Your active task list was preserved across context compression]"
|
||||
)
|
||||
|
||||
|
||||
class TodoStore:
|
||||
@@ -135,7 +140,7 @@ class TodoStore:
|
||||
if not active_items:
|
||||
return None
|
||||
|
||||
lines = ["[Your active task list was preserved across context compression]"]
|
||||
lines = [TODO_INJECTION_HEADER]
|
||||
for item in active_items:
|
||||
marker = markers.get(item["status"], "[?]")
|
||||
lines.append(f"- {marker} {item['id']}. {item['content']} ({item['status']})")
|
||||
|
||||
@@ -567,6 +567,28 @@ class ComputeHost:
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
if route_name in {"session.compress", "slash.compress"}:
|
||||
# The compress mirror defers the context-engine boundary
|
||||
# notification until the host commits. If anything raises
|
||||
# between queueing and finalize (e.g. building the ack's
|
||||
# session_info), discard the pending notification so it can't
|
||||
# leak onto the agent and fire against a rejected boundary on
|
||||
# a later compress. finalize is exactly-once, so this is a
|
||||
# no-op when the mirror already emitted or discarded it.
|
||||
try:
|
||||
from tui_gateway import server as _server
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
_agent = (_server._sessions.get(sid) or {}).get("agent")
|
||||
if _agent is not None:
|
||||
finalize_context_engine_compression_notification(
|
||||
_agent,
|
||||
committed=False,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
self.emit({"type": "control.error", "sid": sid, "request_id": request_id, "message": str(exc)})
|
||||
|
||||
def _bump_progress(self) -> None:
|
||||
|
||||
+91
-13
@@ -2067,7 +2067,15 @@ def _persist_branch_seed(session: dict) -> None:
|
||||
return
|
||||
try:
|
||||
for msg in seed:
|
||||
db.append_message(session_id=key, role=msg.get("role", "user"), content=msg.get("content"))
|
||||
db.append_message(
|
||||
session_id=key,
|
||||
role=msg.get("role", "user"),
|
||||
content=msg.get("content"),
|
||||
# Preserve the parent's original message timestamps —
|
||||
# append_message would otherwise stamp time.time() and the
|
||||
# branch's copied history would all appear authored "now".
|
||||
timestamp=msg.get("timestamp"),
|
||||
)
|
||||
session["_branch_seed_persisted"] = True
|
||||
except Exception:
|
||||
logger.debug("branch seed persist failed", exc_info=True)
|
||||
@@ -3553,6 +3561,9 @@ def _compress_session_history(
|
||||
before_messages: list | None = None,
|
||||
history_version: int | None = None,
|
||||
) -> tuple[int, dict]:
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
from agent.model_metadata import estimate_request_tokens_rough
|
||||
|
||||
agent = session["agent"]
|
||||
@@ -3581,16 +3592,28 @@ def _compress_session_history(
|
||||
# cached prompt (which already contains the agent identity block)
|
||||
# makes the rebuild append the identity a second time. Mirrors the
|
||||
# CLI's _manual_compress fix for issue #15281.
|
||||
compressed, _ = agent._compress_context(
|
||||
history,
|
||||
None,
|
||||
approx_tokens=approx_tokens,
|
||||
focus_topic=focus_topic or None,
|
||||
)
|
||||
try:
|
||||
compressed, _ = agent._compress_context(
|
||||
history,
|
||||
None,
|
||||
approx_tokens=approx_tokens,
|
||||
focus_topic=focus_topic or None,
|
||||
defer_context_engine_notification=True,
|
||||
)
|
||||
except Exception:
|
||||
finalize_context_engine_compression_notification(
|
||||
agent,
|
||||
committed=False,
|
||||
)
|
||||
raise
|
||||
with session["history_lock"]:
|
||||
if int(session.get("history_version", 0)) != history_version:
|
||||
# External mutation during compaction — drop the compressed
|
||||
# result so we don't clobber concurrent edits.
|
||||
finalize_context_engine_compression_notification(
|
||||
agent,
|
||||
committed=False,
|
||||
)
|
||||
usage = _get_usage(agent)
|
||||
return 0, usage
|
||||
session["history"] = compressed
|
||||
@@ -8992,6 +9015,10 @@ def _(rid, params: dict) -> dict:
|
||||
return _err(
|
||||
rid, 4009, "session busy — /interrupt the current turn before /compress"
|
||||
)
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
sid = params.get("session_id", "")
|
||||
focus_topic = str(params.get("focus_topic", "") or "").strip()
|
||||
try:
|
||||
@@ -9059,6 +9086,10 @@ def _(rid, params: dict) -> dict:
|
||||
)
|
||||
info = _session_info(agent, session)
|
||||
_emit("session.info", sid, info)
|
||||
finalize_context_engine_compression_notification(
|
||||
agent,
|
||||
committed=True,
|
||||
)
|
||||
return _ok(
|
||||
rid,
|
||||
{
|
||||
@@ -9080,6 +9111,10 @@ def _(rid, params: dict) -> dict:
|
||||
# no-op, or raised.
|
||||
_status_update(sid, "ready")
|
||||
except Exception as e:
|
||||
finalize_context_engine_compression_notification(
|
||||
session["agent"],
|
||||
committed=False,
|
||||
)
|
||||
return _err(rid, 5005, str(e))
|
||||
|
||||
|
||||
@@ -9200,6 +9235,7 @@ def _(rid, params: dict) -> dict:
|
||||
session_id=new_key,
|
||||
role=msg.get("role", "user"),
|
||||
content=msg.get("content"),
|
||||
timestamp=msg.get("timestamp"),
|
||||
)
|
||||
db.set_session_title(new_key, title)
|
||||
except Exception as e:
|
||||
@@ -11946,7 +11982,7 @@ def _(rid, params: dict) -> dict:
|
||||
)
|
||||
return _ok(rid, {"key": key, "value": nv})
|
||||
|
||||
if key == "compact":
|
||||
if key == "density":
|
||||
raw = str(value or "").strip().lower()
|
||||
cfg0 = _load_cfg()
|
||||
d0 = cfg0.get("display") if isinstance(cfg0.get("display"), dict) else {}
|
||||
@@ -11958,7 +11994,7 @@ def _(rid, params: dict) -> dict:
|
||||
elif raw == "off":
|
||||
nv_b = False
|
||||
else:
|
||||
return _err(rid, 4002, f"unknown compact value: {value}")
|
||||
return _err(rid, 4002, f"unknown density value: {value}")
|
||||
_write_config_key("display.tui_compact", nv_b)
|
||||
return _ok(rid, {"key": key, "value": "on" if nv_b else "off"})
|
||||
|
||||
@@ -12828,7 +12864,7 @@ def _(rid, params: dict) -> dict:
|
||||
)
|
||||
nv = "full" if dm == "expanded" else "collapsed"
|
||||
return _ok(rid, {"value": nv})
|
||||
if key == "compact":
|
||||
if key == "density":
|
||||
on = bool((_load_cfg().get("display") or {}).get("tui_compact", False))
|
||||
return _ok(rid, {"value": "on" if on else "off"})
|
||||
if key == "theme":
|
||||
@@ -13273,7 +13309,7 @@ _TUI_HIDDEN: frozenset[str] = frozenset(
|
||||
)
|
||||
|
||||
_TUI_EXTRA: list[tuple[str, str, str]] = [
|
||||
("/compact", "Toggle compact display mode", "TUI"),
|
||||
("/density", "Toggle compact display mode", "TUI"),
|
||||
("/logs", "Show recent gateway log lines", "TUI"),
|
||||
(
|
||||
"/mouse",
|
||||
@@ -13342,6 +13378,13 @@ def _(rid, params: dict) -> dict:
|
||||
cat_map[cat].append([c, desc])
|
||||
|
||||
for name, desc, cat in _TUI_EXTRA:
|
||||
# Dedup guard: skip TUI extras that collide with a registry
|
||||
# command or one of its aliases (e.g. the historical /compact
|
||||
# collision, #57133, or /sessions which the registry also
|
||||
# advertises). The registry entry is canonical.
|
||||
if name.lower() in canon:
|
||||
continue
|
||||
canon[name.lower()] = name
|
||||
all_pairs.append([name, desc])
|
||||
if cat not in cat_map:
|
||||
cat_map[cat] = []
|
||||
@@ -13943,6 +13986,10 @@ def _(rid, params: dict) -> dict:
|
||||
return _err(
|
||||
rid, 4009, "session busy — /interrupt the current turn before /compress"
|
||||
)
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
sid = params.get("session_id", "")
|
||||
if _session_uses_compute_host(session):
|
||||
command = f"/{name}" + (f" {arg}" if arg else "")
|
||||
@@ -14016,6 +14063,10 @@ def _(rid, params: dict) -> dict:
|
||||
compression_state=getattr(_agent, "context_compressor", None),
|
||||
)
|
||||
_emit("session.info", sid, _session_info(session.get("agent"), session))
|
||||
finalize_context_engine_compression_notification(
|
||||
_agent,
|
||||
committed=True,
|
||||
)
|
||||
return _ok(
|
||||
rid,
|
||||
{
|
||||
@@ -14026,6 +14077,10 @@ def _(rid, params: dict) -> dict:
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
finalize_context_engine_compression_notification(
|
||||
session["agent"],
|
||||
committed=False,
|
||||
)
|
||||
return _err(rid, 5009, f"compress failed: {exc}")
|
||||
|
||||
return _err(rid, 4018, f"not a quick/plugin/bundle/skill command: {name}")
|
||||
@@ -14498,8 +14553,8 @@ def _(rid, params: dict) -> dict:
|
||||
text_lower = text.lower()
|
||||
extras = [
|
||||
{
|
||||
"text": "/compact",
|
||||
"display": "/compact",
|
||||
"text": "/density",
|
||||
"display": "/density",
|
||||
"meta": "Toggle compact display mode",
|
||||
},
|
||||
{
|
||||
@@ -15006,6 +15061,13 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
|
||||
(parts[1].strip() if len(parts) > 1 else ""),
|
||||
session.get("agent"),
|
||||
)
|
||||
if name == "compact":
|
||||
# /compact is an alias of /compress in every host. The compute-host
|
||||
# slash.compress control forwards the user's raw alias verbatim, so
|
||||
# without normalizing here the child mirror silently no-ops — the
|
||||
# session never compresses and the deferred context-engine
|
||||
# notification wiring below is never exercised for that route.
|
||||
name = "compress"
|
||||
|
||||
# Reject agent-mutating commands during an in-flight turn. These
|
||||
# all do read-then-mutate on live agent/session state that the
|
||||
@@ -15051,6 +15113,9 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
|
||||
# while CLI and gateway both did.
|
||||
from agent.manual_compression_feedback import summarize_manual_compression
|
||||
from agent.model_metadata import estimate_request_tokens_rough
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
with session["history_lock"]:
|
||||
_before_messages = list(session.get("history", []))
|
||||
@@ -15090,6 +15155,10 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
|
||||
_lines = [_fb["headline"], _fb["token_line"]]
|
||||
if _fb.get("note"):
|
||||
_lines.append(_fb["note"])
|
||||
finalize_context_engine_compression_notification(
|
||||
agent,
|
||||
committed=True,
|
||||
)
|
||||
return "\n".join(_lines)
|
||||
elif name == "fast" and agent:
|
||||
mode = arg.lower()
|
||||
@@ -15105,6 +15174,15 @@ def _mirror_slash_side_effects(sid: str, session: dict, command: str) -> str:
|
||||
|
||||
process_registry.kill_all()
|
||||
except Exception as e:
|
||||
if name == "compress" and agent:
|
||||
from agent.conversation_compression import (
|
||||
finalize_context_engine_compression_notification,
|
||||
)
|
||||
|
||||
finalize_context_engine_compression_notification(
|
||||
agent,
|
||||
committed=False,
|
||||
)
|
||||
return f"live session sync failed: {e}"
|
||||
return ""
|
||||
|
||||
|
||||
+1
-1
@@ -225,7 +225,7 @@ The following commands are handled directly by the TUI client. Unrecognized comm
|
||||
|
||||
### Core (`core.ts`)
|
||||
`/help`, `/quit` (alias `/exit`), `/update`, `/clear` (alias `/new`),
|
||||
`/compact`, `/copy`, `/paste`, `/details` (alias `/detail`),
|
||||
`/density`, `/copy`, `/paste`, `/details` (alias `/detail`),
|
||||
`/statusbar` (alias `/sb`), `/queue` (alias `/q`), `/logs`, `/history`,
|
||||
`/save`, `/undo`, `/retry`, `/steer`, `/mouse` (alias `/scroll`),
|
||||
`/status`, `/title`, `/fortune`, `/redraw`, `/terminal-setup`
|
||||
|
||||
@@ -278,19 +278,19 @@ export const coreCommands: SlashCommand[] = [
|
||||
},
|
||||
|
||||
{
|
||||
help: 'toggle compact transcript',
|
||||
name: 'compact',
|
||||
help: 'toggle compact display',
|
||||
name: 'density',
|
||||
run: (arg, ctx) => {
|
||||
const next = flagFromArg(arg, ctx.ui.compact)
|
||||
|
||||
if (next === null) {
|
||||
return ctx.transcript.sys('usage: /compact [on|off|toggle]')
|
||||
return ctx.transcript.sys('usage: /density [on|off|toggle]')
|
||||
}
|
||||
|
||||
patchUiState({ compact: next })
|
||||
ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'compact', value: next ? 'on' : 'off' }).catch(() => {})
|
||||
ctx.gateway.rpc<ConfigSetResponse>('config.set', { key: 'density', value: next ? 'on' : 'off' }).catch(() => {})
|
||||
|
||||
queueMicrotask(() => ctx.transcript.sys(`compact ${next ? 'on' : 'off'}`))
|
||||
queueMicrotask(() => ctx.transcript.sys(`density ${next ? 'on' : 'off'}`))
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
@@ -82,6 +82,9 @@ All compression settings are read from `config.yaml` under the `compression` key
|
||||
compression:
|
||||
enabled: true # Enable/disable compression (default: true)
|
||||
threshold: 0.50 # Fraction of context window (default: 0.50 = 50%)
|
||||
# model_thresholds: # Per-model threshold overrides (substring match,
|
||||
# "glm-5.2": 0.40 # longest key wins). See "Per-model threshold
|
||||
# "claude-sonnet": 0.35 # overrides" below.
|
||||
target_ratio: 0.20 # How much of threshold to keep as tail (default: 0.20)
|
||||
protect_last_n: 20 # Minimum protected tail messages (default: 20)
|
||||
codex_gpt55_autoraise: true # gpt-5.5 on Codex OAuth: raise trigger to 85% (default: true)
|
||||
@@ -101,6 +104,7 @@ auxiliary:
|
||||
| Parameter | Default | Range | Description |
|
||||
|-----------|---------|-------|-------------|
|
||||
| `threshold` | `0.50` | 0.0-1.0 | Compression triggers when prompt tokens ≥ `threshold × context_length` |
|
||||
| `model_thresholds` | `{}` | map | Per-model overrides of `threshold`. Keys are substring-matched against the model name (longest match wins). The small-context floor still applies on top (see below) |
|
||||
| `target_ratio` | `0.20` | 0.10-0.80 | Controls tail protection token budget: `threshold_tokens × target_ratio` |
|
||||
| `protect_last_n` | `20` | ≥1 | Minimum number of recent messages always preserved |
|
||||
| `protect_first_n` | `3` | (hardcoded) | System prompt + first exchange always preserved |
|
||||
@@ -108,6 +112,39 @@ auxiliary:
|
||||
| `codex_gpt55_autoraise_notice` | `true` | bool | Show the one-time Codex gpt-5.5 autoraise notice. Set `false` to keep the 85% autoraise but suppress the banner |
|
||||
| `codex_app_server_auto` | `native` | `native`, `hermes`, `off` | Thread-compaction mode for Codex app-server sessions (see below) |
|
||||
|
||||
### Per-model threshold overrides
|
||||
|
||||
`compression.model_thresholds` lets you trigger compaction at different points
|
||||
depending on the active model — useful when you swap between models with very
|
||||
different context windows (e.g. a 1M-context model can compress later while a
|
||||
128K model should compress earlier):
|
||||
|
||||
```yaml
|
||||
compression:
|
||||
threshold: 0.50
|
||||
model_thresholds:
|
||||
"glm-5.2": 0.40
|
||||
"glm-5.2-1M": 0.25
|
||||
"claude-sonnet": 0.35
|
||||
```
|
||||
|
||||
Resolution rules:
|
||||
|
||||
- Keys are **substring-matched** against the model name; the **longest
|
||||
matching key wins** (`glm-5.2-1M` beats `glm-5.2` for model `glm-5.2-1M`).
|
||||
- When no key matches (or the map is empty), the global `threshold` applies.
|
||||
- The override is re-resolved on every `/model` switch; switching to a model
|
||||
with no matching key falls back to the global `threshold`.
|
||||
- The **small-context floor still applies on top** of overrides (raise-only):
|
||||
models with context windows below 512K are floored at `0.75`, so an
|
||||
override below the floor is raised to `0.75`, while an override above it
|
||||
(e.g. `0.80`) wins.
|
||||
|
||||
Plugin context engines can reuse the same resolution logic via
|
||||
`from agent.context_compressor import resolve_model_threshold`; engines that
|
||||
override `update_model()` own their own compaction policy and may ignore the
|
||||
map.
|
||||
|
||||
### Codex gpt-5.5 threshold autoraise
|
||||
|
||||
The ChatGPT Codex OAuth backend hard-caps gpt-5.5 at a **272K** context window
|
||||
|
||||
@@ -165,7 +165,7 @@ context:
|
||||
engine: "lcm" # must match your engine's name property
|
||||
```
|
||||
|
||||
The `compression` config block (`compression.threshold`, `compression.protect_last_n`, etc.) is specific to the built-in `ContextCompressor`. Your engine should define its own config format if needed, reading from `config.yaml` during initialization.
|
||||
The `compression` config block (`compression.threshold`, `compression.protect_last_n`, etc.) is specific to the built-in `ContextCompressor`, with one explicit exception: `compression.model_thresholds` (per-model threshold overrides) is part of the context-engine contract. The host assigns the resolved map to `engine.model_thresholds` *before* the initial `update_model()` call, and the base-class `update_model()` applies it (longest substring match, falling back to the engine's configured threshold). Engines that override `update_model()` own their own compaction policy and may honor or ignore the map — `from agent.context_compressor import resolve_model_threshold` to reuse the same resolution logic. For everything else, your engine should define its own config format if needed, reading from `config.yaml` during initialization.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -194,7 +194,7 @@ This is often the most efficient pattern: `execute_code` handles the 10+ sequent
|
||||
|
||||
## Inherited Tool Access
|
||||
|
||||
Subagents inherit the parent's enabled toolsets. `delegate_task` does not accept a model-facing `toolsets` parameter, so delegated work cannot grant itself capabilities that the parent does not have. Configure the parent's tools before starting the conversation when a delegated task needs web, terminal, file, or other access. Hermes still strips child-blocked tools such as `clarify`, `memory`, and `execute_code`.
|
||||
Subagents inherit the parent's enabled toolsets. `delegate_task` does not accept a model-facing `toolsets` parameter, so delegated work cannot grant itself capabilities that the parent does not have. Configure the parent's tools before starting the conversation when a delegated task needs web, terminal, file, or other access. Hermes still strips child-blocked tools such as `clarify`, `memory`, and `send_message`; children keep `execute_code` for programmatic tool calling.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -264,7 +264,7 @@ For **durable execution** that must survive session closure or process restart,
|
||||
- Each subagent gets its **own terminal session** (separate from the parent)
|
||||
- Subagents inherit the parent's enabled toolsets; the model cannot select or widen them per call
|
||||
- **Nested delegation is opt-in** — only `role="orchestrator"` children can delegate further, and only when `max_spawn_depth` is raised from its default of 1 (flat). Disable globally with `orchestrator_enabled: false`.
|
||||
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `execute_code`. Orchestrator subagents retain `delegate_task` but still cannot use the other three.
|
||||
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `send_message`, `cronjob`. Orchestrator subagents retain `delegate_task` but keep the other blocks. Both roles retain `execute_code` (programmatic tool calling) so children can batch mechanical work instead of burning reasoning iterations.
|
||||
- **Cancellation follows ownership** — `/stop` or closing/resetting the owning session cancels its background children; synchronous descendants under orchestrators follow their parent's interrupt state
|
||||
- Only the final summary enters the parent's context, keeping token usage efficient
|
||||
- Subagents inherit the parent's **API key, provider configuration, and credential pool** (enabling key rotation on rate limits)
|
||||
|
||||
Reference in New Issue
Block a user