Compare commits

..
Author SHA1 Message Date
emozilla 9f61698369 feat(desktop,cli,docker): serve the desktop renderer at /app on the gateway
The web bridge (previous commit) made the renderer boot in a browser; this
makes an instance actually serve it, end to end, plus the Phase 1 UI gating.

Web bridge + renderer:
- petOverlay/terminal/updates/uninstall/themes become OPTIONAL bridge
  members, absent on web rather than stubbed inert — every caller already
  ?.-guards, so features hide through existing branches (typecheck-verified)
- Gateway settings nav entry absent on web (connection IS the serving
  origin); About hides the Updates section behind a capability check on
  window.hermesDesktop?.updates (version + release notes stay)
- notify requests Notification permission at point of first use
- getVersion reports the renderer package version via a vite define

Serving (web_server.py):
- APP_DIST/mount_app: /app/assets static mount, /app 307-canonicalized to
  /app/ (relative asset URLs), index served with the same bootstrap-global
  injection contract as the dashboard SPA (token in loopback mode, cookie +
  ws-ticket in gated mode); same traversal guard, same auth-gate coverage
- Mounts ONLY when HERMES_APP_DIST is explicitly set: no path fallback, so
  a regular desktop build leaving a dist at apps/desktop/dist can't silently
  enable /app. Env set but dist missing logs a WARNING (a Docker image with
  a broken frontend build should not surface as a mystery 404)

Build + image:
- apps/desktop build:web script: renderer-only build (typecheck + vite),
  no Electron packaging steps
- Dockerfile: desktop manifest joins the cached npm-install layer
  (ELECTRON_SKIP_BINARY_DOWNLOAD=1 keeps the ~100MB binary out), build:web
  appended to the frontend-build layer, HERMES_APP_DIST set at runtime
- .dockerignore: apps/desktop source enters the build context; node_modules/
  dist/build/release re-excluded after the negation (last-match-wins would
  otherwise re-include them — a stale local dist would clobber the
  image-built one via COPY . ., and a Windows node_modules would clobber
  the image's Linux natives)

Verified: build:web dist served at /app/ from a real gateway (token injected
server-side, host:'web', gateway WS connected, zero console errors); mount
contract curl-verified in both directions (no env var = /app falls through
to the dashboard SPA even with a dist present on disk). Image build itself
pending a docker-capable machine.

No-op for every install that doesn't set HERMES_APP_DIST; zero behavior
change for Electron users.
2026-07-11 01:14:35 -04:00
emozilla a014dec94a Merge branch 'main' into feat/desktop-web-bridge 2026-07-10 22:02:52 -04:00
emozilla 09c7854662 feat(desktop): web bridge — run the renderer in a plain browser
Add a web implementation of window.hermesDesktop (src/web-bridge) so the
desktop renderer boots in a browser against a gateway with no Electron.
main.tsx installs it only when no preload has claimed the window; under
Electron it is a no-op.

The bridge is same-origin only and mirrors the dashboard SPA's auth in
both modes: gated/OAuth (session cookie + single-use /api/auth/ws-ticket
mint per connect) and loopback/token (injected session token). The
synthesized connection reports mode 'remote', so fs/git/session surfaces
route to the backend through the same paths the Electron app uses
against a remote gateway. fs/git/clipboard-image/session-window methods
map to existing gateway endpoints; Electron-only surface is either an
inert stub (WEB_STUBBED_SURFACE) or deliberately absent so ?.-guarded
callers hide the feature (WEB_OMITTED_SURFACE), each with its reason.

Both bridges now carry a host marker ('electron' | 'web'; absent means
electron) so UI can gate host-specific surface on one field.

parity.test.ts pins the contract between the two bridges: it loads the
real preload with 'electron' mocked and asserts every preload key is
web-implemented or a registered omission, that registries carry no
stale entries, and that shared namespaces expose the same methods. A
new preload method fails the suite until its web story is decided.

vite.config.ts gains an opt-in dev proxy (HERMES_SPIKE_BACKEND) that
forwards /api (HTTP + WS) to a running 'hermes dashboard' for browser
dev; unset, the Electron dev flow is unchanged.

Verified: tsc -b, eslint, parity suite (6/6), and a live browser boot
against a local dashboard — gateway WS open, sessions loaded, message
streamed end-to-end.
2026-07-10 22:00:31 -04:00
2254 changed files with 30046 additions and 286493 deletions
+19 -3
View File
@@ -66,12 +66,28 @@ runtime/
# ---------- Not needed inside the Docker image ----------
# Desktop app source (Tauri/Electron); never installed in the container.
# apps/shared is the dashboard↔desktop websocket helper and is linked from
# web/package.json as a file: workspace dep — keep it in the build context.
# Desktop app source (Tauri/Electron): only apps/shared and apps/desktop are
# needed in the container.
# - apps/shared is the dashboard↔desktop websocket helper, linked from
# web/package.json as a file: workspace dep.
# - apps/desktop is the desktop renderer, built browser-only inside the image
# (`npm run build:web`) and served at /app (HERMES_APP_DIST). Its local
# build outputs are re-excluded below so a developer's stale dist can't
# overwrite the image-built one via `COPY . .` (same failure mode as
# ui-tui/dist/ above), and release/ (packaged Electron apps, ~GBs) never
# belongs in a build context.
apps/
!apps/shared/
!apps/shared/**
!apps/desktop/
!apps/desktop/**
# Re-exclusions: !apps/desktop/** above re-includes ANYTHING under the tree —
# including node_modules already excluded by the global rule at the top
# (last matching rule wins). Each build output must be re-excluded here.
apps/desktop/node_modules/
apps/desktop/dist/
apps/desktop/build/
apps/desktop/release/
# Test suite — not shipped in production images
tests/
-20
View File
@@ -1,13 +1,6 @@
# Hermes Agent Environment Configuration
# Copy this file to .env and fill in your API keys
# =============================================================================
# LLM PROVIDER (Fireworks AI)
# =============================================================================
# Get your key at: https://app.fireworks.ai/settings/users/api-keys
# Address models directly by catalog ID, e.g.
# accounts/fireworks/models/kimi-k2p6, accounts/fireworks/models/glm-5p2
# FIREWORKS_API_KEY=
# =============================================================================
# LLM PROVIDER (OpenRouter)
# =============================================================================
@@ -115,10 +108,6 @@
# HF_BASE_URL=https://router.huggingface.co/v1 # Override default base URL
# OPENCODE_GO_BASE_URL=https://opencode.ai/zen/go/v1 # Override default base URL
# DeepInfra — 100+ top open models, pay-per-use.
# Get your key at: https://deepinfra.com/dash/api_keys
# DEEPINFRA_API_KEY=
# =============================================================================
# LLM PROVIDER (Qwen OAuth)
# =============================================================================
@@ -136,15 +125,6 @@
# Optional base URL override:
# XIAOMI_BASE_URL=https://api.xiaomimimo.com/v1
# =============================================================================
# LLM PROVIDER (Upstage Solar)
# =============================================================================
# Upstage provides access to Upstage Solar models.
# Get your key at: https://console.upstage.ai/api-keys
# UPSTAGE_API_KEY=your_key_here
# Optional base URL override:
# UPSTAGE_BASE_URL=https://api.upstage.ai/v1
# =============================================================================
# TOOL API KEYS
# =============================================================================
+6 -46
View File
@@ -5,18 +5,12 @@ description: >-
the sub-workflows a PR can affect. Outputs are always "true" on push/dispatch
events and fail open (everything "true") when the diff cannot be computed.
inputs:
github-token:
description: Token for the GitHub API (gh CLI). Pass steps.app-token.outputs.token from the calling workflow.
required: false
default: ${{ github.token }}
outputs:
python:
description: Run Python tests / ruff / ty / windows-footguns.
value: ${{ steps.classify.outputs.python }}
frontend:
description: Run the TypeScript testing matrix + desktop build.
description: Run the TypeScript typecheck matrix + desktop build.
value: ${{ steps.classify.outputs.frontend }}
docker_meta:
description: Docker setup and meta files have changed.
@@ -30,15 +24,9 @@ outputs:
deps:
description: Check pyproject.toml dependency upper bounds.
value: ${{ steps.classify.outputs.deps }}
npm_lock:
description: Post/update the semantic package-lock.json diff PR comment.
value: ${{ steps.classify.outputs.npm_lock }}
mcp_catalog:
description: Require MCP catalog security review label.
value: ${{ steps.classify.outputs.mcp_catalog }}
ci_review:
description: Require CI-sensitive file review label.
value: ${{ steps.classify.outputs.ci_review }}
runs:
using: composite
@@ -47,12 +35,7 @@ runs:
id: classify
shell: bash
env:
# Fall back to the built-in read-only token when the caller passes an
# empty value. Fork PRs get no repo secrets, so AUTOFIX_BOT_PAT is ""
# there, and an input `default:` only applies when the input is omitted,
# not when it's passed empty. Without this fallback the compare API
# fails on forks and the classifier fails open (every lane forced on).
GH_TOKEN: ${{ inputs.github-token || github.token }}
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
EVENT_NAME: ${{ github.event_name }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
@@ -68,33 +51,10 @@ runs:
# event payload instead of the "current PR files" endpoint. The SHAs
# are frozen at trigger time, so the file list is deterministic even
# if the PR receives a new push between trigger and detect.
#
# Retried: a rate-limit blip or eventual-consistency 404 on a
# freshly-pushed HEAD would otherwise silently fall open (all lanes
# run — safe, but wasteful and it masks the API failure).
#
# `.files[]?` (null-safe): with --paginate, a PR more than 100
# commits ahead of its merge-base paginates the compare, and pages
# after the first carry `files: null` — bare `.files[]` makes jq
# die with "cannot iterate over: null", which fails every retry
# and forces the fail-open path (seen on stacked PRs). The full
# file list (up to the API's 300-file cap) is on page one.
CHANGED=""
for i in 1 2 3; do
if CHANGED="$(gh api \
--paginate \
"repos/${REPO}/compare/${BASE_SHA}...${HEAD_SHA}" \
--jq '.files[]?.filename')"; then
break
fi
if [ "$i" = 3 ]; then
echo "::warning::compare API failed after 3 attempts — failing open (all lanes run)"
CHANGED=""
break
fi
echo "::warning::compare API failed (attempt $i); retrying in 10s"
sleep 10
done
CHANGED="$(gh api \
--paginate \
"repos/${REPO}/compare/${BASE_SHA}...${HEAD_SHA}" \
--jq '.files[].filename' || true)"
fi
echo "Changed files:"
-59
View File
@@ -1,59 +0,0 @@
name: Get GitHub App Token
description: >-
Mint a short-lived (1-hour) installation access token from the repo's
GitHub App, replacing the long-lived AUTOFIX_BOT_PAT. App tokens get
5,000 req/hr per installation (vs 1,000 for the default GITHUB_TOKEN)
and are scoped to the App's installation permissions, not a user account.
Falls back to the built-in GITHUB_TOKEN when APP_CLIENT_ID is not set —
this happens on fork PRs where repo secrets are unavailable. The fallback
ensures classification, timings, and review comments still work on
forks (with the lower GITHUB_TOKEN rate limit).
Composite actions cannot access the secrets context directly, so the
calling workflow must pass secrets.APP_CLIENT_ID and secrets.APP_PRIVATE_KEY
as inputs. When both are empty (fork PRs), the fallback fires.
inputs:
client-id:
description: GitHub App Client ID. Pass secrets.APP_CLIENT_ID from the calling workflow.
required: false
default: ''
private-key:
description: GitHub App private key PEM. Pass secrets.APP_PRIVATE_KEY from the calling workflow.
required: false
default: ''
outputs:
token:
description: A GitHub App installation access token (1-hour TTL), or GITHUB_TOKEN on forks.
value: ${{ steps.app-token.outputs.token || steps.fallback.outputs.token }}
runs:
using: composite
steps:
- name: Check if App credentials exist
id: check
shell: bash
env:
CLIENT_ID: ${{ inputs.client-id }}
run: |
if [ -n "$CLIENT_ID" ]; then
echo "has_app=true" >> "$GITHUB_OUTPUT"
else
echo "has_app=false" >> "$GITHUB_OUTPUT"
fi
- name: Create GitHub App token
id: app-token
if: steps.check.outputs.has_app == 'true'
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ inputs.client-id }}
private-key: ${{ inputs.private-key }}
- name: Fall back to GITHUB_TOKEN
id: fallback
if: steps.check.outputs.has_app != 'true'
shell: bash
run: echo "token=${{ github.token }}" >> "$GITHUB_OUTPUT"
+3 -23
View File
@@ -3,8 +3,7 @@ description: >-
Run a shell command, retrying on non-zero exit. For dependency installs
(npm ci, uv sync) whose only failures are transient network/toolchain
flakes — a node-gyp header fetch, a registry blip — so CI self-heals
instead of needing a manual re-run. Can also capture stdout as a step
output for commands whose result must be consumed by later steps.
instead of needing a manual re-run.
inputs:
command:
@@ -20,16 +19,10 @@ inputs:
description: Directory to run in.
default: "."
outputs:
stdout:
description: Captured stdout from the successful attempt (empty if not needed).
value: ${{ steps.retry.outputs.stdout }}
runs:
using: composite
steps:
- id: retry
shell: bash
- shell: bash
working-directory: ${{ inputs.working-directory }}
# command goes through env, never interpolated into the script body, so
# a command with quotes/specials can't break or inject into the runner.
@@ -39,25 +32,12 @@ runs:
_DELAY: ${{ inputs.delay }}
run: |
set -uo pipefail
_OUTFILE="$(mktemp)"
trap 'rm -f "$_OUTFILE"' EXIT
n=0
while :; do
n=$((n + 1))
echo "::group::attempt $n/$_ATTEMPTS: $_CMD"
# Run the command, capturing stdout to a temp file while still
# streaming to the log. We redirect first, then tee the file to
# stdout — this avoids pipefail + tee exit-code interactions that
# can cause the if-branch to be skipped under set -e.
if bash -c "$_CMD" > "$_OUTFILE"; then
cat "$_OUTFILE"
if bash -c "$_CMD"; then
echo "::endgroup::"
# Preserve newlines in the output via heredoc delimiter.
{
echo 'stdout<<__RETRY_STDOUT_EOF__'
cat "$_OUTFILE"
echo '__RETRY_STDOUT_EOF__'
} >> "$GITHUB_OUTPUT"
exit 0
fi
echo "::endgroup::"
+16 -177
View File
@@ -17,7 +17,7 @@ on:
permissions:
contents: read
pull-requests: write # needed by lint (PR comment) + supply-chain review_status
pull-requests: write # needed by lint (PR comment) + supply-chain (PR comment)
actions: read # needed by osv-scanner (SARIF upload)
security-events: write # needed by osv-scanner (SARIF upload)
packages: write # needed by docker build
@@ -35,33 +35,20 @@ jobs:
detect:
name: Detect affected areas
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
python: ${{ steps.classify.outputs.python }}
frontend: ${{ steps.classify.outputs.frontend }}
site: ${{ steps.classify.outputs.site }}
scan: ${{ steps.classify.outputs.scan }}
deps: ${{ steps.classify.outputs.deps }}
npm_lock: ${{ steps.classify.outputs.npm_lock }}
docker_meta: ${{ steps.classify.outputs.docker_meta }}
mcp_catalog: ${{ steps.classify.outputs.mcp_catalog }}
ci_review: ${{ steps.classify.outputs.ci_review }}
event_name: ${{ github.event_name }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Detect affected areas
id: classify
uses: ./.github/actions/detect-changes
with:
# The get-app-token composite action falls back to GITHUB_TOKEN
# on fork PRs where APP_ID is unavailable.
github-token: ${{ steps.app-token.outputs.token }}
# ─────────────────────────────────────────────────────────────────────
# Lane-gated sub-workflows. Each runs in parallel after detect finishes.
@@ -74,7 +61,6 @@ jobs:
uses: ./.github/workflows/tests.yml
with:
slice_count: 8
secrets: inherit
lint:
name: Python lints
@@ -83,61 +69,41 @@ jobs:
uses: ./.github/workflows/lint.yml
with:
event_name: ${{ needs.detect.outputs.event_name }}
secrets: inherit
js-tests:
name: JS & TS checks
typecheck:
name: TypeScript
needs: detect
if: needs.detect.outputs.frontend == 'true'
uses: ./.github/workflows/js-tests.yml
secrets: inherit
e2e-desktop:
name: Desktop E2E
needs: detect
if: needs.detect.outputs.python == 'true' || needs.detect.outputs.frontend == 'true'
uses: ./.github/workflows/e2e-desktop.yml
uses: ./.github/workflows/typecheck.yml
docs-site:
name: Docs Site
needs: detect
if: needs.detect.outputs.site == 'true'
uses: ./.github/workflows/docs-site-checks.yml
secrets: inherit
history-check:
name: Deny unrelated histories
needs: detect
if: needs.detect.outputs.event_name == 'pull_request'
uses: ./.github/workflows/history-check.yml
secrets: inherit
contributor-check:
name: Check contributors
needs: detect
if: needs.detect.outputs.python == 'true'
uses: ./.github/workflows/contributor-check.yml
secrets: inherit
uv-lockfile:
name: Check uv.lock
needs: detect
uses: ./.github/workflows/uv-lockfile-check.yml
secrets: inherit
lockfile-diff:
name: package-lock.json diff
needs: detect
if: needs.detect.outputs.event_name == 'pull_request' && needs.detect.outputs.npm_lock == 'true'
uses: ./.github/workflows/lockfile-diff.yml
secrets: inherit
docker-lint:
name: Lint Docker scripts
needs: detect
if: needs.detect.outputs.docker_meta == 'true'
uses: ./.github/workflows/docker-lint.yml
secrets: inherit
docker:
name: Build&Test Docker image
@@ -149,103 +115,17 @@ jobs:
supply-chain:
name: Supply-chain scan
needs: detect
if: needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.scan == 'true' || needs.detect.outputs.deps == 'true')
if: needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.scan == 'true' || needs.detect.outputs.deps == 'true' || needs.detect.outputs.mcp_catalog == 'true')
uses: ./.github/workflows/supply-chain-audit.yml
with:
event_name: ${{ needs.detect.outputs.event_name }}
scan: ${{ needs.detect.outputs.scan == 'true' }}
deps: ${{ needs.detect.outputs.deps == 'true' }}
review-labels:
name: Review label gate
needs: [detect, supply-chain]
if: always() && needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.ci_review == 'true' || needs.detect.outputs.mcp_catalog == 'true' || needs.supply-chain.outputs.critical_findings == 'true')
uses: ./.github/workflows/review-labels.yml
with:
ci_review: ${{ needs.detect.outputs.ci_review == 'true' }}
mcp_catalog: ${{ needs.detect.outputs.mcp_catalog == 'true' }}
supply_chain: ${{ needs.supply-chain.outputs.critical_findings == 'true' }}
secrets: inherit
osv-scanner:
name: OSV scan
uses: ./.github/workflows/osv-scanner.yml
secrets: inherit
# ─────────────────────────────────────────────────────────────────────
# Live-updating PR review comment.
#
# A single ``comment-live`` job polls the GitHub Actions API every 15s
# for job statuses in this run, re-assembles the review comment from
# whatever results are available, and upserts it via the
# ``<!-- hermes-ci-review-bot -->`` marker.
#
# The poller exits when all non-infra jobs are completed (or on
# timeout). ci-timings' review_status is picked up automatically when
# its artifact becomes available — the poller downloads and merges it.
# ─────────────────────────────────────────────────────────────────────
comment-live:
name: CI review comment (live)
needs: [detect, review-labels, lockfile-diff, supply-chain, osv-scanner, uv-lockfile, history-check, contributor-check]
if: always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork != true
runs-on: ubuntu-latest
timeout-minutes: 40
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Run live comment poller
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
PR_NUMBER: ${{ github.event.pull_request.number }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# Commit info for the review comment header.
COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
COMMIT_MESSAGE: ${{ github.event.pull_request.head.commit.message }}
COMMIT_URL: ${{ github.server_url }}/${{ github.repository }}/pull/${{ github.event.pull_request.number }}/commits/${{ github.event.pull_request.head.sha }}
# Structured review statuses from workflow_call jobs.
# Each job outputs a JSON array of {source, results: [...]} objects
# that the assembler renders directly — no hardcoded job-name
# matching. We merge all available outputs into one array.
REVIEW_STATUSES: ${{ toJSON(needs.*.outputs.review_status) }}
run: |
set -uo pipefail
# REVIEW_STATUSES is a JSON array of strings (some may be empty
# when a job was skipped). Parse each string and merge into one
# flat array for the assembler.
python3 - <<'PYEOF'
import json, os, sys
raw = os.environ.get("REVIEW_STATUSES", "")
merged = []
if raw:
try:
arr = json.loads(raw)
except (json.JSONDecodeError, TypeError):
arr = []
for item in arr:
if not item:
continue
try:
statuses = json.loads(item)
except (json.JSONDecodeError, TypeError):
continue
if isinstance(statuses, list):
merged.extend(statuses)
# Write merged array to a temp file the poller reads.
with open("/tmp/review_statuses.json", "w") as f:
json.dump(merged, f)
print(f"Merged {len(merged)} review status entries")
PYEOF
python3 scripts/ci/live_comment.py \
--interval 15 \
--timeout 2100 \
--review-statuses-file /tmp/review_statuses.json
# ─────────────────────────────────────────────────────────────────────
# Gate: runs after everything. ``if: always()`` ensures it reports a
@@ -253,56 +133,35 @@ jobs:
# results cause it to fail; ``skipped`` is treated as success.
#
# Branch protection should require ONLY this check.
#
# Outputs ``needs-json`` — a compact ``{job_name: result}`` dict — so
# the live comment poller can list failed jobs in the PR comment.
# ─────────────────────────────────────────────────────────────────────
all-checks-pass:
name: All required checks pass
needs:
- detect
- tests
- lint
- js-tests
- e2e-desktop
- typecheck
- docs-site
- history-check
- contributor-check
- uv-lockfile
- lockfile-diff
- docker-lint
- supply-chain
- review-labels
- osv-scanner
# comment-live is a polling job — it doesn't block the gate.
# we don't require docker to pass rn because it's so slow lol
# - docker
if: always()
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
needs-json: ${{ steps.evaluate.outputs.needs-json }}
steps:
- name: Evaluate job results
id: evaluate
env:
NEEDS: ${{ toJSON(needs) }}
RESULTS: ${{ toJSON(needs.*.result) }}
run: |
echo "$NEEDS" | python3 -c "
echo "$RESULTS" | python3 -c "
import json, sys
needs = json.load(sys.stdin)
# Emit compact {job_name: result} for the comment assembler.
compact = {name: info['result'] for name, info in needs.items()}
print(f'needs-json={json.dumps(compact)}')
with open('$GITHUB_OUTPUT', 'a') as f:
f.write(f'needs-json={json.dumps(compact)}\n')
failed = [name for name, info in needs.items() if info['result'] == 'failure']
for name, info in sorted(needs.items()):
result = info['result']
icon = '✅' if result in ('success', 'skipped') else '❌'
print(f'{icon} {name}: {result}')
results = json.load(sys.stdin)
failed = [r for r in results if r == 'failure']
if failed:
print(f'::error::{len(failed)} job(s) failed: {\", \".join(failed)}')
print(f'::error::{len(failed)} job(s) failed')
sys.exit(1)
print('All checks passed (or were skipped)')
"
@@ -312,27 +171,16 @@ jobs:
# cache them on main (as a baseline), and on PRs generate an HTML diff
# report with a gantt chart + per-step breakdown. The report is uploaded
# as an artifact and a markdown summary is written to $GITHUB_STEP_SUMMARY.
#
# The live comment poller picks up ci-timings' completion automatically —
# it reads review-status.json from the artifact when the job finishes.
# ─────────────────────────────────────────────────────────────────────
ci-timings:
name: CI timing report
needs: [all-checks-pass, docker]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Restore baseline cache (PR only)
if: github.event_name == 'pull_request'
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -346,31 +194,22 @@ jobs:
- name: Collect timings and generate report
env:
# Forks get no repo secrets (AUTOFIX_BOT_PAT is empty); fall back to
# the built-in read-only token so the timings API read still works
# there instead of hard-failing this advisory job on every fork PR.
# The get-app-token composite action falls back to GITHUB_TOKEN
# on fork PRs where APP_ID is unavailable.
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 scripts/ci/timings_report.py \
--baseline ci-timings-baseline.json \
--output ci-timings-report.html \
--json-out ci-timings.json \
--summary-out ci-timings-summary.md \
--review-status-out review-status.json
--summary-out ci-timings-summary.md
- name: Upload HTML report + review status
# Advisory report — artifact-service blips must not fail the job.
continue-on-error: true
- name: Upload HTML report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
id: ci-timings-artifact
with:
name: ci-timings-report
path: |
ci-timings-report.html
review-status.json
path: ci-timings-report.html
retention-days: 14
archive: false
- name: Output summary
env:
+7 -31
View File
@@ -2,10 +2,6 @@ name: Contributor Attribution Check
on:
workflow_call:
outputs:
review_status:
description: "JSON array of review status objects"
value: ${{ jobs.check-attribution.outputs.review_status }}
permissions:
contents: read
@@ -13,16 +9,12 @@ permissions:
jobs:
check-attribution:
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
review_status: ${{ steps.check-emails.outputs.review_status }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0 # Full history needed for git log
- name: Check for unmapped contributor emails
id: check-emails
run: |
# Get the merge base between this PR and main
MERGE_BASE=$(git merge-base origin/main HEAD)
@@ -32,13 +24,10 @@ jobs:
if [ -z "$NEW_EMAILS" ]; then
echo "No new commits to check."
echo "review_status=[]" >> "$GITHUB_OUTPUT"
exit 0
fi
# An email is mapped if it has a file in contributors/emails/
# (one file per email — conflict-free) or an entry in the frozen
# legacy AUTHOR_MAP in scripts/release.py.
# Check each email against AUTHOR_MAP in release.py
MISSING=""
while IFS= read -r email; do
# Skip teknium and bot emails
@@ -47,12 +36,9 @@ jobs:
continue ;;
esac
# Check if email is in AUTHOR_MAP (either as a key or matches noreply pattern)
if echo "$email" | grep -qP '\+.*@users\.noreply\.github\.com'; then
continue # GitHub id+login noreply emails auto-resolve
fi
if [ -f "contributors/emails/${email}" ]; then
continue # mapped via the contributors directory
continue # GitHub noreply emails auto-resolve
fi
if ! grep -qF "\"${email}\"" scripts/release.py 2>/dev/null; then
@@ -63,29 +49,19 @@ jobs:
if [ -n "$MISSING" ]; then
echo ""
echo "⚠️ New contributor email(s) without a mapping:"
echo "⚠️ New contributor email(s) not in AUTHOR_MAP:"
echo -e "$MISSING"
echo ""
echo "Add a mapping file (do NOT edit AUTHOR_MAP in release.py):"
echo "Please add mappings to scripts/release.py AUTHOR_MAP:"
echo -e "$MISSING" | while read -r line; do
email=$(echo "$line" | sed 's/^ *//' | cut -d' ' -f1)
[ -z "$email" ] && continue
echo " python3 scripts/add_contributor.py ${email} <github-username>"
echo " \"${email}\": \"<github-username>\","
done
echo ""
echo "To find the GitHub username for an email:"
echo " gh api 'search/users?q=EMAIL+in:email' --jq '.items[0].login'"
# Emit review_status for unmapped emails
DETAIL=$(echo -e "$MISSING" | sed '/^$/d; s/^ //')
HOW_TO_FIX=$'Add mappings to scripts/release.py AUTHOR_MAP:\n```\n"<email>": "<github-username>",\n```\nTo find the GitHub username for an email:\n```\ngh api \'search/users?q=EMAIL+in:email\' --jq \'.items[0].login\'\n```\n'
REVIEW_STATUS=$(jq -nc \
--arg detail "$DETAIL" \
--arg how_to_fix "$HOW_TO_FIX" \
'[{"source":"contributor attribution","results":[{"kind":"action_required","title":"Unmapped contributor email(s)","summary":"New contributor email(s) are not in AUTHOR_MAP.","detail":$detail,"how_to_fix":$how_to_fix}]}]')
echo "review_status=$REVIEW_STATUS" >> "$GITHUB_OUTPUT"
exit 1
else
echo "✅ All contributor emails are mapped."
echo "✅ All contributor emails are mapped in AUTHOR_MAP."
fi
+6 -19
View File
@@ -41,28 +41,19 @@ jobs:
# doesn't auto-deploy via the deploy-docs path.
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Trigger Vercel Deploy
run: curl -fsS --retry 3 --retry-delay 10 -X POST "${{ secrets.VERCEL_DEPLOY_HOOK }}"
run: curl -X POST "${{ secrets.VERCEL_DEPLOY_HOOK }}"
deploy-docs:
if: github.repository == 'NousResearch/hermes-agent'
runs-on: ubuntu-latest
timeout-minutes: 30
environment:
name: github-pages
url: ${{ steps.deploy.outputs.page_url }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
@@ -74,14 +65,12 @@ jobs:
python-version: '3.11'
- name: Install PyYAML for skill extraction
uses: ./.github/actions/retry
with:
command: pip install pyyaml==6.0.2 httpx==0.28.1
run: pip install pyyaml==6.0.2 httpx==0.28.1
- name: Prepare skills index (unified multi-source catalog)
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GH_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ github.token }}
SKILLS_INDEX_RUN_ID: ${{ github.event.inputs.skills_index_run_id || '' }}
REBUILD_SKILLS_INDEX: ${{ github.event.inputs.rebuild_skills_index || 'false' }}
run: |
@@ -161,10 +150,8 @@ jobs:
run: python3 website/scripts/generate-skill-docs.py
- name: Install dependencies
uses: ./.github/actions/retry
with:
command: npm ci
working-directory: website
run: npm ci
working-directory: website
- name: Build Docusaurus
run: npm run build
+13 -22
View File
@@ -127,13 +127,12 @@ jobs:
run: uv python install 3.11
- name: Install Python dependencies (for docker tests)
# ``dev`` extra pulls in pytest, pytest-asyncio —
# everything tests/docker/ needs. We deliberately avoid ``all``
# here because the docker tests only drive the container via
# subprocess and don't import hermes_agent's optional deps.
uses: ./.github/actions/retry
with:
command: uv sync --locked --python 3.11 --extra dev
run: |
# ``dev`` extra pulls in pytest, pytest-asyncio —
# everything tests/docker/ needs. We deliberately avoid ``all``
# here because the docker tests only drive the container via
# subprocess and don't import hermes_agent's optional deps.
uv sync --locked --python 3.11 --extra dev
- name: Run docker integration tests
env:
@@ -189,23 +188,15 @@ jobs:
args+=("${IMAGE_NAME}@sha256:${digest_file}")
done
if [ "${{ github.event_name }}" = "release" ]; then
tags=(-t "${IMAGE_NAME}:${RELEASE_TAG}")
docker buildx imagetools create \
-t "${IMAGE_NAME}:${RELEASE_TAG}" \
"${args[@]}"
else
tags=(-t "${IMAGE_NAME}:main" -t "${IMAGE_NAME}:latest")
docker buildx imagetools create \
-t "${IMAGE_NAME}:main" \
-t "${IMAGE_NAME}:latest" \
"${args[@]}"
fi
# Retry: Docker Hub API + just-pushed digest eventual consistency
# can transiently fail the create; the operation is idempotent.
for i in 1 2 3; do
if docker buildx imagetools create "${tags[@]}" "${args[@]}"; then
break
fi
if [ "$i" = 3 ]; then
echo "::error::imagetools create failed after 3 attempts"
exit 1
fi
echo "::warning::imagetools create failed (attempt $i); retrying in 20s"
sleep 20
done
- name: Inspect image
env:
-1
View File
@@ -9,7 +9,6 @@ permissions:
jobs:
docs-site-checks:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-212
View File
@@ -1,212 +0,0 @@
name: E2E Desktop
on:
workflow_call:
permissions:
contents: read
concurrency:
group: e2e-desktop-${{ github.ref }}
cancel-in-progress: true
jobs:
e2e:
name: Playwright E2E (Linux)
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# ── System deps for Electron on headless Ubuntu ───────────────────
# Electron needs GTK, NSS,atk, etc. even under xvfb. Playwright's
# install-deps covers browsers; for Electron we install the apt
# packages directly.
- name: Install system dependencies for Electron
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq \
xvfb \
libgtk-3-0 libnotify4 libnss3 libxss1 libxtst6 \
xdg-utils libatspi2.0-0 libdrm2 libgbm1 libasound2t64
# ── Node ───────────────────────────────────────────────────────────
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
# Full npm ci (not --ignore-scripts): electron's postinstall
# downloads the binary we launch, and node-pty's native build is
# needed for the terminal pane.
- uses: ./.github/actions/retry
with:
command: npm ci
# ── Python (for the hermes serve backend) ──────────────────────────
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # 8.2.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python 3.11
run: uv python install 3.11
- name: Install Python dependencies
uses: ./.github/actions/retry
with:
command: uv sync --locked --python 3.11 --extra all --extra dev
# ── Build desktop app ─────────────────────────────────────────────
- run: npm run --prefix apps/desktop build
# ── Restore visual baseline screenshots from main ──────────────────
# Baselines are generated on main (via --update-snapshots) and cached.
# On PRs, we restore them so toHaveScreenshot has something to compare
# against. The cache key is keyed on the desktop source files so a
# UI change naturally invalidates it — but we fall back to the main
# cache to avoid cold starts on unrelated PRs.
- name: Restore visual baseline screenshots
id: restore-baselines
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: apps/desktop/e2e/*-snapshots
key: visual-baselines-${{ github.ref_name }}
restore-keys: |
visual-baselines-main
# ── Run Playwright E2E under xvfb ─────────────────────────────────
# xvfb runs at a fixed 1280x1024 screen so the 1220x800 Electron
# window always has a consistent viewport for screenshot comparison.
# On main, we run with --update-snapshots to generate baselines.
- name: Run Playwright E2E tests
working-directory: apps/desktop
run: |
if [ "${{ github.ref_name }}" = "main" ]; then
echo "On main — generating/updating baseline screenshots"
xvfb-run -a --server-args="-screen 0 1280x1024x24" \
npx playwright test --reporter=list --update-snapshots
else
echo "On PR — comparing against cached baselines"
xvfb-run -a --server-args="-screen 0 1280x1024x24" \
npx playwright test --reporter=list
fi
env:
CI: "true"
# Ensure no real API keys leak into the test env.
OPENROUTER_API_KEY: ""
OPENAI_API_KEY: ""
NOUS_API_KEY: ""
# ── Save updated baselines to cache (main only) ───────────────────
- name: Save updated baselines to cache
if: github.ref_name == 'main' && always()
uses: actions/cache/save@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: apps/desktop/e2e/*-snapshots
key: visual-baselines-main
# ── Upload Playwright report (HTML + traces) ──────────────────────
- name: Upload Playwright report
id: upload-report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-${{ github.sha }}
path: apps/desktop/playwright-report
retention-days: 14
overwrite: true
# ── Upload test results (screenshots, traces, diffs) ───────────────
- name: Upload test results
id: upload-results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-test-results-${{ github.sha }}
path: apps/desktop/test-results
retention-days: 14
overwrite: true
# ── Upload just the visual diffs (small, fast to review) ──────────
- name: Upload visual diffs
id: upload-diffs
if: always() && github.ref_name != 'main'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: visual-diffs-${{ github.sha }}
path: |
apps/desktop/test-results/**/*-diff.png
apps/desktop/test-results/**/*-actual.png
apps/desktop/test-results/**/*-expected.png
retention-days: 14
overwrite: true
if-no-files-found: ignore
# ── Generate step summary with visual diff info ───────────────────
# Parse the JSON report + scan for diff images, then post a summary
# to the GitHub Actions step output so reviewers can see what changed
# without downloading artifacts. Runs AFTER uploads so it can link
# the artifact download URLs from their step outputs.
- name: Generate visual diff summary
if: always()
working-directory: apps/desktop
env:
REPORT_URL: ${{ steps.upload-report.outputs.artifact-url }}
RESULTS_URL: ${{ steps.upload-results.outputs.artifact-url }}
DIFFS_URL: ${{ steps.upload-diffs.outputs.artifact-url }}
run: |
echo "## Desktop E2E — Visual Diff Report" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
# Count diff images (playwright writes *-diff.png on mismatch)
DIFF_COUNT=$(find test-results -name '*-diff.png' 2>/dev/null | wc -l)
ACTUAL_COUNT=$(find test-results -name '*-actual.png' 2>/dev/null | wc -l)
if [ "$DIFF_COUNT" -eq 0 ]; then
echo "✅ All $ACTUAL_COUNT screenshot(s) matched their baselines (or no baselines existed yet)." >> "$GITHUB_STEP_SUMMARY"
else
echo "📸 **$DIFF_COUNT of $ACTUAL_COUNT screenshot(s) differ from baseline:**" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Test | Diff | Actual | Expected |" >> "$GITHUB_STEP_SUMMARY"
echo "|------|------|--------|----------|" >> "$GITHUB_STEP_SUMMARY"
# List each diff image with a link to the artifact
for diff in $(find test-results -name '*-diff.png' 2>/dev/null | sort); do
base=$(echo "$diff" | sed 's/-diff\.png$//')
test_name=$(basename "$base")
echo "| $test_name | [diff]($diff) | [actual](${base}-actual.png) | [expected](${base}-expected.png) |" >> "$GITHUB_STEP_SUMMARY"
done
fi
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "📥 **Artifacts:**" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
if [ -n "$RESULTS_URL" ]; then
echo "- [playwright-test-results]($RESULTS_URL) — all screenshots (actual + expected + diff) + traces" >> "$GITHUB_STEP_SUMMARY"
fi
if [ -n "$REPORT_URL" ]; then
echo "- [playwright-report]($REPORT_URL) — interactive HTML report" >> "$GITHUB_STEP_SUMMARY"
fi
if [ -n "$DIFFS_URL" ]; then
echo "- [visual-diffs]($DIFFS_URL) — just the diffed screenshots (small, fast to review)" >> "$GITHUB_STEP_SUMMARY"
fi
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "**To update baselines:** merge to main (baselines auto-update on main runs) or run \`npx playwright test --update-snapshots\` locally." >> "$GITHUB_STEP_SUMMARY"
# Also parse the JSON report for pass/fail counts
if [ -f playwright-report/results.json ]; then
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "### Test Results" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
node -e "
const r = require('./playwright-report/results.json');
const stats = r.stats || {};
console.log('| Status | Count |');
console.log('|--------|-------|');
console.log('| ✅ Passed | ' + (stats.expected || 0) + ' |');
console.log('| ❌ Failed | ' + (stats.unexpected || 0) + ' |');
console.log('| ⏭️ Skipped | ' + (stats.skipped || 0) + ' |');
console.log('| 🔄 Flaky | ' + (stats.flaky || 0) + ' |');
" >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
fi
+1 -12
View File
@@ -15,10 +15,6 @@ name: History Check
on:
workflow_call:
outputs:
review_status:
description: "JSON array of review_status objects for the synthesizer."
value: ${{ jobs.check-common-ancestor.outputs.review_status }}
permissions:
contents: read
@@ -26,24 +22,18 @@ permissions:
jobs:
check-common-ancestor:
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
review_status: ${{ steps.merge-base-check.outputs.review_status }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0 # full history both sides for merge-base
- id: merge-base-check
name: Reject PRs with no common ancestor on main
- name: Reject PRs with no common ancestor on main
run: |
# `git merge-base` exits non-zero AND prints nothing when the two
# commits share no ancestor. We check both conditions explicitly
# so the failure message is clear regardless of which signal fires
# first.
if ! BASE=$(git merge-base origin/main HEAD 2>/dev/null) || [ -z "$BASE" ]; then
STATUS='[{"source":"unrelated histories","results":[{"kind":"action_required","title":"Unrelated histories","summary":"This PR has no common ancestor with main.","detail":"","how_to_fix":"Rebase your changes onto current main:\n```\ngit fetch origin main\ngit checkout -b fix-branch origin/main\n# re-apply your changes (cherry-pick, copy files, etc.)\ngit push -f origin fix-branch\n```\n"}]}]'
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
echo ""
echo "::error::This PR has no common ancestor with main."
echo ""
@@ -65,4 +55,3 @@ jobs:
exit 1
fi
echo "::notice::Common ancestor with main: $BASE"
echo "review_status=[]" >> "$GITHUB_OUTPUT"
-258
View File
@@ -1,258 +0,0 @@
name: auto-fix lint issues & formatting
# On push to main (or manual trigger), run `npm run fix` on each workspace
# package and apply any changes via a PR.
#
# Fixable lint issues (import sorting, unused imports, curly braces, etc.) are
# auto-corrected on merge so PRs aren't blocked by them. The PR-time eslint
# check in typecheck.yml fails only when un-fixable errors remain.
#
# NOTE: App token pushes DO trigger further workflow runs (unlike
# secrets.GITHUB_TOKEN). The concurrency group (ts-autofix-${{ github.ref }})
# with cancel-in-progress: true prevents an infinite loop — a re-triggered
# run cancels the in-flight one, and since the second run finds no new fixes
# (the first run already applied them), it exits with an empty patch.
#
# ── Security model: two-job split ───────────────────────────────────────────
#
# The eslint process executes repo code (eslint.config.mjs, package.json
# scripts, installed plugins). To prevent a malicious PR from getting arbitrary
# code execution on a runner with push access, the work is split:
#
# 1. generate-patch (unprivileged, contents: read only)
# Checks out, installs deps, runs eslint --fix, produces a .patch artifact.
# Worst case: malicious code runs here on an ephemeral runner with zero
# push permissions.
#
# 2. apply-patch (privileged, contents: write + pull-requests: write)
# Checks out, downloads the patch artifact, applies it, pushes to the
# bot/js-autofix branch, creates/updates a PR, and enables auto-merge.
# This job never runs npm, never installs anything, never executes any
# repo code. The only input it trusts is the patch artifact.
# Skipped entirely when generate-patch reports no fixes (has-fixes != true).
# The PR auto-merges (squash) once CI passes. If CI fails or main moves,
# the PR is auto-closed and the branch deleted — the next run re-applies
# on the current state.
on:
push:
branches: [main]
paths:
- '**/*.js'
- '**/*.cjs'
- '**/*.mjs'
- '**/*.ts'
- '**/*.tsx'
- 'package.json'
- 'package-lock.json'
workflow_dispatch:
permissions:
contents: read # default; apply-patch job overrides to write
concurrency:
group: ts-autofix-${{ github.ref }}
cancel-in-progress: true
jobs:
generate-patch:
name: Generate eslint --fix patch
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
has-fixes: ${{ steps.produce-patch.outputs.has-fixes }}
# No permissions override → inherits workflow-level contents: read.
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
# --ignore-scripts: eslint only needs TS sources + eslint packages.
- uses: ./.github/actions/retry
with:
command: npm ci --ignore-scripts
- name: npm run fix in all workspaces
# continue-on-error: if un-fixable errors exist on main, we still want
# to commit whatever fixes were applied. The PR-time check in
# typecheck.yml is what blocks un-fixable errors from landing.
continue-on-error: true
run: npm run fix
- name: Produce patch
id: produce-patch
run: |
if git diff --quiet; then
echo "No fixes needed."
echo "has-fixes=false" >> "$GITHUB_OUTPUT"
# Empty patch signals "nothing to do" to apply-patch.
: > js-fix.patch
else
git diff > js-fix.patch
echo "has-fixes=true" >> "$GITHUB_OUTPUT"
echo "Patch size: $(wc -c < js-fix.patch) bytes"
# Reject patches that touch anything outside JS/TS/JSON sources.
# `npm run fix` should only ever modify those; anything else means
# eslint/prettier or a plugin went rogue and we refuse to ship it.
BAD=$(git diff --name-only | grep -vE '\.(js|cjs|mjs|ts|tsx|json)$' || true)
if [ -n "$BAD" ]; then
echo "::error::Refusing to upload patch — touches disallowed files:"
echo "$BAD"
exit 1
fi
fi
- name: Upload patch artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: js-fix-patch
path: js-fix.patch
retention-days: 1
include-hidden-files: true
apply-patch:
name: Apply patch
needs: generate-patch
# Skip entirely when generate-patch found no fixes — saves a runner,
# avoids a redundant checkout/download, and keeps the job graph honest.
if: needs.generate-patch.outputs.has-fixes == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write # needed to push to bot/js-autofix
pull-requests: write # needed for PR creation + auto-merge
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Download patch
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: js-fix-patch
# ${{ runner.temp }} expands in with: params (shell-style $VAR does not).
# download-artifact's path is a *directory* — the artifact's js-fix.patch
# file lands inside it, so $RUNNER_TEMP/js-fix.patch resolves correctly
# in the run step below.
path: ${{ runner.temp }}
- name: Apply patch and push to bot branch
env:
BOT_BRANCH: bot/js-autofix
run: |
set -euo pipefail
# Empty patch = nothing to do.
if [ ! -s "$RUNNER_TEMP/js-fix.patch" ]; then
echo "Patch is empty. No fixes to apply."
exit 0
fi
# Apply the patch produced by the unprivileged job.
git apply --check "$RUNNER_TEMP/js-fix.patch" || {
echo "::error::Patch does not apply cleanly. Branch may have moved."
exit 1
}
git apply "$RUNNER_TEMP/js-fix.patch"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add -A
git commit -m "fmt(js): \`npm run fix\` on merge"
# Push to the dedicated bot branch. Force-push is safe here:
# bot/js-autofix is a bot-only branch that gets rewritten each run.
# If the branch was deleted after a previous PR merge, this
# recreates it.
git push --force origin HEAD:"$BOT_BRANCH"
- name: Create/update PR and enable auto-merge
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
BOT_BRANCH: bot/js-autofix
run: |
set -euo pipefail
# Create PR if one doesn't exist. If it already exists, the
# force-push above already updated it with the latest fixes.
PR_NUM=$(gh pr list --head "$BOT_BRANCH" --state open --json number --jq '.[0].number' 2>/dev/null || true)
if [ -z "$PR_NUM" ]; then
# gh pr create prints the PR URL. Extract the number from it
# (https://github.com/<org>/<repo>/pull/<number>).
PR_URL=$(gh pr create \
--head "$BOT_BRANCH" --base main \
--title 'fmt(js): `npm run fix` auto-fix' \
--body 'Auto-generated by the `auto-fix lint issues & formatting` workflow. Auto-merges (squash) once CI passes. If CI fails or `main` moves, the PR is auto-closed and the branch deleted — the next run re-applies on the current state.')
PR_NUM=$(echo "$PR_URL" | grep -oE '[0-9]+$')
fi
# Enable auto-merge (squash). If already enabled, this is a no-op.
gh pr merge "$PR_NUM" --auto --squash || true
- name: Wait for merge, auto-close on failure or stale
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
START_SHA: ${{ github.sha }}
run: |
set -euo pipefail
PR_NUM=$(gh pr list --head bot/js-autofix --state open --json number --jq '.[0].number' 2>/dev/null || true)
if [ -z "$PR_NUM" ]; then
echo "No open PR. Nothing to wait for."
exit 0
fi
echo "Waiting for PR #$PR_NUM to merge..."
# Poll every 15s for up to ~10 minutes. Auto-merge will handle the
# PR even if this job times out — the polling is for cleanup only
# (auto-close on CI failure, conflicts, or main moving).
for i in $(seq 1 40); do
sleep 15
STATE=$(gh pr view "$PR_NUM" --json state --jq '.state')
if [ "$STATE" = "MERGED" ] || [ "$STATE" = "CLOSED" ]; then
echo "PR #$PR_NUM is $STATE."
exit 0
fi
# If main moved, the PR may have already merged (which moves
# main) or another commit landed. Re-check state first.
CURRENT_SHA=$(gh api "repos/${{ github.repository }}/branches/main" --jq '.commit.sha')
if [ "$CURRENT_SHA" != "$START_SHA" ]; then
STATE=$(gh pr view "$PR_NUM" --json state --jq '.state')
if [ "$STATE" = "MERGED" ]; then
echo "PR #$PR_NUM merged (main moved to $CURRENT_SHA)."
exit 0
fi
echo "Main moved ($START_SHA → $CURRENT_SHA). Closing stale PR."
gh pr close "$PR_NUM" --delete-branch || true
exit 0
fi
# If CI checks failed, close + delete the branch.
if gh pr checks "$PR_NUM" 2>/dev/null | grep -qi "fail"; then
echo "CI failed on PR #$PR_NUM. Closing + deleting branch."
gh pr close "$PR_NUM" --delete-branch
exit 0
fi
# If PR is conflicted, close + delete the branch.
MERGEABLE=$(gh pr view "$PR_NUM" --json mergeable --jq '.mergeable')
if [ "$MERGEABLE" = "CONFLICTING" ]; then
echo "PR #$PR_NUM is conflicted. Closing + deleting branch."
gh pr close "$PR_NUM" --delete-branch
exit 0
fi
done
echo "Timeout reached. Auto-merge will handle PR #$PR_NUM if CI passes."
-51
View File
@@ -1,51 +0,0 @@
# .github/workflows/js-tests.yml
name: JS Tests
on:
workflow_call:
jobs:
workspaces:
name: List npm workspaces
runs-on: ubuntu-latest
timeout-minutes: 20
outputs:
packages: ${{ steps.set-matrix.outputs.packages }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
- uses: ./.github/actions/retry
with:
command: npm ci --ignore-scripts
- id: set-matrix
run: |
PACKAGES=$(npm query .workspace | jq -c '[.[].location]')
if [ "$PACKAGES" = "[]" ] || [ -z "$PACKAGES" ]; then
echo "::error::Workspace discovery produced an empty package list — refusing to emit a zero-length matrix (would skip all JS/TS checks silently)."
exit 1
fi
echo "packages=$PACKAGES" >> "$GITHUB_OUTPUT"
check:
name: Typecheck & Test
needs: workspaces
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
matrix:
package: ${{ fromJson(needs.workspaces.outputs.packages) }}
fail-fast: false # report all failures, not just the first one
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
- uses: ./.github/actions/retry
with:
command: npm ci
- run: npm run --prefix ${{ matrix.package }} check
- run: npm run --prefix ${{ matrix.package }} fix
-81
View File
@@ -1,81 +0,0 @@
name: Label rerun
# When the ``ci-reviewed`` label is added to a PR, rerun all failed jobs in
# the latest CI run. This re-evaluates ``review-labels`` (which now sees the
# label) and GitHub automatically reruns dependent jobs (``comment-live``,
# ``all-checks-pass``) — so the review comment gets updated too.
#
# If the CI run is still in progress when the label is added, we wait for it
# to finish before rerunning (``gh run rerun`` only works on completed runs).
# The wait can be long (20+ min for a full CI run), but it's better than
# silently failing and leaving the reviewer stuck.
on:
pull_request:
types: [labeled]
permissions:
actions: write
pull-requests: read
concurrency:
group: label-rerun-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
rerun-review-labels:
name: Rerun review-labels job
if: github.event.label.name == 'ci-reviewed'
runs-on: ubuntu-latest
timeout-minutes: 40
steps:
- name: Wait for CI run to finish, then rerun failed jobs
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -uo pipefail
# Find the latest CI run for this PR's head SHA.
RUN_ID=$(gh run list \
--repo "$REPO" \
--commit "$HEAD_SHA" \
--workflow ci.yml \
--limit 1 \
--json databaseId,status \
--jq '.[0] | "\(.databaseId) \(.status)"' 2>/dev/null || true)
if [ -z "$RUN_ID" ]; then
echo "No CI run found for this PR — nothing to rerun."
exit 0
fi
# Split "RUN_ID STATUS" into two vars.
RUN_ID="${RUN_ID%% *}"
STATUS="${RUN_ID##* }"
echo "Latest CI run: $RUN_ID (status: $STATUS)"
# If the run is still in progress, wait for it to finish.
# gh run rerun only works on completed runs — if we try while it's
# running, GitHub rejects with "cannot be rerun; This workflow is
# already running".
if [ "$STATUS" != "completed" ]; then
echo "Run is $STATUS — waiting for completion (this may take a while)..."
# gh run watch --exit-status exits non-zero if the run fails,
# which is expected (the label gate fails). Don't let that kill
# the workflow — we WANT to rerun failed jobs.
timeout 2100 gh run watch "$RUN_ID" --repo "$REPO" --interval 15 || true
# Verify it's actually completed now.
STATUS=$(gh run view "$RUN_ID" --repo "$REPO" --json status --jq '.status' 2>/dev/null || echo "unknown")
if [ "$STATUS" != "completed" ]; then
echo "Run is still $STATUS after wait — giving up."
exit 0
fi
fi
echo "Run completed. Rerunning all failed jobs..."
gh run rerun "$RUN_ID" --repo "$REPO" --failed || true
echo "Done. GitHub will rerun review-labels and all dependent jobs."
+4 -6
View File
@@ -2,14 +2,11 @@ name: Lint (ruff + ty)
# Two things here:
# 1. Advisory diff — ruff + ty diagnostics as a diff vs the target branch.
# Writes a Markdown summary to the run page. Exit zero always.
# Posts a Markdown summary and a PR comment. Exit zero always.
# 2. Blocking ``ruff check .`` — enforces the explicit rules in
# ``[tool.ruff.lint.select]`` (currently PLW1514). Failure blocks merge.
# Separate job so the advisory diff still runs even when enforcement
# fails.
#
# CI-sensitive file review was previously here as a ``ci-review`` job but
# has moved to ``review-labels.yml`` so it can be rerun independently.
# Separate job so the advisory diff still runs and posts even when
# enforcement fails.
on:
workflow_call:
@@ -21,6 +18,7 @@ on:
permissions:
contents: read
pull-requests: write # needed to post/update PR comments
concurrency:
group: lint-${{ github.ref }}
-95
View File
@@ -1,95 +0,0 @@
name: Lockfile diff
# Advisory PR comment showing the *semantic* diff of package-lock.json
# changes — which packages were added/removed/updated and their versions.
# The raw textual diff of a lockfile is unreadable (npm reorders entries
# and rewrites integrity hashes), so scripts/ci/lockfile_diff.py parses
# the ``packages`` map at the merge base and at HEAD and set-diffs the
# {install path: version} maps instead.
#
# The semantic diff is exposed as a workflow_call output ``review_status``
# (a JSON array in the unified status format) and an artifact
# (``lockfile-diff`` containing the markdown fragment) for the step
# summary.
#
# Never blocking — this is review signal, not enforcement.
on:
workflow_call:
outputs:
changed:
description: Whether package-lock.json changed relative to the target branch.
value: ${{ jobs.diff.outputs.changed }}
review_status:
description: JSON array of review status objects for the unified PR comment.
value: ${{ jobs.diff.outputs.review_status }}
permissions:
contents: read
concurrency:
group: lockfile-diff-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
diff:
name: package-lock.json semantic diff
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
changed: ${{ steps.diff.outputs.changed }}
review_status: ${{ steps.emit-status.outputs.review_status }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0 # need history for the merge base
- name: Generate semantic lockfile diff
id: diff
run: |
set -euo pipefail
# Three-dot semantics by hand: diff from the merge base with the
# target branch to the PR head, so changes that landed on main
# after the branch point don't show up as this PR's doing.
BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
echo "Merge base: ${BASE_SHA}"
python3 scripts/ci/lockfile_diff.py \
--base "$BASE_SHA" \
--head HEAD \
--output /tmp/lockfile-diff.md
if [ -s /tmp/lockfile-diff.md ]; then
echo "changed=true" >> "$GITHUB_OUTPUT"
{
echo "## package-lock.json semantic diff"
echo ""
cat /tmp/lockfile-diff.md
} >> "$GITHUB_STEP_SUMMARY"
else
echo "changed=false" >> "$GITHUB_OUTPUT"
: > /tmp/lockfile-diff.md
fi
- name: Emit review_status
id: emit-status
run: |
set -euo pipefail
CHANGED="${{ steps.diff.outputs.changed }}"
STATUS="[]"
if [ "$CHANGED" = "true" ]; then
CONTENT=$(cat /tmp/lockfile-diff.md | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))")
STATUS="[{\"source\":\"lockfile-diff\",\"results\":[{\"kind\":\"action_required\",\"title\":\"package-lock.json\",\"summary\":\"Locked npm dependency versions changed.\",\"detail\":${CONTENT},\"how_to_fix\":\"Add the \`ci-reviewed\` label after verifying the version changes are expected.\"}]}"
else
STATUS="[]"
fi
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
- name: Upload diff artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: lockfile-diff
path: /tmp/lockfile-diff.md
retention-days: 1
overwrite: true
+2 -79
View File
@@ -18,21 +18,17 @@ name: OSV-Scanner
# Findings land in the repo's Security tab (Code Scanning > OSV-Scanner).
# fail-on-vuln is disabled so the job does not block merges on pre-existing
# vulnerabilities in pinned deps that we may need to patch deliberately.
#
# The reusable workflow can't emit custom outputs, so a wrapper job
# downloads the SARIF result and summarizes the vulnerability count into
# a review_status for the unified PR comment.
on:
workflow_call:
schedule:
# Weekly scan against main — catches CVEs published after merge for
# deps that haven't changed since.
- cron: '0 9 * * 1'
- cron: "0 9 * * 1"
workflow_dispatch:
permissions:
# Required to upload SARIF file to CodeQL. See: https://github.com/github/codeql-action/issues/2117
# Required by the reusable workflow to upload SARIF to the Security tab.
actions: read
contents: read
security-events: write
@@ -49,76 +45,3 @@ jobs:
--lockfile=package-lock.json
--lockfile=website/package-lock.json
fail-on-vuln: false
emit-status:
name: Emit review status
runs-on: ubuntu-latest
needs: scan
if: always()
outputs:
review_status: ${{ steps.emit.outputs.review_status }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download SARIF result
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: osv-results
path: /tmp/osv-results
continue-on-error: true
- name: Emit review_status
id: emit
run: |
set -euo pipefail
STATUS="[]"
if [ -f /tmp/osv-results/osv-results.sarif ]; then
# Count vulnerabilities from the SARIF file
VULN_COUNT=$(python3 -c "
import json, sys
try:
with open('/tmp/osv-results/osv-results.sarif') as f:
data = json.load(f)
count = 0
vulns = []
for run in data.get('runs', []):
for result in run.get('results', []):
count += 1
rule_id = result.get('ruleId', 'unknown')
message = result.get('message', {}).get('text', '')
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
vulns.append(f'- {rule_id} in {loc}: {message}')
print(count)
if vulns:
print('\n'.join(vulns[:20]), file=sys.stderr)
except Exception:
print(0)
")
VULN_DETAIL=""
if [ "$VULN_COUNT" -gt 0 ] 2>/dev/null; then
VULN_PLURAL=$([ "$VULN_COUNT" -eq 1 ] && echo "y" || echo "ies")
VULN_DETAIL=$(python3 -c "
import json, sys
try:
with open('/tmp/osv-results/osv-results.sarif') as f:
data = json.load(f)
vulns = []
for run in data.get('runs', []):
for result in run.get('results', []):
rule_id = result.get('ruleId', 'unknown')
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
vulns.append(f'- {rule_id} in {loc}')
print(json.dumps('\n'.join(vulns[:20])))
except Exception:
print(json.dumps(''))
")
STATUS="[{\"source\":\"osv scan\",\"results\":[{\"kind\":\"warning\",\"title\":\"OSV vulnerability scan\",\"summary\":\"${VULN_COUNT} known vulnerabilit${VULN_PLURAL} found in pinned dependencies.\",\"detail\":${VULN_DETAIL},\"how_to_fix\":\"Review the findings in the [Security tab](../../security/code-scanning). Update the affected dependencies if a patched version is available.\"}]}]"
else
STATUS="[]"
fi
fi
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
-98
View File
@@ -1,98 +0,0 @@
name: Review labels
# Require explicit maintainer review when CI-sensitive files or the MCP
# catalog change. Previously this was split across two jobs in two
# workflows: ``ci-review`` in lint.yml (gated on ``ci_review``) and
# ``mcp-catalog-review`` in supply-chain-audit.yml (gated on
# ``mcp_catalog``). Both checked for their own label.
#
# Now consolidated: a single ``ci-reviewed`` label covers both. The
# comment sections tell the reviewer exactly what to verify per area,
# so one label is enough — the human reads the comment, not the label
# name.
#
# Outputs:
# ci_reviewed — "true" / "false" / "" (empty when neither lane ran)
# review_status — JSON array of status objects consumed by the review
# comment assembler. See scripts/ci/emit_review_status.py.
on:
workflow_call:
inputs:
ci_review:
description: Whether CI-sensitive files (eslint config, workflows, actions) changed.
type: boolean
default: false
mcp_catalog:
description: Whether the MCP catalog / installer changed.
type: boolean
default: false
supply_chain:
description: Whether the critical supply-chain scan found a risk requiring review.
type: boolean
default: false
outputs:
ci_reviewed:
description: Whether the ci-reviewed label is present. Empty when neither input was true.
value: ${{ jobs.check.outputs.ci_reviewed }}
review_status:
description: JSON array of status objects for the review comment assembler.
value: ${{ jobs.check.outputs.review_status }}
permissions:
contents: read
pull-requests: read # read PR labels
jobs:
check:
name: Review label gate
if: inputs.ci_review || inputs.mcp_catalog || inputs.supply_chain
runs-on: ubuntu-latest
timeout-minutes: 2
outputs:
ci_reviewed: ${{ steps.label-check.outputs.ci_reviewed }}
review_status: ${{ steps.build-status.outputs.review_status }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Check ci-reviewed label
id: label-check
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
PR="${{ github.event.pull_request.number }}"
LABELS=$(gh pr view "$PR" --repo "$REPO" --json labels --jq '.labels[].name' || true)
if echo "$LABELS" | grep -Fxq 'ci-reviewed'; then
echo "ci-reviewed label present."
echo "ci_reviewed=true" >> "$GITHUB_OUTPUT"
else
echo "ci-reviewed label missing."
echo "ci_reviewed=false" >> "$GITHUB_OUTPUT"
fi
- name: Build review_status JSON
id: build-status
env:
CI_REVIEW: ${{ inputs.ci_review }}
MCP_CATALOG: ${{ inputs.mcp_catalog }}
SUPPLY_CHAIN: ${{ inputs.supply_chain }}
LABEL_PRESENT: ${{ steps.label-check.outputs.ci_reviewed }}
run: |
set -euo pipefail
args=()
if [ "$CI_REVIEW" = "true" ]; then args+=(--ci-review); fi
if [ "$MCP_CATALOG" = "true" ]; then args+=(--mcp-catalog); fi
if [ "$SUPPLY_CHAIN" = "true" ]; then args+=(--supply-chain); fi
if [ "$LABEL_PRESENT" = "true" ]; then args+=(--label-present); fi
python3 scripts/ci/emit_review_status.py "${args[@]}" --output "$GITHUB_OUTPUT"
- name: Fail on missing label
if: steps.label-check.outputs.ci_reviewed != 'true'
run: |
echo "::error::CI-sensitive changes require the ci-reviewed label. Add the label and re-run this check."
exit 1
+2 -11
View File
@@ -20,7 +20,6 @@ jobs:
check-freshness:
if: github.repository == 'NousResearch/hermes-agent'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Probe live index
id: probe
@@ -29,7 +28,7 @@ jobs:
URL="https://hermes-agent.nousresearch.com/docs/api/skills-index.json"
echo "Probing $URL"
# -L follows redirects; -f fails on HTTP errors; -s suppresses progress
if ! curl -fsSL --retry 3 --retry-delay 10 -o /tmp/skills-index.json "$URL"; then
if ! curl -fsSL -o /tmp/skills-index.json "$URL"; then
echo "status=fetch-failed" >> "$GITHUB_OUTPUT"
echo "detail=Could not download $URL" >> "$GITHUB_OUTPUT"
exit 0
@@ -108,18 +107,10 @@ jobs:
echo "Summary: ${{ steps.probe.outputs.summary }}"
fi
- name: Get GitHub App token
if: steps.probe.outputs.status != 'ok'
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Open issue on degraded / failed probe
if: steps.probe.outputs.status != 'ok'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
STATUS: ${{ steps.probe.outputs.status }}
DETAIL: ${{ steps.probe.outputs.detail }}
run: |
+3 -20
View File
@@ -20,29 +20,19 @@ jobs:
# Only run on the upstream repository, not on forks
if: github.repository == 'NousResearch/hermes-agent'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- name: Install dependencies
uses: ./.github/actions/retry
with:
command: pip install httpx==0.28.1 pyyaml==6.0.2
run: pip install httpx==0.28.1 pyyaml==6.0.2
- name: Build skills index
env:
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python scripts/build_skills_index.py
- name: Upload index artifact
@@ -59,15 +49,8 @@ jobs:
needs: build-index
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Trigger Deploy Site workflow
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: gh workflow run deploy-site.yml --repo ${{ github.repository }} -f skills_index_run_id=${{ github.run_id }}
+72 -104
View File
@@ -10,18 +10,9 @@ name: Supply Chain Audit
# advisory-only workflow instead.
#
# Path-gating is handled centrally by the ``ci.yml`` orchestrator's
# ``detect`` job. The orchestrator passes ``scan`` / ``deps`` booleans as
# inputs; this workflow's jobs gate on those inputs instead of re-computing
# the diff. MCP catalog review was previously here but has moved to
# ``review-labels.yml`` so it can be rerun independently.
#
# Outputs:
# review_status — JSON array of status objects consumed by the review
# comment assembler (scripts/ci/assemble_review_comment.py).
# critical_findings — "true" when the narrow critical-pattern scan found
# something. The review-label gate consumes this and
# owns the action-required result, so adding
# ``ci-reviewed`` can heal the run on rerun.
# ``detect`` job. The orchestrator passes ``scan`` / ``deps`` /
# ``mcp_catalog`` booleans as inputs; this workflow's jobs gate on those
# inputs instead of re-computing the diff.
on:
workflow_call:
@@ -38,13 +29,10 @@ on:
description: Whether pyproject.toml changed.
type: boolean
required: true
outputs:
review_status:
description: JSON array of review status objects for the review comment assembler.
value: ${{ jobs.aggregate.outputs.review_status }}
critical_findings:
description: Whether the critical-pattern scan found a risk requiring maintainer review.
value: ${{ jobs.aggregate.outputs.critical_findings }}
mcp_catalog:
description: Whether the MCP catalog / installer changed.
type: boolean
required: true
permissions:
pull-requests: write
@@ -55,27 +43,16 @@ jobs:
name: Scan PR for critical supply chain risks
if: inputs.scan
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
review_status: ${{ steps.emit-status.outputs.review_status }}
critical_findings: ${{ steps.scan.outputs.found }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Scan diff for critical patterns
id: scan
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
@@ -83,7 +60,7 @@ jobs:
HEAD="${{ github.event.pull_request.head.sha }}"
# Added lines only, excluding lockfiles.
# Three-point diff (base...head) diffs from the merge base to HEAD,
# Three-dot diff (base...head) diffs from the merge base to HEAD,
# so only changes introduced by this PR are included — not changes
# that landed on main after the PR branched off.
DIFF=$(git diff "$BASE"..."$HEAD" -- . ':!uv.lock' ':!*.lock' ':!package-lock.json' ':!yarn.lock' || true)
@@ -161,32 +138,32 @@ jobs:
echo "found=false" >> "$GITHUB_OUTPUT"
fi
- name: Emit review_status
id: emit-status
if: always()
- name: Post critical finding comment
if: steps.scan.outputs.found == 'true'
env:
FOUND: ${{ steps.scan.outputs.found }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 - <<'PYEOF'
import json, os
BODY="## 🚨 CRITICAL Supply Chain Risk Detected
# The review-label gate renders and blocks critical findings. Keep
# this scan a fact-finder so adding ci-reviewed can rerun the gate
# without requiring the scanner itself to fail again.
status = []
This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
f.write(f"review_status={json.dumps(status)}\n")
PYEOF
$(cat /tmp/findings.md)
---
*Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.*"
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs — GITHUB_TOKEN is read-only)"
- name: Fail on critical findings
if: steps.scan.outputs.found == 'true'
run: |
echo "::error::CRITICAL supply chain risk patterns detected in this PR. See the PR comment for details."
exit 1
dep-bounds:
name: Check PyPI dependency upper bounds
if: inputs.deps
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
review_status: ${{ steps.emit-status.outputs.review_status }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -209,7 +186,7 @@ jobs:
exit 0
fi
# Match PyPI dep specs that have >= and no < ceiling.
# Match PyPI dep specs that have >= but no < ceiling.
# Pattern: "package>=version" without a following ",<" bound.
# Excludes git+ URLs (which use commit SHAs) and comments.
UNBOUNDED=$(echo "$ADDED" | grep -oE '"[a-zA-Z0-9_-]+(\[[^\]]*\])?>=[ 0-9.]+"' | grep -v ',<' || true)
@@ -221,36 +198,26 @@ jobs:
echo "found=false" >> "$GITHUB_OUTPUT"
fi
- name: Emit review_status
id: emit-status
if: always()
- name: Post unbounded dep warning
if: steps.bounds.outputs.found == 'true'
env:
FOUND: ${{ steps.bounds.outputs.found }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 - <<'PYEOF'
import json, os
BODY="## ⚠️ Unbounded PyPI Dependency Detected
found = os.environ.get("FOUND", "") == "true"
This PR adds PyPI dependencies without a \`<next_major\` upper bound. Per our [supply chain policy](../blob/main/CONTRIBUTING.md#dependency-pinning-policy-supply-chain-hardening), all PyPI deps must be pinned as \`>=floor,<next_major\`.
if found:
with open("/tmp/unbounded.txt", encoding="utf-8") as f:
detail = f.read()
status = [{
"source": "supply chain",
"results": [{
"kind": "action_required",
"title": "Unbounded PyPI dependencies",
"summary": "This PR adds PyPI dependencies without upper bounds.",
"detail": detail,
"how_to_fix": 'Add a `<next_major` upper bound, e.g. `"package>=1.2.0,<2"`. See CONTRIBUTING.md dependency pinning policy.'
}]
}]
else:
status = []
**Unbounded specs found:**
\`\`\`
$(cat /tmp/unbounded.txt)
\`\`\`
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
f.write(f"review_status={json.dumps(status)}\n")
PYEOF
**Fix:** Add an upper bound, e.g. \`"package>=1.2.0,<2"\`
---
*See PR #2810 and CONTRIBUTING.md for the full policy rationale.*"
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs)"
- name: Fail on unbounded deps
if: steps.bounds.outputs.found == 'true'
@@ -258,39 +225,40 @@ jobs:
echo "::error::PyPI dependencies without upper bounds detected. Add <next_major ceiling per CONTRIBUTING.md policy."
exit 1
aggregate:
name: Aggregate review statuses
needs: [scan, dep-bounds]
if: always()
mcp-catalog-review:
name: MCP catalog security review
if: inputs.mcp_catalog
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
review_status: ${{ steps.merge.outputs.review_status }}
critical_findings: ${{ steps.merge.outputs.critical_findings }}
steps:
- name: Merge review statuses
id: merge
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Require explicit MCP catalog review label
env:
SCAN_STATUS: ${{ needs.scan.outputs.review_status }}
DEP_STATUS: ${{ needs.dep-bounds.outputs.review_status }}
CRITICAL_FINDINGS: ${{ needs.scan.outputs.critical_findings }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 - <<'PYEOF'
import json, os
set -euo pipefail
PR="${{ github.event.pull_request.number }}"
LABELS=$(gh pr view "$PR" --json labels --jq '.labels[].name' || true)
if echo "$LABELS" | grep -Fxq 'mcp-catalog-reviewed'; then
echo "MCP catalog review label present."
exit 0
fi
merged = []
for key in ("SCAN_STATUS", "DEP_STATUS"):
raw = os.environ.get(key, "")
if not raw:
continue
try:
data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
continue
if isinstance(data, list):
merged.extend(data)
BODY="## ⚠️ MCP catalog security review required
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
f.write(f"review_status={json.dumps(merged)}\n")
f.write("critical_findings=" + os.environ.get("CRITICAL_FINDINGS", "false") + "\n")
PYEOF
This PR changes the bundled MCP catalog or MCP catalog installer code. MCP entries can define local commands that users later install into \`mcp_servers\`, so this needs explicit maintainer review before merge.
A maintainer should verify:
- any new/changed \`optional-mcps/**/manifest.yaml\` command and args are expected,
- stdio transports do not use shell+egress/exfiltration payloads,
- git install refs are pinned and bootstrap commands are minimal,
- requested env vars/secrets match the upstream MCP's documented needs.
After review, add the \`mcp-catalog-reviewed\` label and re-run this check."
gh pr comment "$PR" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs)"
echo "::error::MCP catalog changes require the mcp-catalog-reviewed label."
exit 1
-11
View File
@@ -20,7 +20,6 @@ jobs:
generate:
name: "Generate slices"
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
matrix: ${{ steps.matrix.outputs.matrix }}
steps:
@@ -32,12 +31,6 @@ jobs:
with:
path: test_durations.json
key: test-durations
# Saves use test-durations-${run_id}, so the exact key above never
# matches — without this prefix fallback the cache ALWAYS missed,
# LPT slicing ran on no data, and unbalanced slices pushed heavy
# files toward the per-file timeout under load.
restore-keys: |
test-durations-
- name: Generate test slices
id: matrix
@@ -121,9 +114,6 @@ jobs:
NOUS_API_KEY: ""
- name: Upload per-slice durations
# Advisory artifact (feeds slice balancing) — a transient artifact-
# service blip must not fail an otherwise-green test slice.
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-durations-slice-${{ matrix.slice.index }}
@@ -136,7 +126,6 @@ jobs:
needs: test
if: needs.test.result == 'success' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Download all slice durations
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+51
View File
@@ -0,0 +1,51 @@
# .github/workflows/typecheck.yml
name: Typecheck
on:
workflow_call:
jobs:
typecheck:
name: Check TypeScript
runs-on: ubuntu-latest
strategy:
matrix:
package:
[ui-tui, web, apps/bootstrap-installer, apps/desktop, apps/shared]
fail-fast: false # report all failures, not just the first one
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
# --ignore-scripts: typecheck only needs the TS sources + type defs, not
# native builds. Skipping install scripts drops node-pty's node-gyp
# header fetch — the transient flake that killed this job pre-`tsc` — and
# is faster. retry covers the remaining registry blips.
- uses: ./.github/actions/retry
with:
command: npm ci --ignore-scripts
- run: npm run --prefix ${{ matrix.package }} typecheck
# Production build of the desktop renderer. `typecheck` runs `tsc` only,
# which does NOT exercise Vite/Rolldown module resolution — so an
# unresolvable package export (e.g. a transitive @assistant-ui/tap that no
# longer exports "./react-shim") slips past typecheck and only explodes when
# users build apps/desktop from source on install/update. Run the real
# `vite build` here so that class of break fails in CI instead.
desktop-build:
name: Build desktop app
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
# Keep install scripts here: the production build may need node-pty's
# native binary. retry handles the transient install-time fetch flakes.
- uses: ./.github/actions/retry
with:
command: npm ci
- run: npm run --prefix apps/desktop build
+4 -28
View File
@@ -26,7 +26,6 @@ jobs:
build:
name: Build distribution 📦
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
@@ -57,24 +56,10 @@ jobs:
node-version: "22"
- name: Build web dashboard
uses: ./.github/actions/retry
with:
command: npm ci
working-directory: web
- name: Compile web dashboard
run: npm run build
working-directory: web
run: cd web && npm ci && npm run build
- name: Build TUI bundle
uses: ./.github/actions/retry
with:
command: npm ci
working-directory: ui-tui
- name: Compile TUI bundle
run: npm run build
working-directory: ui-tui
run: cd ui-tui && npm ci && npm run build
- name: Bundle TUI into hermes_cli
run: |
@@ -105,7 +90,6 @@ jobs:
name: Publish to PyPI
needs: build
runs-on: ubuntu-latest
timeout-minutes: 30
environment:
name: pypi
url: https://pypi.org/p/hermes-agent
@@ -131,7 +115,6 @@ jobs:
if: startsWith(github.ref, 'refs/tags/')
needs: publish
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write # attach assets to the existing release
id-token: write # sigstore signing
@@ -143,16 +126,9 @@ jobs:
name: python-package-distributions
path: dist/
- name: Get GitHub App token
id: app-token
uses: ./.github/actions/get-app-token
with:
client-id: ${{ secrets.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Wait for GitHub Release to exist
env:
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ github.token }}
# release.py creates the GitHub Release after pushing the tag,
# but this workflow starts from the tag push — wait for it.
run: |
@@ -178,7 +154,7 @@ jobs:
- name: Attach signed artifacts to GitHub Release
if: env.skip_sign != 'true'
env:
GITHUB_TOKEN: ${{ steps.app-token.outputs.token }}
GITHUB_TOKEN: ${{ github.token }}
# release.py already created the GitHub Release — just upload
# the Sigstore signatures alongside the existing assets.
run: >-
+1 -25
View File
@@ -45,10 +45,6 @@ name: uv.lock check
on:
workflow_call:
outputs:
review_status:
description: "JSON review status for the review-status aggregator"
value: ${{ jobs.check.outputs.review_status }}
permissions:
contents: read
@@ -62,8 +58,6 @@ jobs:
name: uv lock --check
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
review_status: ${{ steps.verify.outputs.review_status }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -79,22 +73,8 @@ jobs:
# of this file) — failures often mean "your branch is behind main,
# rebase and regenerate uv.lock."
- name: Verify uv.lock is up-to-date
id: verify
run: |
# uv lock --check re-resolves against PyPI (network). Retry so a
# registry blip doesn't read as "lockfile stale". A genuinely stale
# lockfile fails all attempts (deterministic), costing only seconds.
ok=false
for i in 1 2 3; do
if uv lock --check; then
ok=true
break
fi
[ "$i" = 3 ] && break
echo "::warning::uv lock --check failed (attempt $i); retrying in 10s"
sleep 10
done
if [ "$ok" != true ]; then
if ! uv lock --check; then
cat <<'EOF' >> "$GITHUB_STEP_SUMMARY"
## ❌ uv.lock is out of sync with pyproject.toml
@@ -124,9 +104,5 @@ jobs:
on `main` post-merge.
EOF
echo "::error title=uv.lock out of sync::Run \`uv lock\` locally and commit the result. If on a PR, sync with main first."
review_status='[{"source":"uv.lock check","results":[{"kind":"action_required","title":"uv.lock out of sync","summary":"uv.lock is out of sync with pyproject.toml.","how_to_fix":"Run `uv lock` locally and commit the result. If on a PR, sync with main first:\n```\ngit fetch origin main\ngit rebase origin/main\nuv lock\ngit add uv.lock\ngit commit -m \"chore: refresh uv.lock\"\n```\n"}]}]'
echo "review_status=${review_status}" >> "$GITHUB_OUTPUT"
exit 1
fi
review_status='[]'
echo "review_status=${review_status}" >> "$GITHUB_OUTPUT"
+2 -24
View File
@@ -4,8 +4,6 @@
/_pycache/
*.pyc*
__pycache__/
act/
.act-sandbox-agent.*
.venv/
.venv
.vscode/
@@ -44,10 +42,7 @@ run_datagen_sonnet.sh
source-data/*
run_datagen_megascience_glm4-6.sh
data/*
# No trailing slash: also matches node_modules SYMLINKS (worktrees often
# symlink node_modules to the main checkout; the dir-only pattern let one
# slip into a commit and break `npm ci` on CI with ENOTDIR).
node_modules
node_modules/
browser-use/
agent-browser/
# Private keys
@@ -59,10 +54,6 @@ __pycache__/
hermes_agent.egg-info/
wandb/
testlogs
playwright-report/
test-results/
# Playwright visual regression baselines — cached from main in CI, not committed
*-snapshots/
# CLI config (may contain sensitive SSH paths)
cli-config.yaml
@@ -75,23 +66,10 @@ environments/benchmarks/evals/
# Web UI build output
hermes_cli/web_dist/
# Cross-process web UI build lock (flock target, always empty)
.web_ui_build.lock
apps/desktop/build/
apps/desktop/dist/
# tsc-emitted artifacts (a stray `tsc -b` compiles into src/, and vite then
# resolves the stale .js OVER the .tsx — never track these)
apps/desktop/src/**/*.js
apps/desktop/src/**/*.js.map
apps/desktop/src/**/*.d.ts
!apps/desktop/src/global.d.ts
!apps/desktop/src/vite-env.d.ts
apps/shared/src/**/*.js
apps/shared/src/**/*.js.map
apps/shared/src/**/*.d.ts
apps/desktop/release/
*.tsbuildinfo
apps/desktop/*.tsbuildinfo
# Web UI assets — synced from @nous-research/ui at build time via
# `npm run sync-assets` (see web/package.json).
-4
View File
@@ -1,4 +0,0 @@
# Lockfiles must never be reformatted — main has a repo rule requiring
# team approval when lockfiles change, so an autofix PR touching one
# would hang waiting for review.
package-lock.json
+13 -91
View File
@@ -491,18 +491,18 @@ The dashboard embeds the real `hermes --tui` — **not** a rewrite. See `hermes
### Electron Desktop Chat App (`apps/desktop/`)
A **separate** chat surface from both the classic CLI and the dashboard's embedded TUI. It is an Electron + React + nanostore renderer (`@assistant-ui/react`) that talks to a `tui_gateway` backend over JSON-RPC (`requestGateway(method, params)`). The WebSocket/JSON-RPC transport lives in the framework-agnostic `apps/shared` package (`@hermes/shared` — `JsonRpcGatewayClient` + WS URL helpers), which the web dashboard (`web/`) also consumes; **desktop has no build/runtime dependency on the dashboard frontend** — it spawns a headless `hermes serve` backend server (the same gateway `dashboard` serves, minus the browser UI entirely: `serve` sets `headless_backend=True`, so `cmd_dashboard` skips `_build_web_ui` AND exports `HERMES_SERVE_HEADLESS=1` so `mount_spa()` disables the SPA even if a stray `web_dist/` exists — only the JSON-RPC/WS/API surface is reachable). `dashboard` and `serve` share `cmd_dashboard`/`start_server` but are independent surfaces — neither launches the other. The one exception is a backward-compat *fallback*: `serve` is newer, so the desktop spawn (`electron/backend-command.ts` + `backendSupportsServe()` in `electron/main.ts`) detects whether the resolved runtime registers `serve` and, only when it does not (an older managed install / PATH `hermes` the app hasn't updated yet), rewrites the argv to the legacy `dashboard --no-open`. Without that, a new app against an un-upgraded runtime would crash on an unknown subcommand and brick every mid-upgrade user. It does NOT embed `hermes --tui` — it has its own composer, transcript, and slash-command pipeline. For scoped Desktop architecture, state, resolver, transport, and testing rules, read `apps/desktop/AGENTS.md`.
A **separate** chat surface from both the classic CLI and the dashboard's embedded TUI. It is an Electron + React + nanostore renderer (`@assistant-ui/react`) that talks to a `tui_gateway` backend over JSON-RPC (`requestGateway(method, params)`). The WebSocket/JSON-RPC transport lives in the framework-agnostic `apps/shared` package (`@hermes/shared` — `JsonRpcGatewayClient` + WS URL helpers), which the web dashboard (`web/`) also consumes; **desktop has no build/runtime dependency on the dashboard frontend** — it spawns a headless `hermes serve` backend server (the same gateway `dashboard` serves, minus the browser UI entirely: `serve` sets `headless_backend=True`, so `cmd_dashboard` skips `_build_web_ui` AND exports `HERMES_SERVE_HEADLESS=1` so `mount_spa()` disables the SPA even if a stray `web_dist/` exists — only the JSON-RPC/WS/API surface is reachable). `dashboard` and `serve` share `cmd_dashboard`/`start_server` but are independent surfaces — neither launches the other. The one exception is a backward-compat *fallback*: `serve` is newer, so the desktop spawn (`electron/backend-command.cjs` + `backendSupportsServe()` in `main.cjs`) detects whether the resolved runtime registers `serve` and, only when it does not (an older managed install / PATH `hermes` the app hasn't updated yet), rewrites the argv to the legacy `dashboard --no-open`. Without that, a new app against an un-upgraded runtime would crash on an unknown subcommand and brick every mid-upgrade user. It does NOT embed `hermes --tui` — it has its own composer, transcript, and slash-command pipeline. Route desktop bugs to the `hermes-desktop-app-work` skill, not `hermes-dashboard-work`.
**Slash commands in the desktop app are curated client-side, then dispatched to the backend.** The pipeline:
- **Backend already provides everything.** `tui_gateway/server.py` `commands.catalog` (empty-query list) and `complete.slash` (typed-query completions) both include built-in commands, user `quick_commands`, AND skill-derived commands (`scan_skill_commands()` / `get_skill_commands()`). The desktop app does not need a new RPC to see skills.
- **The renderer curates via `apps/desktop/src/lib/desktop-slash-commands.ts`.** This is the load-bearing file. It holds `DESKTOP_COMMAND_SPECS` (the built-ins and their Desktop surfaces) plus `NO_DESKTOP_SURFACE` block-lists for terminal-only / messaging-only / picker-owned / settings-owned / advanced commands that should NOT clutter the desktop popover.
- **The renderer curates via `apps/desktop/src/lib/desktop-slash-commands.ts`.** This is the load-bearing file. It holds `DESKTOP_COMMANDS` (the ~19 built-ins shown in the palette) plus block-lists for terminal-only / messaging-only / picker-owned / settings-owned / advanced commands that should NOT clutter the desktop popover.
- `isDesktopSlashCommand(name)` — gates **execution**. Returns true for built-ins AND for any non-built-in (skill / quick command), so typed extension commands run.
- `isDesktopSlashSuggestion(name)` — gates **discovery/completion**. Used by BOTH completion paths in `app/chat/composer/hooks/use-slash-completions.ts` (empty-query catalog filter + typed-query `complete.slash` filter) and by `filterDesktopCommandsCatalog`.
- `isDesktopSlashExtensionCommand(name)` — true when the command is NOT a known Hermes built-in (i.e. a skill or user quick command). Both suggestion and catalog-filter paths allow extensions through so skill commands surface in the palette. (Added when fixing "skill commands missing from the desktop slash palette" — the curated allow-list was silently dropping every skill/quick command from completions even though they executed fine when typed.)
- **Dispatch** lives in `app/session/hooks/use-prompt-actions/slash.ts` (`runSlash`): built-ins that the desktop owns (`/skin`, `/help`, `/new`, …) are handled locally or via `commands.catalog`; everything else goes to `slash.exec`, falling back to `command.dispatch` (which the gateway resolves into skill / alias / exec directives). A skill command resolves to `{type: "skill", message}` and is submitted as a normal prompt.
- **Dispatch** lives in `app/session/hooks/use-prompt-actions.ts` (`runSlash`): built-ins that the desktop owns (`/skin`, `/help`, `/new`, …) are handled locally or via `commands.catalog`; everything else goes to `slash.exec`, falling back to `command.dispatch` (which the gateway resolves into skill / alias / exec directives). A skill command resolves to `{type: "skill", message}` and is submitted as a normal prompt.
**Rule:** the desktop slash palette's curation is about hiding noise (terminal-only / messaging-only built-ins), NOT about hiding user-activated extensions. Skill commands and `quick_commands` are extensions the backend surfaces — they belong in completions. If you tighten `desktop-slash-commands.ts`, keep `isDesktopSlashExtensionCommand` flowing into both the suggestion and catalog-filter paths. Tests: from `apps/desktop`, run `npx vitest run src/lib/desktop-slash-commands.test.ts` (workspace dependencies are installed at the repo root).
**Rule:** the desktop slash palette's curation is about hiding noise (terminal-only / messaging-only built-ins), NOT about hiding user-activated extensions. Skill commands and `quick_commands` are extensions the backend surfaces — they belong in completions. If you tighten `desktop-slash-commands.ts`, keep `isDesktopSlashExtensionCommand` flowing into both the suggestion and catalog-filter paths. Tests: `apps/desktop/src/lib/desktop-slash-commands.test.ts` (run via the repo-root `vitest`, since `apps/desktop` resolves deps from the root workspace install).
---
@@ -998,8 +998,7 @@ Two shapes:
Roles:
- `role="leaf"` (default) — focused worker. Cannot call `delegate_task`,
`clarify`, `memory`, `send_message`, `cronjob`. Retains `execute_code`
(programmatic tool calling).
`clarify`, `memory`, `send_message`, `execute_code`.
- `role="orchestrator"` — retains `delegate_task` so it can spawn its
own workers. Gated by `delegation.orchestrator_enabled` (default true)
and bounded by `delegation.max_spawn_depth` (default 2).
@@ -1095,16 +1094,14 @@ kanban task.
- **CLI:** `hermes_cli/kanban.py` wires `hermes kanban` with verbs
`init`, `create`, `list` (alias `ls`), `show`, `assign`, `link`,
`unlink`, `comment`, `attach`, `attachments`, `attach-rm`, `complete`,
`block`, `unblock`, `archive`, `tail`, plus less-commonly-used `watch`,
`stats`, `runs`, `log`, `assignees`, `heartbeat`, `notify-*`,
`dispatch`, `daemon`, `gc`.
`unlink`, `comment`, `complete`, `block`, `unblock`, `archive`,
`tail`, plus less-commonly-used `watch`, `stats`, `runs`, `log`,
`assignees`, `heartbeat`, `notify-*`, `dispatch`, `daemon`, `gc`.
- **Worker/orchestrator toolset:** `tools/kanban_tools.py` exposes
`kanban_show`, `kanban_complete`, `kanban_block`, `kanban_heartbeat`,
`kanban_comment`, `kanban_create`, `kanban_link`, `kanban_attach`,
`kanban_attach_url`, `kanban_attachments`; profiles that explicitly
enable the `kanban` toolset outside a dispatcher-spawned task also get
`kanban_list` and `kanban_unblock` for board routing.
`kanban_comment`, `kanban_create`, `kanban_link`; profiles that
explicitly enable the `kanban` toolset outside a dispatcher-spawned
task also get `kanban_list` and `kanban_unblock` for board routing.
- **Dispatcher:** long-lived loop that (default every 60s) reclaims
stale claims, promotes ready tasks, atomically claims, and spawns
assigned profiles. Runs **inside the gateway** by default via
@@ -1281,7 +1278,6 @@ def profile_env(tmp_path, monkeypatch):
## Testing
### Python
**ALWAYS use `scripts/run_tests.sh`** — do not call `pytest` directly. The script enforces
hermetic environment parity with CI (unset credential vars, TZ=UTC, LANG=C.UTF-8,
`-n auto` xdist workers, in-tree subprocess-isolation plugin). Direct `pytest`
@@ -1295,20 +1291,12 @@ scripts/run_tests.sh tests/agent/test_foo.py::test_x # one test
scripts/run_tests.sh -v --tb=long # pass-through pytest flags
```
**Flake policy:** the runner auto-retries a failing test FILE once in a fresh
subprocess (`--file-retries`, default 1; `HERMES_TEST_FILE_RETRIES=0` to
disable). Pass-on-retry counts as green but is printed in a `⚠ FLAKY` summary
section with both attempts' output. A FLAKY report is a bug to fix, not noise
to ignore — timing-sensitive tests must not assume a quiet runner (loose
wall-clock bounds ≥ 2s, event-based sync, no `assert not _wait_until(...)`
negative-timing races).
#### Subprocess-per-test-file isolation
### Subprocess-per-test-file isolation
Every test file runs in a freshly-spawned Python subprocess via `run_tests_parallel.py`. This means module-level dicts/sets and
ContextVars from one test file cannot leak into the next.
#### Why the wrapper
### Why the wrapper
| | Without wrapper | With wrapper |
| ------------------- | ------------------------------------------- | ----------------------------------------- |
@@ -1317,17 +1305,6 @@ ContextVars from one test file cannot leak into the next.
| Timezone | Local TZ (PDT etc.) | UTC |
| Locale | Whatever is set | C.UTF-8 |
### Where to place what tests
The CI change classifier (`scripts/ci/classify_changes.py`) runs specific jobs based on what files changed. A Python test that asserts
about the contents of `package.json`, `package-lock.json`, `.ts`/`.tsx`
source, or any other JS-side artifact will not run on a PR that only touches
those files. This means a regression can go green on a PR and red on `main` (where the
classifier fails open and runs everything).
Any test that reads or asserts about `package.json`,
`package-lock.json`, `tsconfig.json`, `.ts`/`.tsx`/`.js`/`.mjs`/`.cjs`
source files configuration belongs in the JS (vitest) test suite, not in `tests/*.py`.
### Don't write change-detector tests
@@ -1377,58 +1354,3 @@ not the specific names.
Reviewers should reject new change-detector tests; authors should convert
them into invariants before re-requesting review.
### Never read source code in tests
A test that reads a source file's text is testing *the shape of the
source code*, not its behavior. This is a hard antipattern, banned outright.
Any test that reads a .py, .ts, .tsx, etc., file is suspect.
**Why it's actively harmful, not just weak:**
- It passes when the implementation is subtly broken (the regex matches a
call site that exists but is wired wrong) and fails when a correct
refactor changes formatting, variable names, or control flow with
identical runtime behavior. Both directions of failure are wrong.
- It can't be run against a built/bundled/minified artifact, so it silently
stops testing anything the moment code moves, gets renamed, or a
dependency reformats it.
- It actively blocks refactors: reviewers see "keeps a pattern intact" tests
fail during pure structural cleanup with no behavior change, and either
hand-wave the failure (dangerous) or waste time updating regexes that add
nothing (waste).
- It gives false confidence. a green suite full of source-regex tests
looks like coverage but has never once executed the code path it claims
to guard.
**Do not write:**
```ts
const source = fs.readFileSync(path.join(__dirname, 'main.ts'), 'utf8')
test('backend spawn hides the Windows console', () => {
assert.match(source, /spawn\(\s*backend\.command,\s*backend\.args[\s\S]{0,300}hiddenWindowsChildOptions/)
})
```
**Do write — extract the logic into a small pure/DI-testable function and
call it for real:**
```ts
// backend-spawn.ts
export function hiddenWindowsChildOptions(options: SpawnOptionsLike = {}, isWindows = process.platform === 'win32') {
if (!isWindows || 'windowsHide' in options) return options
return { ...options, windowsHide: true }
}
// backend-spawn.test.ts
test('windowsHide defaults to true on Windows, is left alone elsewhere', () => {
assert.equal(hiddenWindowsChildOptions({}, true).windowsHide, true)
assert.equal(hiddenWindowsChildOptions({}, false).windowsHide, undefined)
assert.equal(hiddenWindowsChildOptions({ windowsHide: false }, true).windowsHide, false)
})
```
If the logic lives inline in a god-file (`main.ts`, `cli.py`,
`gateway/run.py`) and extracting it feels disruptive: that's the actual
signal to do the extraction, not to regex around it.
+60 -39
View File
@@ -26,8 +26,8 @@ ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
# replaces tini with s6-overlay's /init (PID 1 = s6-svscan), which reaps
# zombies non-blockingly on SIGCHLD and additionally supervises the main
# hermes process, the dashboard, and per-profile gateways.
RUN apt-get -o Acquire::Retries=3 update && \
apt-get -o Acquire::Retries=3 install -y --no-install-recommends \
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates curl iputils-ping python3 python-is-python3 ripgrep ffmpeg gcc g++ make cmake python3-dev python3-venv libffi-dev libolm-dev procps git openssh-client docker-cli xz-utils && \
rm -rf /var/lib/apt/lists/*
@@ -40,30 +40,33 @@ RUN apt-get -o Acquire::Retries=3 update && \
# we map between them inline. The noarch + symlinks tarballs are
# architecture-independent and reused as-is.
#
# We use `curl` instead of `ADD` for ALL three tarballs: `ADD` evaluates its
# URL at parse time (no ARG / TARGETARCH substitution) and — critically for
# CI reliability — cannot retry, so a single GitHub-release CDN blip fails
# the whole 15-45 min build. curl -fsSL --retry 3 self-heals those blips,
# and every tarball is still checksum-verified below before extraction.
# We use `curl` instead of `ADD` for the per-arch tarball because `ADD`
# evaluates its URL at parse time, before any ARG / TARGETARCH substitution
# — splitting one URL per arch into two ADDs would download both on every
# build and leave dead bytes in the cache. A single curl + arch-keyed URL
# is simpler and cache-friendlier.
#
# Supply-chain integrity: every tarball is checksum-verified against the
# upstream-published SHA256. To bump S6_OVERLAY_VERSION, fetch the four
# `.sha256` files from the corresponding release and update the ARGs. The
# checksum lookup happens during build, so a compromised release artifact
# fails the build loudly instead of silently producing a tampered image.
ARG TARGETARCH
ARG S6_OVERLAY_VERSION=3.2.3.0
ARG S6_OVERLAY_NOARCH_SHA256=b720f9d9340efc8bb07528b9743813c836e4b02f8693d90241f047998b4c53cf
ARG S6_OVERLAY_X86_64_SHA256=a93f02882c6ed46b21e7adb5c0add86154f01236c93cd82c7d682722e8840563
ARG S6_OVERLAY_AARCH64_SHA256=0952056ff913482163cc30e35b2e944b507ba1025d78f5becbb89367bf344581
ARG S6_OVERLAY_SYMLINKS_SHA256=a60dc5235de3ecbcf874b9c1f18d73263ab99b289b9329aa950e8729c4789f0e
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz /tmp/
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-symlinks-noarch.tar.xz /tmp/
RUN set -eu; \
case "${TARGETARCH:-amd64}" in \
amd64) s6_arch="x86_64"; s6_arch_sha="${S6_OVERLAY_X86_64_SHA256}" ;; \
arm64) s6_arch="aarch64"; s6_arch_sha="${S6_OVERLAY_AARCH64_SHA256}" ;; \
*) echo "Unsupported TARGETARCH=${TARGETARCH} for s6-overlay" >&2; exit 1 ;; \
esac; \
base="https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}"; \
curl -fsSL --retry 3 -o /tmp/s6-overlay-noarch.tar.xz \
"${base}/s6-overlay-noarch.tar.xz"; \
curl -fsSL --retry 3 -o /tmp/s6-overlay-symlinks-noarch.tar.xz \
"${base}/s6-overlay-symlinks-noarch.tar.xz"; \
curl -fsSL --retry 3 -o /tmp/s6-overlay-arch.tar.xz \
"${base}/s6-overlay-${s6_arch}.tar.xz"; \
"https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${s6_arch}.tar.xz"; \
{ \
printf '%s %s\n' "${S6_OVERLAY_NOARCH_SHA256}" /tmp/s6-overlay-noarch.tar.xz; \
printf '%s %s\n' "${s6_arch_sha}" /tmp/s6-overlay-arch.tar.xz; \
@@ -73,19 +76,17 @@ RUN set -eu; \
tar -C / -Jxpf /tmp/s6-overlay-noarch.tar.xz; \
tar -C / -Jxpf /tmp/s6-overlay-arch.tar.xz; \
tar -C / -Jxpf /tmp/s6-overlay-symlinks-noarch.tar.xz; \
rm /tmp/s6-overlay-*.tar.xz /tmp/s6-overlay.sha256
# #34192 / #66679: backward-compat shim for orchestration templates that
# still reference the legacy /usr/bin/tini entrypoint (Hostinger's
# 'Hermes WebUI' catalog, NAS compose projects that preserve an old
# entrypoint on image update, etc.). A plain symlink to /init made the
# path exist, but forwarded tini flags like `-g` into s6-overlay's
# rc.init as the container CMD (`rc.init: 91: -g: not found`) and
# boot-looped any `restart: unless-stopped` deploy. The shim strips the
# tini CLI surface, then exec's /init + main-wrapper — see
# docker/tini-shim.sh. Safe to drop once the affected catalogs are
# updated.
COPY --chmod=0755 docker/tini-shim.sh /usr/bin/tini
rm /tmp/s6-overlay-*.tar.xz /tmp/s6-overlay.sha256; \
# #34192: backward-compat shim for orchestration templates that still\
# reference the legacy /usr/bin/tini entrypoint (e.g. Hostinger's\
# 'Hermes WebUI' catalog). The image has moved to s6-overlay /init\
# as PID 1 (see ENTRYPOINT below + the migration comment at the top\
# of this file), but external wrappers pinned to /usr/bin/tini will\
# crash with 'tini: No such file or directory' on startup. The shim\
# symlinks /usr/bin/tini -> /init so legacy wrappers exec the right\
# PID-1 reaper without behavior change for users on the current\
# ENTRYPOINT. Safe to drop once the affected catalogs are updated.\
ln -sf /init /usr/bin/tini
# Non-root user for runtime; UID can be overridden via HERMES_UID at runtime
RUN useradd -u 10000 -m -d /opt/data hermes
@@ -117,6 +118,7 @@ WORKDIR /opt/hermes
COPY package.json package-lock.json ./
COPY web/package.json web/
COPY ui-tui/package.json ui-tui/
COPY apps/desktop/package.json apps/desktop/
COPY ui-tui/packages/hermes-ink/ ui-tui/packages/hermes-ink/
# apps/shared/ is copied IN FULL because web/package.json references it as a
# `file:` workspace dependency (same pattern as hermes-ink above).
@@ -134,11 +136,16 @@ COPY apps/shared/ apps/shared/
# guards against a future regression if the source npm version changes.
ENV npm_config_install_links=false
RUN npm install --prefer-offline --no-audit --fetch-retries=5 && \
for i in 1 2 3; do \
npx playwright install --with-deps chromium --only-shell && break || \
{ [ "$i" = 3 ] && exit 1; echo "playwright install failed (attempt $i); retrying in 10s"; sleep 10; }; \
done && \
# `ELECTRON_SKIP_BINARY_DOWNLOAD=1`: the desktop workspace lists `electron` as
# a devDependency, but the image only ever runs its *renderer* as a browser
# bundle (`build:web` below) — the ~100MB Electron runtime binary would be
# downloaded, never executed, and baked into the layer. Skipping the download
# keeps the postinstall a no-op; node-pty's native compile still runs (the
# gcc/make toolchain is already present for python-olm above).
ENV ELECTRON_SKIP_BINARY_DOWNLOAD=1
RUN npm install --prefer-offline --no-audit && \
npx playwright install --with-deps chromium --only-shell && \
npm cache clean --force
# ---------- Layer-cached Python dependency install ----------
@@ -190,8 +197,14 @@ RUN uv sync --frozen --no-install-project --extra all --extra messaging --extra
COPY web/ web/
COPY ui-tui/ ui-tui/
COPY apps/shared/ apps/shared/
# The desktop renderer, built browser-only (no Electron packaging steps) and
# served by the gateway at /app — HERMES_APP_DIST below points at this dist.
# .dockerignore excludes any local dist/build/release so this build is always
# from source.
COPY apps/desktop/ apps/desktop/
RUN cd web && npm run build && \
cd ../ui-tui && npm run build
cd ../ui-tui && npm run build && \
cd ../apps/desktop && npm run build:web
# ---------- Source code ----------
# .dockerignore excludes node_modules, so the installs above survive.
@@ -276,6 +289,13 @@ COPY --chmod=0755 docker/cont-init.d/02-reconcile-profiles /etc/cont-init.d/02-r
# ---------- Runtime ----------
ENV HERMES_WEB_DIST=/opt/hermes/hermes_cli/web_dist
# The desktop renderer web build (Layer above: `apps/desktop && npm run
# build:web`), served by the gateway at /app. Explicitly opting in via this
# env var is the ONLY thing that mounts /app — web_server.py has no path
# fallback — so the browser UI is a fact of the image, never of a source
# checkout. The dist survives `COPY . .` because .dockerignore excludes
# apps/desktop/dist from the build context.
ENV HERMES_APP_DIST=/opt/hermes/apps/desktop/dist
# Point the TUI launcher at the prebuilt bundle baked at build time (Layer 8:
# `ui-tui && npm run build`). This makes _make_tui_argv take the prebuilt-bundle
# fast path (`node --expose-gc /opt/hermes/ui-tui/dist/entry.js`) and skip the
@@ -284,13 +304,14 @@ ENV HERMES_WEB_DIST=/opt/hermes/hermes_cli/web_dist
#
# Why this is required (not just an optimization): the root package-lock.json
# describes the WHOLE monorepo workspace set (root + web + ui-tui + apps/*),
# but the image only installs root/web/ui-tui (apps/* — the desktop app — is
# never `npm install`ed here). So the actualized node_modules permanently
# disagrees with the canonical lock, _tui_need_npm_install() returns True on
# every launch, and the runtime `npm install` it triggers (a) can never
# converge against the partial monorepo and (b) races itself across concurrent
# embedded-chat (/api/pty) connections → ENOTEMPTY → the chat tab dies with a
# 502 / "[session ended]". Pointing at the prebuilt bundle sidesteps the whole
# but the image's npm install only actualizes the workspaces present in the
# build context (apps/desktop joined for the /app renderer build; any other
# apps/* stay excluded). So the actualized node_modules can disagree with the
# canonical lock, _tui_need_npm_install() returns True on every launch, and
# the runtime `npm install` it triggers (a) can never converge against the
# partial monorepo and (b) races itself across concurrent embedded-chat
# (/api/pty) connections → ENOTEMPTY → the chat tab dies with a 502 /
# "[session ended]". Pointing at the prebuilt bundle sidesteps the whole
# check. (A separate launcher hardening is tracked independently.)
ENV HERMES_TUI_DIR=/opt/hermes/ui-tui
ENV HERMES_HOME=/opt/data
-1
View File
@@ -1,7 +1,6 @@
graft skills
graft optional-skills
graft optional-mcps
graft hermes_cli/web_dist
graft locales
# Bundled plugin manifests (plugin.yaml / plugin.yml). Without these the
# PluginManager scan (hermes_cli/plugins.py) finds zero plugins on installs
-1
View File
@@ -109,7 +109,6 @@ hermes # Interactive CLI — start a conversation
hermes model # Choose your LLM provider and model
hermes tools # Configure which tools are enabled
hermes config set # Set individual config values
hermes config get # Print individual config values
hermes gateway # Start the messaging gateway (Telegram, Discord, etc.)
hermes setup # Run the full setup wizard (configures everything at once)
hermes claw migrate # Migrate from OpenClaw (if coming from OpenClaw)
+9 -16
View File
@@ -38,22 +38,19 @@ def _permission_option_supports_kind(kind: str) -> bool:
return True
def _build_permission_options(
*, allow_permanent: bool, smart_denied: bool = False,
) -> list[PermissionOption]:
def _build_permission_options(*, allow_permanent: bool) -> list[PermissionOption]:
"""Return ACP options that match Hermes approval semantics."""
options = [PermissionOption(
option_id="allow_once", kind="allow_once", name="Allow once",
)]
if not smart_denied:
options.append(PermissionOption(
options = [
PermissionOption(option_id="allow_once", kind="allow_once", name="Allow once"),
PermissionOption(
option_id="allow_session",
# ACP has no session-scoped kind, so use the closest persistent
# hint while keeping Hermes semantics in the option id.
kind="allow_always",
name="Allow for session",
))
if allow_permanent and not smart_denied:
),
]
if allow_permanent:
options.append(
PermissionOption(
option_id="allow_always",
@@ -62,7 +59,7 @@ def _build_permission_options(
),
)
options.append(PermissionOption(option_id="deny", kind="reject_once", name="Deny"))
if not smart_denied and _permission_option_supports_kind("reject_always"):
if _permission_option_supports_kind("reject_always"):
options.append(
PermissionOption(
option_id="deny_always",
@@ -132,15 +129,11 @@ def make_approval_callback(
description: str,
*,
allow_permanent: bool = True,
smart_denied: bool = False,
**_: object,
) -> str:
from agent.async_utils import safe_schedule_threadsafe
options = _build_permission_options(
allow_permanent=allow_permanent,
smart_denied=smart_denied,
)
options = _build_permission_options(allow_permanent=allow_permanent)
tool_call = _build_permission_tool_call(command, description)
coro = request_permission_fn(
+7 -34
View File
@@ -456,7 +456,7 @@ class HermesACPAgent(acp.Agent):
"tools": "List available tools",
"context": "Show conversation context info",
"reset": "Clear conversation history",
"compress": "Compress conversation context",
"compact": "Compress conversation context",
"steer": "Inject guidance into the currently running agent turn",
"queue": "Queue a prompt to run after the current turn finishes",
"version": "Show Hermes version",
@@ -485,7 +485,7 @@ class HermesACPAgent(acp.Agent):
"description": "Clear conversation history",
},
{
"name": "compress",
"name": "compact",
"description": "Compress conversation context",
},
{
@@ -1617,28 +1617,12 @@ class HermesACPAgent(acp.Agent):
self._send_session_info_update(session_id),
)
# Snapshot the runtime identity; the validator lets the
# background titler skip its LLM call if the session's model
# changed before it fires (#19027).
_title_model = getattr(state.agent, "model", None)
_title_provider = getattr(state.agent, "provider", None)
maybe_auto_title(
self.session_manager._get_db(),
session_id,
user_text,
final_response,
state.history,
main_runtime={
"model": getattr(state.agent, "model", None),
"provider": getattr(state.agent, "provider", None),
"base_url": getattr(state.agent, "base_url", None),
"api_key": getattr(state.agent, "api_key", None),
"api_mode": getattr(state.agent, "api_mode", None),
},
runtime_validator=lambda: (
getattr(state.agent, "model", None) == _title_model
and getattr(state.agent, "provider", None) == _title_provider
),
title_callback=_notify_title_update,
)
except Exception:
@@ -1756,7 +1740,7 @@ class HermesACPAgent(acp.Agent):
"tools": self._cmd_tools,
"context": self._cmd_context,
"reset": self._cmd_reset,
"compress": self._cmd_compress,
"compact": self._cmd_compact,
"steer": self._cmd_steer,
"queue": self._cmd_queue,
"version": self._cmd_version,
@@ -1898,7 +1882,7 @@ class HermesACPAgent(acp.Agent):
lines.append(
f"Compression: due now (threshold ~{threshold_tokens:,}"
+ (f", {threshold_pct:.0f}%" if threshold_pct else "")
+ "). Run /compress."
+ "). Run /compact."
)
else:
lines.append(
@@ -1913,27 +1897,16 @@ class HermesACPAgent(acp.Agent):
if getattr(agent, "compression_enabled", True) is False:
lines.append("Compression is disabled for this agent.")
else:
lines.append("Tip: run /compress to compress manually before the threshold.")
lines.append("Tip: run /compact to compress manually before the threshold.")
return "\n".join(lines)
def _cmd_reset(self, args: str, state: SessionState) -> str:
state.history.clear()
reset_failed = False
try:
reset_session_state = getattr(state.agent, "reset_session_state", None)
if callable(reset_session_state):
reset_session_state()
except Exception:
reset_failed = True
logger.warning("ACP session state reset failed for %s", state.session_id, exc_info=True)
finally:
self.session_manager.save_session(state.session_id)
if reset_failed:
return "Conversation history cleared. Agent session state reset failed; see logs."
self.session_manager.save_session(state.session_id)
return "Conversation history cleared."
def _cmd_compress(self, args: str, state: SessionState) -> str:
def _cmd_compact(self, args: str, state: SessionState) -> str:
if not state.history:
return "Nothing to compress — conversation is empty."
try:
+22 -17
View File
@@ -26,18 +26,31 @@ from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
def _win_path_to_wsl(path: str) -> str | None:
"""Convert a Windows drive path to its WSL /mnt/<drive>/... equivalent."""
match = re.match(r"^([A-Za-z]):[\\/](.*)$", path)
if not match:
return None
drive = match.group(1).lower()
tail = match.group(2).replace("\\", "/")
return f"/mnt/{drive}/{tail}"
def _translate_acp_cwd(cwd: str) -> str:
"""Translate Windows ACP cwd values when Hermes itself is running in WSL.
Windows ACP clients can launch ``hermes acp`` inside WSL while still sending
editor workspaces as Windows drive paths (``E:\\Projects``) or
``\\\\wsl.localhost\\`` UNC paths. Store and execute against the POSIX form so
agents, tools, and persisted ACP sessions all agree on the usable workspace.
Native Linux/macOS keeps the original cwd unchanged.
editor workspaces as Windows drive paths such as ``E:\\Projects``. Store
and execute against the WSL mount path so agents, tools, and persisted ACP
sessions all agree on the usable workspace. Native Linux/macOS keeps the
original cwd unchanged.
"""
from hermes_constants import translate_cwd_for_wsl_backend
from hermes_constants import is_wsl
return translate_cwd_for_wsl_backend(str(cwd))
if not is_wsl():
return cwd
translated = _win_path_to_wsl(str(cwd))
return translated if translated is not None else cwd
def _normalize_cwd_for_compare(cwd: str | None) -> str:
@@ -48,9 +61,7 @@ def _normalize_cwd_for_compare(cwd: str | None) -> str:
# Normalize Windows drive paths into the equivalent WSL mount form so
# ACP history filters match the same workspace across Windows and WSL.
from hermes_constants import windows_path_to_wsl
translated = windows_path_to_wsl(expanded)
translated = _win_path_to_wsl(expanded)
if translated is not None:
expanded = translated
elif re.match(r"^/mnt/[A-Za-z]/", expanded):
@@ -534,15 +545,9 @@ class SessionManager:
model = row.get("model") or None
# Load conversation history. repair_alternation: this restore feeds
# LIVE REPLAY — the loaded list becomes the resumed agent's working
# conversation. A durable ``user;user`` violation left in state.db would
# otherwise re-fire the pre-request defensive repair on every request
# for the rest of the session (see hermes_state.get_messages_as_conversation).
# Load conversation history.
try:
history = db.get_messages_as_conversation(
session_id, repair_alternation=True
)
history = db.get_messages_as_conversation(session_id)
except Exception:
logger.warning("Failed to load messages for ACP session %s", session_id, exc_info=True)
history = []
+1 -52
View File
@@ -3,7 +3,6 @@
from __future__ import annotations
import json
import logging
import uuid
from typing import Any, Dict, List, Optional
@@ -15,8 +14,6 @@ from acp.schema import (
ToolKind,
)
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Map hermes tool names -> ACP ToolKind
# ---------------------------------------------------------------------------
@@ -387,24 +384,6 @@ def _format_execute_code_result(result: Optional[str]) -> Optional[str]:
error = str(data.get("error") or "")
exit_code = data.get("exit_code")
parts = [f"Exit code: {exit_code}" if exit_code is not None else "Execution complete"]
if data.get("stdout_truncated"):
total = data.get("stdout_bytes_total")
captured = data.get("stdout_bytes_captured")
omitted = data.get("stdout_bytes_omitted")
if all(isinstance(v, int) for v in (captured, total, omitted)):
parts.extend([
"",
(
"Output truncated: "
f"captured {captured:,} of {total:,} bytes "
f"({omitted:,} omitted)."
),
])
else:
parts.extend(["", "Output truncated."])
warning = str(data.get("warning") or "").strip()
if warning:
parts.extend(["", "Warning:", warning])
if output:
parts.extend(["", "Output:", output])
if error:
@@ -1047,37 +1026,7 @@ def build_tool_start(
*,
edit_diff: Any = None,
) -> ToolCallStart:
"""Create a ToolCallStart event for the given hermes tool invocation.
A malformed tool argument (e.g. a non-string ``command``/``path`` from a
model that ignores the schema) must never abort the ACP tool-call render —
``build_tool_start`` runs on the live tool-progress callback and during
session history replay. On any failure in the title/content/location
builders, fall back to a minimal, valid start event. Mirrors
``get_cute_tool_message`` in ``agent/display.py``, wrapped for the same
reason on the CLI side.
"""
try:
return _build_tool_start(
tool_call_id, tool_name, arguments, edit_diff=edit_diff
)
except Exception as exc: # noqa: BLE001 — a tool-call render must never abort the turn
logger.debug("ACP tool-start render failed for %r: %s", tool_name, exc)
safe_name = tool_name if isinstance(tool_name, str) and tool_name else "tool"
return acp.start_tool_call(
tool_call_id, safe_name, kind=get_tool_kind(safe_name),
content=None, locations=[], raw_input=None,
)
def _build_tool_start(
tool_call_id: str,
tool_name: str,
arguments: Dict[str, Any],
*,
edit_diff: Any = None,
) -> ToolCallStart:
"""Build the ToolCallStart event (unguarded; see ``build_tool_start``)."""
"""Create a ToolCallStart event for the given hermes tool invocation."""
kind = get_tool_kind(tool_name)
title = build_tool_title(tool_name, arguments)
locations = extract_locations(arguments)
+2 -2
View File
@@ -1,7 +1,7 @@
{
"id": "hermes-agent",
"name": "Hermes Agent",
"version": "0.19.0",
"version": "0.18.2",
"description": "Self-improving open-source AI agent by Nous Research with ACP editor integration, persistent memory, skills, and rich tool support.",
"repository": "https://github.com/NousResearch/hermes-agent",
"website": "https://hermes-agent.nousresearch.com/docs/user-guide/features/acp",
@@ -9,7 +9,7 @@
"license": "MIT",
"distribution": {
"uvx": {
"package": "hermes-agent[acp]==0.19.0",
"package": "hermes-agent[acp]==0.18.2",
"args": ["hermes-acp"]
}
}
+11 -205
View File
@@ -214,7 +214,7 @@ def build_nous_credits_snapshot(account_info) -> Optional[AccountUsageSnapshot]:
return None
details.append(f"Top up: {nous_portal_topup_url(account_info)}")
details.append("(or run /topup)")
details.append("(or run /credits)")
plan = getattr(sub, "plan", None) if sub is not None else None
return AccountUsageSnapshot(
@@ -340,7 +340,7 @@ def _snapshot_from_credits_state(state) -> Optional[AccountUsageSnapshot]:
@dataclass(frozen=True)
class CreditsView:
"""Surface-agnostic data for the ``/topup`` balance view.
"""Surface-agnostic data for the ``/credits`` command.
One portal fetch, one parse — consumed identically by the CLI panel, the
gateway button, and any other money surface. Fail-open: when not logged in
@@ -356,11 +356,11 @@ class CreditsView:
def build_credits_view(*, markdown: bool = False, timeout: float = 10.0) -> CreditsView:
"""Build the /topup balance view: balance block + identity line + top-up URL.
"""Build the /credits view: balance block + identity line + top-up URL.
Reuses the same account fetch + snapshot + URL builder as the /usage credits
block, so the numbers always match. The balance block is the rendered
snapshot MINUS its trailing top-up/command-hint lines (the /topup surface
snapshot MINUS its trailing top-up/command-hint lines (the /credits surface
supplies its own affordance). Fail-open → ``CreditsView(logged_in=False)``.
"""
not_logged_in = CreditsView(logged_in=False)
@@ -386,7 +386,7 @@ def build_credits_view(*, markdown: bool = False, timeout: float = 10.0) -> Cred
timeout=timeout
)
except Exception:
logger.debug("credits ▸ /topup portal fetch failed (fail-open)", exc_info=True)
logger.debug("credits ▸ /credits portal fetch failed (fail-open)", exc_info=True)
return not_logged_in
if account is None or not getattr(account, "logged_in", False):
@@ -394,8 +394,8 @@ def build_credits_view(*, markdown: bool = False, timeout: float = 10.0) -> Cred
snapshot = build_nous_credits_snapshot(account)
# Balance lines = the snapshot block minus the two trailing affordance lines
# ("Top up: <url>" + "(or run /topup)") that build_nous_credits_snapshot
# appends for the /usage surface. /topup renders its own button/panel.
# ("Top up: <url>" + "(or run /credits)") that build_nous_credits_snapshot
# appends for the /usage surface. /credits renders its own button/panel.
balance_lines: list[str] = []
if snapshot is not None:
rendered = render_account_usage_lines(snapshot, markdown=markdown)
@@ -425,28 +425,15 @@ def build_credits_view(*, markdown: bool = False, timeout: float = 10.0) -> Cred
)
def _codex_backend_urls(base_url: str) -> tuple[str, str, str]:
"""Resolve the Codex backend endpoints (usage, reset-credits list, consume).
Mirrors the Codex CLI's PathStyle split (codex-rs backend-client): base URLs
containing ``/backend-api`` use the ChatGPT ``/wham/...`` paths; everything
else uses ``/api/codex/...``.
"""
def _resolve_codex_usage_url(base_url: str) -> str:
normalized = (base_url or "").strip().rstrip("/")
if not normalized:
normalized = "https://chatgpt.com/backend-api/codex"
if normalized.endswith("/codex"):
normalized = normalized[: -len("/codex")]
prefix = normalized + ("/wham" if "/backend-api" in normalized else "/api/codex")
return (
prefix + "/usage",
prefix + "/rate-limit-reset-credits",
prefix + "/rate-limit-reset-credits/consume",
)
def _resolve_codex_usage_url(base_url: str) -> str:
return _codex_backend_urls(base_url)[0]
if "/backend-api" in normalized:
return normalized + "/wham/usage"
return normalized + "/api/codex/usage"
def _resolve_codex_usage_credentials(
@@ -538,14 +525,6 @@ def _fetch_codex_account_usage(
)
)
details: list[str] = []
reset_credits = payload.get("rate_limit_reset_credits") or {}
banked = reset_credits.get("available_count")
if isinstance(banked, (int, float)) and int(banked) > 0:
count = int(banked)
plural = "s" if count != 1 else ""
details.append(
f"You have {count} reset{plural} banked - use /usage reset to activate"
)
credits = payload.get("credits") or {}
if credits.get("has_credits"):
balance = credits.get("balance")
@@ -563,179 +542,6 @@ def _fetch_codex_account_usage(
)
@dataclass(frozen=True)
class CodexResetRedeemResult:
"""Outcome of a `/usage reset` attempt against the Codex backend."""
status: str # reset | nothing_to_reset | no_credit | already_redeemed |
# not_exhausted | no_credits_banked | unavailable
message: str
available_count: int = 0
windows_reset: int = 0
@property
def redeemed(self) -> bool:
return self.status == "reset"
# Client-side guard threshold: a rate-limit window only counts as exhausted
# when it is fully used. Below this, redeeming a banked reset wastes most of
# its value, so we block and point at --force instead.
_CODEX_WINDOW_EXHAUSTED_PERCENT = 100.0
def redeem_codex_reset_credit(
*,
base_url: Optional[str] = None,
api_key: Optional[str] = None,
force: bool = False,
) -> CodexResetRedeemResult:
"""Redeem one banked Codex rate-limit reset credit (`/usage reset`).
Flow (mirrors the Codex CLI's reset-credits picker, codex-rs
``backend-client``):
1. ``GET .../usage`` — read the current windows + banked credit count.
2. Guard: zero banked credits → refuse. No window fully used and not
``force`` → refuse with a warning (a banked reset restores the WHOLE
5h + weekly allowance; burning it early wastes it). The backend has
the same protection (``nothing_to_reset`` doesn't consume the
credit), but failing fast client-side gives a clearer message.
3. ``POST .../rate-limit-reset-credits/consume`` with a fresh UUID
idempotency key (``redeem_request_id``). No ``credit_id`` — the
backend picks the next available credit, exactly like the CLI's
default "Full reset" option.
Never raises: every failure mode returns a ``CodexResetRedeemResult``
with a user-renderable message.
"""
import uuid
try:
token, resolved_base_url, account_id = _resolve_codex_usage_credentials(base_url, api_key)
except Exception:
return CodexResetRedeemResult(
status="unavailable",
message="No Codex credentials available. Run `hermes auth` to sign in with your ChatGPT account.",
)
usage_url, _credits_url, consume_url = _codex_backend_urls(resolved_base_url)
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/json",
"User-Agent": "codex-cli",
}
if account_id:
headers["ChatGPT-Account-Id"] = account_id
try:
with httpx.Client(timeout=15.0) as client:
usage_resp = client.get(usage_url, headers=headers)
usage_resp.raise_for_status()
payload = usage_resp.json() or {}
reset_credits = payload.get("rate_limit_reset_credits") or {}
raw_count = reset_credits.get("available_count")
available = int(raw_count) if isinstance(raw_count, (int, float)) else 0
if available <= 0:
return CodexResetRedeemResult(
status="no_credits_banked",
message="No banked reset credits on this account — nothing to redeem.",
)
rate_limit = payload.get("rate_limit") or {}
worst_used: Optional[float] = None
for key in ("primary_window", "secondary_window"):
used = (rate_limit.get(key) or {}).get("used_percent")
if isinstance(used, (int, float)):
worst_used = max(worst_used or 0.0, float(used))
exhausted = worst_used is not None and worst_used >= _CODEX_WINDOW_EXHAUSTED_PERCENT
if not exhausted and not force:
usage_note = (
f"your busiest window is only {worst_used:.0f}% used"
if worst_used is not None
else "your current usage could not be confirmed as exhausted"
)
plural = "s" if available != 1 else ""
return CodexResetRedeemResult(
status="not_exhausted",
message=(
f"⚠️ Not redeeming: {usage_note}. A banked reset restores your FULL "
f"5h + weekly limits, so spending it now would waste most of it. "
f"You have {available} reset{plural} banked. "
f"Use `/usage reset --force` to redeem anyway."
),
available_count=available,
)
consume_resp = client.post(
consume_url,
headers={**headers, "Content-Type": "application/json"},
json={"redeem_request_id": str(uuid.uuid4())},
)
consume_resp.raise_for_status()
body = consume_resp.json() or {}
except httpx.HTTPStatusError as exc:
code = exc.response.status_code
if code in (401, 403):
return CodexResetRedeemResult(
status="unavailable",
message=(
"Codex backend rejected the request (HTTP "
f"{code}). Reset credits require ChatGPT-account (OAuth) auth — "
"run `hermes auth` and sign in with your ChatGPT account."
),
)
return CodexResetRedeemResult(
status="unavailable",
message=f"Codex backend error (HTTP {code}) — try again shortly.",
)
except Exception as exc:
return CodexResetRedeemResult(
status="unavailable",
message=f"Could not reach the Codex backend: {exc}",
)
code = str(body.get("code", "") or "").strip().lower()
windows_reset = body.get("windows_reset")
windows_reset = int(windows_reset) if isinstance(windows_reset, (int, float)) else 0
remaining = max(0, available - 1)
plural = "s" if remaining != 1 else ""
if code == "reset":
return CodexResetRedeemResult(
status="reset",
message=(
f"✅ Reset redeemed — your usage limits have been reset. "
f"{remaining} banked reset{plural} remaining."
),
available_count=remaining,
windows_reset=windows_reset,
)
if code == "nothing_to_reset":
return CodexResetRedeemResult(
status="nothing_to_reset",
message=(
"Backend reports nothing to reset — your limits aren't exhausted. "
"The credit was NOT spent."
),
available_count=available,
)
if code == "no_credit":
return CodexResetRedeemResult(
status="no_credit",
message="Backend reports no available reset credit on this account.",
)
if code == "already_redeemed":
return CodexResetRedeemResult(
status="already_redeemed",
message="This redemption was already processed — no additional credit was spent.",
available_count=remaining,
)
return CodexResetRedeemResult(
status="unavailable",
message=f"Unexpected response from the Codex backend: {body!r}",
)
def _fetch_anthropic_account_usage() -> Optional[AccountUsageSnapshot]:
token = (resolve_anthropic_token() or "").strip()
if not token:
+17 -468
View File
@@ -28,7 +28,7 @@ import time
import uuid
from datetime import datetime
from typing import Any, Callable, Dict, List, Optional
from urllib.parse import parse_qs, urlparse, urlunparse
from urllib.parse import urlparse, parse_qs, urlunparse
from agent.context_compressor import ContextCompressor
from agent.iteration_budget import IterationBudget
@@ -48,7 +48,6 @@ from agent.tool_guardrails import (
ToolGuardrailDecision,
)
from hermes_cli.config import cfg_get
from hermes_cli.route_identity import normalize_route_base_url
from hermes_cli.timeouts import get_provider_request_timeout
from hermes_constants import get_hermes_home
from utils import base_url_host_matches, is_truthy_value
@@ -69,151 +68,18 @@ def _ra():
return run_agent
def _normalize_route_base_url(base_url: Any) -> str:
"""Canonicalize an endpoint URL for model-route identity comparisons."""
return normalize_route_base_url(base_url)
def _provider_default_routes(provider: str) -> set[str]:
"""Return known exact default routes for a canonical provider id."""
routes: set[str] = set()
try:
from hermes_cli.providers import HERMES_OVERLAYS, get_provider
overlay = HERMES_OVERLAYS.get(provider)
provider_def = get_provider(provider)
for value in (
getattr(overlay, "base_url_override", ""),
getattr(provider_def, "base_url", ""),
):
route = _normalize_route_base_url(value)
if route:
routes.add(route)
except Exception:
pass
try:
from providers import get_provider_profile
profile = get_provider_profile(provider)
route = _normalize_route_base_url(
getattr(profile, "base_url", "")
)
if route:
routes.add(route)
except Exception:
pass
try:
from hermes_cli.auth import PROVIDER_REGISTRY
from hermes_cli.models import normalize_provider as normalize_model_provider
from hermes_cli.providers import normalize_provider as normalize_registry_provider
for provider_id, config in PROVIDER_REGISTRY.items():
canonical_id = normalize_registry_provider(
normalize_model_provider(provider_id)
)
if canonical_id != provider:
continue
route = _normalize_route_base_url(
getattr(config, "inference_base_url", "")
)
if route:
routes.add(route)
except Exception:
pass
if provider == "gemini":
routes.update(
f"{route.rstrip('/')}/openai"
for route in list(routes)
)
return routes
def _context_route_mismatch(
configured_base_url: Any,
active_base_url: Any,
configured_provider: Any,
active_provider: Any,
*,
already_normalized: bool = False,
) -> bool:
"""Return whether a context pin's configured route differs from runtime."""
if already_normalized:
configured_route = str(configured_base_url or "")
active_route = str(active_base_url or "")
else:
configured_route = _normalize_route_base_url(configured_base_url)
active_route = _normalize_route_base_url(active_base_url)
if configured_route:
return configured_route != active_route
configured_provider = str(configured_provider or "").strip()
active_provider = str(active_provider or "").strip()
if not configured_provider:
return False
try:
from hermes_cli.models import normalize_provider as normalize_model_provider
configured_provider = normalize_model_provider(configured_provider)
active_provider = normalize_model_provider(active_provider)
except Exception:
configured_provider = configured_provider.lower()
active_provider = active_provider.lower()
try:
from hermes_cli.providers import normalize_provider as normalize_registry_provider
configured_provider = normalize_registry_provider(configured_provider)
active_provider = normalize_registry_provider(active_provider)
except Exception:
pass
if active_route:
configured_routes = _provider_default_routes(configured_provider)
return not configured_routes or active_route not in configured_routes
return bool(
configured_provider
and active_provider
and configured_provider != active_provider
)
def _normalize_custom_provider_name(value: Any) -> str:
"""Mirror runtime normalization for a requested custom-provider identity."""
return str(value or "").strip().lower().replace(" ", "-")
def _custom_provider_runtime_ids(value: Any) -> set[str]:
"""Return raw/menu identities that runtime accepts for a configured name."""
normalized = _normalize_custom_provider_name(value)
if not normalized:
return set()
return {normalized, f"custom:{normalized}"}
def _build_codex_gpt5_autoraise_notice(
autoraise: Dict[str, Any], context_length: Optional[int] = None
) -> str:
def _build_codex_gpt5_autoraise_notice(autoraise: Dict[str, Any]) -> str:
"""Build the one-time notice shown when Codex gpt-5.x raises compaction.
``autoraise`` is ``{"model": <slug>, "from": <old_ratio>, "to": <new_ratio>}``.
``context_length`` is the live-resolved window from the context compressor
(Codex's /models catalog is authoritative and can change server-side, e.g.
the gpt-5.6 family's 272K → 372K → 272K shifts in July 2026), so the banner
reports what this session actually got rather than a hardcoded cap. The
same text is printed inline for CLI users and replayed via
The same text is printed inline for CLI users and replayed via
``status_callback`` for gateway users, so it must be self-contained and
include the exact opt-back-out command.
"""
model = str(autoraise.get("model") or "gpt-5.4/5.5").strip().lower().rsplit("/", 1)[-1]
if isinstance(context_length, int) and context_length > 0:
cap = f"{round(context_length / 1000)}K"
else:
# Static fallback when the resolved window isn't available:
# gpt-5.3-codex-spark has a native 128K window; the gpt-5.4/5.5/5.6
# family is capped at 272K by the Codex OAuth backend.
cap = "128K" if model.startswith("gpt-5.3-codex-spark") else "272K"
# gpt-5.3-codex-spark has a native 128K window; the gpt-5.4/5.5/5.6 family
# is capped at 272K by the Codex OAuth backend.
cap = "128K" if model.startswith("gpt-5.3-codex-spark") else "272K"
from_pct = int(round(autoraise["from"] * 100))
to_pct = int(round(autoraise["to"] * 100))
return (
@@ -321,26 +187,10 @@ def _normalized_custom_base_url(value: Any) -> str:
def _custom_provider_model_matches(agent_model: str, entry: Dict[str, Any]) -> bool:
agent_model_norm = str(agent_model or "").strip().lower()
# Multi-model entries (v12+ `providers.<name>.models` mapping / legacy
# `models:` list): the agent's model matching ANY catalog entry counts.
# Without this, a provider whose `model`/`default_model` differs from the
# session model silently fails to match and per-provider request settings
# (extra_body, e.g. OpenAI service_tier) are dropped — billing the whole
# session at the wrong tier (July 2026 sweeper incident: flex config
# ignored, ~2.3x overbilling).
models = entry.get("models")
catalog: List[str] = []
if isinstance(models, dict):
catalog = [str(k).strip().lower() for k in models.keys()]
elif isinstance(models, (list, tuple)):
catalog = [str(m).strip().lower() for m in models]
if catalog and agent_model_norm in catalog:
return True
provider_model = str(entry.get("model", "") or "").strip().lower()
if not provider_model and not catalog:
if not provider_model:
return True
return provider_model == agent_model_norm
return provider_model == str(agent_model or "").strip().lower()
def _custom_provider_extra_body_for_agent(
@@ -562,13 +412,13 @@ def init_agent(
agent.skip_context_files = skip_context_files
agent.load_soul_identity = load_soul_identity
agent.pass_session_id = pass_session_id
agent._credential_pool = credential_pool
agent.log_prefix_chars = log_prefix_chars
agent.log_prefix = f"{log_prefix} " if log_prefix else ""
# Store effective base URL for feature detection (prompt caching, reasoning, etc.)
agent.base_url = base_url or ""
provider_name = provider.strip().lower() if isinstance(provider, str) and provider.strip() else None
agent.provider = provider_name or ""
agent._credential_pool = credential_pool
agent.acp_command = acp_command or command
agent.acp_args = list(acp_args or args or [])
if api_mode in {"chat_completions", "codex_responses", "anthropic_messages", "bedrock_converse", "codex_app_server"}:
@@ -604,24 +454,6 @@ def init_agent(
else:
agent.api_mode = "chat_completions"
# Credential-pool validation runs AFTER provider auto-detection so
# a pool scoped to e.g. "anthropic" is not rejected when the agent
# was constructed with provider=None and an anthropic.com URL.
# Regression from #63048 which placed this check before the
# URL-based auto-detection block above (fixed #63425).
if credential_pool is not None:
try:
from agent.credential_pool import credential_pool_matches_provider
if not credential_pool_matches_provider(
credential_pool,
agent.provider,
base_url=agent.base_url,
):
agent._credential_pool = None
except Exception:
agent._credential_pool = None
# Eagerly warm the transport cache so import errors surface at init,
# not mid-conversation. Also validates the api_mode is registered.
try:
@@ -877,25 +709,6 @@ def init_agent(
# commentary when the provider later returns it as a completed interim
# assistant message.
agent._current_streamed_assistant_text = ""
# Completed interim messages delivered during the current user turn.
# Unlike token-stream tracking, this spans Codex continuation/tool calls so
# repeated commentary is not re-sent before normalization can deduplicate it.
agent._delivered_interim_texts: set[str] = set()
# Single-writer guard for the streaming delta sink (#65991). A stale/
# superseded stream (e.g. one the stale-stream detector reconnected past,
# whose socket abort raced and never actually stopped the old worker) must
# NOT keep writing tokens into the turn alongside the retry's stream —
# otherwise two coherent responses interleave token-by-token into one
# transcript. Every streaming attempt claims a monotonic writer token; the
# delta sink drops chunks whose calling thread holds a stale token. The
# threading.local means threads that never claimed (non-streaming callers)
# are never fenced, so the guard can only ever drop a superseded stream,
# never the single legitimate writer.
agent._stream_writer_lock = threading.Lock()
agent._stream_writer_token = 0
agent._stream_writer_tls = threading.local()
agent._stream_writer_dropped = 0
# Optional current-turn user-message override used when the API-facing
# user message intentionally differs from the persisted transcript
@@ -1470,14 +1283,6 @@ def init_agent(
# SQLite session store (optional -- provided by CLI or gateway)
agent._session_db = session_db
agent._parent_session_id = parent_session_id
# A close flush and the worker's turn-start flush can overlap. The durable
# marker is attached to each in-memory message dict, so its test-and-append
# sequence must be serialized per agent rather than relying on SQLite alone.
agent._session_persist_lock = threading.RLock()
# CLI retains its just-accepted user dict until turn setup can reuse it.
# This preserves the message-local durable marker if close persistence wins
# the race before the agent's normal early turn flush.
agent._pending_cli_user_message = None
agent._last_flushed_db_idx = 0 # tracks DB-write cursor to prevent duplicate writes
agent._session_db_created = False # DB row deferred to run_conversation()
# Most agents own their session row and should finalize it on close().
@@ -1507,40 +1312,6 @@ def init_agent(
_agent_cfg = _load_agent_config()
except Exception:
_agent_cfg = {}
# Codex commentary visibility (display.show_commentary, default true).
# When true, completed Codex phase=commentary messages are delivered as
# visible mid-turn updates through the interim message path. When false,
# commentary falls back to the reasoning channel (visible only with
# show_reasoning enabled).
agent.show_commentary = True
try:
_display_section = _agent_cfg.get("display", {})
if isinstance(_display_section, dict):
agent.show_commentary = bool(_display_section.get("show_commentary", True))
except Exception:
agent.show_commentary = True
# LM Studio can either be explicitly preloaded through LM Studio's
# management API (the historical Hermes behavior) or left to LM Studio's
# just-in-time / Auto-Evict chat-completions path. Keep the default
# explicit for backward compatibility; users with LM Studio Auto-Evict can
# opt into JIT via ``model.lmstudio_load_mode: jit``.
agent.lmstudio_load_mode = "explicit"
try:
_model_section = _agent_cfg.get("model", {})
if isinstance(_model_section, dict):
_load_mode = str(_model_section.get("lmstudio_load_mode", "explicit") or "explicit").strip().lower()
if _load_mode in {"explicit", "jit"}:
agent.lmstudio_load_mode = _load_mode
else:
logger.warning(
"Invalid model.lmstudio_load_mode=%r; expected 'explicit' or 'jit'. Using explicit.",
_model_section.get("lmstudio_load_mode"),
)
except Exception:
agent.lmstudio_load_mode = "explicit"
try:
agent._tool_guardrails = ToolCallGuardrailController(
ToolCallGuardrailConfig.from_mapping(
@@ -1561,14 +1332,7 @@ def init_agent(
agent._memory_nudge_interval = 10
agent._turns_since_memory = 0
agent._iters_since_skill = 0
# A flush/background agent may pass skip_memory=True to avoid spinning up an
# external memory *provider*, but if the caller also explicitly enables the
# "memory" toolset it still needs the built-in file-backed store — otherwise
# the memory tool dispatches with store=None and every call fails (#65429).
# So the built-in store is created unless memory is globally disabled, while
# the external-provider block below stays gated on skip_memory.
_memory_toolset_requested = "memory" in (agent.enabled_toolsets or [])
if not skip_memory or _memory_toolset_requested:
if not skip_memory:
try:
mem_config = _agent_cfg.get("memory", {})
agent._memory_enabled = mem_config.get("memory_enabled", False)
@@ -1788,34 +1552,6 @@ def init_agent(
compression_enabled = str(_compression_cfg.get("enabled", True)).lower() in {"true", "1", "yes"}
compression_target_ratio = float(_compression_cfg.get("target_ratio", 0.20))
compression_protect_last = int(_compression_cfg.get("protect_last_n", 20))
# Cap on compression retry rounds before a turn gives up with "max
# compression attempts reached" (compression.max_attempts). Hardcoding 3
# strands sessions that legitimately need more rounds — e.g. a restart
# history reload whose incompressible tool schemas keep the request
# estimate above the threshold even though the messages compress fine
# (the #62605 failure class). Default 3 preserves current behavior, so
# an unset key is behavior-neutral; validated >= 1, hard-capped at 10,
# and any non-int-like value falls back to 3. Booleans are rejected
# (bool subclasses int, so int(True) would silently become 1) and
# fractional floats are rejected rather than truncated — "4.7 attempts"
# is a config mistake, not a request for 4.
_raw_max_attempts = _compression_cfg.get("max_attempts", 3)
if isinstance(_raw_max_attempts, bool):
compression_max_attempts = 3
elif isinstance(_raw_max_attempts, int):
compression_max_attempts = _raw_max_attempts
elif isinstance(_raw_max_attempts, float):
compression_max_attempts = (
int(_raw_max_attempts) if _raw_max_attempts.is_integer() else 3
)
else:
try:
compression_max_attempts = int(str(_raw_max_attempts).strip())
except (TypeError, ValueError):
compression_max_attempts = 3
if compression_max_attempts < 1:
compression_max_attempts = 3
compression_max_attempts = min(compression_max_attempts, 10)
# protect_first_n is the number of non-system messages to protect at
# the head, in addition to the system prompt (which is always
# implicitly protected by the compressor). Floor at 0 — a value of
@@ -1828,17 +1564,6 @@ def init_agent(
compression_abort_on_summary_failure = str(
_compression_cfg.get("abort_on_summary_failure", False)
).lower() in {"true", "1", "yes"}
# Per-model threshold overrides: keys are substring-matched against the
# model name (longest match wins). Empty dict = use the global threshold
# for all models (backward compatible).
_raw_model_thresholds = _compression_cfg.get("model_thresholds", {})
if isinstance(_raw_model_thresholds, dict):
compression_model_thresholds = {
str(k): float(v) for k, v in _raw_model_thresholds.items()
if isinstance(v, (int, float)) and not isinstance(v, bool)
}
else:
compression_model_thresholds = {}
# In-place compaction: when True, compress_context() rewrites the message
# list + rebuilds the system prompt WITHOUT rotating the session id (no
# parent_session_id chain, no `name #N` renumber). See #38763 and
@@ -1927,9 +1652,8 @@ def init_agent(
)
_config_context_length = None
# Resolve custom_providers once before route-scoping a global context pin:
# a named custom provider may keep its base URL only in this list rather
# than repeating it under ``model``.
# Resolve custom_providers list once for reuse below (startup
# context-length override and plugin context-engine init).
try:
from hermes_cli.config import get_compatible_custom_providers
_custom_providers = get_compatible_custom_providers(_agent_cfg)
@@ -1938,163 +1662,6 @@ def init_agent(
if not isinstance(_custom_providers, list):
_custom_providers = []
# ``model.context_length`` describes the configured default model. A
# process launched directly with ``--model`` / ``-m`` has already replaced
# ``agent.model`` before this initializer loads config, so carrying the
# default model's explicit window into that different runtime is stale. The
# live switch/fallback paths already clear this override; keep direct-start
# overrides consistent with them and let provider metadata resolve the
# active model's window instead.
if _config_context_length is not None and isinstance(_model_cfg, dict):
_configured_default_model = str(_model_cfg.get("default") or "").strip()
_configured_default_runtime_model = _configured_default_model
_active_runtime_model = agent.model
if _configured_default_model:
try:
from hermes_cli.model_normalize import normalize_model_for_provider
_configured_default_runtime_model = normalize_model_for_provider(
_configured_default_model, agent.provider
)
_active_runtime_model = normalize_model_for_provider(
agent.model, agent.provider
)
except Exception:
pass
_configured_provider = str(_model_cfg.get("provider") or "").strip()
_configured_base_url = _normalize_route_base_url(
_model_cfg.get("base_url")
)
_configured_provider_norm = _normalize_custom_provider_name(
_configured_provider
)
_custom_provider_candidate = bool(_configured_provider_norm)
_runtime_first_provider_ids = {
"auto",
"moa",
"vertex",
"google-vertex",
"vertex-ai",
"gcp-vertex",
"vertexai",
}
if _configured_provider_norm in _runtime_first_provider_ids:
_custom_provider_candidate = False
elif (
_custom_provider_candidate
and _configured_provider_norm != "custom"
and not _configured_provider_norm.startswith("custom:")
):
try:
from hermes_cli.auth import resolve_provider as resolve_auth_provider
_resolved_auth_provider = resolve_auth_provider(
_configured_provider_norm
)
_custom_provider_candidate = (
str(_resolved_auth_provider or "").strip().lower()
!= _configured_provider_norm
)
except Exception:
pass
if not _configured_base_url and _custom_provider_candidate:
_configured_custom_provider = _normalize_custom_provider_name(
_configured_provider
)
_user_providers = _agent_cfg.get("providers")
_disabled_custom_provider_ids: set[str] = set()
if isinstance(_user_providers, dict):
from hermes_cli.config import is_provider_enabled
for _provider_key, _provider_entry in _user_providers.items():
if not isinstance(_provider_entry, dict):
continue
_entry_name = str(
_provider_entry.get("name") or ""
).strip()
_entry_provider_ids = _custom_provider_runtime_ids(
_provider_key
) | _custom_provider_runtime_ids(_entry_name)
if not is_provider_enabled(_provider_entry):
_disabled_custom_provider_ids.update(
provider_id
for provider_id in _entry_provider_ids
if provider_id
)
continue
if _configured_custom_provider not in _entry_provider_ids:
continue
_configured_base_url = _normalize_route_base_url(
_provider_entry.get("api")
or _provider_entry.get("url")
or _provider_entry.get("base_url")
)
if _configured_base_url:
break
if not _configured_base_url:
for _provider_entry in _custom_providers:
if not isinstance(_provider_entry, dict):
continue
_entry_name = str(
_provider_entry.get("name") or ""
).strip()
_entry_provider_key = str(
_provider_entry.get("provider_key") or ""
).strip().lower()
_entry_provider_ids = _custom_provider_runtime_ids(
_entry_name
) | _custom_provider_runtime_ids(_entry_provider_key)
if (
_entry_provider_key
and _custom_provider_runtime_ids(_entry_provider_key)
& _disabled_custom_provider_ids
):
continue
if _configured_custom_provider not in _entry_provider_ids:
continue
_configured_base_url = _normalize_route_base_url(
_provider_entry.get("base_url")
)
if _configured_base_url:
break
_active_route_url = str(agent.base_url or "")
_requested_route_url = str(base_url or "")
if "?" in _requested_route_url.split("#", 1)[0]:
try:
_requested_parts = urlparse(_requested_route_url)
_requested_without_query = urlunparse(
_requested_parts._replace(query="")
)
if _normalize_route_base_url(
_requested_without_query
) == _normalize_route_base_url(_active_route_url):
_active_route_url = _requested_route_url
except (TypeError, ValueError):
pass
_active_base_url = _normalize_route_base_url(_active_route_url)
_route_mismatch = _context_route_mismatch(
_configured_base_url,
_active_base_url,
_configured_provider,
agent.provider,
already_normalized=True,
)
_model_mismatch = bool(
_configured_default_runtime_model
and _configured_default_runtime_model != _active_runtime_model
)
if _model_mismatch or _route_mismatch:
_ra().logger.debug(
"Ignoring model.context_length=%s for startup runtime %s at %s "
"(configured default is %s at %s)",
_config_context_length,
agent.model,
_active_base_url or agent.provider,
_configured_default_model,
_configured_base_url or _model_cfg.get("provider"),
)
_config_context_length = None
# Store for reuse by _check_compression_model_feasibility (auxiliary
# compression model context-length detection needs the same list).
agent._custom_providers = _custom_providers
@@ -2117,11 +1684,11 @@ def init_agent(
# Surface a clear warning if the user set a context_length but it
# wasn't a valid positive int — the helper silently skips those.
if _config_context_length is None:
_target = _normalize_route_base_url(agent.base_url)
_target = agent.base_url.rstrip("/") if agent.base_url else ""
for _cp_entry in _custom_providers:
if not isinstance(_cp_entry, dict):
continue
_cp_url = _normalize_route_base_url(_cp_entry.get("base_url"))
_cp_url = (_cp_entry.get("base_url") or "").rstrip("/")
if _target and _cp_url == _target:
_cp_models = _cp_entry.get("models", {})
if isinstance(_cp_models, dict):
@@ -2234,16 +1801,6 @@ def init_agent(
provider=agent.provider,
custom_providers=_custom_providers,
)
# Per-model threshold overrides are part of the explicit
# context-engine contract: assign them BEFORE the initial
# update_model() call so the first resolution (which derives
# threshold_percent/threshold_tokens for the initial model) already
# sees the overrides. Assigning after update_model() left the initial
# model on the engine's global threshold until the first /model
# switch. Engines that override update_model() own their own policy
# and may ignore the attribute.
if compression_model_thresholds:
agent.context_compressor.model_thresholds = compression_model_thresholds
agent.context_compressor.update_model(
model=agent.model,
context_length=_plugin_ctx_len,
@@ -2270,7 +1827,6 @@ def init_agent(
api_mode=agent.api_mode,
abort_on_summary_failure=compression_abort_on_summary_failure,
max_tokens=agent.max_tokens,
model_thresholds=compression_model_thresholds,
)
_bind_session_state = getattr(agent.context_compressor, "bind_session_state", None)
if callable(_bind_session_state):
@@ -2281,7 +1837,6 @@ def init_agent(
agent.compression_enabled = compression_enabled
agent.compression_in_place = compression_in_place
agent.codex_app_server_auto_compaction = codex_app_server_auto_compaction
agent.max_compression_attempts = compression_max_attempts
# Reject models whose context window is below the minimum required
# for reliable tool-calling workflows (64K tokens).
@@ -2466,7 +2021,7 @@ def init_agent(
# autoraised model) updates the marker state and re-notifies once. The
# config display gate (compression.codex_gpt55_autoraise_notice) still
# suppresses the banner entirely without disabling the threshold autoraise.
_autoraise = getattr(agent, "_compression_threshold_autoraised", None) or {}
_autoraise = getattr(agent, "_compression_threshold_autoraised", None)
_show_autoraise_notice = (
bool(_autoraise)
and compression_enabled
@@ -2489,10 +2044,7 @@ def init_agent(
# for CLI users; gateway users get the same text replayed via
# _compression_warning on turn 1 (set below).
if _show_autoraise_notice:
print(_build_codex_gpt5_autoraise_notice(
_autoraise,
context_length=getattr(agent.context_compressor, "context_length", None),
))
print(_build_codex_gpt5_autoraise_notice(_autoraise))
# Check immediately so CLI users see the warning at startup.
# Gateway status_callback is not yet wired, so any warning is stored
@@ -2502,10 +2054,7 @@ def init_agent(
# above only reaches the CLI, so stash the same text here to be replayed
# through status_callback on the first turn (Telegram/Discord/Slack/etc.).
if _show_autoraise_notice:
agent._compression_warning = _build_codex_gpt5_autoraise_notice(
_autoraise,
context_length=getattr(agent.context_compressor, "context_length", None),
)
agent._compression_warning = _build_codex_gpt5_autoraise_notice(_autoraise)
# Mark shown so repeated inits in this profile (e.g. every gateway message)
# stay silent. Recorded once, whether the notice went to the CLI print or
+34 -255
View File
@@ -26,11 +26,10 @@ import copy
import json
import logging
import re
import threading
import time
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
from typing import Any, Dict, List, Optional
from hermes_cli.timeouts import get_provider_request_timeout
from agent.prompt_builder import format_steer_marker
@@ -38,7 +37,6 @@ from agent.tool_dispatch_helpers import _trajectory_normalize_msg, make_tool_res
from agent.trajectory import convert_scratchpad_to_think
from agent.credential_pool import STATUS_EXHAUSTED
from agent.error_classifier import FailoverReason
from agent.turn_context import drop_stale_api_content
from utils import base_url_host_matches, base_url_hostname, env_var_enabled, atomic_json_write
logger = logging.getLogger(__name__)
@@ -359,108 +357,6 @@ def sanitize_tool_call_arguments(
return repaired
# Session-scoped in-flight registry backing note_turn_start's cross-agent
# check. The per-agent marker catches a second turn on the SAME AIAgent
# object, but the gateway caches agents per *routing key* (``_agent_cache``
# in gateway/run.py) while the durable transcript is keyed by *session_id* —
# and the key→id mapping is many-to-one (``switch_session``: /resume from a
# second chat/topic, CLI-continuity rebinding, async-delegation pinning,
# topic-binding tip-walks). Two routing keys mapped to one session_id run
# concurrent turns on two different agent objects, which per-agent state can
# never see (#64934). Keyed by session_id so that route produces the same
# named warning. Process-local by design — same visibility scope as the
# per-agent marker it extends.
_INFLIGHT_TURNS_BY_SESSION: Dict[str, Tuple[str, float]] = {}
_INFLIGHT_TURNS_LOCK = threading.Lock()
def note_turn_start(agent, turn_id: str):
"""Tripwire: detect a turn starting while a previous turn of the same
agent — or of the same underlying *session* on a different agent object —
has not completed its turn-end persist.
Two turns interleaving on one session corrupt the durable transcript:
their flushes race (user rows can persist out of arrival order), a row
can be swallowed by the identity-marker dedup over shared history dicts,
and the second turn runs on a history base that never saw the first
turn's exchange. This helper does NOT prevent any of that — it names the
occurrence, with both turn ids, so the dispatch route that let the
second turn through the busy guard can be identified from logs.
Returns the previous in-flight turn_id when an overlap is detected,
else None. Takes ownership of the in-flight slot either way, so a turn
that crashed before its persist produces at most one warning."""
prev = getattr(agent, "_inflight_turn_id", None)
prev_started = getattr(agent, "_inflight_turn_started", 0.0)
agent._inflight_turn_id = turn_id
agent._inflight_turn_started = time.time()
overlap = None
if prev and prev != turn_id:
logger.warning(
"turn %s starting while turn %s (started %.0fs ago) has not "
"completed its turn-end persist (session=%s) — concurrent turns "
"on one session; transcript writes may interleave",
turn_id,
prev,
time.time() - prev_started if prev_started else -1.0,
getattr(agent, "session_id", None) or "-",
)
overlap = prev
# Cross-agent leg: same session_id in flight under a different agent
# object means two routing keys resolve to one durable session — the
# busy guard (keyed by routing key) cannot see this overlap at all.
# Persist-disabled agents (background-review forks) deliberately share
# the live parent's session_id for prompt-cache warmth but can never
# write to the transcript — they must not register here (would warn a
# false overlap against the parent's real turn) nor pop the parent's
# slot at their persist (note_turn_persisted skips them symmetrically).
session_id = getattr(agent, "session_id", None)
if session_id and not getattr(agent, "_persist_disabled", False):
now = time.time()
with _INFLIGHT_TURNS_LOCK:
entry = _INFLIGHT_TURNS_BY_SESSION.get(session_id)
_INFLIGHT_TURNS_BY_SESSION[session_id] = (turn_id, now)
# Stamp the session id this turn registered under: compression can
# rotate agent.session_id mid-turn, and the persist-time clear must
# pop the slot the turn actually holds, not the rotated id.
agent._inflight_turn_session_id = session_id
if entry and entry[0] not in (turn_id, prev):
logger.warning(
"turn %s starting while turn %s (started %.0fs ago) is still "
"in flight on session %s under a different agent object — "
"two routing keys are mapped to one session_id; concurrent "
"turns on one session; transcript writes may interleave",
turn_id,
entry[0],
now - entry[1] if entry[1] else -1.0,
session_id,
)
overlap = overlap or entry[0]
return overlap
def note_turn_persisted(agent):
"""Clear the in-flight marker at turn-end persist (see note_turn_start).
Called from the single persist funnel; unconditional by design — when two
turns genuinely overlap, the first persist clears the second turn's slot
and the tripwire under-reports instead of double-reporting. A diagnostic
must never be noisier than the defect it hunts."""
agent._inflight_turn_id = None
# Symmetric with note_turn_start's cross-agent leg: persist-disabled
# forks never registered a session slot, and their persist funnel still
# runs — popping here would steal the live parent turn's slot and make
# the tripwire under-report the real overlap it exists to catch.
if not getattr(agent, "_persist_disabled", False):
session_id = getattr(agent, "_inflight_turn_session_id", None) or getattr(
agent, "session_id", None
)
if session_id:
with _INFLIGHT_TURNS_LOCK:
_INFLIGHT_TURNS_BY_SESSION.pop(session_id, None)
agent._inflight_turn_session_id = None
def repair_message_sequence(agent, messages: List[Dict]) -> int:
"""Collapse malformed role-alternation left in the live history.
@@ -530,12 +426,6 @@ def repair_message_sequence(agent, messages: List[Dict]) -> int:
or m.get("finish_reason") == "incomplete"
)
def _is_verification_candidate(m: Dict) -> bool:
return m.get("finish_reason") in {
"verification_required",
"verify_hook_continue",
}
collapsed: List[Dict] = []
for msg in messages:
if (
@@ -548,16 +438,6 @@ def repair_message_sequence(agent, messages: List[Dict]) -> int:
and not _is_codex_interim(collapsed[-1])
):
prev = collapsed[-1]
# Verification candidate collapsing: when the earlier assistant
# message is a provisional candidate (finish_reason =
# verification_required / verify_hook_continue), the later
# response supersedes it for model replay — replace rather than
# union. Both remain durable in state.db; this only affects the
# in-memory sequence sent to the model. (#65919 §7)
if _is_verification_candidate(prev):
collapsed[-1] = msg
repairs += 1
continue
# Union tool_calls (preserve order, both may carry them).
prev_calls = list(prev.get("tool_calls") or [])
new_calls = list(msg.get("tool_calls") or [])
@@ -665,10 +545,6 @@ def repair_message_sequence(agent, messages: List[Dict]) -> int:
if prev_content and new_content
else (prev_content or new_content)
)
# Merged content invalidates the api_content sidecar (exact
# bytes previously sent for the pre-merge message) — drop it
# so replay can't substitute stale bytes.
drop_stale_api_content(prev)
repairs += 1
continue
merged.append(msg)
@@ -725,16 +601,16 @@ def strip_think_blocks(agent, content: str) -> str:
"""Remove reasoning/thinking blocks from content, returning only visible text.
Handles four cases:
1. Closed tag pairs (`` <think>… ``) — the common path when
1. Closed tag pairs (``<think>…</think>``) — the common path when
the provider emits complete reasoning blocks.
2. Unterminated open tag at a block boundary (start of text or
after a newline) — e.g. MiniMax M2.7 / NIM endpoints where the
closing tag is dropped. Everything from the open tag to end
of string is stripped. The block-boundary check mirrors
``gateway/stream_consumer.py``'s filter so models that mention
`` <think>`` in prose aren't over-stripped.
``<think>`` in prose aren't over-stripped.
3. Stray orphan open/close tags that slip through.
4. Tag variants: `` <think>``, ``<thinking>``, ``<reasoning>``,
4. Tag variants: ``<think>``, ``<thinking>``, ``<reasoning>``,
``<REASONING_SCRATCHPAD>``, ``<thought>`` (Gemma 4), all
case-insensitive.
@@ -754,39 +630,6 @@ def strip_think_blocks(agent, content: str) -> str:
"""
if not content:
return ""
# Coerce non-string content to text before any regex runs. Providers
# that return assistant ``content`` as a list of blocks (Anthropic via
# OpenRouter emits ``[{"type":"text",...}, {"type":"thinking",...}]``) or
# as a dict flow into this shared helper from several callers — most
# notably ``_interim_assistant_visible_text`` reading a *stored* history
# message whose content was persisted as a list. A raw list/dict reaching
# ``re.sub`` below raises ``TypeError: expected string or bytes-like
# object, got 'list'``, which the outer conversation loop swallows and
# retries forever (observed as an infinite "preparing terminal…" loop on
# Anthropic models via OpenRouter). Flatten here so every caller is safe.
if not isinstance(content, str):
if isinstance(content, list):
_parts: list[str] = []
for _part in content:
if isinstance(_part, str):
_parts.append(_part)
elif isinstance(_part, dict):
_ptype = str(_part.get("type") or "").strip().lower()
# Drop reasoning/thinking blocks outright — this function's
# whole job is to strip them, and their text lives under
# different keys ("thinking", "reasoning") per provider.
if _ptype in {"thinking", "reasoning", "redacted_thinking"}:
continue
_text = _part.get("text")
if isinstance(_text, str) and _text:
_parts.append(_text)
content = "".join(_parts)
elif isinstance(content, dict):
content = str(content.get("text") or content.get("content") or "")
else:
content = str(content)
if not content:
return ""
# 1. Closed tag pairs — case-insensitive for all variants so
# mixed-case tags (<THINK>, <Thinking>) don't slip through to
# the unterminated-tag pass and take trailing content with them.
@@ -952,14 +795,7 @@ def recover_with_credential_pool(
if effective_reason == FailoverReason.billing:
rotate_status = status_code if status_code is not None else 402
next_entry = pool.mark_exhausted_and_rotate(
status_code=rotate_status,
error_context=error_context,
# Runtime credentials can be resolved by a separate pool instance,
# leaving this recovery pool without ``current_id``. Match the key
# that actually failed instead of quarantining a different account.
api_key_hint=getattr(agent, "api_key", None),
)
next_entry = pool.mark_exhausted_and_rotate(status_code=rotate_status, error_context=error_context)
if next_entry is not None:
_ra().logger.info(
"Credential %s (billing) — rotated to pool entry %s",
@@ -1372,42 +1208,7 @@ def restore_primary_runtime(agent) -> bool:
api_mode=rt.get("compressor_api_mode", ""),
)
# ── Rebind and re-select the primary credential pool ──
# A cross-provider fallback attaches the fallback provider's pool. The
# runtime fields above restore the primary, but leaving that pool in
# place makes the next primary 401/429 hit the provider-mismatch guard
# and disables credential rotation. Reload the primary pool first; if
# auth storage is temporarily unreadable, clear the mismatched pool.
primary_provider = str(rt.get("provider") or "").strip().lower()
pool = getattr(agent, "_credential_pool", None)
pool_provider = str(getattr(pool, "provider", "") or "").strip().lower()
pool_matches_primary = pool_provider == primary_provider
if (
primary_provider == "custom"
and pool_provider.startswith("custom:")
):
try:
from agent.credential_pool import get_custom_provider_pool_key
primary_key = (
get_custom_provider_pool_key(str(rt.get("base_url") or "")) or ""
).strip().lower()
pool_matches_primary = bool(primary_key) and primary_key == pool_provider
except Exception:
pool_matches_primary = False
if pool is not None and pool_provider and not pool_matches_primary:
agent._credential_pool = None
try:
from agent.credential_pool import load_pool
agent._credential_pool = load_pool(primary_provider)
except Exception as exc:
logger.warning(
"Restore could not reload primary credential pool for %s: %s",
primary_provider,
exc,
)
# ── Re-select from the credential pool if one is available ──
# The snapshot's api_key was captured at construction time. Across
# turns the pool may have rotated (token revocation, billing/rate-limit
# exhaustion, cooldown), leaving the snapshot key stale. Restoring it
@@ -1421,6 +1222,7 @@ def restore_primary_runtime(agent) -> bool:
entry = pool.select()
if entry is not None:
entry_provider = str(getattr(entry, "provider", "") or "").strip().lower()
primary_provider = str(rt.get("provider") or "").strip().lower()
entry_matches_primary = entry_provider == primary_provider
# Custom endpoints all carry the generic ``custom`` provider on
# the agent while the pool entry is keyed ``custom:<name>`` (see
@@ -1469,13 +1271,6 @@ def restore_primary_runtime(agent) -> bool:
primary_provider or "?",
)
# ── Restore reasoning_config if it was saved ──
# switch_model saves reasoning_config in _primary_runtime. If the
# snapshot predates that (older sessions), keep the current value.
saved_reasoning = rt.get("reasoning_config")
if saved_reasoning is not None:
agent.reasoning_config = dict(saved_reasoning)
# ── Reset fallback chain for the new turn ──
agent._fallback_activated = False
agent._fallback_index = 0
@@ -2063,9 +1858,6 @@ def switch_model(agent, new_model, new_provider, api_key='', base_url='', api_mo
old_norm = (old_provider or "").strip().lower()
new_norm = (new_provider or "").strip().lower()
if old_norm != new_norm or getattr(agent, "_credential_pool", None) is None:
# A pool bound to the old provider is worse than no pool: the
# recovery guard rejects it and every later 401/429 skips rotation.
agent._credential_pool = None
try:
from agent.credential_pool import load_pool
agent._credential_pool = load_pool(new_provider)
@@ -2236,24 +2028,6 @@ def switch_model(agent, new_model, new_provider, api_key='', base_url='', api_mo
api_mode=agent.api_mode,
)
# ── Re-resolve reasoning_config from per-model override ──
# The new model may have a different reasoning_effort override. Re-read
# config so the override takes effect immediately on /model switch —
# resolved through the shared chokepoint (per-model > global; YAML
# boolean False = disabled).
try:
from hermes_constants import resolve_reasoning_config
from hermes_cli.config import load_config as _sm_load_config
_reasoning_cfg = _sm_load_config() or {}
agent.reasoning_config = resolve_reasoning_config(_reasoning_cfg, agent.model)
logger.info(
"switch_model: reasoning_config resolved for %s: %s",
agent.model, agent.reasoning_config,
)
except Exception as _reasoning_err:
logger.debug("switch_model: could not re-resolve reasoning_config: %s", _reasoning_err)
# ── Invalidate cached system prompt so it rebuilds next turn ──
agent._cached_system_prompt = None
@@ -2276,7 +2050,6 @@ def switch_model(agent, new_model, new_provider, api_key='', base_url='', api_mo
"client_kwargs": dict(agent._client_kwargs),
"use_prompt_caching": agent._use_prompt_caching,
"use_native_cache_layout": agent._use_native_cache_layout,
"reasoning_config": dict(agent.reasoning_config) if getattr(agent, "reasoning_config", None) else None,
"compressor_model": getattr(_cc, "model", agent.model) if _cc else agent.model,
"compressor_base_url": getattr(_cc, "base_url", agent.base_url) if _cc else agent.base_url,
"compressor_api_key": getattr(_cc, "api_key", "") if _cc else "",
@@ -3256,10 +3029,6 @@ def extract_api_error_context(error: Exception) -> Dict[str, Any]:
if isinstance(reason, str) and reason.strip():
context["reason"] = reason.strip()
message = payload.get("message") or payload.get("error_description")
if not message and isinstance(payload.get("error"), str):
# xAI uses a top-level string ``error`` beside a structured
# ``code`` (for example personal-team-blocked:spending-limit).
message = payload.get("error")
if isinstance(message, str) and message.strip():
context["message"] = message.strip()
for key in ("resets_at", "reset_at"):
@@ -3392,21 +3161,20 @@ def apply_pending_steer_to_tool_results(agent, messages: list, num_tool_msgs: in
def force_close_tcp_sockets(client: Any) -> int:
"""Abort in-flight TCP I/O by shutting down sockets WITHOUT closing FDs.
def force_close_tcp_sockets(client: Any, *, release_fds: bool = False) -> int:
"""Abort in-flight TCP I/O by shutting down pool sockets.
When a provider drops a connection mid-stream — or the user issues an
interrupt — we want to unblock httpx's reader/writer immediately rather
than waiting for the kernel's per-connection timeout. ``shutdown(SHUT_RDWR)``
achieves that: it sends FIN, breaks any pending ``recv``/``send`` with EOF
or ``EPIPE``, but does NOT release the file descriptor.
or ``EPIPE``.
Historically this helper also called ``socket.close()`` so the FD got
released immediately, but that's unsafe when (as is the case for both the
interrupt-abort path and stale-call kill path) the helper runs on a
different thread than the one driving the request:
By default (``release_fds=False``) this helper does **not** call
``socket.close()`` / release the FD. That default is load-bearing for
cross-thread abort paths (#29507):
* The Python ``socket.socket`` we close here is the SAME object held by
* The Python ``socket.socket`` we close is the SAME object held by
httpx's pool, so closing it via Python sets its ``_fd`` to -1 and
future operations on that Python object fail safely.
* BUT the SSL wrapper (``ssl.SSLSocket``'s underlying OpenSSL ``BIO``)
@@ -3418,15 +3186,20 @@ def force_close_tcp_sockets(client: Any) -> int:
wrong file (issue #29507: 24-byte TLS application-data record
clobbering SQLite header bytes 5..28).
The fix is to let the owning thread own the close. ``shutdown()`` from any
thread is FD-safe; ``close()`` is not. The httpx connection's own close
path — which runs from the worker thread when it unwinds — will release
the FD via the same ``socket.socket`` object, and because Python's socket
close atomically swaps ``_fd`` to -1 *before* issuing ``os.close``, there
is no FD-aliasing window when only one thread closes.
``shutdown()`` from any thread is FD-safe; ``close()`` is not when a
stranger thread still has the BIO holding the raw FD.
Returns the number of sockets shut down. (Field kept as
``tcp_force_closed=N`` in the log line for backwards-compatible parsing.)
When the **owning** thread is disposing of a client that is no longer
shared (``_close_openai_client`` after replace / request-complete), pass
``release_fds=True``. httpx's own ``client.close()`` does not reliably
``os.close()`` sockets that were already ``shutdown()``'d, so without an
explicit ``sock.close()`` those FDs stay in kernel CLOSED state forever
and accumulate under long-lived gateways (issue #61979 — ~1 CLOSED fd
per ~6 minutes through a local proxy path).
Returns the number of sockets shut down (and optionally closed). Field
kept as ``tcp_force_closed=N`` in log lines for backwards-compatible
parsing.
"""
import socket as _socket
@@ -3438,7 +3211,13 @@ def force_close_tcp_sockets(client: Any) -> int:
except OSError:
# Already shut down / not connected / FD invalid — all benign.
pass
# IMPORTANT (#29507): do NOT call sock.close() here. See docstring.
# IMPORTANT (#29507): never release FDs from stranger-thread
# abort paths. Only the owning-thread close path may opt in.
if release_fds:
try:
sock.close()
except OSError:
pass
shutdown_count += 1
except Exception as exc:
_ra().logger.debug("Force-close TCP sockets sweep error: %s", exc)
+27 -64
View File
@@ -65,7 +65,6 @@ THINKING_BUDGET = {"xhigh": 32000, "high": 16000, "medium": 8000, "low": 4000}
# maps to low on every model. See:
# https://platform.claude.com/docs/en/about-claude/models/migration-guide
ADAPTIVE_EFFORT_MAP = {
"ultra": "max",
"max": "max",
"xhigh": "xhigh",
"high": "high",
@@ -127,8 +126,6 @@ _FAST_MODE_SUPPORTED_SUBSTRINGS = ("opus-4-6", "opus-4.6")
_ANTHROPIC_OUTPUT_LIMITS = {
# Mythos-class named models (claude-fable-5, …) — 1M context, reasoning
"claude-fable": 128_000,
# Claude Sonnet 5
"claude-sonnet-5": 128_000,
# Claude 4.8
"claude-opus-4-8": 128_000,
# Claude 4.7
@@ -249,13 +246,7 @@ def _supports_adaptive_thinking(model: str) -> bool:
only returns False for the explicit legacy list of older Claude families
that require manual budget-based thinking. Non-Claude Anthropic-Messages
models (minimax, qwen3, …) return False so they keep the manual path.
Kimi / Moonshot models are the exception: their Anthropic-compatible
endpoints implement the adaptive contract (``thinking.type="adaptive"``
+ ``output_config.effort``, including ``xhigh`` and ``display``).
"""
if _model_name_is_kimi_family(model):
return True
if not _is_claude_model(model):
return False
m = model.lower()
@@ -457,8 +448,7 @@ def _is_kimi_coding_endpoint(base_url: str | None) -> bool:
# Model-name prefixes that identify the Kimi / Moonshot family. Covers
# - official slugs: ``kimi-k2.5``, ``kimi_thinking``, ``moonshot-v1-8k``
# - common release lines: ``k1.5-...``, ``k2-thinking``, ``k25-...``, ``k2.5-...``,
# and the bare Coding Plan slug ``k3`` (plus ``k3.x``/``k3-...`` variants)
# - common release lines: ``k1.5-...``, ``k2-thinking``, ``k25-...``, ``k2.5-...``
# Matched case-insensitively against the post-``normalize_model_name`` form,
# so a caller's ``provider/vendor/model`` slug is handled the same as a
# bare name.
@@ -468,14 +458,8 @@ _KIMI_FAMILY_MODEL_PREFIXES = (
"k1.", "k1-",
"k2.", "k2-",
"k25", "k2.5",
"k3.", "k3-",
)
# Bare release slugs with no separator suffix (Kimi Coding Plan serves K3
# as the exact slug ``k3``). Kept exact-match so unrelated model names that
# merely start with the same characters don't get misclassified.
_KIMI_FAMILY_EXACT_SLUGS = frozenset({"k3"})
def _model_name_is_kimi_family(model: str | None) -> bool:
if not isinstance(model, str):
@@ -486,8 +470,6 @@ def _model_name_is_kimi_family(model: str | None) -> bool:
# Strip vendor prefix (e.g. ``moonshotai/kimi-k2.5`` → ``kimi-k2.5``)
if "/" in m:
m = m.rsplit("/", 1)[-1]
if m in _KIMI_FAMILY_EXACT_SLUGS:
return True
return m.startswith(_KIMI_FAMILY_MODEL_PREFIXES)
@@ -551,9 +533,8 @@ def _requires_bearer_auth(base_url: str | None) -> bool:
Some third-party /anthropic endpoints implement Anthropic's Messages API but
require Authorization: Bearer instead of Anthropic's native x-api-key header.
MiniMax's global and China Anthropic-compatible endpoints, Azure AI
Foundry's Anthropic-style endpoint, and Palantir Foundry's LLM proxy
follow this pattern.
MiniMax's global and China Anthropic-compatible endpoints, and Azure AI
Foundry's Anthropic-style endpoint follow this pattern.
"""
normalized = _normalize_base_url_text(base_url)
if not normalized:
@@ -562,11 +543,6 @@ def _requires_bearer_auth(base_url: str | None) -> bool:
return (
normalized.startswith(("https://api.minimax.io/anthropic", "https://api.minimaxi.com/anthropic"))
or "azure.com" in normalized
# Palantir Foundry LLM proxy (<org>.palantirfoundry.com/api/v2/llm/proxy/anthropic)
# rejects x-api-key with 401 and requires Authorization: Bearer.
# Hostname match (not substring) so e.g. evil.com/palantirfoundry
# paths don't trigger Bearer auth.
or base_url_host_matches(normalized, "palantirfoundry.com")
)
@@ -1591,10 +1567,7 @@ def _is_bedrock_model_id(model: str) -> bool:
"""
lower = model.lower()
# Regional inference-profile prefixes
if any(lower.startswith(p) for p in (
"global.", "us.", "eu.", "apac.", "ap.", "au.", "jp.",
"ca.", "sa.", "me.", "af.",
)):
if any(lower.startswith(p) for p in ("global.", "us.", "eu.", "ap.", "jp.")):
return True
# Bare Bedrock model IDs: provider.model-family
if lower.startswith("anthropic."):
@@ -2296,6 +2269,13 @@ def _manage_thinking_signatures(
"""
_THINKING_TYPES = frozenset(("thinking", "redacted_thinking"))
_is_third_party = _is_third_party_anthropic_endpoint(base_url)
# Kimi / DeepSeek share a contract: strip signed Anthropic blocks
# (neither upstream can validate Anthropic signatures), preserve unsigned
# ones synthesised from reasoning_content. See #13848, #16748.
_preserve_unsigned_thinking = (
_is_kimi_family_endpoint(base_url, model)
or _is_deepseek_anthropic_endpoint(base_url)
)
last_assistant_idx = None
for i in range(len(result) - 1, -1, -1):
@@ -2307,12 +2287,8 @@ def _manage_thinking_signatures(
if m.get("role") != "assistant" or not isinstance(m.get("content"), list):
continue
if _is_kimi_family_endpoint(base_url, model):
# Kimi does not enforce thinking signatures — replay as-is
# (shared cleanup below still strips cache markers + the internal flag).
pass
elif _is_deepseek_anthropic_endpoint(base_url):
# DeepSeek: strip signed, preserve unsigned.
if _preserve_unsigned_thinking:
# Kimi / DeepSeek: strip signed, preserve unsigned.
new_content = []
for b in m["content"]:
if not isinstance(b, dict) or b.get("type") not in _THINKING_TYPES:
@@ -2412,24 +2388,6 @@ def _evict_old_screenshots(result: List[Dict[str, Any]]) -> None:
]
def _ensure_leading_user_turn(result: List[Dict[str, Any]]) -> None:
"""Anthropic requires messages[0] to have role=user.
After a second context compaction on the auto path the summary can be
emitted as role=assistant with nothing in front of it (the system prompt
lives outside messages[] or is extracted into the separate ``system``
param), so messages[0] ends up assistant and the Messages API rejects
the request with HTTP 400 — often masked by a misleading
"tool_use ids were found without tool_result blocks" error (#52160).
Mirror the Bedrock Converse adapter, which unconditionally prepends a
minimal user turn when the first message is not user
(convert_messages_to_converse).
"""
if result and result[0].get("role") != "user":
result.insert(0, {"role": "user", "content": [{"type": "text", "text": " "}]})
def convert_messages_to_anthropic(
messages: List[Dict],
base_url: str | None = None,
@@ -2488,7 +2446,6 @@ def convert_messages_to_anthropic(
_strip_orphaned_tool_blocks(result)
result = _merge_consecutive_roles(result)
_ensure_leading_user_turn(result)
_manage_thinking_signatures(result, base_url, model)
_evict_old_screenshots(result)
@@ -2663,19 +2620,25 @@ def build_anthropic_kwargs(
# MiniMax Anthropic-compat endpoints support thinking (manual mode only,
# not adaptive). Haiku does NOT support extended thinking — skip entirely.
#
# Kimi / Moonshot models also use adaptive thinking: their
# Anthropic-compatible endpoints (api.moonshot.cn/anthropic,
# api.kimi.com/coding) accept ``thinking.type="adaptive"`` +
# ``output_config.effort``, and the replay-validation 400s that
# originally motivated dropping the parameter (#13848) no longer
# occur. (Kimi on chat_completions enables thinking via extra_body
# in the ChatCompletionsTransport — see #13503.)
# Kimi's /coding endpoint speaks the Anthropic Messages protocol but has
# its own thinking semantics: when ``thinking.enabled`` is sent, Kimi
# validates the message history and requires every prior assistant
# tool-call message to carry OpenAI-style ``reasoning_content``. The
# Anthropic path never populates that field, and
# ``convert_messages_to_anthropic`` strips all Anthropic thinking blocks
# on third-party endpoints — so the request fails with HTTP 400
# "thinking is enabled but reasoning_content is missing in assistant
# tool call message at index N". Kimi's reasoning is driven server-side
# on the /coding route, so skip Anthropic's thinking parameter entirely
# for that host. (Kimi on chat_completions enables thinking via
# extra_body in the ChatCompletionsTransport — see #13503.)
#
# On 4.7+ the `thinking.display` field defaults to "omitted", which
# silently hides reasoning text that Hermes surfaces in its CLI. We
# request "summarized" so the reasoning blocks stay populated — matching
# 4.6 behavior and preserving the activity-feed UX during long tool runs.
if reasoning_config and isinstance(reasoning_config, dict):
_is_kimi_coding = _is_kimi_family_endpoint(base_url, model)
if reasoning_config and isinstance(reasoning_config, dict) and not _is_kimi_coding:
if reasoning_config.get("enabled") is not False and "haiku" not in model.lower():
effort = str(reasoning_config.get("effort", "medium")).lower()
budget = THINKING_BUDGET.get(effort, 8000)
-16
View File
@@ -66,19 +66,3 @@ def safe_schedule_threadsafe(
coro.close()
log.log(log_level, "%s: %s", log_message, exc)
return None
def consume_detached_task_result(task: "asyncio.Future[Any]") -> None:
"""Retrieve a detached task's result without surfacing cancellation.
Used as an ``add_done_callback`` on tasks that were cancelled and
detached (e.g. an adapter close path that swallows ``CancelledError``
past its teardown deadline). Observing ``task.exception()`` prevents
"exception was never retrieved" noise on the event loop; cancellation
and any terminal error are deliberately swallowed — the task's owner
already gave up on it.
"""
try:
task.exception()
except (asyncio.CancelledError, Exception):
pass
-138
View File
@@ -1,138 +0,0 @@
"""Ambient session-accounting context for auxiliary LLM calls.
Auxiliary calls (vision, compression, title generation, web_extract,
session_search, ...) funnel through ``agent.auxiliary_client`` which has no
session handle — so their token usage was historically discarded, leaving
dashboard analytics blind to aux model spend (issue #23270).
Instead of threading ``session_db``/``session_id`` parameters through every
aux call site, the agent loop publishes them here (mirroring the Nous Portal
conversation context in ``agent.portal_tags``) and the auxiliary client
records usage at its single response-validation chokepoint.
ContextVar semantics give us the right isolation for free:
* concurrent agents in one process (gateway sessions, delegate subagents)
never see each other's accounting context;
* worker threads spawned via ``tools.thread_context.propagate_context_to_thread``
(MoA fan-out, background review) inherit the parent turn's context;
* asyncio tasks inherit the context of the code that created them.
MoA reference/aggregator slots are explicitly EXCLUDED from recording:
``agent/conversation_loop.py`` already folds MoA advisor usage and cost into
the main loop's ``update_token_counts`` delta, so recording them here would
double-count (see ``_EXCLUDED_TASKS``).
"""
from __future__ import annotations
import logging
from contextvars import ContextVar
from typing import Any, Optional
logger = logging.getLogger(__name__)
# (session_db, session_id) for the active agent turn, or None outside one.
_accounting: ContextVar[Optional[tuple]] = ContextVar(
"aux_accounting_context", default=None
)
# Aux tasks whose usage is already accounted by the main loop — recording
# them here would double-count. MoA advisor/aggregator usage is folded into
# conversation_loop's update_token_counts delta (tokens AND cost).
_EXCLUDED_TASKS = frozenset({"moa_reference", "moa_aggregator"})
def set_accounting_context(session_db: Any, session_id: Optional[str]):
"""Publish the active session's accounting handles for aux usage recording.
Called by the agent loop at turn entry. Returns the ContextVar token so
callers can ``reset_accounting_context(token)`` on turn exit. Publishing
``None`` handles (no DB / no session id) clears the context.
"""
if session_db is None or not session_id:
return _accounting.set(None)
return _accounting.set((session_db, session_id))
def reset_accounting_context(token) -> None:
"""Restore the previous accounting context (pair with ``set_...``)."""
try:
_accounting.reset(token)
except Exception:
_accounting.set(None)
def get_accounting_context() -> Optional[tuple]:
"""Return ``(session_db, session_id)`` for the active turn, or ``None``."""
return _accounting.get()
def record_aux_usage(
response: Any,
task: Optional[str],
*,
provider: Optional[str] = None,
base_url: Optional[str] = None,
) -> None:
"""Record an auxiliary response's token usage against the ambient session.
Called from the auxiliary client's response-validation chokepoint. Strictly
best-effort: any failure is swallowed (accounting must never break an aux
call). No-ops when:
* no accounting context is published (call is outside any agent turn),
* the task is main-loop-accounted (MoA slots — see ``_EXCLUDED_TASKS``),
* the response carries no usage object.
The model is read from ``response.model`` (accurate even after the aux
client's provider-fallback chains); *provider*/*base_url* reflect the
originally-resolved route and are best-effort.
"""
try:
if not task or task in _EXCLUDED_TASKS:
return
ctx = _accounting.get()
if ctx is None:
return
session_db, session_id = ctx
raw_usage = getattr(response, "usage", None)
if raw_usage is None:
return
from agent.usage_pricing import estimate_usage_cost, normalize_usage
usage = normalize_usage(raw_usage, provider=provider)
if not (
usage.input_tokens or usage.output_tokens
or usage.cache_read_tokens or usage.cache_write_tokens
or usage.reasoning_tokens
):
return
model = str(getattr(response, "model", "") or "") or "unknown"
estimated_cost = None
try:
cost = estimate_usage_cost(
model, usage, provider=provider, base_url=base_url
)
if cost.amount_usd is not None:
estimated_cost = float(cost.amount_usd)
except Exception:
logger.debug("Aux usage cost estimation failed", exc_info=True)
session_db.record_auxiliary_usage(
session_id,
task,
model=model,
billing_provider=provider,
billing_base_url=base_url,
input_tokens=usage.input_tokens,
output_tokens=usage.output_tokens,
cache_read_tokens=usage.cache_read_tokens,
cache_write_tokens=usage.cache_write_tokens,
reasoning_tokens=usage.reasoning_tokens,
estimated_cost_usd=estimated_cost,
)
except Exception:
logger.debug("Aux usage recording failed (non-fatal)", exc_info=True)
+116 -674
View File
File diff suppressed because it is too large Load Diff
-13
View File
@@ -696,19 +696,6 @@ def _run_review_in_thread(
if isinstance(_rt.get("command"), str) and _rt["command"]:
_fork_kwargs["acp_command"] = _rt["command"]
_fork_kwargs["acp_args"] = _rt.get("args") or []
# Match parent's reasoning config so the fork's ``thinking`` /
# ``output_config`` are byte-identical in the request body —
# Anthropic's cache key is namespaced by ``thinking`` presence.
# Same-model path only: when routed to a different aux model the
# cache is cold regardless (parity buys nothing) and the parent's
# effort vocabulary may not be valid for the routed model/provider
# (e.g. OpenRouter ``extra_body.reasoning.effort`` is forwarded
# unclamped; codex_responses passes ``max``/``ultra`` through
# unmapped except on gpt-5.6/xAI). Let the routed fork use
# provider defaults — matching the ``not _routed`` gate on
# _cached_system_prompt below.
if not _routed:
_fork_kwargs["reasoning_config"] = getattr(agent, "reasoning_config", None)
review_agent = AIAgent(
model=_rt.get("model") or agent.model,
max_iterations=16,
-131
View File
@@ -1,131 +0,0 @@
"""System-battery read-out for the CLI/TUI status bar.
Reads the host battery through ``psutil`` (already a Hermes dependency) and
exposes a compact, colour-coded label. Everything degrades to "unavailable"
when there is no battery (desktops, servers, VMs) or when the read fails, so
callers can render the result unconditionally and simply show nothing.
The status bar repaints often (every keystroke and on a ~1s idle refresh), so
:func:`read_battery` memoises the last reading for a few seconds instead of
hitting ``psutil`` on every frame.
"""
from __future__ import annotations
import time
from dataclasses import dataclass
from typing import Optional
@dataclass(frozen=True)
class BatteryStatus:
"""A single battery reading.
``available`` is False on machines without a battery (or when the read
failed). ``percent`` is clamped to 0-100. ``plugged`` is True when on AC
power, False on battery, and None when the platform can't tell.
"""
available: bool
percent: Optional[int] = None
plugged: Optional[bool] = None
@property
def charging(self) -> bool:
return bool(self.plugged)
UNAVAILABLE = BatteryStatus(available=False)
# Colour buckets, mirroring the status-bar context styles but inverted (a full
# battery is "good", an empty one is "critical").
CATEGORY_GOOD = "good"
CATEGORY_WARN = "warn"
CATEGORY_BAD = "bad"
CATEGORY_CRITICAL = "critical"
CATEGORY_DIM = "dim"
_CACHE_TTL_SECONDS = 8.0
_cache: Optional[tuple[float, BatteryStatus]] = None
def _read_battery_uncached() -> BatteryStatus:
try:
import psutil
except Exception:
return UNAVAILABLE
# ``sensors_battery`` is missing on some platforms/builds of psutil.
reader = getattr(psutil, "sensors_battery", None)
if reader is None:
return UNAVAILABLE
try:
batt = reader()
except Exception:
return UNAVAILABLE
if batt is None:
return UNAVAILABLE
percent: Optional[int] = None
raw_percent = getattr(batt, "percent", None)
if raw_percent is not None:
try:
percent = max(0, min(100, int(round(float(raw_percent)))))
except (TypeError, ValueError):
percent = None
plugged = getattr(batt, "power_plugged", None)
if plugged is not None:
plugged = bool(plugged)
return BatteryStatus(available=True, percent=percent, plugged=plugged)
def read_battery(use_cache: bool = True) -> BatteryStatus:
"""Return the current battery status (cached for a few seconds)."""
global _cache
if use_cache and _cache is not None:
ts, cached = _cache
if time.monotonic() - ts < _CACHE_TTL_SECONDS:
return cached
status = _read_battery_uncached()
_cache = (time.monotonic(), status)
return status
def clear_cache() -> None:
"""Drop the memoised reading (used by tests)."""
global _cache
_cache = None
def battery_category(status: BatteryStatus) -> str:
"""Bucket a reading into a colour category: good/warn/bad/critical/dim."""
if not status.available or status.percent is None:
return CATEGORY_DIM
# On AC power the level isn't a concern — always read as healthy.
if status.charging:
return CATEGORY_GOOD
pct = status.percent
if pct <= 10:
return CATEGORY_CRITICAL
if pct <= 20:
return CATEGORY_BAD
if pct <= 50:
return CATEGORY_WARN
return CATEGORY_GOOD
def battery_glyph(status: BatteryStatus) -> str:
"""Return the leading glyph: a bolt while charging, else a battery."""
return "\u26a1" if status.charging else "\U0001f50b" # ⚡ / 🔋
def format_battery(status: BatteryStatus) -> str:
"""Return a compact label like ``🔋 82%`` / ``⚡ 82%`` (empty if N/A)."""
if not status.available or status.percent is None:
return ""
return f"{battery_glyph(status)} {status.percent}%"
+23 -204
View File
@@ -448,10 +448,7 @@ def is_anthropic_bedrock_model(model_id: str) -> bool:
"""
model_lower = model_id.lower()
# Strip regional prefix if present
for prefix in (
"global.", "us.", "eu.", "apac.", "ap.", "au.", "jp.",
"ca.", "sa.", "me.", "af.",
):
for prefix in ("us.", "global.", "eu.", "ap.", "jp."):
if model_lower.startswith(prefix):
model_lower = model_lower[len(prefix):]
break
@@ -493,26 +490,6 @@ def convert_tools_to_converse(tools: List[Dict]) -> List[Dict]:
return result
# Bedrock's Converse API rejects any text content block whose text is empty
# OR whitespace-only (ValidationException: "text content blocks must contain
# non-whitespace text"). A lone space is whitespace and is rejected too — the
# placeholder MUST itself be non-whitespace. Ref: issue #9486.
_EMPTY_TEXT_PLACEHOLDER = "(empty)"
def _safe_text(text) -> str:
"""Return ``text`` if it's non-whitespace, else a non-whitespace placeholder.
Handles None, empty string, and whitespace-only string (spaces, tabs,
newlines) — all of which Bedrock's Converse API rejects as text content.
"""
if text is None:
return _EMPTY_TEXT_PLACEHOLDER
if not isinstance(text, str):
text = str(text)
return text if text.strip() else _EMPTY_TEXT_PLACEHOLDER
def _convert_content_to_converse(content) -> List[Dict]:
"""Convert OpenAI message content (string or list) to Converse content blocks.
@@ -520,27 +497,26 @@ def _convert_content_to_converse(content) -> List[Dict]:
- Plain text strings → [{"text": "..."}]
- Content arrays with text/image_url parts → mixed text/image blocks
Replaces empty/whitespace-only text blocks with a non-whitespace
placeholder — Bedrock's Converse API rejects messages where a text
content block is empty or whitespace-only (ValidationException:
"text content blocks must contain non-whitespace text"). Ref: issue #9486.
Filters out empty text blocks — Bedrock's Converse API rejects messages
where a text content block has an empty ``text`` field (ValidationException:
"text content blocks must be non-empty"). Ref: issue #9486.
"""
if content is None:
return [{"text": _safe_text(content)}]
return [{"text": " "}]
if isinstance(content, str):
return [{"text": _safe_text(content)}]
return [{"text": content}] if content.strip() else [{"text": " "}]
if isinstance(content, list):
blocks = []
for part in content:
if isinstance(part, str):
blocks.append({"text": _safe_text(part)})
blocks.append({"text": part})
continue
if not isinstance(part, dict):
continue
part_type = part.get("type", "")
if part_type == "text":
text = part.get("text", "")
blocks.append({"text": _safe_text(text)})
blocks.append({"text": text if text else " "})
elif part_type == "image_url":
image_url = part.get("image_url", {})
url = image_url.get("url", "") if isinstance(image_url, dict) else ""
@@ -552,27 +528,18 @@ def _convert_content_to_converse(content) -> List[Dict]:
mime_part = header[5:].split(";")[0]
if mime_part:
media_type = mime_part
# Decode base64 to raw bytes — boto3 re-encodes at the
# wire layer, so passing the base64 string directly
# results in double-encoding and Bedrock rejects it with
# "Failed to sanitize image". Ref: #33317.
import base64
try:
raw_bytes = base64.b64decode(data)
except Exception:
raw_bytes = data.encode("utf-8")
blocks.append({
"image": {
"format": media_type.split("/")[-1] if "/" in media_type else "jpeg",
"source": {"bytes": raw_bytes},
"source": {"bytes": data},
}
})
else:
# Remote URL — Converse doesn't support URLs directly,
# include as text reference for the model.
blocks.append({"text": f"[Image: {url}]"})
return blocks if blocks else [{"text": _EMPTY_TEXT_PLACEHOLDER}]
return [{"text": _safe_text(content)}]
return blocks if blocks else [{"text": " "}]
return [{"text": str(content)}]
def convert_messages_to_converse(
@@ -602,18 +569,14 @@ def convert_messages_to_converse(
content = msg.get("content")
if role == "system":
# System messages become the system prompt. Blank/whitespace-only
# parts are dropped entirely (not placeholder-filled) since a
# system prompt made up of only placeholder text is meaningless.
# System messages become the system prompt
if isinstance(content, str) and content.strip():
system_blocks.append({"text": content})
elif isinstance(content, list):
for part in content:
if isinstance(part, dict) and part.get("type") == "text":
text = part.get("text", "")
if isinstance(text, str) and text.strip():
system_blocks.append({"text": text})
elif isinstance(part, str) and part.strip():
system_blocks.append({"text": part.get("text", "")})
elif isinstance(part, str):
system_blocks.append({"text": part})
continue
@@ -624,7 +587,7 @@ def convert_messages_to_converse(
tool_result_block = {
"toolResult": {
"toolUseId": tool_call_id,
"content": [{"text": _safe_text(result_content)}],
"content": [{"text": result_content}],
}
}
# In Converse, tool results go in a "user" role message
@@ -663,7 +626,7 @@ def convert_messages_to_converse(
})
if not content_blocks:
content_blocks = [{"text": _EMPTY_TEXT_PLACEHOLDER}]
content_blocks = [{"text": " "}]
# Merge with previous assistant message if needed (strict alternation)
if converse_msgs and converse_msgs[-1]["role"] == "assistant":
@@ -689,11 +652,11 @@ def convert_messages_to_converse(
# Converse requires the first message to be from the user
if converse_msgs and converse_msgs[0]["role"] != "user":
converse_msgs.insert(0, {"role": "user", "content": [{"text": _EMPTY_TEXT_PLACEHOLDER}]})
converse_msgs.insert(0, {"role": "user", "content": [{"text": " "}]})
# Converse requires the last message to be from the user
if converse_msgs and converse_msgs[-1]["role"] != "user":
converse_msgs.append({"role": "user", "content": [{"text": _EMPTY_TEXT_PLACEHOLDER}]})
converse_msgs.append({"role": "user", "content": [{"text": " "}]})
return (system_blocks if system_blocks else None, converse_msgs)
@@ -817,7 +780,6 @@ def stream_converse_with_callbacks(
on_tool_start=None,
on_reasoning_delta=None,
on_interrupt_check=None,
on_event=None,
) -> SimpleNamespace:
"""Process a Bedrock ConverseStream event stream with real-time callbacks.
@@ -837,12 +799,6 @@ def stream_converse_with_callbacks(
on supported models (Claude 4.6+).
on_interrupt_check: Called on each event. Should return True if the
agent has been interrupted and streaming should stop.
on_event: Called once at the top of the loop body for EVERY yielded
Bedrock event (text/tool-input/reasoning/metadata deltas alike),
before any branching. Provides a wire-level liveness signal so an
external watchdog can distinguish "still receiving events" from
"stream wedged with no data". Errors raised by the callback are
swallowed so a liveness hook can never abort the stream.
Returns:
An OpenAI-compatible SimpleNamespace response, identical in shape to
@@ -858,15 +814,6 @@ def stream_converse_with_callbacks(
usage_data: Dict[str, int] = {}
for event in event_stream.get("stream", []):
# Wire-level liveness signal: fire on EVERY yielded event (text, tool
# input, reasoning, metadata) before branching so an external watchdog
# can tell a still-flowing stream from a wedged one. Best-effort — a
# liveness callback must never be able to abort the stream.
if on_event is not None:
try:
on_event()
except Exception:
pass
# Check for interrupt
if on_interrupt_check and on_interrupt_check():
break
@@ -1349,24 +1296,9 @@ def classify_bedrock_error(error_message: str) -> str:
# detection is unavailable.
BEDROCK_CONTEXT_LENGTHS: Dict[str, int] = {
# Anthropic Claude models on Bedrock.
# Context windows per Anthropic's official models comparison
# (https://platform.claude.com/docs/en/about-claude/models/overview).
# Fable / Sonnet 5 / Opus 4.8 / 4.7 / 4.6 / Sonnet 4.6 have 1M generally
# available (no beta header required as of April 2026). Sonnet 4.5 and
# Sonnet 4 had their `context-1m-2025-08-07` beta retired on
# April 30, 2026, so they are standard 200K; Haiku 4.5 is 200K.
# These 1M entries must match agent/model_metadata.py
# DEFAULT_CONTEXT_LENGTHS or the agent compresses context prematurely.
# Keys are matched by longest-substring, so the versioned 4-6/4-7/4-8
# entries win over the generic "anthropic.claude-opus-4" fallback.
"anthropic.claude-fable-5": 1_000_000,
"anthropic.claude-fable": 1_000_000,
"anthropic.claude-sonnet-5": 1_000_000,
"anthropic.claude-opus-4-8": 1_000_000,
"anthropic.claude-opus-4-7": 1_000_000,
"anthropic.claude-opus-4-6": 1_000_000,
"anthropic.claude-sonnet-4-6": 1_000_000,
# Anthropic Claude models on Bedrock
"anthropic.claude-opus-4-6": 200_000,
"anthropic.claude-sonnet-4-6": 200_000,
"anthropic.claude-sonnet-4-5": 200_000,
"anthropic.claude-haiku-4-5": 200_000,
"anthropic.claude-opus-4": 200_000,
@@ -1393,22 +1325,9 @@ BEDROCK_CONTEXT_LENGTHS: Dict[str, int] = {
# Default for unknown Bedrock models
BEDROCK_DEFAULT_CONTEXT_LENGTH = 128_000
# Probe tiers (in tokens). We send a request padded just past each tier and
# read the real window from Bedrock's length-validation error. Two reasons
# this is tiered rather than one giant request:
# 1. A wildly oversized payload (e.g. 5M tokens) makes Bedrock return an
# opaque InternalServerException after retries instead of a clean
# ValidationException — so we must stay within a sane overage.
# 2. Stepping up lets us discover larger windows (2M+) without over-padding
# smaller ones.
# Each tier value is the *padding target*; the error reports the true maximum,
# which is what we actually return.
_BEDROCK_PROBE_TIERS = (1_300_000, 2_200_000)
_WORDS_PER_TOKEN = 0.9 # conservative: ensures the padded prompt clears the tier
def _static_bedrock_context_length(model_id: str) -> int:
"""Longest-substring-match lookup against the static fallback table.
def get_bedrock_context_length(model_id: str) -> int:
"""Look up the context window size for a Bedrock model.
Uses substring matching so versioned IDs like
``anthropic.claude-sonnet-4-6-20250514-v1:0`` resolve correctly.
@@ -1421,103 +1340,3 @@ def _static_bedrock_context_length(model_id: str) -> int:
best_key = key
best_val = val
return best_val
def probe_bedrock_context_length(model_id: str, region: str) -> Optional[int]:
"""Discover a Bedrock model's real context window by provoking a length error.
Bedrock does not expose the context window via any metadata API
(``get-foundation-model`` omits it, ``Converse`` metrics omit it,
``CountTokens`` is unsupported on several models). The only authoritative
source is the ``ValidationException`` raised when a prompt exceeds the
window:
"The model returned the following errors: prompt is too long:
1300032 tokens > 1000000 maximum"
Length validation happens *before* inference, so an oversized request is
rejected immediately and cheaply — no tokens are generated and no input is
actually processed. We pad a request just past each tier in
``_BEDROCK_PROBE_TIERS`` and parse the reported ``maximum``. Tiers exist
because (a) a *wildly* oversized payload makes Bedrock fail with an opaque
InternalServerException instead of a clean length error, and (b) stepping
up discovers larger windows without over-padding smaller ones.
Returns the detected window, or ``None`` if the probe could not run
(missing credentials, network error, or no parseable limit) so the caller
can fall back to the static table.
"""
try:
from agent.model_metadata import parse_context_limit_from_error
except ImportError: # pragma: no cover — same package
return None
try:
client = _get_bedrock_runtime_client(region)
except Exception as exc: # boto3 missing / credential resolution failure
logger.debug("Bedrock context probe skipped for %s: %s", model_id, exc)
return None
last_error = ""
for tier_tokens in _BEDROCK_PROBE_TIERS:
pad_words = int(tier_tokens / _WORDS_PER_TOKEN)
oversized = "data " * pad_words
try:
client.converse(
modelId=model_id,
messages=[{"role": "user", "content": [{"text": oversized}]}],
inferenceConfig={"maxTokens": 8},
)
# Accepted a prompt this large → the window is at least this tier.
# Returning the tier as a lower bound is safe and avoids inventing
# a number we can't confirm.
logger.debug(
"Bedrock context probe for %s accepted ~%s-token prompt; "
"window is at least that", model_id, f"{tier_tokens:,}",
)
return tier_tokens
except Exception as exc:
msg = str(exc)
last_error = msg
limit = parse_context_limit_from_error(msg)
if limit and limit >= 1024:
logger.info(
"Probed Bedrock context window for %s: %s tokens",
model_id, f"{limit:,}",
)
return limit
# No parseable limit at this tier (opaque server error, auth,
# throttle). Try the next, smaller-overage strategy is N/A here —
# tiers ascend — so just continue; if all fail we return None.
continue
logger.debug(
"Bedrock context probe for %s returned no parseable limit: %s",
model_id, last_error[:200],
)
return None
def get_bedrock_context_length(model_id: str, region: str = "", probe: bool = True) -> int:
"""Resolve the context window for a Bedrock model.
Resolution order:
1. Live probe against Bedrock (authoritative; cached by the caller).
2. Static fallback table (longest-substring match).
3. Conservative default.
The static table is intentionally a *fallback*, not the primary source:
AWS ships new model versions (opus-4-7, opus-4-8, ...) faster than the
table can track, and a stale entry silently caps the window (e.g. a
1M-token Opus pinned to 200K via an ``opus-4`` substring match). The
probe asks Bedrock directly so every model — current or future — gets its
real window with no table maintenance.
``probe=False`` (or an empty ``region``) skips the network call and uses
the static table only — used by pure-offline/display code paths.
"""
if probe and region:
probed = probe_bedrock_context_length(model_id, region)
if probed:
return probed
return _static_bedrock_context_length(model_id)
-323
View File
@@ -1,323 +0,0 @@
"""Shared dollar-denominated usage model for the billing/subscription surfaces.
The single source of truth behind the ``/usage`` and ``/subscription`` usage
bars (TUI + CLI). User feedback (Jun 2026): the terminal surfaces show
**dollars**, never "credits", and every usage bar must make the monthly
subscription allowance and separately-purchased top-up dollars distinctly
visible.
Data source: the NAS account-info fetch (``NousPortalAccountInfo``), whose
``paid_service_access_info`` carries the three dollar magnitudes we render
(despite the legacy ``*_credits`` field names, these are USD floats):
- ``subscription_credits_remaining`` -> plan dollars left this month
- ``purchased_credits_remaining`` -> top-up dollars left (rolls over)
- ``total_usable_credits`` -> total spendable
plus ``subscription.monthly_credits`` (the plan's monthly $ allowance, the
denominator for the "% used" plan bar) and ``current_period_end`` (renewal).
Design: two SEPARATE bars (decided with the user) rather than one crammed
three-segment bar — at terminal widths three same-glyph density segments are
unreadable. The plan bar is "spent vs allowance this month" (carries % used);
the top-up bar is "money you bought, doesn't expire". Each gets full
resolution and a single fill glyph, so the bar is never ambiguous and never
relies on color.
Fail-open everywhere: any missing/non-finite field degrades to fewer bars or a
magnitudes-only view; a logged-out / unreachable portal yields
``available=False`` and the surface shows nothing.
"""
from __future__ import annotations
import logging
import math
import os
from dataclasses import dataclass, field
from typing import Any, Optional
logger = logging.getLogger(__name__)
# Below this TOTAL spendable ($), a paid account is flagged "low" — the alert
# state that nudges top-up/upgrade before a mid-run cutoff. Product threshold
# (user feedback): "any amount below $5 should be an alert status."
LOW_BALANCE_THRESHOLD_USD = 5.0
def _finite(value: Any) -> Optional[float]:
"""Return value as a float iff it's a real finite number (not bool/NaN/Inf)."""
if isinstance(value, bool) or not isinstance(value, (int, float)):
return None
f = float(value)
return f if math.isfinite(f) else None
def _fmt_usd(value: Optional[float]) -> str:
"""``$X.YY`` for display. ``None`` -> ``$0.00`` (callers gate on presence)."""
return f"${(value or 0.0):,.2f}"
def format_renews(value: Optional[str]) -> Optional[str]:
"""Format an ISO date/timestamp as a human date, e.g. ``Jul 24, 2026``.
Accepts ``2026-07-24``, ``2026-07-24T11:05:01.000Z``, etc. Returns the raw
string unchanged if it can't be parsed (never raises), and ``None`` for
empty input.
"""
if not value:
return None
from datetime import datetime
text = str(value).strip()
if not text:
return None
iso = text[:-1] + "+00:00" if text.endswith("Z") else text
try:
dt = datetime.fromisoformat(iso)
except ValueError:
# Fall back to a bare date prefix (YYYY-MM-DD) if present.
try:
dt = datetime.strptime(text[:10], "%Y-%m-%d")
except ValueError:
return text
# %-d isn't portable to Windows; build the day without a leading zero.
return f"{dt.strftime('%b')} {dt.day}, {dt.year}"
@dataclass(frozen=True)
class UsageBar:
"""One full-resolution bar: ``spent`` of ``total``, plus a remaining figure.
``kind`` is ``"plan"`` (monthly allowance, shows % used) or ``"topup"``
(purchased dollars, no denominator — ``spent`` is 0 and ``total`` ==
``remaining`` so it renders as a full bar of available balance).
"""
kind: str # "plan" | "topup"
remaining_usd: float
total_usd: float
spent_usd: float = 0.0
@property
def pct_used(self) -> Optional[int]:
if self.kind != "plan" or self.total_usd <= 0:
return None
return max(0, min(100, round(self.spent_usd / self.total_usd * 100)))
@property
def fill_fraction(self) -> float:
"""Fraction of the bar that should read as 'remaining' (filled)."""
if self.total_usd <= 0:
return 0.0
return max(0.0, min(1.0, self.remaining_usd / self.total_usd))
@dataclass(frozen=True)
class UsageModel:
"""Surface-agnostic dollar usage model shared by /usage and /subscription.
``status`` classifies the account for copy selection:
- ``"free"`` : no paid access / no subscription (free models only)
- ``"low"`` : paid, but total spendable < $5 (ALERT)
- ``"healthy"`` : paid, total spendable >= $5
- ``"depleted"`` : paid access lost (balance exhausted)
"""
available: bool
status: str = "free"
plan_name: Optional[str] = None
renews_at: Optional[str] = None
renews_display: Optional[str] = None
subscription_remaining_usd: Optional[float] = None
topup_remaining_usd: Optional[float] = None
total_spendable_usd: Optional[float] = None
plan_bar: Optional[UsageBar] = None
topup_bar: Optional[UsageBar] = None
@property
def has_topup(self) -> bool:
return bool(self.topup_remaining_usd and self.topup_remaining_usd > 0)
def usage_model_from_account(account_info: Any) -> UsageModel:
"""Build a :class:`UsageModel` from a ``NousPortalAccountInfo``. Fail-open.
Returns ``UsageModel(available=False)`` when there's no usable account info
(logged out, no entitlement block). Never raises.
"""
try:
if account_info is None or not getattr(account_info, "logged_in", False):
return UsageModel(available=False)
access = getattr(account_info, "paid_service_access_info", None)
sub = getattr(account_info, "subscription", None)
paid = getattr(account_info, "paid_service_access", None)
sub_remaining = _finite(getattr(access, "subscription_credits_remaining", None)) if access else None
topup_remaining = _finite(getattr(access, "purchased_credits_remaining", None)) if access else None
total_usable = _finite(getattr(access, "total_usable_credits", None)) if access else None
plan_name = getattr(sub, "plan", None) if sub is not None else None
renews_at = getattr(sub, "current_period_end", None) if sub is not None else None
monthly = _finite(getattr(sub, "monthly_credits", None)) if sub is not None else None
has_subscription = bool(plan_name) or (monthly is not None and monthly > 0)
# Total spendable: prefer the server's total; else sum the parts we have.
if total_usable is not None:
total_spendable = total_usable
else:
parts = [v for v in (sub_remaining, topup_remaining) if v is not None]
total_spendable = sum(parts) if parts else None
# Status classification.
if paid is False:
status = "depleted"
elif not has_subscription and not (topup_remaining and topup_remaining > 0):
# No plan and no purchased balance -> free-models-only.
status = "free"
elif total_spendable is not None and total_spendable < LOW_BALANCE_THRESHOLD_USD:
status = "low"
else:
status = "healthy"
# Plan bar — only with a positive monthly allowance AND a remaining we
# can place on it. spent = cap - remaining, clamped (a debt/over-cap
# balance reads as fully spent rather than a nonsensical negative).
plan_bar: Optional[UsageBar] = None
if monthly is not None and monthly > 0 and sub_remaining is not None:
remaining = max(0.0, min(monthly, sub_remaining))
plan_bar = UsageBar(
kind="plan",
remaining_usd=remaining,
total_usd=monthly,
spent_usd=max(0.0, monthly - sub_remaining),
)
# Top-up bar — only when there are purchased dollars to show. No
# denominator (top-up has no monthly cap), so it renders full = balance.
topup_bar: Optional[UsageBar] = None
if topup_remaining is not None and topup_remaining > 0:
topup_bar = UsageBar(
kind="topup",
remaining_usd=topup_remaining,
total_usd=topup_remaining,
spent_usd=0.0,
)
return UsageModel(
available=True,
status=status,
plan_name=plan_name,
renews_at=renews_at,
renews_display=format_renews(renews_at),
subscription_remaining_usd=sub_remaining,
topup_remaining_usd=topup_remaining,
total_spendable_usd=total_spendable,
plan_bar=plan_bar,
topup_bar=topup_bar,
)
except Exception:
logger.debug("usage ▸ model build failed (fail-open)", exc_info=True)
return UsageModel(available=False)
def build_usage_model(*, timeout: float = 10.0) -> UsageModel:
"""Fetch account-info and build the shared usage model. Fail-open.
Dev override: ``HERMES_DEV_CREDITS_FIXTURE`` short-circuits to a fixture so
every usage state is testable without a live account (mirrors the existing
``/usage`` credits-block fixture path).
"""
fixture = _dev_fixture_usage_model()
if fixture is not None:
return fixture
try:
from hermes_cli.auth import get_provider_auth_state
tok = (get_provider_auth_state("nous") or {}).get("access_token")
if not (isinstance(tok, str) and tok.strip()):
return UsageModel(available=False)
except Exception:
return UsageModel(available=False)
try:
import concurrent.futures
from hermes_cli.nous_account import get_nous_portal_account_info
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
account = pool.submit(get_nous_portal_account_info, force_fresh=True).result(timeout=timeout)
return usage_model_from_account(account)
except Exception:
logger.debug("usage ▸ portal fetch failed (fail-open)", exc_info=True)
return UsageModel(available=False)
# =============================================================================
# Dev fixtures (throwaway scaffolding — env-var driven, no live portal)
# =============================================================================
def _dev_fixture_usage_model() -> Optional[UsageModel]:
"""Map ``HERMES_DEV_CREDITS_FIXTURE`` to a usage model for offline UX work.
Recognized names: ``free | healthy | low | topup | depleted``. Returns
``None`` when the env var is unset (real portal path runs).
"""
name = (os.getenv("HERMES_DEV_CREDITS_FIXTURE") or "").strip().lower()
if not name:
return None
if name == "free":
return UsageModel(available=True, status="free", plan_name=None)
if name in ("healthy", "mid"):
return UsageModel(
available=True,
status="healthy",
plan_name="Plus",
renews_at="2026-07-01",
subscription_remaining_usd=14.0,
total_spendable_usd=14.0,
plan_bar=UsageBar(kind="plan", remaining_usd=14.0, total_usd=20.0, spent_usd=6.0),
)
if name in ("topup", "top-up"):
return UsageModel(
available=True,
status="healthy",
plan_name="Plus",
renews_at="2026-07-01",
subscription_remaining_usd=14.0,
topup_remaining_usd=12.0,
total_spendable_usd=26.0,
plan_bar=UsageBar(kind="plan", remaining_usd=14.0, total_usd=20.0, spent_usd=6.0),
topup_bar=UsageBar(kind="topup", remaining_usd=12.0, total_usd=12.0, spent_usd=0.0),
)
if name == "low":
return UsageModel(
available=True,
status="low",
plan_name="Plus",
renews_at="2026-07-01",
subscription_remaining_usd=3.4,
total_spendable_usd=3.4,
plan_bar=UsageBar(kind="plan", remaining_usd=3.4, total_usd=20.0, spent_usd=16.6),
)
if name == "depleted":
return UsageModel(
available=True,
status="depleted",
plan_name="Plus",
renews_at="2026-07-01",
subscription_remaining_usd=0.0,
total_spendable_usd=0.0,
plan_bar=UsageBar(kind="plan", remaining_usd=0.0, total_usd=20.0, spent_usd=20.0),
)
return None
+9 -172
View File
@@ -1,4 +1,4 @@
"""Surface-agnostic core for the Phase 2b Remote Spending screens.
"""Surface-agnostic core for the Phase 2b terminal-billing screens.
One fetch/parse per concern, consumed identically by the CLI handler
(``cli.py::_show_billing``), the TUI JSON-RPC methods
@@ -15,7 +15,6 @@ We keep them as :class:`decimal.Decimal` end-to-end and only format for display.
from __future__ import annotations
import logging
import os
import uuid
from dataclasses import dataclass, field
from decimal import Decimal, InvalidOperation
@@ -65,47 +64,15 @@ def format_money(value: Optional[Decimal]) -> str:
# =============================================================================
# resolvedVia → the human answer to "why THIS card?". Keys are the server's card
# resolution rungs (NAS card-on-file ladder); absent/unknown rungs render no label
# so the display degrades cleanly on servers that don't send resolvedVia yet.
_CARD_PROVENANCE_LABELS = {
"subPin": "the card on your subscription",
"customerDefault": "your default card saved on the portal",
"autoRefill": "your auto-reload card",
}
@dataclass(frozen=True)
class CardInfo:
brand: str
last4: str
# NAS card-on-file field (post card-resolver): which ladder rung found the
# card. Defaults off so pre-resolver payloads parse unchanged.
resolved_via: Optional[str] = None
@property
def masked(self) -> str:
# A Link payment method has no card number (last4 = "") — render the
# brand alone, not "Link ····".
if not self.last4:
return self.brand
return f"{self.brand} ····{self.last4}"
@property
def provenance(self) -> Optional[str]:
"""Human label for why this card was picked, or None (unknown rung /
server too old to say)."""
if self.resolved_via is None:
return None
return _CARD_PROVENANCE_LABELS.get(self.resolved_via)
@property
def display(self) -> str:
"""The one-line card display: ``Visa ····4242 — the card on your
subscription`` (or just the masked card when provenance is unknown)."""
label = self.provenance
return f"{self.masked} — {label}" if label else self.masked
@dataclass(frozen=True)
class MonthlyCap:
@@ -114,20 +81,11 @@ class MonthlyCap:
is_default_ceiling: bool = False
@dataclass(frozen=True)
class AutoReloadCard:
kind: str # "canonical" | "distinct" | "none"
payment_method_id: Optional[str] = None
brand: Optional[str] = None
last4: Optional[str] = None
@dataclass(frozen=True)
class AutoReload:
enabled: bool = False
threshold_usd: Optional[Decimal] = None
reload_to_usd: Optional[Decimal] = None
card: Optional[AutoReloadCard] = None
@dataclass(frozen=True)
@@ -142,8 +100,7 @@ class BillingState:
org_id: Optional[str] = None
org_slug: Optional[str] = None
org_name: Optional[str] = None
role: Optional[str] = None # "OWNER" | "ADMIN" | "FINANCE_ADMIN" | "SECURITY_ADMIN" | "MEMBER"
can_change_plan_raw: Optional[bool] = None
role: Optional[str] = None # "OWNER" | "ADMIN" | "MEMBER"
balance_usd: Optional[Decimal] = None
cli_billing_enabled: bool = False
charge_presets: tuple[Decimal, ...] = ()
@@ -158,33 +115,17 @@ class BillingState:
@property
def is_admin(self) -> bool:
"""Deprecated/display only — a legacy OWNER/ADMIN check.
NOT a capability check; use :attr:`can_change_plan` for gating billing
plan-change actions.
"""
"""True for OWNER/ADMIN — the roles that can manage billing."""
return (self.role or "").upper() in ("OWNER", "ADMIN")
@property
def can_change_plan(self) -> bool:
"""Server capability when supplied; otherwise the legacy role fallback."""
if self.can_change_plan_raw is not None:
return self.can_change_plan_raw
return self.is_admin
@property
def can_charge(self) -> bool:
"""True when the UI should offer charge/auto-reload actions.
Uses the server-granted plan-change capability (``can_change_plan``,
which itself falls back to the legacy OWNER/ADMIN role check when the
server omits ``canChangePlan``) AND the per-org kill-switch. This lets
the server grant charge capability to non-OWNER/ADMIN roles (e.g.
FINANCE_ADMIN) via ``canChangePlan``, instead of hard-coding the
deprecated 3-role admin check. (The server still enforces; this is
just for graying out actions the user can't take.)
Admin role AND the per-org kill-switch on. (The server still enforces;
this is just for graying out actions the user can't take.)
"""
return self.can_change_plan and self.cli_billing_enabled
return self.is_admin and self.cli_billing_enabled
def _parse_card(raw: Any) -> Optional[CardInfo]:
@@ -192,13 +133,9 @@ def _parse_card(raw: Any) -> Optional[CardInfo]:
return None
brand = raw.get("brand")
last4 = raw.get("last4")
if not (isinstance(brand, str) and isinstance(last4, str)):
return None
# Post-resolver fields — all optional so both payload generations parse.
resolved_via = raw.get("resolvedVia")
if not isinstance(resolved_via, str):
resolved_via = None
return CardInfo(brand=brand, last4=last4, resolved_via=resolved_via)
if isinstance(brand, str) and isinstance(last4, str):
return CardInfo(brand=brand, last4=last4)
return None
def _parse_monthly_cap(raw: Any) -> Optional[MonthlyCap]:
@@ -218,27 +155,6 @@ def _parse_auto_reload(raw: Any) -> Optional[AutoReload]:
enabled=bool(raw.get("enabled")),
threshold_usd=parse_money(raw.get("thresholdUsd")),
reload_to_usd=parse_money(raw.get("reloadToUsd")),
card=_parse_auto_reload_card(raw.get("card")),
)
def _parse_auto_reload_card(raw: Any) -> Optional[AutoReloadCard]:
if not isinstance(raw, dict):
return None
kind = raw.get("kind")
if kind not in ("canonical", "distinct", "none"):
return None
if kind in ("canonical", "none"):
return AutoReloadCard(kind=kind)
payment_method_id = raw.get("paymentMethodId")
brand = raw.get("brand")
last4 = raw.get("last4")
return AutoReloadCard(
kind=kind,
payment_method_id=payment_method_id if isinstance(payment_method_id, str) else None,
brand=brand if isinstance(brand, str) else None,
last4=last4 if isinstance(last4, str) else None,
)
@@ -263,11 +179,6 @@ def billing_state_from_payload(
org_slug=org.get("slug"),
org_name=org.get("name"),
role=org.get("role"),
can_change_plan_raw=(
payload.get("canChangePlan")
if isinstance(payload.get("canChangePlan"), bool)
else None
),
balance_usd=parse_money(payload.get("balanceUsd")),
cli_billing_enabled=bool(payload.get("cliBillingEnabled")),
charge_presets=tuple(presets),
@@ -291,15 +202,7 @@ def build_billing_state(*, timeout: float = 15.0) -> BillingState:
Returns ``BillingState(logged_in=False)`` when not logged in. On a portal/HTTP
failure, returns ``logged_in=False`` with ``error`` set so the surface can show
a clear message rather than crashing.
Dev override: ``HERMES_DEV_BILLING_FIXTURE`` short-circuits to a fixture so the
card-on-file / admin / scope states are testable offline (mirrors
``HERMES_DEV_CREDITS_FIXTURE`` for the usage model).
"""
fixture = _dev_fixture_billing_state()
if fixture is not None:
return fixture
try:
from hermes_cli.nous_billing import (
BillingAuthError,
@@ -340,72 +243,6 @@ def _fallback_portal_url(base: str) -> str:
return f"{base.rstrip('/')}/billing?topup=open"
# =============================================================================
# Dev fixtures (throwaway scaffolding — env-var driven, no live portal)
# =============================================================================
def _dev_fixture_billing_state() -> Optional[BillingState]:
"""Map ``HERMES_DEV_BILLING_FIXTURE`` to a :class:`BillingState` for offline UX.
Recognized names::
nocard logged in · billing on · admin · NO card on file
card card on file · auto-reload off
card-autoreload card on file · auto-reload on
notadmin logged in · MEMBER role (billing actions disabled)
billing-off logged in · admin · per-org kill-switch OFF
logged-out not logged in
Returns ``None`` when the env var is unset (the real portal path runs).
Mirrors ``HERMES_DEV_CREDITS_FIXTURE``; the usage *bar* still comes from
``HERMES_DEV_CREDITS_FIXTURE`` (set both to pair a bar with a billing state).
"""
name = (os.getenv("HERMES_DEV_BILLING_FIXTURE") or "").strip().lower()
if not name:
return None
# Shared fixture portal host (matches subscription_view._DEV_FIXTURE_PORTAL —
# prod host, not staging; the ?topup=open suffix is the /topup deep-link).
portal = "https://portal.nousresearch.com/billing?topup=open"
common: dict[str, Any] = dict(
org_id="org_acme",
org_slug="acme",
org_name="Acme Inc",
role="OWNER",
balance_usd=Decimal("3.40"),
cli_billing_enabled=True,
charge_presets=(Decimal("10"), Decimal("25"), Decimal("50")),
min_usd=Decimal("5"),
max_usd=Decimal("500"),
portal_url=portal,
)
card = CardInfo(brand="Visa", last4="4242")
autoreload_on = AutoReload(enabled=True, threshold_usd=Decimal("5"), reload_to_usd=Decimal("25"))
if name in ("logged-out", "logged_out", "loggedout"):
return BillingState(logged_in=False)
if name == "nocard":
return BillingState(logged_in=True, card=None, **common)
if name == "card":
return BillingState(logged_in=True, card=card, **common)
if name in ("card-sub", "card_sub"):
# Post-resolver: the card came from the subscription (provenance label).
_sub_card = CardInfo(brand="Visa", last4="4242", resolved_via="subPin")
return BillingState(logged_in=True, card=_sub_card, **common)
if name in ("card-autoreload", "card_autoreload", "autoreload"):
return BillingState(logged_in=True, card=card, auto_reload=autoreload_on, **common)
if name in ("notadmin", "not-admin", "member"):
opts = {**common, "role": "MEMBER"}
return BillingState(logged_in=True, card=card, **opts)
if name in ("billing-off", "billing_off", "off"):
opts = {**common, "cli_billing_enabled": False}
return BillingState(logged_in=True, card=None, **opts)
# Unknown name → logged-out so the misconfiguration is visible.
return BillingState(logged_in=False, error=f"unknown HERMES_DEV_BILLING_FIXTURE: {name}")
# =============================================================================
# Idempotency
# =============================================================================
File diff suppressed because it is too large Load Diff
+18 -132
View File
@@ -288,13 +288,6 @@ _RESPONSES_BUILTIN_TOOL_TYPES = {
_RESPONSE_MESSAGE_STATUSES = {"completed", "incomplete", "in_progress"}
# The Responses API rejects input[].id longer than this with a non-retryable
# HTTP 400 ("string too long"). Codex-issued assistant message ids are
# server-assigned base64 blobs that can run 400+ chars, while Hermes-minted
# ids (msg_...) stay well under this cap and are worth keeping for
# prefix-cache hits. Drop only the oversized ones on replay.
_MAX_RESPONSES_ITEM_ID_LENGTH = 64
def _normalize_responses_message_status(value: Any, *, default: str = "completed") -> str:
"""Normalize a Responses assistant message status for replay.
@@ -314,7 +307,6 @@ def _chat_messages_to_responses_input(
messages: List[Dict[str, Any]],
*,
is_xai_responses: bool = False,
is_github_responses: bool = False,
replay_encrypted_reasoning: bool = True,
current_issuer_kind: Optional[str] = None,
) -> List[Dict[str, Any]]:
@@ -339,16 +331,6 @@ def _chat_messages_to_responses_input(
items from the conversation history and threads ``replay_enabled=False``
through this converter so subsequent turns send no reasoning items.
``is_github_responses`` drops the ``id`` field from replayed
``codex_message_items`` regardless of length. The Copilot backend
(api.githubcopilot.com/responses) binds these ids to a specific
backend "connection" — credential-pool rotation, a gateway restart,
or routine load-balancer churn between turns all invalidate it — and
rejects a stale id with HTTP 401 "input item ID does not belong to
this connection" even for short ids (see #32716). ``phase``/
``status``/``content`` are still replayed; only ``id`` is unsafe to
reuse across a Copilot connection.
``current_issuer_kind`` enables a per-item cross-issuer guard. The
Responses API's ``encrypted_content`` blob is decryptable only by the
endpoint that minted it — replaying a Codex-issued blob against xAI
@@ -481,14 +463,8 @@ def _chat_messages_to_responses_input(
"content": normalized_content_parts,
}
item_id = raw_item.get("id")
if (
not is_github_responses
and isinstance(item_id, str)
and item_id.strip()
):
stripped_id = item_id.strip()
if len(stripped_id) <= _MAX_RESPONSES_ITEM_ID_LENGTH:
replay_item["id"] = stripped_id
if isinstance(item_id, str) and item_id.strip():
replay_item["id"] = item_id.strip()
phase = raw_item.get("phase")
if isinstance(phase, str) and phase.strip():
replay_item["phase"] = phase.strip()
@@ -600,11 +576,7 @@ def _chat_messages_to_responses_input(
# Input preflight / validation
# ---------------------------------------------------------------------------
def _preflight_codex_input_items(
raw_items: Any,
*,
is_github_responses: bool = False,
) -> List[Dict[str, Any]]:
def _preflight_codex_input_items(raw_items: Any) -> List[Dict[str, Any]]:
if not isinstance(raw_items, list):
raise ValueError("Codex Responses input must be a list of input items.")
@@ -745,14 +717,8 @@ def _preflight_codex_input_items(
"content": normalized_content,
}
item_id = item.get("id")
if (
not is_github_responses
and isinstance(item_id, str)
and item_id.strip()
):
stripped_id = item_id.strip()
if len(stripped_id) <= _MAX_RESPONSES_ITEM_ID_LENGTH:
normalized_item["id"] = stripped_id
if isinstance(item_id, str) and item_id.strip():
normalized_item["id"] = item_id.strip()
phase = item.get("phase")
if isinstance(phase, str) and phase.strip():
normalized_item["phase"] = phase.strip()
@@ -824,7 +790,6 @@ def _preflight_codex_api_kwargs(
api_kwargs: Any,
*,
allow_stream: bool = False,
is_github_responses: bool = False,
) -> Dict[str, Any]:
if not isinstance(api_kwargs, dict):
raise ValueError("Codex Responses request must be a dict.")
@@ -846,10 +811,7 @@ def _preflight_codex_api_kwargs(
instructions = str(instructions)
instructions = instructions.strip() or DEFAULT_AGENT_IDENTITY
normalized_input = _preflight_codex_input_items(
api_kwargs.get("input"),
is_github_responses=is_github_responses,
)
normalized_input = _preflight_codex_input_items(api_kwargs.get("input"))
tools = api_kwargs.get("tools")
normalized_tools = None
@@ -1118,22 +1080,6 @@ def _normalize_codex_response(
differs from the one that minted the encrypted_content blob and drop
the item instead of triggering HTTP 400 invalid_encrypted_content.
"""
response_status = getattr(response, "status", None)
if isinstance(response_status, str):
response_status = response_status.strip().lower()
else:
response_status = None
incomplete_details = getattr(response, "incomplete_details", None)
incomplete_reason = ""
if isinstance(incomplete_details, dict):
incomplete_reason = str(incomplete_details.get("reason") or "").strip().lower()
elif incomplete_details is not None:
incomplete_reason = str(getattr(incomplete_details, "reason", "") or "").strip().lower()
response_incomplete_content_filter = (
response_status == "incomplete" and incomplete_reason == "content_filter"
)
output = getattr(response, "output", None)
if not isinstance(output, list) or not output:
# The Codex backend can return empty output when the answer was
@@ -1150,18 +1096,15 @@ def _normalize_codex_response(
content=[SimpleNamespace(type="output_text", text=out_text.strip())],
)]
response.output = output
elif response_incomplete_content_filter:
# This is a deterministic provider safety block, not a partial
# answer. Synthesize an empty message so finish_reason below becomes
# content_filter and the conversation loop can fallback/surface it
# instead of burning three continuation attempts.
output = [SimpleNamespace(
type="message", role="assistant", status="completed", content=[]
)]
response.output = output
else:
raise RuntimeError("Responses API returned no output items")
response_status = getattr(response, "status", None)
if isinstance(response_status, str):
response_status = response_status.strip().lower()
else:
response_status = None
if response_status in {"failed", "cancelled"}:
error_obj = getattr(response, "error", None)
error_msg = _format_responses_error(error_obj, response_status)
@@ -1379,45 +1322,6 @@ def _normalize_codex_response(
# so the model keeps its chain-of-thought on the retry.
final_text = ""
# ── Reasoning-channel answer salvage (xAI grok) ──────────────
# grok-4.x on the xAI /v1/responses surface sometimes emits its final
# answer inside the reasoning item instead of as a ``message`` output
# item, marking where the answer starts with grok's internal
# ``<response>`` delimiter. Without salvage, the reasoning-only rule
# below classifies the turn ``incomplete`` — and because reasoning
# items on this surface carry no ``encrypted_content``, the interim
# message replays as nothing, so every continuation request is
# byte-identical to the one that just failed. The turn burns its 3
# retries and dies with "Codex response remained incomplete after 3
# continuation attempts" even though the answer was produced on the
# first attempt. Observed live with grok-4.20 on xai-oauth
# (2026-07-13). Promote the delimited tail to assistant content and
# keep the untagged prefix as thinking text.
if (
issuer_kind == "xai_responses"
and not final_text
and not tool_calls
and reasoning_parts
):
joined_reasoning = "\n\n".join(reasoning_parts)
marker = joined_reasoning.rfind("<response>")
if marker != -1:
salvaged = joined_reasoning[marker + len("<response>"):]
closing = salvaged.find("</response>")
if closing != -1:
salvaged = salvaged[:closing]
salvaged = salvaged.strip()
if salvaged:
logger.warning(
"xAI response delivered its final answer inside the "
"reasoning channel (<response> delimiter); promoting "
"%d chars to assistant content.",
len(salvaged),
)
final_text = salvaged
reasoning_prefix = joined_reasoning[:marker].strip()
reasoning_parts = [reasoning_prefix] if reasoning_prefix else []
assistant_message = SimpleNamespace(
content=final_text,
tool_calls=tool_calls,
@@ -1430,8 +1334,6 @@ def _normalize_codex_response(
if tool_calls:
finish_reason = "tool_calls"
elif response_incomplete_content_filter:
finish_reason = "content_filter"
elif leaked_tool_call_text:
finish_reason = "incomplete"
elif saw_streaming_or_item_incomplete:
@@ -1440,28 +1342,12 @@ def _normalize_codex_response(
finish_reason = "incomplete"
elif (reasoning_items_raw or reasoning_parts or saw_reasoning_item) and not final_text:
# Response contains only reasoning (encrypted thinking state and/or
# human-readable summary) with no visible content or tool calls.
#
# For the specially-handled backends (Codex, xAI, GitHub/Copilot),
# reasoning-only with status="completed" means "the model is still
# thinking and needs another turn" — treat it as incomplete so the
# Codex continuation path retries instead of falling into the
# empty-content retry loop.
#
# For all other backends (other:<base_url>, etc.), trust the provider's
# own response.status signal. When status == "completed" and no items
# are queued/in_progress/incomplete, reasoning alone is a valid final
# state — forcing "incomplete" causes multi-minute stalls as the
# continuation path re-issues calls (3 retries × up to 240s each).
# See https://github.com/NousResearch/hermes-agent/issues/64434
if response_status == "completed" and issuer_kind not in (
"codex_backend",
"xai_responses",
"github_responses",
):
finish_reason = "stop"
else:
finish_reason = "incomplete"
# human-readable summary) with no visible content or tool calls. The
# model is still thinking and needs another turn to produce the actual
# answer. Marking this as "stop" would send it into the empty-content
# retry loop which burns retries then fails — treat it as incomplete so
# the Codex continuation path handles it correctly.
finish_reason = "incomplete"
else:
finish_reason = "stop"
return assistant_message, finish_reason
+85 -477
View File
@@ -16,18 +16,70 @@ compatibility.
from __future__ import annotations
import json
import logging
import os
import time
from types import SimpleNamespace
from typing import Any, Callable, Dict, List
from agent.stream_single_writer import claim_stream_writer, stream_writer_is_current
from typing import Any, Dict, List
logger = logging.getLogger(__name__)
def _codex_note_to_tool_progress(note: dict) -> tuple[str, str, dict] | None:
"""Map a Codex app-server ``item/started`` notification to a Hermes
tool-progress event ``(tool_name, preview, args)``.
The Codex app-server runtime processes ``item/started`` notifications for
command execution, file changes, and MCP/dynamic tool calls, but never
surfaced them as Hermes tool-progress events — so gateways (Telegram, etc.)
showed no verbose "running X" breadcrumbs on this route while every other
provider did (#38835). Returns None for items that aren't tool-shaped.
"""
if not isinstance(note, dict) or note.get("method") != "item/started":
return None
params = note.get("params") or {}
item = params.get("item") or {}
if not isinstance(item, dict):
return None
item_type = item.get("type") or ""
if item_type == "commandExecution":
command = item.get("command") or ""
return "exec_command", command, {"command": command, "cwd": item.get("cwd") or ""}
if item_type == "fileChange":
changes = item.get("changes") or []
preview = "file changes"
if isinstance(changes, list) and changes:
paths = [
str(change.get("path"))
for change in changes
if isinstance(change, dict) and change.get("path")
]
if paths:
preview = ", ".join(paths[:3])
if len(paths) > 3:
preview += f", +{len(paths) - 3} more"
return "apply_patch", preview, {"changes": changes}
if item_type == "mcpToolCall":
server = item.get("server") or "mcp"
tool = item.get("tool") or "unknown"
args = item.get("arguments") or {}
if not isinstance(args, dict):
args = {"arguments": args}
return f"mcp.{server}.{tool}", tool, args
if item_type == "dynamicToolCall":
tool = item.get("tool") or "unknown"
args = item.get("arguments") or {}
if not isinstance(args, dict):
args = {"arguments": args}
return tool, tool, args
return None
def _coerce_usage_int(value: Any) -> int:
if isinstance(value, bool):
return 0
@@ -61,15 +113,6 @@ def _record_codex_app_server_usage(agent, turn) -> dict[str, Any]:
usage = getattr(turn, "token_usage_last", None)
if not isinstance(usage, dict) or not usage:
compressor = getattr(agent, "context_compressor", None)
if (
compressor is not None
and getattr(compressor, "awaiting_real_usage_after_compression", False)
):
# No usage means this turn cannot adjudicate the pending compaction.
# Consume the marker so a later unrelated reading is not charged to
# it and preflight deferral cannot stay latched indefinitely.
compressor.update_from_response({})
if agent._session_db and agent.session_id:
try:
if not agent._session_db_created:
@@ -77,9 +120,6 @@ def _record_codex_app_server_usage(agent, turn) -> dict[str, Any]:
agent._session_db.update_token_counts(
agent.session_id,
model=agent.model,
billing_provider=agent.provider,
billing_base_url=agent.base_url,
billing_mode="subscription_included",
api_call_count=1,
)
except Exception as exc:
@@ -227,18 +267,6 @@ def _record_codex_app_server_compaction(
compressor, "compression_count", 0
) + 1
compressor.last_compression_rough_tokens = approx_tokens or 0
# The app server has already completed a real compaction boundary. Its
# usage update (when supplied) is therefore the same real-vs-real
# effectiveness verdict used by the normal compression path.
record_boundary = getattr(
type(compressor), "record_completed_compaction", None
)
if callable(record_boundary):
# Codex owns this summary. A prior Hermes deterministic-fallback
# flag must not leak into the native boundary's quality verdict.
record_boundary(compressor, used_fallback=False)
elif hasattr(compressor, "_verify_compaction_cleared_threshold"):
compressor._verify_compaction_cleared_threshold = True
if not getattr(turn, "token_usage_last", None):
compressor.last_prompt_tokens = -1
compressor.last_completion_tokens = 0
@@ -270,348 +298,6 @@ def _record_codex_app_server_compaction(
return True
# ---------------------------------------------------------------------------
# Codex app-server → Hermes UI bridge (#33200)
#
# The codex_app_server runtime hands the entire turn to a subprocess and
# bypasses the normal Hermes tool loop. Without this bridge gateway
# adapters (Discord, Telegram, TUI) never see live tool-progress bubbles
# or interim assistant commentary while codex is working — the user just
# stares at a quiet channel until the final answer lands. The bridge
# translates raw codex JSON-RPC notifications into the same three agent
# callbacks the standard runtime fires:
# - tool_progress_callback("tool.started"|"tool.completed", name, ...)
# - _fire_stream_delta(text) for streaming agentMessage chunks
# - _emit_interim_assistant_message({...}) for completed agentMessages
# ---------------------------------------------------------------------------
# Codex item types that map to a Hermes tool_call in the projector (and
# therefore deserve a tool_progress bubble pair). The projector lives in
# agent/transports/codex_event_projector.py — keep these in sync so the
# tool name shown in the UI matches the name recorded in messages.
# webSearch is codex's built-in web search tool — it has no projector
# entry (codex handles it internally) but still deserves a bubble.
_CODEX_TOOL_ITEM_TYPES = frozenset(
{"commandExecution", "fileChange", "mcpToolCall", "dynamicToolCall", "webSearch"}
)
# Internal MCP server that wraps Hermes' native tools for codex. When
# codex calls back through it, the inner dispatch runs in a SEPARATE
# hermes-tools-mcp-server subprocess that has no access to the parent
# agent's tool_progress_callback — so the inner call can never surface
# its own native progress event. The codex-level mcpToolCall event IS
# the display event for those calls; we strip the mcp.hermes-tools.*
# namespacing and emit the bare tool name (web_search, browser_navigate,
# vision_analyze, ...) since the user thinks of these as Hermes tools,
# not as MCP calls.
_INTERNAL_MCP_SERVER = "hermes-tools"
def _codex_item_to_tool_name(item: dict) -> str:
"""Synthetic Hermes tool name for a codex item. Mirrors
CodexEventProjector so the progress bubble and the projected
tool_calls entry use the same identifier."""
item_type = item.get("type") or ""
if item_type == "commandExecution":
return "exec_command"
if item_type == "fileChange":
return "apply_patch"
if item_type == "mcpToolCall":
server = item.get("server") or "mcp"
tool = item.get("tool") or "unknown"
if server == _INTERNAL_MCP_SERVER:
return tool
return f"mcp.{server}.{tool}"
if item_type == "dynamicToolCall":
return item.get("tool") or "dynamic"
if item_type == "webSearch":
return "web_search"
return item_type or "unknown"
def _codex_item_to_args(item: dict) -> dict:
"""Args dict surfaced to tool_progress_callback("tool.started", ...).
Mirrors the projector's _project_command / _project_file_change /
_project_mcp_tool_call / _project_dynamic_tool_call shapes."""
item_type = item.get("type") or ""
if item_type == "commandExecution":
return {"command": item.get("command") or "",
"cwd": item.get("cwd") or ""}
if item_type == "fileChange":
return {"changes": [
{"kind": (c.get("kind") or {}).get("type") or "update",
"path": c.get("path") or ""}
for c in (item.get("changes") or []) if isinstance(c, dict)
]}
if item_type in {"mcpToolCall", "dynamicToolCall"}:
args = item.get("arguments") or {}
return args if isinstance(args, dict) else {"arguments": args}
if item_type == "webSearch":
return {"query": item.get("query") or ""}
return {}
def _codex_item_to_preview(item: dict) -> Any:
"""Short human-readable preview for the tool.started bubble. Returns
None when no useful preview is available (Hermes' UI tolerates None)."""
item_type = item.get("type") or ""
if item_type == "commandExecution":
cmd = item.get("command") or ""
return cmd[:120] if cmd else None
if item_type == "fileChange":
paths = [c.get("path") for c in (item.get("changes") or [])
if isinstance(c, dict) and c.get("path")]
if not paths:
return None
preview = ", ".join(paths[:3])
if len(paths) > 3:
preview += f", +{len(paths) - 3} more"
return preview
if item_type in {"mcpToolCall", "dynamicToolCall"}:
args = item.get("arguments") or {}
if not isinstance(args, dict) or not args:
return None
try:
return json.dumps(args, ensure_ascii=False)[:120]
except (TypeError, ValueError):
return None
if item_type == "webSearch":
query = item.get("query") or ""
return query[:120] if query else None
return None
def _codex_item_completion_payload(item: dict) -> tuple[str, bool]:
"""Return (result_text, is_error) for a completed codex tool item.
Mirrors the projector's tool-result content so the bubble shows the
same outcome string that ends up in the messages list."""
item_type = item.get("type") or ""
if item_type == "commandExecution":
out = item.get("aggregatedOutput") or ""
exit_code = item.get("exitCode")
is_error = bool(exit_code is not None and exit_code != 0)
if is_error:
out = f"[exit {exit_code}]\n{out}"
return out, is_error
if item_type == "fileChange":
status = item.get("status") or "unknown"
n = len(item.get("changes") or [])
return (
f"apply_patch status={status}, {n} change(s)",
status not in {"completed", "applied", "success"},
)
if item_type == "mcpToolCall":
error = item.get("error")
if error:
return (
f"[error] {json.dumps(error, ensure_ascii=False)[:1000]}",
True,
)
result = item.get("result")
return (
json.dumps(result, ensure_ascii=False)[:4000]
if result is not None else "",
False,
)
if item_type == "dynamicToolCall":
content_items = item.get("contentItems") or []
if isinstance(content_items, list) and content_items:
return (
json.dumps(content_items, ensure_ascii=False)[:4000],
not bool(item.get("success", True)),
)
success = item.get("success", True)
return f"success={success}", not bool(success)
return "", False
def make_codex_app_server_event_bridge(agent) -> Callable[[dict], None]:
"""Build an ``on_event`` callback that wires codex app-server JSON-RPC
notifications into Hermes' gateway UI callbacks.
Returns a single-argument callable suitable for
``CodexAppServerSession(on_event=...)``.
Translation map:
* ``item/started`` for tool-shaped items → ``tool_progress_callback(
"tool.started", name, preview, args)``
* ``item/completed`` for tool-shaped items → ``tool_progress_callback(
"tool.completed", name, None, None, duration=..., is_error=...,
result=...)``
* ``item/agentMessage/delta`` → ``_fire_stream_delta(text)`` so chat
adapters can render the assistant's reply as it streams.
* ``item/reasoning/delta`` → ``_fire_reasoning_delta(text)``
* ``item/completed`` for ``agentMessage`` →
``_emit_interim_assistant_message({"role": "assistant",
"content": text})``. The gateway's ``already_streamed`` check
dedupes against any text the stream-delta callback already
rendered for the same message.
All callback invocations are guarded — a buggy display callback must
not tear down the codex turn loop. Errors are logged at DEBUG so the
notification stream keeps flowing regardless.
"""
# item_id -> (tool_name, args, started_wall_time). Populated on
# item/started and consumed on item/completed so duration is correct
# even when codex doesn't report durationMs.
started: dict[str, tuple[str, dict, float]] = {}
def _stable_call_id(item: dict, name: str) -> str:
"""Deterministic tool_call id mirroring CodexEventProjector, so a
live TUI tool card correlates with the same tool call after the
session is resumed and history is projected."""
from agent.transports.codex_event_projector import _deterministic_call_id
item_id = item.get("id") or ""
item_type = item.get("type") or ""
if item_type == "commandExecution":
return _deterministic_call_id("exec", item_id)
if item_type == "fileChange":
return _deterministic_call_id("apply_patch", item_id)
if item_type == "mcpToolCall":
server = item.get("server") or "mcp"
tool = item.get("tool") or "unknown"
return _deterministic_call_id(f"mcp__{server}__{tool}", item_id)
if item_type == "dynamicToolCall":
tool = item.get("tool") or "unknown"
return _deterministic_call_id(f"dyn_{tool}", item_id)
return _deterministic_call_id(name, item_id)
def _fire_tool_started(item: dict) -> None:
item_id = item.get("id") or ""
name = _codex_item_to_tool_name(item)
args = _codex_item_to_args(item)
if item_id:
started[item_id] = (name, args, time.monotonic())
cb = getattr(agent, "tool_progress_callback", None)
if cb is not None:
try:
cb("tool.started", name, _codex_item_to_preview(item), args)
except Exception:
logger.debug(
"tool_progress_callback raised on tool.started for %s",
name, exc_info=True,
)
# Authoritative stable-ID tool card (TUI / desktop). Fires
# alongside tool_progress so surfaces that render structured tool
# cards (not just progress bubbles) stay correlated with the
# projected history entry after a resume.
start_cb = getattr(agent, "tool_start_callback", None)
if start_cb is not None:
try:
start_cb(_stable_call_id(item, name), name, args)
except Exception:
logger.debug(
"tool_start_callback raised for %s", name, exc_info=True,
)
def _fire_tool_completed(item: dict) -> None:
item_id = item.get("id") or ""
name = _codex_item_to_tool_name(item)
prior = started.pop(item_id, None)
# Prefer codex's own durationMs when present so the bubble shows
# exact tool wall-time; fall back to our started timestamp; fall
# back to None if we never saw an item/started (some codex
# versions only emit completed for fast items).
duration: Any = None
codex_ms = item.get("durationMs")
if isinstance(codex_ms, (int, float)) and codex_ms >= 0:
duration = codex_ms / 1000.0
elif prior is not None:
duration = time.monotonic() - prior[2]
result, is_error = _codex_item_completion_payload(item)
cb = getattr(agent, "tool_progress_callback", None)
if cb is not None:
try:
cb("tool.completed", name, None, None,
duration=duration, is_error=is_error, result=result)
except Exception:
logger.debug(
"tool_progress_callback raised on tool.completed for %s",
name, exc_info=True,
)
complete_cb = getattr(agent, "tool_complete_callback", None)
if complete_cb is not None:
args = prior[1] if prior is not None else _codex_item_to_args(item)
try:
complete_cb(_stable_call_id(item, name), name, args, result)
except Exception:
logger.debug(
"tool_complete_callback raised for %s", name, exc_info=True,
)
def _fire_text_delta(params: dict) -> None:
text = params.get("delta") or params.get("text") or ""
if not isinstance(text, str) or not text:
return
fn = getattr(agent, "_fire_stream_delta", None)
if fn is None:
return
try:
fn(text)
except Exception:
logger.debug("_fire_stream_delta raised", exc_info=True)
def _fire_reasoning_delta(params: dict) -> None:
text = params.get("delta") or params.get("text") or ""
if not isinstance(text, str) or not text:
return
fn = getattr(agent, "_fire_reasoning_delta", None)
if fn is None:
return
try:
fn(text)
except Exception:
logger.debug("_fire_reasoning_delta raised", exc_info=True)
def _fire_agent_message_completed(item: dict) -> None:
text = item.get("text") or ""
if not isinstance(text, str) or not text.strip():
return
# display.show_commentary=false — mid-turn narration stays off the
# visible interim path on this runtime too (same contract as the
# codex_responses commentary channel).
if not getattr(agent, "show_commentary", True):
return
emit = getattr(agent, "_emit_interim_assistant_message", None)
if emit is None:
return
try:
emit({"role": "assistant", "content": text})
except Exception:
logger.debug(
"_emit_interim_assistant_message raised", exc_info=True,
)
def on_event(note: dict) -> None:
if not isinstance(note, dict):
return
method = note.get("method") or ""
params = note.get("params") or {}
if not isinstance(params, dict):
params = {}
if method == "item/agentMessage/delta":
_fire_text_delta(params)
return
if method in {"item/reasoning/delta", "item/reasoning/summaryDelta"}:
_fire_reasoning_delta(params)
return
item = params.get("item")
if not isinstance(item, dict):
return
item_type = item.get("type") or ""
if method == "item/started" and item_type in _CODEX_TOOL_ITEM_TYPES:
_fire_tool_started(item)
return
if method == "item/completed":
if item_type in _CODEX_TOOL_ITEM_TYPES:
_fire_tool_completed(item)
elif item_type == "agentMessage":
_fire_agent_message_completed(item)
return on_event
def run_codex_app_server_turn(
agent,
*,
@@ -670,13 +356,22 @@ def run_codex_app_server_turn(
exc_info=True,
)
# Bridge codex JSON-RPC notifications (item/started, item/completed,
# item/agentMessage/delta, ...) into Hermes' gateway UI callbacks
# (tool_progress_callback, _fire_stream_delta,
# _emit_interim_assistant_message). Without this, Discord/Telegram
# users see no live tool-progress or interim commentary while
# codex_app_server is running — only the final answer (#33200).
# Supersedes the narrower item/started-only bridge from #38835.
def _on_codex_event(note: dict) -> None:
# Bridge Codex app-server item/started notifications to Hermes
# tool-progress so gateways show verbose "running X" breadcrumbs
# on this route too (#38835).
progress_callback = getattr(agent, "tool_progress_callback", None)
if progress_callback is None:
return
mapped = _codex_note_to_tool_progress(note)
if mapped is None:
return
tool_name, preview, args = mapped
try:
progress_callback("tool.started", tool_name, preview, args)
except Exception:
logger.debug("codex tool-progress callback raised", exc_info=True)
agent._codex_session = CodexAppServerSession(
cwd=cwd,
approval_callback=approval_callback,
@@ -684,7 +379,7 @@ def run_codex_app_server_turn(
auto_approve_exec=auto_approve_requests,
auto_approve_apply_patch=auto_approve_requests,
),
on_event=make_codex_app_server_event_bridge(agent),
on_event=_on_codex_event,
)
# NOTE: the user message is ALREADY appended to messages by the
@@ -887,37 +582,15 @@ def _item_field(item: Any, name: str, default: Any = None) -> Any:
def _raise_stream_error(event: Any) -> None:
"""Raise a ``_StreamErrorEvent`` from a ``type=error`` SSE frame.
The Responses spec puts the failure details at the top level of the
frame (``{"type": "error", "code": ..., "message": ..., "param": ...}``),
but the official OpenAI SDK and several OpenAI-compatible proxies wrap
them in an HTTP-style nested envelope instead
(``{"type": "error", "error": {"code": ..., "message": ..., "param": ...}}``).
Read the top-level fields first, then fall back to the nested envelope so
the error classifier sees the provider's real code/message (rate-limit vs
context-overflow vs entitlement) rather than the generic placeholder.
Port of anomalyco/opencode#36130.
Imported lazily so this module stays importable from places that don't
pull in ``run_agent`` (e.g. plugin code, doc tools).
"""
from run_agent import _StreamErrorEvent
nested = _event_field(event, "error")
def _error_field(name: str) -> Any:
value = _event_field(event, name)
if value is None and nested is not None:
value = _item_field(nested, name)
return value
raw_message = _error_field("message")
if raw_message is not None and not isinstance(raw_message, str):
raw_message = str(raw_message)
message = (raw_message or "stream emitted error event").strip() or "stream emitted error event"
message = (_event_field(event, "message", "") or "stream emitted error event").strip()
raise _StreamErrorEvent(
message,
code=_error_field("code"),
param=_error_field("param"),
code=_event_field(event, "code"),
param=_event_field(event, "param"),
)
@@ -927,7 +600,6 @@ def _consume_codex_event_stream(
model: str,
on_text_delta=None,
on_reasoning_delta=None,
on_commentary_message=None,
on_first_delta=None,
on_event=None,
interrupt_check=None,
@@ -959,11 +631,7 @@ def _consume_codex_event_stream(
* ``on_text_delta(str)`` — fires per ``response.output_text.delta``, suppressed
once a function_call event is seen (so tool-call turns don't bleed text
into the chat).
* ``on_reasoning_delta(str)`` — fires per ``response.reasoning.*.delta`` and
``phase=analysis`` message deltas. When no dedicated commentary callback
is supplied, commentary also uses this legacy fallback.
* ``on_commentary_message(str)`` — fires once per completed
``phase=commentary`` message, before any following tool item executes.
* ``on_reasoning_delta(str)`` — fires per ``response.reasoning.*.delta``.
* ``on_first_delta()`` — one-shot, fires on the first text delta only.
* ``on_event(event)`` — fires for every event before any other processing.
Used for watchdog activity, debug logging, anything wire-shape-agnostic.
@@ -974,7 +642,6 @@ def _consume_codex_event_stream(
has_tool_calls = False
first_delta_fired = False
active_message_phase: str | None = None
commentary_text_deltas: List[str] = []
terminal_status: str = "completed"
terminal_usage: Any = None
terminal_response_id: str = None
@@ -1019,8 +686,6 @@ def _consume_codex_event_stream(
if item_type == "message":
phase = _item_field(item, "phase", None)
active_message_phase = phase.strip().lower() if isinstance(phase, str) else None
if active_message_phase == "commentary":
commentary_text_deltas = []
else:
active_message_phase = None
if "function_call" in str(item_type):
@@ -1029,16 +694,10 @@ def _consume_codex_event_stream(
if "output_text.delta" in event_type or event_type == "response.output_text.delta":
delta_text = _event_field(event, "delta", "")
if delta_text and active_message_phase == "commentary":
commentary_text_deltas.append(delta_text)
# Preserve CLI/backward compatibility when no first-class
# commentary consumer is installed.
if on_commentary_message is None and on_reasoning_delta is not None:
try:
on_reasoning_delta(delta_text)
except Exception:
logger.debug("Codex stream on_reasoning_delta raised", exc_info=True)
elif delta_text and active_message_phase == "analysis":
is_commentary_delta = active_message_phase in {"commentary", "analysis"}
if delta_text and is_commentary_delta:
# Commentary streams through the reasoning channel, not the
# visible answer stream (and stays out of output_text).
if on_reasoning_delta is not None:
try:
on_reasoning_delta(delta_text)
@@ -1078,27 +737,6 @@ def _consume_codex_event_stream(
done_item = _event_field(event, "item")
if done_item is not None:
collected_output_items.append(done_item)
done_phase = _item_field(done_item, "phase", None)
done_phase = done_phase.strip().lower() if isinstance(done_phase, str) else None
if done_phase == "commentary" and on_commentary_message is not None:
commentary_text = "".join(commentary_text_deltas).strip()
if not commentary_text:
content_parts = _item_field(done_item, "content", [])
if isinstance(content_parts, list):
commentary_text = "".join(
str(_item_field(part, "text", "") or "")
for part in content_parts
if _item_field(part, "type", "") == "output_text"
).strip()
if commentary_text:
try:
on_commentary_message(commentary_text)
except Exception:
logger.debug(
"Codex stream on_commentary_message raised",
exc_info=True,
)
commentary_text_deltas = []
continue
if event_type in _TERMINAL_EVENT_TYPES:
@@ -1199,14 +837,14 @@ def run_codex_stream(agent, api_kwargs: dict, client: Any = None, on_first_delta
def _on_reasoning_delta(text: str) -> None:
agent._fire_reasoning_delta(text)
def _on_commentary_message(text: str) -> None:
agent._fire_streamed_codex_commentary(text)
def _on_event(event: Any) -> None:
# TTFB watchdog and activity touch — runs once per SSE event.
agent._codex_stream_last_event_ts = time.time()
agent._touch_activity("receiving stream response")
def _interrupt_check() -> bool:
return bool(agent._interrupt_requested)
for attempt in range(max_stream_retries + 1):
if agent._interrupt_requested:
raise InterruptedError("Agent interrupted before Codex stream retry")
@@ -1226,27 +864,6 @@ def run_codex_stream(agent, api_kwargs: dict, client: Any = None, on_first_delta
continue
raise
# Claim the delta sink for THIS attempt (#65991) — parity with the
# chat_completions/anthropic/bedrock paths. If a prior attempt's
# stream is somehow still alive, this claim supersedes it so its
# late deltas are fenced out of the turn; conversely, a newer
# attempt supersedes us and the interrupt_check below stops our
# consumption immediately.
_writer_token = claim_stream_writer(agent)
def _interrupt_or_superseded(_tok=_writer_token) -> bool:
if agent._interrupt_requested:
return True
if not stream_writer_is_current(agent, _tok):
logger.warning(
"Codex streaming attempt superseded by a newer stream; "
"stopping consumption to preserve the single-writer "
"invariant (model=%s).",
api_kwargs.get("model", "unknown"),
)
return True
return False
try:
# Compatibility: some mocks/providers return a concrete response
# instead of an iterable. Pass it straight through.
@@ -1259,17 +876,9 @@ def run_codex_stream(agent, api_kwargs: dict, client: Any = None, on_first_delta
model=api_kwargs.get("model"),
on_text_delta=_on_text_delta,
on_reasoning_delta=_on_reasoning_delta,
on_commentary_message=(
_on_commentary_message
if (
getattr(agent, "interim_assistant_callback", None) is not None
and getattr(agent, "show_commentary", True)
)
else None
),
on_first_delta=on_first_delta,
on_event=_on_event,
interrupt_check=_interrupt_or_superseded,
interrupt_check=_interrupt_check,
)
except (_httpx.RemoteProtocolError, _httpx.ReadTimeout, _httpx.ConnectError, ConnectionError) as exc:
if attempt < max_stream_retries:
@@ -1318,5 +927,4 @@ __all__ = [
"run_codex_stream",
"run_codex_create_stream_fallback",
"_consume_codex_event_stream",
"make_codex_app_server_event_bridge",
]
+14 -9
View File
@@ -55,12 +55,13 @@ import json
import logging
import os
import re
import subprocess
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Optional
from hermes_cli._subprocess_compat import bounded_git_probe
from hermes_cli._subprocess_compat import IS_WINDOWS, windows_hide_flags
logger = logging.getLogger("hermes.coding_context")
@@ -688,14 +689,18 @@ def _enabled_mcp_servers(config: Optional[dict[str, Any]]) -> list[str]:
def _git(cwd: Path, *args: str) -> str:
"""``git -C <cwd> <args>`` → stripped stdout, or ``""`` on any failure.
Uses the shared :func:`bounded_git_probe` so the post-kill cleanup is bounded
on Windows — a plain ``subprocess.run(timeout=...)`` here deadlocked the agent
turn inside ``build_coding_workspace_block`` when a killed git left a suspended
descendant holding the pipe handles (issue #66037).
"""
return bounded_git_probe(["git", "-C", str(cwd), *args], timeout=_GIT_TIMEOUT)
_popen_kwargs = {"creationflags": windows_hide_flags()} if IS_WINDOWS else {}
try:
out = subprocess.run(
["git", "-C", str(cwd), *args],
capture_output=True,
text=True,
timeout=_GIT_TIMEOUT,
**_popen_kwargs,
)
except (OSError, subprocess.SubprocessError):
return ""
return out.stdout.strip() if out.returncode == 0 else ""
def _parse_status(porcelain: str) -> tuple[dict[str, str], dict[str, int]]:
File diff suppressed because it is too large Load Diff
+3 -50
View File
@@ -26,31 +26,7 @@ Lifecycle:
"""
from abc import ABC, abstractmethod
from typing import Any, Dict, List, Optional
from agent.redact import redact_sensitive_text
MEMORY_CONTEXT_MAX_CHARS = 6_000
_MEMORY_CONTEXT_HEAD_CHARS = 4_000
_MEMORY_CONTEXT_TAIL_CHARS = 1_500
_MEMORY_CONTEXT_TRUNCATION_MARKER = "\n...[memory provider context truncated]...\n"
def sanitize_memory_context(memory_context: str) -> str:
"""Prepare provider context for a context-engine/LLM egress boundary."""
sanitized = redact_sensitive_text(
memory_context.strip(),
force=True,
redact_url_credentials=True,
)
if len(sanitized) <= MEMORY_CONTEXT_MAX_CHARS:
return sanitized
return (
sanitized[:_MEMORY_CONTEXT_HEAD_CHARS]
+ _MEMORY_CONTEXT_TRUNCATION_MARKER
+ sanitized[-_MEMORY_CONTEXT_TAIL_CHARS:]
)
from typing import Any, Dict, List
class ContextEngine(ABC):
@@ -111,10 +87,8 @@ class ContextEngine(ABC):
def compress(
self,
messages: List[Dict[str, Any]],
current_tokens: Optional[int] = None,
focus_topic: Optional[str] = None,
force: bool = False,
memory_context: str = "",
current_tokens: int = None,
focus_topic: str = None,
) -> List[Dict[str, Any]]:
"""Compact the message list and return the new message list.
@@ -129,12 +103,6 @@ class ContextEngine(ABC):
Engines that support guided compression should prioritise
preserving information related to this topic. Engines that
don't support it may simply ignore this argument.
force: Whether a user-requested compression should bypass an
engine-owned cooldown. Engines without cooldowns may ignore it.
memory_context: Text returned by memory providers immediately before
compaction. Summarizing engines should include non-empty text in
their handoff prompt. Older engines may omit this parameter; the
host filters unsupported optional arguments by signature.
"""
# -- Optional: pre-flight check ----------------------------------------
@@ -260,19 +228,4 @@ class ContextEngine(ABC):
(e.g. recalculate DAG budgets, switch summary models).
"""
self.context_length = context_length
# Apply per-model threshold overrides if set (longest substring match).
# Falls back to _config_threshold_percent (the raw config value) when
# no override matches. Plugin engines that override update_model() can
# call resolve_model_threshold() for the same logic.
from agent.context_compressor import resolve_model_threshold
if not hasattr(self, "_config_threshold_percent"):
# Snapshot the pre-override percent ONCE so repeated model
# switches fall back to the engine's configured value, not the
# previous model's override.
self._config_threshold_percent = self.threshold_percent
self._base_threshold_percent = resolve_model_threshold(
model, getattr(self, "model_thresholds", {}),
self._config_threshold_percent,
)
self.threshold_percent = self._base_threshold_percent
self.threshold_tokens = int(context_length * self.threshold_percent)
File diff suppressed because it is too large Load Diff
+110 -664
View File
File diff suppressed because it is too large Load Diff
+5 -4
View File
@@ -26,7 +26,7 @@ from openai.types.chat.chat_completion_message_tool_call import (
Function,
)
from agent.file_safety import get_read_block_error, get_write_denied_error
from agent.file_safety import get_read_block_error, is_write_denied
from agent.redact import redact_sensitive_text
from tools.environments.local import hermes_subprocess_env
@@ -727,9 +727,10 @@ class CopilotACPClient:
elif method == "fs/write_text_file":
try:
path = _ensure_path_within_cwd(str(params.get("path") or ""), cwd)
denied = get_write_denied_error(str(path))
if denied:
raise PermissionError(denied)
if is_write_denied(str(path)):
raise PermissionError(
f"Write denied: '{path}' is a protected system/credential file."
)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(str(params.get("content") or ""))
response = {
+55 -273
View File
@@ -43,19 +43,11 @@ logger = logging.getLogger(__name__)
def _load_config_safe() -> Optional[dict]:
"""Load config.yaml read-only, returning None on any error.
Uses ``load_config_readonly()``: every consumer in this module only reads
(``get_pool_strategy``, ``_iter_custom_providers``, the model-config seed),
and the deepcopy that ``load_config()`` pays per call is what made
credential-pool checks the dominant cost of ``model.options`` — the picker
calls ``load_pool()`` once per provider row, each of which loaded (and
deep-copied) the full config again.
"""
"""Load config.yaml, returning None on any error."""
try:
from hermes_cli.config import load_config_readonly
from hermes_cli.config import load_config
return load_config_readonly()
return load_config()
except Exception:
return None
@@ -122,20 +114,6 @@ EXHAUSTED_TTL_401_SECONDS = 5 * 60 # 5 minutes
EXHAUSTED_TTL_429_SECONDS = 60 * 60 # 1 hour
EXHAUSTED_TTL_DEFAULT_SECONDS = 60 * 60 # 1 hour
# Throttle window for the "no available entries" INFO line. Credential
# selection runs on a hot path (every model call, plus auxiliary tasks like
# compression/moa/titles), so when a pool is empty or fully exhausted the
# un-throttled log fires on *every* selection. On Windows several Hermes
# processes share one rotating log guarded by concurrent-log-handler's
# cross-process lock; that per-selection volume storms the lock
# (``RuntimeError: Cannot acquire lock after 20 attempts``), pegs a core, and
# stalls the asyncio event loop long enough to fail the Desktop backend
# readiness handshake ("Timed out connecting to Hermes backend after
# 15000ms"). Logging the condition at most once per window preserves the
# signal while removing the storm — same class of fix as the warn-once
# dedup in #58265.
NO_AVAILABLE_ENTRIES_LOG_THROTTLE_SECONDS = 60.0
# Pool key prefix for custom OpenAI-compatible endpoints.
# Custom endpoints all share provider='custom' but are keyed by their
# custom_providers name: 'custom:<normalized_name>'.
@@ -150,17 +128,6 @@ _EXTRA_KEYS = frozenset({
})
def _normalize_pool_auth_type(provider: str, token: Any, auth_type: Any) -> str:
"""Infer pool auth metadata for token formats with one unambiguous meaning."""
if (
provider == "anthropic"
and isinstance(token, str)
and token.startswith("sk-ant-oat")
):
return AUTH_TYPE_OAUTH
return str(auth_type or AUTH_TYPE_API_KEY)
@dataclass
class PooledCredential:
provider: str
@@ -190,11 +157,6 @@ class PooledCredential:
def __post_init__(self):
if self.extra is None:
self.extra = {}
self.auth_type = _normalize_pool_auth_type(
self.provider,
self.access_token,
self.auth_type,
)
def __getattr__(self, name: str):
if name in _EXTRA_KEYS:
@@ -483,44 +445,6 @@ def get_pool_strategy(provider: str) -> str:
return STRATEGY_FILL_FIRST
def credential_pool_matches_provider(
pool_or_provider: Any,
provider: Optional[str],
*,
base_url: Optional[str] = None,
) -> bool:
"""Return whether a pool belongs to the requested runtime provider.
Named custom endpoints intentionally use two identities: the live agent is
``custom`` while its pool is keyed ``custom:<name>``. Accept that pair only
when the runtime base URL resolves to the exact same custom pool key.
Empty string identities fail closed. Legacy pool adapters without a
``provider`` attribute remain compatible; production pools are scoped.
"""
raw_pool_provider = getattr(pool_or_provider, "provider", None)
if raw_pool_provider is None:
if isinstance(pool_or_provider, str):
raw_pool_provider = pool_or_provider
else:
# Backward compatibility for lightweight/unscoped pool adapters.
# Production CredentialPool instances always carry ``provider``;
# old plugins and tests may expose only select()/has_credentials().
return True
pool_provider = str(raw_pool_provider or "").strip().lower()
provider_norm = str(provider or "").strip().lower()
if not pool_provider or not provider_norm:
return False
if pool_provider == provider_norm:
return True
if provider_norm != "custom" or not pool_provider.startswith(CUSTOM_POOL_PREFIX):
return False
try:
matched_pool = get_custom_provider_pool_key(base_url or "")
except Exception:
return False
return str(matched_pool or "").strip().lower() == pool_provider
DEFAULT_MAX_CONCURRENT_PER_CREDENTIAL = 1
@@ -565,12 +489,14 @@ def _write_through_provider_state_to_global_root(
except Exception:
return
try:
auth_mod._persist_provider_state_to_store(
provider_id,
state,
global_path,
set_active=False,
)
if global_path.exists():
global_store = _load_auth_store(global_path)
else:
global_store = {}
if not isinstance(global_store, dict):
return
_store_provider_state(global_store, provider_id, dict(state), set_active=False)
auth_mod._save_auth_store(global_store, global_path)
except Exception as exc: # pragma: no cover - best effort
logger.debug(
"%s pool refresh: write-through to global root failed: %s",
@@ -588,12 +514,6 @@ class CredentialPool:
self._lock = threading.Lock()
self._active_leases: Dict[str, int] = {}
self._max_concurrent = DEFAULT_MAX_CONCURRENT_PER_CREDENTIAL
# Monotonic timestamp of the last "no available entries" log, used to
# throttle that message so an empty/exhausted pool cannot storm the
# shared rotating log (see NO_AVAILABLE_ENTRIES_LOG_THROTTLE_SECONDS).
# Re-armed to None on every successful selection so a recover→re-exhaust
# transition logs promptly instead of being swallowed by a stale window.
self._last_no_entries_log_at: Optional[float] = None
def has_credentials(self) -> bool:
return bool(self._entries)
@@ -852,45 +772,6 @@ class CredentialPool:
logger.debug("Failed to sync xAI OAuth entry from auth.json: %s", exc)
return entry
def _sync_xai_oauth_entry_from_pool_store(
self, entry: PooledCredential
) -> PooledCredential:
"""Adopt a token pair rotated by another pool instance.
Direct xAI integrations load a fresh ``CredentialPool`` for each
request. Their in-memory locks therefore cannot protect xAI's
single-use refresh token across concurrent requests or processes.
This helper is called while the shared auth-store lock is held and
re-reads the exact persisted row before a refresh POST is attempted.
"""
if self.provider != "xai-oauth":
return entry
try:
persisted = next(
(
payload
for payload in read_credential_pool(self.provider)
if isinstance(payload, dict) and payload.get("id") == entry.id
),
None,
)
if not isinstance(persisted, dict):
return entry
stored = PooledCredential.from_dict(self.provider, persisted)
if (
stored.access_token != entry.access_token
or stored.refresh_token != entry.refresh_token
):
logger.debug(
"Pool entry %s: adopting xAI OAuth tokens rotated by another pool instance",
entry.id,
)
self._replace_entry(entry, stored)
return stored
except Exception as exc:
logger.debug("Failed to sync xAI OAuth entry from credential pool: %s", exc)
return entry
def _sync_nous_entry_from_auth_store(self, entry: PooledCredential) -> PooledCredential:
"""Sync a Nous pool entry from auth.json if tokens differ.
@@ -1084,58 +965,31 @@ class CredentialPool:
self._mark_exhausted(entry, None)
return None
# Codex and xAI OAuth refresh tokens are single-use. The
# sync→POST→write-back sequence below must run atomically across Hermes
# processes: otherwise two processes can both adopt the same on-disk
# token, both POST it, and the loser gets ``refresh_token_reused``.
# Serialize the whole sequence through the shared cross-process
# auth-store flock (the same lock and extended-timeout pattern used by
# resolve_codex_runtime_credentials()). When a waiter finally acquires
# the lock, the in-lock re-sync below picks up the rotated token the
# winner persisted and skips the POST.
if self.provider in ("openai-codex", "xai-oauth"):
sync_entry = (
self._sync_codex_entry_from_auth_store
if self.provider == "openai-codex"
else self._sync_xai_oauth_entry_from_pool_store
# Codex OAuth refresh tokens are single-use. The sync→POST→write-back
# sequence below must run atomically across Hermes processes: otherwise
# two processes can both adopt the same on-disk token, both POST it, and
# the loser gets ``refresh_token_reused``. Serialize the whole sequence
# through the shared cross-process auth-store flock (the same lock and
# extended-timeout pattern used by resolve_codex_runtime_credentials()).
# When a waiter finally acquires the lock, the in-lock re-sync below
# picks up the rotated token the winner persisted and skips the POST.
if self.provider == "openai-codex":
refresh_timeout_seconds = auth_mod.env_float(
"HERMES_CODEX_REFRESH_TIMEOUT_SECONDS", 20
)
with _auth_store_lock(
timeout_seconds=self._single_use_refresh_lock_timeout()
):
synced = sync_entry(entry)
if self.provider == "openai-codex":
if synced is not entry:
entry = synced
if not force and not self._entry_needs_refresh(entry):
return entry
return self._refresh_entry_impl(entry, force=force)
if (
synced.access_token != entry.access_token
or synced.refresh_token != entry.refresh_token
):
return synced
return self._refresh_entry_impl(synced, force=force)
lock_timeout = max(
float(auth_mod.AUTH_LOCK_TIMEOUT_SECONDS),
float(refresh_timeout_seconds) + 5.0,
)
with _auth_store_lock(timeout_seconds=lock_timeout):
synced = self._sync_codex_entry_from_auth_store(entry)
if synced is not entry:
entry = synced
if not force and not self._entry_needs_refresh(entry):
return entry
return self._refresh_entry_impl(entry, force=force)
return self._refresh_entry_impl(entry, force=force)
def _single_use_refresh_lock_timeout(self) -> float:
"""Lock timeout for single-use-refresh-token providers.
Covers the configured refresh POST timeout plus a margin so a slow
token endpoint cannot make the flock give up before the refresh
resolves. Reads the provider's ``HERMES_*_REFRESH_TIMEOUT_SECONDS``
override.
"""
env_var = (
"HERMES_CODEX_REFRESH_TIMEOUT_SECONDS"
if self.provider == "openai-codex"
else "HERMES_XAI_REFRESH_TIMEOUT_SECONDS"
)
refresh_timeout_seconds = auth_mod.env_float(env_var, 20)
return max(
float(auth_mod.AUTH_LOCK_TIMEOUT_SECONDS),
float(refresh_timeout_seconds) + 5.0,
)
def _refresh_entry_impl(
self, entry: PooledCredential, *, force: bool
) -> Optional[PooledCredential]:
@@ -1524,11 +1378,6 @@ class CredentialPool:
entries_to_prune: List[str] = []
available: List[PooledCredential] = []
for entry in self._entries:
# Borrowed credentials persist as metadata-only references and are
# hydrated from their live source on load. A stale duplicate row
# can remain unhydrated; never lease or select it as an empty key.
if entry.auth_type == AUTH_TYPE_API_KEY and not entry.runtime_api_key:
continue
# For anthropic claude_code entries, sync from the credentials file
# before any status/refresh checks. This picks up tokens refreshed
# by other processes (Claude Code CLI, other Hermes profiles).
@@ -1632,32 +1481,13 @@ class CredentialPool:
self._persist(removed_ids=entries_to_prune)
return available
def _log_no_available_entries(self) -> None:
"""Emit the empty-pool INFO line at most once per throttle window.
Called on every selection while the pool is empty/exhausted. Without
throttling this storms the Windows cross-process log lock and stalls the
event loop (see NO_AVAILABLE_ENTRIES_LOG_THROTTLE_SECONDS).
"""
now = time.monotonic()
last = self._last_no_entries_log_at
if last is not None and (now - last) < NO_AVAILABLE_ENTRIES_LOG_THROTTLE_SECONDS:
return
self._last_no_entries_log_at = now
logger.info("credential pool: no available entries (all exhausted or empty)")
def _select_unlocked(self, *, refresh: bool = True) -> Optional[PooledCredential]:
available = self._available_entries(clear_expired=True, refresh=refresh)
def _select_unlocked(self) -> Optional[PooledCredential]:
available = self._available_entries(clear_expired=True, refresh=True)
if not available:
self._current_id = None
self._log_no_available_entries()
logger.info("credential pool: no available entries (all exhausted or empty)")
return None
# A successful selection means the pool recovered; re-arm the throttle
# so a later re-exhaustion logs immediately rather than being silenced
# by a window opened during the previous empty stretch.
self._last_no_entries_log_at = None
if self._strategy == STRATEGY_RANDOM:
entry = random.choice(available)
self._current_id = entry.id
@@ -1781,35 +1611,6 @@ class CredentialPool:
with self._lock:
return self._try_refresh_current_unlocked()
def try_refresh_matching(
self, api_key_hint: Optional[str] = None
) -> Optional[PooledCredential]:
"""Force-refresh the entry that supplied ``api_key_hint``.
Direct provider integrations may reload the pool after a request has
already failed, so they cannot rely on ``current_id`` identifying the
issuing credential. With no hint, select an entry without first doing
the normal proactive refresh; the forced refresh below must consume a
rotating refresh token exactly once.
"""
with self._lock:
entry = None
if api_key_hint:
entry = next(
(
candidate
for candidate in self._entries
if candidate.runtime_api_key == api_key_hint
),
None,
)
else:
entry = self.current() or self._select_unlocked(refresh=False)
if entry is None:
return None
self._current_id = entry.id
return self._try_refresh_current_unlocked()
def _try_refresh_current_unlocked(self) -> Optional[PooledCredential]:
entry = self.current()
if entry is None:
@@ -1893,15 +1694,11 @@ class CredentialPool:
def _upsert_entry(entries: List[PooledCredential], provider: str, source: str, payload: Dict[str, Any]) -> bool:
matching_indices = []
existing_idx = None
for idx, entry in enumerate(entries):
if entry.source == source:
matching_indices.append(idx)
existing_idx = matching_indices[0] if matching_indices else None
duplicate_indices = set(matching_indices[1:])
if duplicate_indices:
entries[:] = [entry for idx, entry in enumerate(entries) if idx not in duplicate_indices]
existing_idx = idx
break
if existing_idx is None:
payload.setdefault("id", uuid.uuid4().hex[:6])
@@ -1933,8 +1730,8 @@ def _upsert_entry(entries: List[PooledCredential], provider: str, source: str, p
# Runtime-only borrowed secret updates should refresh the in-memory
# entry without forcing auth.json churn when the disk-safe payload is
# unchanged (for example env keys with the same fingerprint).
return bool(duplicate_indices) or existing.to_dict() != updated.to_dict()
return bool(duplicate_indices)
return existing.to_dict() != updated.to_dict()
return False
def _normalize_pool_priorities(provider: str, entries: List[PooledCredential]) -> bool:
@@ -2309,10 +2106,9 @@ def _seed_from_env(provider: str, entries: List[PooledCredential]) -> Tuple[bool
def _get_env_prefer_dotenv(key: str) -> str:
env_file = load_env()
raw = env_file.get(key, "").strip()
scoped_value = (_get_secret(key, "") or "").strip()
env_val = os.environ.get(key, "").strip()
# If .env contains an unresolved op:// reference, prefer the
# already-resolved value supplied by the active secret scope (or by
# os.environ in legacy single-profile mode), set by
# already-resolved value from os.environ (set by
# load_hermes_dotenv() -> apply_onepassword_secrets()). The raw
# "op://Vault/Item/field" string would otherwise win and every
# provider auth attempt would receive a URL instead of a key. This
@@ -2320,9 +2116,9 @@ def _seed_from_env(provider: str, entries: List[PooledCredential]) -> Tuple[bool
# references straight into .env rather than the secrets.onepassword
# config block. For every non-op:// value the original
# .env-takes-precedence behaviour is preserved unchanged.
if raw.startswith("op://") and scoped_value:
return scoped_value
return raw or scoped_value
if raw.startswith("op://") and env_val:
return env_val
return raw or _get_secret(key, "") or env_val
# Honour user suppression — `hermes auth remove <provider> <N>` for an
# env-seeded credential marks the env:<VAR> source as suppressed so it
@@ -2409,6 +2205,12 @@ def _seed_from_env(provider: str, entries: List[PooledCredential]) -> Tuple[bool
if _is_source_suppressed(provider, source):
continue
active_sources.add(source)
# Claude Code OAuth tokens are the only Anthropic credentials that should flow into the OAuth refresh path.
auth_type = (
AUTH_TYPE_OAUTH
if provider == "anthropic" and token.startswith("sk-ant-oat")
else AUTH_TYPE_API_KEY
)
base_url = env_url or pconfig.inference_base_url
if provider == "kimi-coding":
base_url = _resolve_kimi_base_url(token, pconfig.inference_base_url, env_url)
@@ -2423,6 +2225,7 @@ def _seed_from_env(provider: str, entries: List[PooledCredential]) -> Tuple[bool
env_var=env_var,
token=token,
base_url=base_url,
auth_type=auth_type,
),
)
return changed, active_sources
@@ -2550,37 +2353,16 @@ def load_pool(provider: str) -> CredentialPool:
for payload in raw_entries
)
entries = [PooledCredential.from_dict(provider, payload) for payload in raw_entries]
raw_needs_auth_normalization = any(
isinstance(payload, dict)
and _normalize_pool_auth_type(
provider,
payload.get("access_token"),
payload.get("auth_type", AUTH_TYPE_API_KEY),
) != payload.get("auth_type", AUTH_TYPE_API_KEY)
for payload in raw_entries
)
if raw_needs_auth_normalization:
# A profile may be reading this provider from the global-root fallback.
# Keep that fallback read-only: only the store that owns these rows may
# rewrite them. Loading the default/root profile will heal global rows.
active_pool = _load_auth_store().get("credential_pool")
active_entries = active_pool.get(provider) if isinstance(active_pool, dict) else None
raw_needs_auth_normalization = bool(active_entries)
if provider.startswith(CUSTOM_POOL_PREFIX):
# Custom endpoint pool — seed from custom_providers config and model config
custom_changed, custom_sources = _seed_custom_pool(provider, entries)
changed = raw_needs_sanitization or raw_needs_auth_normalization or custom_changed
changed = raw_needs_sanitization or custom_changed
changed |= _prune_stale_seeded_entries(entries, custom_sources)
else:
singleton_changed, singleton_sources = _seed_from_singletons(provider, entries)
env_changed, env_sources = _seed_from_env(provider, entries)
changed = (
raw_needs_sanitization
or raw_needs_auth_normalization
or singleton_changed
or env_changed
)
changed = raw_needs_sanitization or singleton_changed or env_changed
# ``load_pool()`` is a non-destructive read for env-seeded entries: a
# process missing a provider env var must not delete the persisted
# pool entry for every other process (#9331). File-backed singletons
+1 -1
View File
@@ -355,7 +355,7 @@ def evaluate_credits_notices(
if show_depleted and "credits.depleted" not in active:
to_show.append(
AgentNotice(
text="✕ Credit access paused · run /topup to top up",
text="✕ Credit access paused · run /credits to top up",
level="error",
kind=CREDITS_NOTICE_KIND,
key="credits.depleted",
+1 -6
View File
@@ -98,12 +98,7 @@ def _backup_cron_jobs_into(dest: Path) -> Dict[str, Any]:
info["reason"] = "no cron/jobs.json present"
return info
try:
# utf-8-sig: same dialect as cron/jobs.load_jobs — a UTF-8 BOM left
# by Windows editors otherwise survives decoding as U+FEFF, breaks
# json.loads below, and misreports jobs_count as 0 with a spurious
# parse warning. The BOM-less text is also what gets written to the
# backup, so a later rollback restores a loadable file.
raw = src.read_text(encoding="utf-8-sig")
raw = src.read_text(encoding="utf-8")
except OSError as e:
logger.debug("Failed to read cron/jobs.json for backup: %s", e)
info["reason"] = f"read error: {e}"
+3 -50
View File
@@ -462,14 +462,13 @@ def build_tool_preview(tool_name: str, args: dict, max_len: int | None = None) -
sid = args.get("session_id", "")
data = args.get("data", "")
timeout_val = args.get("timeout")
parts = [str(action) if action else ""]
parts = [action]
if sid:
parts.append(str(sid)[:16])
parts.append(sid[:16])
if data:
parts.append(f'"{_oneline(str(data)[:20])}"')
parts.append(f'"{_oneline(data[:20])}"')
if timeout_val and action == "wait":
parts.append(f"{timeout_val}s")
parts = [p for p in parts if p]
return " ".join(parts) if parts else None
if tool_name == "todo":
@@ -645,52 +644,6 @@ def verb_drops_preview(tool_name: str) -> bool:
return tool_name in _TOOL_VERBS_NO_PREVIEW
def build_status_phrase(tool_name: str, args: dict | None, max_len: int = 49) -> str | None:
"""Build a short present-tense status phrase for platform status surfaces.
Used by text-rendering "typing" indicators (Slack's
``assistant.threads.setStatus`` line) to show what the agent is doing
right now: ``is running scripts/run_tests.sh…`` instead of a static
``is thinking...``. The phrase is phrased to follow the bot's display
name ("Hermes is running …"), so it starts lowercase with "is".
Pass ``args=None`` for a verb-only phrase (``is running…``) — used when
``display.live_status`` is ``verb`` to keep argument previews out of
shared channels.
Returns None for the ``_thinking`` pseudo-tool and when friendly labels
are disabled (callers fall back to their static default). ``max_len``
caps the total phrase length; Slack truncates its status line around 50
characters, so the default stays just under that.
"""
if not tool_name or tool_name == "_thinking":
return None
if not _friendly_tool_labels:
return None
verb = _TOOL_VERBS.get(tool_name)
if verb:
head = f"is {verb[0].lower()}{verb[1:]}"
else:
# Custom / plugin / MCP tools: generic but still informative.
head = f"is using {tool_name}"
phrase = head
if args and verb and tool_name not in _TOOL_VERBS_NO_PREVIEW:
preview = build_tool_preview(tool_name, args, max_len=None)
if preview:
# Previews can contain newlines (terminal commands); keep the
# status to the first line.
preview = preview.splitlines()[0].strip()
phrase = f"{head}{tool_verb_connector(tool_name)}{preview}"
if len(phrase) > max_len - 1:
phrase = phrase[: max_len - 2].rstrip() + "…"
else:
phrase = phrase + "…"
return phrase
def build_tool_label(tool_name: str, args: dict, max_len: int | None = None) -> str | None:
"""Build a human-phrased status label for a tool call.
+14 -114
View File
@@ -123,25 +123,6 @@ _BILLING_PATTERNS = [
"not available on the free tier",
]
# xAI's explicit Grok credit-exhaustion code. Keep the HTTP 403 special case
# provider-scoped: other providers' generic billing codes historically remain
# auth failures when they arrive as 403.
_XAI_SPENDING_LIMIT_ERROR_CODE = "personal-team-blocked:spending-limit"
# Structured provider codes that mean the account cannot serve paid traffic
# until credits/subscription capacity is restored. xAI returns its explicit
# Grok spending-limit signal as HTTP 403 rather than 402.
_BILLING_ERROR_CODES = frozenset({
"insufficient_quota",
"billing_not_active",
"payment_required",
"insufficient_credits",
"no_usable_credits",
"balance_depleted",
"model_not_supported_on_free_tier",
_XAI_SPENDING_LIMIT_ERROR_CODE,
})
# Patterns that indicate rate limiting (transient, will resolve)
_RATE_LIMIT_PATTERNS = [
"rate limit",
@@ -269,11 +250,6 @@ _CONTEXT_OVERFLOW_PATTERNS = [
"context window",
"prompt is too long",
"prompt exceeds max length",
# NOTE: bare "max_tokens" is load-bearing — the output-cap-retry path keys
# off it (e.g. "max_tokens: 65536 > context_window: 200000 ..."). Do NOT
# remove it. Provider empty-response advisories also contain "very low
# max_tokens", but those are intercepted by _EMPTY_PROVIDER_RESPONSE_PATTERNS
# BEFORE this list is consulted, so they never mis-route into compression.
"max_tokens",
"maximum number of tokens",
# vLLM / local inference server patterns
@@ -291,8 +267,6 @@ _CONTEXT_OVERFLOW_PATTERNS = [
# Chinese error messages (some providers return these)
"超过最大长度",
"上下文长度",
# Z.AI / Zhipu GLM pattern (English form; error code 1210)
"tokens in request more than max tokens allowed",
# AWS Bedrock Converse API error patterns
"input is too long",
"max input token",
@@ -431,19 +405,6 @@ _THINKING_SIG_PATTERNS = [
# the exception type is generic (e.g. RuntimeError from a local shim that
# wraps a subprocess timeout). Checked before the type-based transport
# heuristics so custom-provider "timed out" errors don't fall through to
# Provider empty-response advisories (OpenRouter / nano-gpt / similar).
# Checked before context-overflow matching because the advisory text often
# mentions "max_tokens" as a possible cause, which historically sat in
# _CONTEXT_OVERFLOW_PATTERNS and sent healthy sessions into a compression
# death spiral ending in "Cannot compress further".
_EMPTY_PROVIDER_RESPONSE_PATTERNS = [
"returned an empty response",
"empty response despite retries",
"provider returned an empty response",
"model returning empty responses",
"empty response stream",
]
# the unknown bucket and get misreported as empty responses.
_TIMEOUT_MESSAGE_PATTERNS = [
"timed out",
@@ -793,14 +754,6 @@ def classify_api_error(
if classified is not None:
return classified
# Local MoA config drift is deterministic: a persisted session can retain
# a preset name that was later renamed/deleted. Retrying the same lookup
# cannot recover and makes a clear config error look like an API outage.
from agent.errors import MoAPresetNotFoundError
if isinstance(error, MoAPresetNotFoundError):
return _result(FailoverReason.model_not_found, retryable=False)
# ── 3. Error code classification ────────────────────────────────
if error_code:
@@ -887,34 +840,12 @@ def classify_api_error(
)
return _result(FailoverReason.timeout, retryable=True)
# ── 7b. Stale-call circuit breaker → failover immediately ──────
# _check_stale_giveup() in agent/chat_completion_helpers.py raises a
# RuntimeError when the provider has been unresponsive for N
# consecutive stale attempts (default 5). The error is NOT a transport
# timeout — the circuit breaker fires *before* any network call to avoid
# an indefinite stall. Without this classification the RuntimeError
# falls through to FailoverReason.unknown (retryable=True), which burns
# all max_retries against the same dead provider (each retry hitting the
# circuit breaker instantly with zero network overhead) before fallback
# is attempted. Classify as non-retryable + should_fallback so the
# retry loop activates the next fallback provider on the first hit.
if (
error_type == "RuntimeError"
and "consecutive stale attempts" in error_msg
and "aborting this call" in error_msg
):
return _result(
FailoverReason.timeout,
retryable=False,
should_fallback=True,
)
# ── 8. Transport / timeout heuristics ───────────────────────────
# ── 7. Transport / timeout heuristics ───────────────────────────
if error_type in _TRANSPORT_ERROR_TYPES or isinstance(error, (TimeoutError, ConnectionError, OSError)):
return _result(FailoverReason.timeout, retryable=True)
# ── 9. Fallback: unknown ────────────────────────────────────────
# ── 8. Fallback: unknown ────────────────────────────────────────
return _result(FailoverReason.unknown, retryable=True)
@@ -953,11 +884,7 @@ def _classify_by_status(
# OpenRouter 403 "key limit exceeded" is actually billing. Other
# providers also use 403 for account-plan or credit exhaustion.
if (
(
provider == "xai-oauth"
and error_code.lower() == _XAI_SPENDING_LIMIT_ERROR_CODE
)
or "key limit exceeded" in error_msg
"key limit exceeded" in error_msg
or "spending limit" in error_msg
or any(p in error_msg for p in _BILLING_PATTERNS)
):
@@ -1095,14 +1022,6 @@ def _classify_by_status(
# remaining explicit context-overflow signal routes into the
# compression-and-retry path (mirroring _classify_400) instead of
# blind server_error retries that exhaust and drop the turn.
# Empty-response advisories that mention "max_tokens" must not enter
# that compression path.
if any(p in error_msg for p in _EMPTY_PROVIDER_RESPONSE_PATTERNS):
return result_fn(
FailoverReason.server_error,
retryable=True,
should_compress=False,
)
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
return result_fn(
FailoverReason.context_overflow,
@@ -1116,12 +1035,6 @@ def _classify_by_status(
# Cloudflare/Tailscale hop relabeling the status). Route explicit
# overflow bodies into compression; otherwise treat as transient
# overload and retry.
if any(p in error_msg for p in _EMPTY_PROVIDER_RESPONSE_PATTERNS):
return result_fn(
FailoverReason.server_error,
retryable=True,
should_compress=False,
)
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
return result_fn(
FailoverReason.context_overflow,
@@ -1234,7 +1147,6 @@ def _classify_400(
"encrypted content for item" in error_msg
and "could not be verified" in error_msg
)
or "could not decrypt the provided encrypted_content" in error_msg
):
return result_fn(
FailoverReason.invalid_encrypted_content,
@@ -1247,8 +1159,8 @@ def _classify_400(
# returns:
# "Unsupported parameter: 'max_tokens' is not supported with this model.
# Use 'max_completion_tokens' instead."
# That string contains the literal substring "max_tokens", which historically
# sat in _CONTEXT_OVERFLOW_PATTERNS — so without this guard the 400 is
# That string contains the literal substring "max_tokens", which is one of
# the _CONTEXT_OVERFLOW_PATTERNS — so without this guard the 400 is
# misclassified as context_overflow, routed into the compression loop,
# re-sent with the same bad parameter, and ends in "Cannot compress
# further". These errors are deterministic (every retry gets the identical
@@ -1270,17 +1182,6 @@ def _classify_400(
should_fallback=True,
)
# Empty-provider-response advisories must not enter compression. They
# often mention "max_tokens" as a possible cause and used to match the
# bare overflow pattern, then thrash compress until "Cannot compress
# further" on an otherwise healthy session (custom endpoints / nano-gpt).
if any(p in error_msg for p in _EMPTY_PROVIDER_RESPONSE_PATTERNS):
return result_fn(
FailoverReason.server_error,
retryable=True,
should_compress=False,
)
# Context overflow from 400
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
return result_fn(
@@ -1368,7 +1269,15 @@ def _classify_by_error_code(
should_rotate_credential=True,
)
if code_lower in _BILLING_ERROR_CODES:
if code_lower in {
"insufficient_quota",
"billing_not_active",
"payment_required",
"insufficient_credits",
"no_usable_credits",
"balance_depleted",
"model_not_supported_on_free_tier",
}:
return result_fn(
FailoverReason.billing,
retryable=False,
@@ -1484,15 +1393,6 @@ def _classify_by_message(
should_fallback=True,
)
# Empty-provider-response advisories (often mention "max_tokens") must
# retry without compression — see the matching 400-path guard above.
if any(p in error_msg for p in _EMPTY_PROVIDER_RESPONSE_PATTERNS):
return result_fn(
FailoverReason.server_error,
retryable=True,
should_compress=False,
)
# Context overflow patterns
if any(p in error_msg for p in _CONTEXT_OVERFLOW_PATTERNS):
return result_fn(
-10
View File
@@ -1,13 +1,3 @@
class SSLConfigurationError(Exception):
"""Raised when SSL/TLS certificate bundle configuration fails."""
pass
class EmptyStreamError(RuntimeError):
"""Raised when a provider closes a stream without yielding a response."""
pass
class MoAPresetNotFoundError(ValueError):
"""Raised when a persisted MoA preset no longer exists in config."""
+8 -41
View File
@@ -46,9 +46,6 @@ def build_write_denied_paths(home: str) -> set[str]:
# Top-level Anthropic PKCE credential store remains sensitive even
# when a profile is active; default/non-profile sessions still read it.
str(hermes_root / ".anthropic_oauth.json"),
# Bitwarden Secrets Manager encrypted disk cache.
str(hermes_home / "cache" / "bws_cache.enc.json"),
str(hermes_root / "cache" / "bws_cache.enc.json"),
os.path.join(home, ".netrc"),
os.path.join(home, ".pgpass"),
os.path.join(home, ".npmrc"),
@@ -98,16 +95,16 @@ def get_safe_write_roots() -> set[str]:
return roots
def _classify_write_denial(path: str) -> Optional[str]:
"""Return ``'credential'``, ``'safe_root'``, or ``None`` if writes are allowed."""
def is_write_denied(path: str) -> bool:
"""Return True if path is blocked by the write denylist or safe root."""
home = os.path.realpath(os.path.expanduser("~"))
resolved = os.path.realpath(os.path.expanduser(str(path)))
if resolved in build_write_denied_paths(home):
return "credential"
return True
for prefix in build_write_denied_prefixes(home):
if resolved.startswith(prefix):
return "credential"
return True
mcp_tokens_dir_name = "mcp-tokens"
@@ -121,27 +118,16 @@ def _classify_write_denial(path: str) -> Optional[str]:
continue
for base_real in hermes_dirs:
# Session transcripts are application-owned state. Letting the agent's
# generic file tools rewrite state.db or legacy JSON snapshots can
# falsify conversation history and invalidate resume/compression state.
try:
if resolved == os.path.realpath(os.path.join(base_real, "state.db")):
return True
sessions_real = os.path.realpath(os.path.join(base_real, "sessions"))
if resolved == sessions_real or resolved.startswith(sessions_real + os.sep):
return True
except Exception:
pass
try:
mcp_real = os.path.realpath(os.path.join(base_real, mcp_tokens_dir_name))
if resolved == mcp_real or resolved.startswith(mcp_real + os.sep):
return "credential"
return True
except Exception:
pass
try:
pairing_real = os.path.realpath(os.path.join(base_real, "pairing"))
if resolved == pairing_real or resolved.startswith(pairing_real + os.sep):
return "credential"
return True
except Exception:
pass
@@ -153,28 +139,9 @@ def _classify_write_denial(path: str) -> Optional[str]:
allowed = True
break
if not allowed:
return "safe_root"
return True
return None
def is_write_denied(path: str) -> bool:
"""Return True if path is blocked by the write denylist or safe root."""
return _classify_write_denial(path) is not None
def get_write_denied_error(path: str, *, verb: str = "Write") -> Optional[str]:
"""Return a user/model-facing error when writes to ``path`` are blocked."""
denial = _classify_write_denial(path)
if denial is None:
return None
if denial == "safe_root":
roots_display = os.pathsep.join(sorted(get_safe_write_roots()))
return (
f"{verb} denied: '{path}' is outside HERMES_WRITE_SAFE_ROOT "
f"({roots_display}). Unset the variable or add this path's directory prefix."
)
return f"{verb} denied: '{path}' is a protected system/credential file."
return False
# Common secret-bearing project-local environment file basenames.
+2 -16
View File
@@ -32,13 +32,6 @@ from agent.gemini_schema import sanitize_gemini_tool_parameters
logger = logging.getLogger(__name__)
try:
import hermes_cli as _hermes_cli
_HERMES_VERSION = str(_hermes_cli.__version__)
except Exception:
_HERMES_VERSION = "0.0.0"
DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta"
# Published max output-token ceiling shared by every current Gemini text model
@@ -106,10 +99,7 @@ def probe_gemini_tier(
url,
params={"key": key},
json=payload,
headers={
"Content-Type": "application/json",
"X-Goog-Api-Client": f"hermes-agent/{_HERMES_VERSION}",
},
headers={"Content-Type": "application/json"},
)
except Exception as exc:
logger.debug("probe_gemini_tier: network error: %s", exc)
@@ -911,11 +901,7 @@ class GeminiNativeClient:
"Content-Type": "application/json",
"Accept": "application/json",
"x-goog-api-key": self.api_key,
# Include Hermes client context following Gemini's partner
# integration guidance.
# See https://ai.google.dev/gemini-api/docs/partner-integration
"User-Agent": f"hermes-agent/{_HERMES_VERSION} (gemini-native)",
"X-Goog-Api-Client": f"hermes-agent/{_HERMES_VERSION}",
"User-Agent": "hermes-agent (gemini-native)",
}
headers.update(self._default_headers)
return headers
-24
View File
@@ -87,30 +87,6 @@ def sanitize_gemini_schema(schema: Any) -> Dict[str, Any]:
if any(not isinstance(item, str) for item in enum_val):
cleaned.pop("enum", None)
# Gemini validates ``required`` strictly against the same node's
# ``properties`` — GenerateContentRequest fails with HTTP 400
# "...items.required[0]: property is not defined" when a required name
# has no matching property in that node. MCP servers routinely emit
# this shape (e.g. the GitHub remote MCP's array item schemas carry
# ``required`` without ``properties``), and one bad tool schema fails
# the ENTIRE request before any model output. Filter ``required`` to
# names that exist in this node's ``properties`` and drop it when
# nothing valid remains. The tool handler still validates required
# fields at execution time, so this only removes what Gemini couldn't
# accept anyway. (Port of Kilo-Org/kilocode#11955.)
required_val = cleaned.get("required")
if isinstance(required_val, list):
props_val = cleaned.get("properties")
prop_names = set(props_val.keys()) if isinstance(props_val, dict) else set()
valid_required = [
name for name in required_val
if isinstance(name, str) and name in prop_names
]
if not valid_required:
cleaned.pop("required", None)
elif len(valid_required) != len(required_val):
cleaned["required"] = valid_required
return cleaned
+3 -5
View File
@@ -267,12 +267,10 @@ def _resolve_inference_base_url(
) -> str:
"""Best-effort base URL for the active inference provider."""
try:
from agent.auxiliary_client import _runtime_main_value
from agent.auxiliary_client import _RUNTIME_MAIN_BASE_URL
runtime = str(_runtime_main_value("base_url") or "").strip()
runtime_provider = str(_runtime_main_value("provider") or "").strip().lower()
requested_provider = str(provider or "").strip().lower()
if runtime and (not requested_provider or requested_provider == runtime_provider):
runtime = str(_RUNTIME_MAIN_BASE_URL or "").strip()
if runtime:
return runtime
except Exception:
pass
+23 -194
View File
@@ -17,7 +17,6 @@ Usage:
"""
import json
import sqlite3
import time
from collections import Counter, defaultdict
from datetime import datetime
@@ -142,8 +141,8 @@ class InsightsEngine:
}
# Compute insights
models = self._compute_model_breakdown(sessions, cutoff, source)
overview = self._compute_overview(sessions, message_stats, models)
overview = self._compute_overview(sessions, message_stats)
models = self._compute_model_breakdown(sessions)
platforms = self._compute_platform_breakdown(sessions)
tools = self._compute_tool_breakdown(tool_usage)
skills = self._compute_skill_breakdown(skill_usage)
@@ -173,7 +172,7 @@ class InsightsEngine:
"message_count, tool_call_count, input_tokens, output_tokens, "
"cache_read_tokens, cache_write_tokens, billing_provider, "
"billing_base_url, billing_mode, estimated_cost_usd, "
"actual_cost_usd, cost_status, cost_source, api_call_count")
"actual_cost_usd, cost_status, cost_source")
# Pre-computed query strings — f-string evaluated once at class definition,
# not at runtime, so no user-controlled value can alter the query structure.
@@ -400,12 +399,7 @@ class InsightsEngine:
# Computation
# =========================================================================
def _compute_overview(
self,
sessions: List[Dict],
message_stats: Dict,
models: Optional[List[Dict]] = None,
) -> Dict:
def _compute_overview(self, sessions: List[Dict], message_stats: Dict) -> Dict:
"""Compute high-level overview statistics."""
total_input = sum(s.get("input_tokens") or 0 for s in sessions)
total_output = sum(s.get("output_tokens") or 0 for s in sessions)
@@ -437,21 +431,6 @@ class InsightsEngine:
else:
models_without_pricing.add(display)
if models:
total_cost = sum(float(m.get("cost") or 0.0) for m in models)
# Token totals likewise: the per-model breakdown includes
# auxiliary usage rows (vision/compression/titles — task
# dimension in session_model_usage, #23270) plus reconciled
# residuals, while the sessions counters carry main-loop usage
# only. Summing the breakdown keeps overview totals consistent
# with the per-model table and stops `hermes insights`
# undercounting aux spend (#58592, #9979).
total_input = sum(int(m.get("input_tokens") or 0) for m in models)
total_output = sum(int(m.get("output_tokens") or 0) for m in models)
total_cache_read = sum(int(m.get("cache_read_tokens") or 0) for m in models)
total_cache_write = sum(int(m.get("cache_write_tokens") or 0) for m in models)
total_tokens = total_input + total_output + total_cache_read + total_cache_write
# Session duration stats (guard against negative durations from clock drift)
durations = []
for s in sessions:
@@ -494,189 +473,39 @@ class InsightsEngine:
"included_cost_sessions": included_cost_sessions,
}
_GET_MODEL_USAGE_WITH_SOURCE = (
"SELECT u.session_id, u.model, u.billing_provider, u.billing_base_url,"
" u.api_call_count, u.input_tokens, u.output_tokens,"
" u.cache_read_tokens, u.cache_write_tokens, u.reasoning_tokens,"
" u.estimated_cost_usd, u.actual_cost_usd, u.cost_status,"
" u.cost_source, u.billing_mode"
" FROM session_model_usage u"
" JOIN sessions s ON s.id = u.session_id"
" WHERE s.started_at >= ? AND s.source = ?"
)
_GET_MODEL_USAGE_ALL = (
"SELECT u.session_id, u.model, u.billing_provider, u.billing_base_url,"
" u.api_call_count, u.input_tokens, u.output_tokens,"
" u.cache_read_tokens, u.cache_write_tokens, u.reasoning_tokens,"
" u.estimated_cost_usd, u.actual_cost_usd, u.cost_status,"
" u.cost_source, u.billing_mode"
" FROM session_model_usage u"
" JOIN sessions s ON s.id = u.session_id"
" WHERE s.started_at >= ?"
)
def _get_model_usage(self, cutoff: float, source: str = None) -> List[Dict]:
"""Fetch per-model usage rows within the window (issue #51607).
Returns an empty list when the table is missing (e.g. a DB opened by
older code that never created it) so the caller can fall back to the
per-session aggregate.
"""
try:
if source:
cursor = self._conn.execute(
self._GET_MODEL_USAGE_WITH_SOURCE, (cutoff, source)
)
else:
cursor = self._conn.execute(self._GET_MODEL_USAGE_ALL, (cutoff,))
return [dict(row) for row in cursor.fetchall()]
except sqlite3.OperationalError:
return []
def _compute_model_breakdown(
self, sessions: List[Dict], cutoff: float, source: str = None
) -> List[Dict]:
"""Break down token usage and cost by model.
Tokens and cost are attributed per model from session_model_usage, so a
session that switched models mid-flight (via ``/model``) splits across
every model it used instead of dumping everything on the initial model
(issue #51607). Sessions without per-model rows — e.g. data written
before this table existed and not yet backfilled — fall back to their
single recorded (model, billing_provider) aggregate so nothing is lost.
Tool calls aren't tied to a specific API invocation, so they stay
attributed to the session's recorded model.
"""
def _compute_model_breakdown(self, sessions: List[Dict]) -> List[Dict]:
"""Break down usage by model."""
model_data = defaultdict(lambda: {
"sessions": set(), "input_tokens": 0, "output_tokens": 0,
"sessions": 0, "input_tokens": 0, "output_tokens": 0,
"cache_read_tokens": 0, "cache_write_tokens": 0,
"reasoning_tokens": 0, "total_tokens": 0, "api_calls": 0,
"tool_calls": 0, "cost": 0.0, "actual_cost": 0.0,
"total_tokens": 0, "tool_calls": 0, "cost": 0.0,
})
def _accumulate(model, provider, base_url, session_id, inp, out,
cache_read, cache_write, reasoning, *,
stored_cost=None, actual_cost=None, cost_status=None):
model = model or "unknown"
for s in sessions:
model = s.get("model") or "unknown"
# Normalize: strip provider prefix for display
display_model = model.split("/")[-1] if "/" in model else model
d: Dict[str, Any] = model_data[display_model]
d["sessions"].add(session_id)
d = model_data[display_model]
d["sessions"] += 1
inp = s.get("input_tokens") or 0
out = s.get("output_tokens") or 0
cache_read = s.get("cache_read_tokens") or 0
cache_write = s.get("cache_write_tokens") or 0
d["input_tokens"] += inp
d["output_tokens"] += out
d["cache_read_tokens"] += cache_read
d["cache_write_tokens"] += cache_write
d["reasoning_tokens"] += reasoning
d["total_tokens"] += inp + out + cache_read + cache_write
if stored_cost is None:
estimate, status = _estimate_cost(
model, inp, out,
cache_read_tokens=cache_read, cache_write_tokens=cache_write,
provider=provider or None, base_url=base_url,
)
else:
estimate = float(stored_cost or 0.0)
status = cost_status or "unknown"
d["tool_calls"] += s.get("tool_call_count") or 0
estimate, status = _estimate_cost(s)
d["cost"] += estimate
d["actual_cost"] += float(actual_cost or 0.0)
d["has_pricing"] = has_known_pricing(model, s.get("billing_provider"), s.get("billing_base_url"))
d["cost_status"] = status
if has_known_pricing(model, provider or None, base_url):
d["has_pricing"] = True
else:
d.setdefault("has_pricing", False)
return display_model
usage_rows = self._get_model_usage(cutoff, source)
usage_totals = defaultdict(lambda: {
"input_tokens": 0, "output_tokens": 0, "cache_read_tokens": 0,
"cache_write_tokens": 0, "reasoning_tokens": 0,
"api_call_count": 0, "estimated_cost_usd": 0.0,
"actual_cost_usd": 0.0,
})
for r in usage_rows:
totals: Dict[str, Any] = usage_totals[r["session_id"]]
for key in (
"input_tokens", "output_tokens", "cache_read_tokens",
"cache_write_tokens", "reasoning_tokens", "api_call_count",
):
totals[key] += r[key] or 0
totals["estimated_cost_usd"] += r["estimated_cost_usd"] or 0.0
totals["actual_cost_usd"] += r["actual_cost_usd"] or 0.0
d = _accumulate(
r["model"], r["billing_provider"], r.get("billing_base_url"),
r["session_id"], r["input_tokens"] or 0, r["output_tokens"] or 0,
r["cache_read_tokens"] or 0, r["cache_write_tokens"] or 0,
r["reasoning_tokens"] or 0,
stored_cost=(
r["estimated_cost_usd"]
if r.get("cost_status") or r.get("cost_source")
else None
),
actual_cost=r["actual_cost_usd"],
cost_status=r.get("cost_status"),
)
model_data[d]["api_calls"] += r["api_call_count"] or 0
# Reconcile against the aggregate row. This covers legacy sessions,
# interrupted migrations, and absolute cumulative updates without
# double-counting already-attributed route deltas.
for s in sessions:
totals = usage_totals[s["id"]]
inp = max(0, (s.get("input_tokens") or 0) - totals["input_tokens"])
out = max(0, (s.get("output_tokens") or 0) - totals["output_tokens"])
cache_read = max(
0, (s.get("cache_read_tokens") or 0) - totals["cache_read_tokens"]
)
cache_write = max(
0, (s.get("cache_write_tokens") or 0) - totals["cache_write_tokens"]
)
residual_cost = max(
0.0, float(s.get("estimated_cost_usd") or 0.0)
- totals["estimated_cost_usd"],
)
residual_actual = max(
0.0, float(s.get("actual_cost_usd") or 0.0)
- totals["actual_cost_usd"],
)
residual_calls = max(
0, (s.get("api_call_count") or 0) - totals["api_call_count"]
)
if not (
inp or out or cache_read or cache_write or residual_cost
or residual_actual or residual_calls
):
continue
d = _accumulate(
s.get("model"), s.get("billing_provider"),
s.get("billing_base_url"), s["id"],
inp, out, cache_read, cache_write, 0,
stored_cost=residual_cost,
actual_cost=residual_actual,
cost_status=s.get("cost_status"),
)
residual_bucket: Dict[str, Any] = model_data[d]
residual_bucket["api_calls"] += residual_calls
# Tool calls are attributed by the session's recorded model.
for s in sessions:
tool_calls = s.get("tool_call_count") or 0
if not tool_calls:
continue
model = s.get("model") or "unknown"
display_model = model.split("/")[-1] if "/" in model else model
model_data[display_model]["tool_calls"] += tool_calls
result = []
for model, data in model_data.items():
entry = {"model": model, **data}
entry["sessions"] = len(data["sessions"])
# Models that surfaced only via tool-call attribution (no token
# rows) won't have these set by _accumulate — default them so the
# output shape is uniform for downstream/JSON consumers.
entry.setdefault("has_pricing", False)
entry.setdefault("cost_status", "unknown")
result.append(entry)
result = [
{"model": model, **data}
for model, data in model_data.items()
]
# Sort by tokens first, fall back to session count when tokens are 0
result.sort(key=lambda x: (x["total_tokens"], x["sessions"]), reverse=True)
return result
-108
View File
@@ -1,108 +0,0 @@
"""Turn-end guard for kanban workers.
Kanban workers must end with ``kanban_complete`` or ``kanban_block``. Models
(especially GLM / Qwen families) sometimes narrate the next step
("Let me write the report now") and stop with ``finish_reason=stop`` and no
tool calls. Hermes treats that as a clean exit → ``rc=0`` → dispatcher
``protocol_violation``.
This module is policy-only: when a kanban worker tries to finish without a
terminal board tool, return a bounded synthetic nudge so the conversation
loop continues instead of exiting.
"""
from __future__ import annotations
import os
from typing import Any, Iterable, Optional
_TERMINAL_KANBAN_TOOLS = frozenset({"kanban_complete", "kanban_block"})
_DEFAULT_MAX_ATTEMPTS = 2
def kanban_stop_nudge_enabled() -> bool:
"""Return whether the kanban stop-guard is active for this process.
On when ``HERMES_KANBAN_TASK`` is set (dispatcher-spawned worker), unless
``HERMES_KANBAN_STOP_NUDGE`` explicitly disables it.
"""
env = os.environ.get("HERMES_KANBAN_STOP_NUDGE")
if env is not None and env.strip().lower() in {"0", "false", "no", "off"}:
return False
task = (os.environ.get("HERMES_KANBAN_TASK") or "").strip()
return bool(task)
def _tool_call_name(tc: Any) -> str:
if isinstance(tc, dict):
fn = tc.get("function")
if isinstance(fn, dict):
return str(fn.get("name") or "")
return str(tc.get("name") or "")
fn = getattr(tc, "function", None)
if fn is not None:
return str(getattr(fn, "name", "") or "")
return str(getattr(tc, "name", "") or "")
def session_called_kanban_terminal(messages: Iterable[dict] | None) -> bool:
"""True if this conversation already invoked a terminal kanban tool."""
if not messages:
return False
for msg in messages:
if not isinstance(msg, dict):
continue
role = msg.get("role")
if role == "assistant":
for tc in msg.get("tool_calls") or []:
if _tool_call_name(tc) in _TERMINAL_KANBAN_TOOLS:
return True
elif role == "tool":
name = str(msg.get("name") or "")
if name in _TERMINAL_KANBAN_TOOLS:
return True
return False
def build_kanban_stop_nudge(
*,
messages: Iterable[dict] | None = None,
attempts: int = 0,
max_attempts: int = _DEFAULT_MAX_ATTEMPTS,
task_id: Optional[str] = None,
) -> Optional[str]:
"""Return a synthetic follow-up when a kanban worker exits without a terminal tool.
Returns ``None`` when the guard should not fire (not a kanban worker,
already completed/blocked, or nudge budget exhausted).
"""
if not kanban_stop_nudge_enabled():
return None
if attempts >= max_attempts:
return None
if session_called_kanban_terminal(messages):
return None
tid = (task_id or os.environ.get("HERMES_KANBAN_TASK") or "").strip() or "this task"
return (
"[System: You are a Hermes kanban worker. A plain-text reply is NOT a "
"terminal state for the board.\n\n"
f"Task `{tid}` is still `running`. Ending now without a board tool "
"causes a protocol violation (clean exit with no "
"`kanban_complete` / `kanban_block`).\n\n"
"Do this immediately in your next response — do not narrate intent:\n"
"1. Finish any remaining deliverable (write the required file(s) now).\n"
"2. Call `kanban_complete(summary=..., artifacts=[...])` if the work "
"is done, OR `kanban_block(reason=...)` if you are blocked.\n\n"
"Never end a turn with only a promise of future action. Repeated "
"protocol violations will block this task and require manual intervention.]"
)
__all__ = [
"build_kanban_stop_nudge",
"kanban_stop_nudge_enabled",
"session_called_kanban_terminal",
]
-12
View File
@@ -20,17 +20,6 @@ _LM_VALID_EFFORTS = {"none", "minimal", "low", "medium", "high", "xhigh"}
# Map them onto the OpenAI-compatible request vocabulary.
_LM_EFFORT_ALIASES = {"off": "none", "on": "medium"}
# Hermes' generic effort ladder grew past LM Studio's vocabulary ("max",
# "ultra"). Clamp the stronger generic levels onto LM Studio's ceiling: left
# alone they miss _LM_VALID_EFFORTS, keep the initialized "medium" default and
# are thereby conflated with unparseable input, so asking for more reasoning
# yields less than "xhigh". Mirrors the ceiling clamp every other provider
# applies (see agent/transports/codex.py).
#
# Deliberately separate from _LM_EFFORT_ALIASES: that mapping is also applied
# to the model's published allowed_options, which must not be rewritten.
_LM_EFFORT_CLAMP = {"max": "xhigh", "ultra": "xhigh"}
def resolve_lmstudio_effort(
reasoning_config: Optional[dict],
@@ -50,7 +39,6 @@ def resolve_lmstudio_effort(
else:
raw = (reasoning_config.get("effort") or "").strip().lower()
raw = _LM_EFFORT_ALIASES.get(raw, raw)
raw = _LM_EFFORT_CLAMP.get(raw, raw)
if raw in _LM_VALID_EFFORTS:
effort = raw
if allowed_options:
+71 -140
View File
@@ -18,15 +18,9 @@ into it via :func:`agent.lsp.manager.LSPService.touch_file`.
Implementation notes:
- All per-document state lives in one :class:`_DocState` keyed by
absolute path. Freshness is tracked with **document versions**,
not timestamps: every didChange bumps ``version``, and each stored
push/pull result is tagged with the version it describes. A
result is fresh iff its tag >= the version being waited on, so a
didChange implicitly invalidates everything older — no clearing,
no clock comparisons, no race windows. This is what prevents
"ghost diagnostics": a slow server's leftovers from the previous
edit can never masquerade as a verdict on the current content.
- Push diagnostics are stored per-URI in :attr:`_push_diagnostics` from
``textDocument/publishDiagnostics`` notifications. Pull diagnostics
go in :attr:`_pull_diagnostics`. The merged view dedupes by content.
- Whole-document sync. Even when the server advertises incremental
sync, we send a single ``contentChanges`` entry replacing the
@@ -51,7 +45,6 @@ import asyncio
import logging
import os
import sys
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Awaitable, Callable, Dict, List, Optional, Set
from urllib.parse import quote, unquote
@@ -131,40 +124,6 @@ def _end_position(text: str) -> Dict[str, int]:
return {"line": last_line, "character": last_col}
@dataclass
class _DocState:
"""Everything the client tracks for one open document.
``version`` is the LSP document version we last sent (didOpen=0,
each didChange +1). It doubles as the freshness token: stored
push/pull results are tagged with the version they describe
(``push_version`` / ``pull_version``), and a result is *fresh*
iff its tag has caught up to ``version``. Bumping the version on
didChange therefore invalidates all older results implicitly —
no store-clearing, no timestamps.
``push_version``/``pull_version`` start at -1 = "no data yet".
Servers that echo a document version in publishDiagnostics get
exact tagging; those that don't are credited with the current
version at receipt time (a push observed after we sent the
change describes the changed content or newer).
"""
version: int = 0
text: str = ""
push: List[Dict[str, Any]] = field(default_factory=list)
pull: List[Dict[str, Any]] = field(default_factory=list)
push_version: int = -1
pull_version: int = -1
seed_seen: bool = False
def fresh_push(self, version: Optional[int] = None) -> bool:
return self.push_version >= (self.version if version is None else version)
def fresh_pull(self, version: Optional[int] = None) -> bool:
return self.pull_version >= (self.version if version is None else version)
class LSPClient:
"""Async LSP client tied to one server process and one workspace root.
@@ -227,10 +186,18 @@ class LSPClient:
# is silently dropped by default.
}
# Per-document state (version, text, diagnostic stores, and
# their freshness tags), keyed by absolute file path (NOT URI).
# See _DocState for the version-based freshness model.
self._docs: Dict[str, _DocState] = {}
# Tracked file state — required for didChange version bumps.
self._files: Dict[str, Dict[str, Any]] = {}
# Diagnostic stores, keyed by file path (NOT URI).
self._push_diagnostics: Dict[str, List[Dict[str, Any]]] = {}
self._pull_diagnostics: Dict[str, List[Dict[str, Any]]] = {}
# Per-path "last published" time so wait-for-fresh logic works.
self._published: Dict[str, float] = {}
# Per-path version of the latest push (matches our didChange
# version when the server respects it).
self._published_version: Dict[str, int] = {}
# First-push seen flag, for typescript-style seed-on-first-push.
self._first_push_seen: Set[str] = set()
# Capability registrations — only diagnostic ones are tracked.
self._diagnostic_registrations: Dict[str, Dict[str, Any]] = {}
@@ -680,25 +647,25 @@ class LSPClient:
if not isinstance(diagnostics, list):
diagnostics = []
version = params.get("version")
loop_time = asyncio.get_event_loop().time()
doc = self._docs.setdefault(path, _DocState(version=-1))
if self._seed_first_push and not doc.seed_seen:
# First push: seed the store WITHOUT a freshness tag. It
# arrives before the user-triggered didChange could've
# produced fresh diagnostics, so it must never satisfy a
# waiter — it's baseline data only.
doc.seed_seen = True
doc.push = diagnostics
if self._seed_first_push and path not in self._first_push_seen:
# First push: seed without firing the event so a waiter
# doesn't resolve on the very first push (which arrives
# before the user-triggered didChange could've produced
# fresh diagnostics).
self._first_push_seen.add(path)
self._push_diagnostics[path] = diagnostics
self._published[path] = loop_time
if isinstance(version, int):
self._published_version[path] = version
return
doc.seed_seen = True
doc.push = diagnostics
# Tag with the echoed document version when the server provides
# one; otherwise credit the current version — a push observed
# after we sent the change describes the changed content (or
# newer). Note doc.version is -1 for never-opened paths
# (e.g. relatedDocuments spillover), keeping them unfresh.
doc.push_version = version if isinstance(version, int) else doc.version
self._push_diagnostics[path] = diagnostics
self._published[path] = loop_time
if isinstance(version, int):
self._published_version[path] = version
self._first_push_seen.add(path)
# Bump the monotonic push counter and wake every waiter. We
# keep the Event sticky-set so any wait already in progress
# resolves; waiters re-check their predicate after waking and
@@ -727,16 +694,16 @@ class LSPClient:
raise LSPProtocolError(f"cannot read {abs_path}: {e}") from e
uri = file_uri(abs_path)
doc = self._docs.get(abs_path)
existing = self._files.get(abs_path)
if doc is not None and doc.version >= 0:
if existing is not None:
# Re-open: bump version, fire didChangeWatchedFiles + didChange.
await self._send_notification(
"workspace/didChangeWatchedFiles",
{"changes": [{"uri": uri, "type": 2}]}, # 2 = CHANGED
)
new_version = doc.version + 1
old_text = doc.text
new_version = existing["version"] + 1
old_text = existing["text"]
content_changes: List[Dict[str, Any]]
if self._sync_kind == 2:
content_changes = [
@@ -757,11 +724,7 @@ class LSPClient:
"contentChanges": content_changes,
},
)
# Bumping the version is the whole invalidation story:
# every stored result tagged with an older version is now
# stale by definition (see _DocState).
doc.version = new_version
doc.text = text
self._files[abs_path] = {"version": new_version, "text": text}
return new_version
# First open: didChangeWatchedFiles CREATED + didOpen.
@@ -769,9 +732,12 @@ class LSPClient:
"workspace/didChangeWatchedFiles",
{"changes": [{"uri": uri, "type": 1}]}, # 1 = CREATED
)
# Fresh doc state — anything stashed under this path by a
# pre-open push (relatedDocuments spillover etc.) is discarded.
self._docs[abs_path] = _DocState(version=0, text=text)
# Clear any stale push/pull entries — fresh open should start
# from scratch.
self._push_diagnostics.pop(abs_path, None)
self._pull_diagnostics.pop(abs_path, None)
self._published.pop(abs_path, None)
self._published_version.pop(abs_path, None)
await self._send_notification(
"textDocument/didOpen",
{
@@ -783,6 +749,7 @@ class LSPClient:
}
},
)
self._files[abs_path] = {"version": 0, "text": text}
return 0
async def save_file(self, path: str) -> None:
@@ -802,19 +769,12 @@ class LSPClient:
async def _pull_document_diagnostics(self, path: str) -> None:
"""Send ``textDocument/diagnostic`` for one file.
Stores results into the doc's pull store, tagged with the
document version captured at request send time. If a didChange
races past the in-flight request, the version bump makes the
stored result stale automatically — no explicit invalidation.
Silently no-ops on errors (server may not support the pull
endpoint).
Stores results into :attr:`_pull_diagnostics`. Silently
no-ops on errors (server may not support the pull endpoint).
"""
abs_path = os.path.abspath(path)
doc = self._docs.get(abs_path)
sent_version = doc.version if doc else -1
try:
params: Dict[str, Any] = {
"textDocument": {"uri": file_uri(abs_path)}
"textDocument": {"uri": file_uri(os.path.abspath(path))}
}
result = await self._send_request_with_retry(
"textDocument/diagnostic",
@@ -828,9 +788,7 @@ class LSPClient:
return
items = result.get("items")
if isinstance(items, list):
doc = self._docs.setdefault(abs_path, _DocState(version=-1))
doc.pull = items
doc.pull_version = sent_version
self._pull_diagnostics[os.path.abspath(path)] = items
related = result.get("relatedDocuments")
if isinstance(related, dict):
for uri, sub in related.items():
@@ -838,11 +796,7 @@ class LSPClient:
continue
sub_items = sub.get("items")
if isinstance(sub_items, list):
rel = self._docs.setdefault(uri_to_path(uri), _DocState(version=-1))
rel.pull = sub_items
# Same send-anchored tagging: fresh only if that
# doc hasn't changed since the request went out.
rel.pull_version = rel.version
self._pull_diagnostics[uri_to_path(uri)] = sub_items
async def wait_for_diagnostics(
self,
@@ -850,36 +804,22 @@ class LSPClient:
version: int,
*,
mode: str = "document",
timeout: Optional[float] = None,
) -> bool:
) -> None:
"""Wait for the server to publish diagnostics for ``path`` at ``version``.
``mode`` is ``"document"`` (5s budget, document pulls) or
``"full"`` (10s budget, also workspace pulls). ``timeout``
overrides the mode's default budget when provided — this is
how the user's ``lsp.wait_timeout`` config reaches the wait
loop (slow servers like tsserver on big projects need more
than the 5s default).
Returns ``True`` when *fresh* diagnostics arrived (a push at
or after our didChange, or a pull answered after it) and
``False`` on timeout. Callers must treat ``False`` as "no
data", NOT as "no errors" — the diagnostic stores may still
hold stale entries from the previous edit at that point.
Best-effort — never throws if the server doesn't support pull
diagnostics; we still get the push side.
``"full"`` (10s budget, also workspace pulls). Best-effort —
returns silently on timeout. Does NOT throw if the server
doesn't support pull diagnostics; we still get the push side.
"""
if timeout is not None and timeout > 0:
budget = timeout
else:
budget = DIAGNOSTICS_FULL_WAIT if mode == "full" else DIAGNOSTICS_DOCUMENT_WAIT
budget = DIAGNOSTICS_FULL_WAIT if mode == "full" else DIAGNOSTICS_DOCUMENT_WAIT
deadline = asyncio.get_event_loop().time() + budget
abs_path = os.path.abspath(path)
while True:
remaining = deadline - asyncio.get_event_loop().time()
if remaining <= 0:
return False
return
# Concurrent: document pull + push wait.
pull_task = asyncio.create_task(self._pull_document_diagnostics(abs_path))
@@ -898,24 +838,26 @@ class LSPClient:
pass
# If we got a fresh push for our version, we're done.
doc = self._docs.get(abs_path)
if doc and doc.fresh_push(version):
return True
current_v = self._published_version.get(abs_path)
if abs_path in self._published and (
current_v is None or current_v >= version
):
return
# Pull may have answered for the current version — that's
# also success.
if doc and doc.fresh_pull(version):
return True
# Pull may have populated _pull_diagnostics — that's also
# success.
if abs_path in self._pull_diagnostics:
return
# Loop until budget runs out.
async def _wait_for_fresh_push(self, path: str, version: int, timeout: float) -> None:
"""Wait until a fresh publishDiagnostics arrives for ``path`` at ``version``+."""
"""Wait until a publishDiagnostics arrives for ``path`` at ``version``+."""
deadline = asyncio.get_event_loop().time() + timeout
baseline = self._push_counter
while True:
doc = self._docs.get(path)
if doc and doc.fresh_push(version):
current_v = self._published_version.get(path)
if path in self._published and (current_v is None or current_v >= version):
# Debounce — wait a tick in case more diagnostics arrive
# immediately after. TS often emits in pairs. We
# snapshot the counter so we wake on a *new* push, not
@@ -946,28 +888,17 @@ class LSPClient:
except asyncio.TimeoutError:
continue
def diagnostics_for(self, path: str, *, fresh_only: bool = False) -> List[Dict[str, Any]]:
def diagnostics_for(self, path: str) -> List[Dict[str, Any]]:
"""Return current merged + deduped diagnostics for one file.
Diagnostics from push and pull stores are concatenated and
deduplicated by ``(severity, code, message, range)`` content
key. Empty list if the server hasn't published anything.
With ``fresh_only=True``, a store only contributes when its
version tag has caught up to the document's current version —
stale leftovers from the previous edit cycle are excluded.
This is what report paths should use: after an edit, "stale
errors" and "no errors" must not be conflated.
"""
doc = self._docs.get(os.path.abspath(path))
if doc is None:
return []
if fresh_only:
return _dedupe(
doc.push if doc.fresh_push() else [],
doc.pull if doc.fresh_pull() else [],
)
return _dedupe(doc.push, doc.pull)
abs_path = os.path.abspath(path)
push = self._push_diagnostics.get(abs_path) or []
pull = self._pull_diagnostics.get(abs_path) or []
return _dedupe(push, pull)
def _dedupe(*lists: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+10 -41
View File
@@ -292,10 +292,7 @@ class LSPService:
if not self.enabled_for(file_path):
return
try:
# Outer join budget must exceed the inner wait budget or a
# slow-but-alive server gets falsely marked broken.
t = max(8.0, self._wait_timeout + 3.0)
diags = self._loop.run(self._snapshot_async(file_path), timeout=t)
diags = self._loop.run(self._snapshot_async(file_path), timeout=8.0)
self._delta_baseline[os.path.abspath(file_path)] = diags or []
except Exception as e: # noqa: BLE001
logger.debug("baseline snapshot failed for %s: %s", file_path, e)
@@ -344,7 +341,7 @@ class LSPService:
try:
t = timeout if timeout is not None else self._wait_timeout + 2.0
diags = self._loop.run(self._open_and_wait_async(file_path), timeout=t)
diags = self._loop.run(self._open_and_wait_async(file_path), timeout=t) or []
except asyncio.TimeoutError as e:
eventlog.log_timeout(server_id, file_path)
logger.debug("LSP diagnostics timeout for %s: %s", file_path, e)
@@ -356,17 +353,6 @@ class LSPService:
self._mark_broken_for_file(file_path, e)
return []
if diags is None:
# The server is alive but never produced diagnostics for the
# post-edit content within the wait budget (common for
# tsserver on large projects). Report "no data" rather than
# whatever stale state is in the stores — surfacing the
# previous edit's errors as if they were current is the
# ghost-diagnostics bug. The server is NOT marked broken:
# slow is not dead, and the next edit may well succeed.
eventlog.log_timeout(server_id, file_path, kind="fresh diagnostics")
return []
abs_path = os.path.abspath(file_path)
if delta:
baseline = self._delta_baseline.get(abs_path) or []
@@ -466,43 +452,26 @@ class LSPService:
return []
try:
version = await client.open_file(file_path, language_id=language_id_for(file_path))
fresh = await client.wait_for_diagnostics(file_path, version, mode=self._wait_mode)
await client.wait_for_diagnostics(file_path, version, mode=self._wait_mode)
except Exception as e: # noqa: BLE001
logger.debug("snapshot open/wait failed: %s", e)
return []
self._last_used[(client.server_id, client.workspace_root)] = time.time()
if not fresh:
# No fresh data for the pre-edit content — an empty baseline
# is safe: worst case the delta filter removes less, never
# more. Never seed the baseline from stale stores.
return []
return list(client.diagnostics_for(file_path, fresh_only=True))
return list(client.diagnostics_for(file_path))
async def _open_and_wait_async(self, file_path: str) -> Optional[List[Dict[str, Any]]]:
"""Open + wait for FRESH diagnostics.
Returns the fresh diagnostic list, or ``None`` when the server
never produced post-change data within the wait budget. The
distinction matters: ``[]`` means "server checked the new
content, it's clean", ``None`` means "no verdict" — the caller
must not substitute stale data for either.
"""
async def _open_and_wait_async(self, file_path: str) -> List[Dict[str, Any]]:
client = await self._get_or_spawn(file_path)
if client is None:
return None
return []
try:
version = await client.open_file(file_path, language_id=language_id_for(file_path))
await client.save_file(file_path)
fresh = await client.wait_for_diagnostics(
file_path, version, mode=self._wait_mode, timeout=self._wait_timeout
)
await client.wait_for_diagnostics(file_path, version, mode=self._wait_mode)
except Exception as e: # noqa: BLE001
logger.debug("open/wait failed for %s: %s", file_path, e)
return None
return []
self._last_used[(client.server_id, client.workspace_root)] = time.time()
if not fresh:
return None
return list(client.diagnostics_for(file_path, fresh_only=True))
return list(client.diagnostics_for(file_path))
async def _current_diags_async(self, file_path: str) -> List[Dict[str, Any]]:
ws, gated = resolve_workspace_for_file(file_path)
@@ -513,7 +482,7 @@ class LSPService:
client = self._clients.get((srv.server_id, ws))
if client is None:
return []
return list(client.diagnostics_for(file_path, fresh_only=True))
return list(client.diagnostics_for(file_path))
async def _get_or_spawn(self, file_path: str) -> Optional[LSPClient]:
srv = find_server_for_file(file_path)
+11 -52
View File
@@ -4,86 +4,45 @@ from __future__ import annotations
from typing import Any, Sequence
from agent.redact import redact_sensitive_text
def summarize_manual_compression(
before_messages: Sequence[dict[str, Any]],
after_messages: Sequence[dict[str, Any]],
before_tokens: int,
after_tokens: int,
*,
compression_state: Any = None,
) -> dict[str, Any]:
"""Return consistent user-facing feedback for manual compression."""
before_count = len(before_messages)
after_count = len(after_messages)
noop = list(after_messages) == list(before_messages)
aborted = (
compression_state is not None
and getattr(compression_state, "_last_compress_aborted", False) is True
)
fallback_used = (
compression_state is not None
and getattr(compression_state, "_last_summary_fallback_used", False) is True
)
failure_reason = (
getattr(compression_state, "_last_summary_error", None)
if compression_state is not None
else None
)
if not isinstance(failure_reason, str) or not failure_reason.strip():
failure_reason = None
if aborted:
headline = f"Compression aborted: {before_count} messages preserved"
elif fallback_used:
headline = (
f"Compressed with fallback: {before_count} → {after_count} messages"
)
elif noop:
if noop:
headline = f"No changes from compression: {before_count} messages"
if after_tokens == before_tokens:
token_line = (
f"Approx request size: ~{before_tokens:,} tokens (unchanged)"
)
else:
token_line = (
f"Approx request size: ~{before_tokens:,} → "
f"~{after_tokens:,} tokens"
)
else:
headline = f"Compressed: {before_count} → {after_count} messages"
if noop and after_tokens == before_tokens:
token_line = f"Approx request size: ~{before_tokens:,} tokens (unchanged)"
else:
token_line = (
f"Approx request size: ~{before_tokens:,} → "
f"~{after_tokens:,} tokens"
)
note = None
if aborted:
note = "Summary generation failed; no messages were removed."
elif fallback_used:
dropped_count = getattr(
compression_state, "_last_summary_dropped_count", None
)
if not isinstance(dropped_count, int) or isinstance(dropped_count, bool):
dropped_count = max(before_count - after_count, 0)
note = (
"Summary generation failed; Hermes used limited fallback context "
f"and removed {dropped_count} message(s)."
)
elif not noop and after_count < before_count and after_tokens > before_tokens:
if not noop and after_count < before_count and after_tokens > before_tokens:
note = (
"Note: fewer messages can still raise this estimate when "
"compression rewrites the transcript into denser summaries."
)
if failure_reason and (aborted or fallback_used):
# This text crosses a user-facing UI boundary. Never let a disabled
# global redaction preference expose credentials embedded in provider
# exception text.
safe_reason = redact_sensitive_text(failure_reason.strip(), force=True)
note = f"{note} Reason: {safe_reason}"
return {
"noop": noop,
"aborted": aborted,
"fallback_used": fallback_used,
"headline": headline,
"token_line": token_line,
"note": note,
+59 -155
View File
@@ -30,7 +30,7 @@ import logging
import re
import inspect
import threading
from concurrent.futures import Future, ThreadPoolExecutor, wait
from concurrent.futures import ThreadPoolExecutor
from typing import Any, Callable, Dict, List, Optional
from agent.memory_provider import MemoryProvider
@@ -44,7 +44,6 @@ logger = logging.getLogger(__name__)
# teardown indefinitely — the worker threads are daemon, so anything still
# running past this window dies with the interpreter.
_SYNC_DRAIN_TIMEOUT_S = 5.0
_EXTERNAL_PREFETCH_TIMEOUT_S = 8.0
def normalize_tool_schema(schema: Any) -> Optional[Dict[str, Any]]:
@@ -358,19 +357,10 @@ class MemoryManager:
provider is allowed. Failures in one provider never block the other.
"""
def __init__(self, *, external_prefetch_timeout: Optional[float] = None) -> None:
def __init__(self) -> None:
self._providers: List[MemoryProvider] = []
self._tool_to_provider: Dict[str, MemoryProvider] = {}
self._has_external: bool = False # True once a non-builtin provider is added
self._external_prefetch_timeout = (
_EXTERNAL_PREFETCH_TIMEOUT_S
if external_prefetch_timeout is None
else float(external_prefetch_timeout)
)
if self._external_prefetch_timeout <= 0:
raise ValueError("external_prefetch_timeout must be positive")
self._external_prefetch_threads: Dict[str, threading.Thread] = {}
self._external_prefetch_lock = threading.Lock()
# Background executor for end-of-turn sync/prefetch. Lazily created on
# first use so the common builtin-only path spawns no extra threads.
# A single worker serializes a provider's writes (turn N must land
@@ -378,16 +368,6 @@ class MemoryManager:
# _submit_background() and the sync_all/queue_prefetch_all rationale.
self._sync_executor: Optional[ThreadPoolExecutor] = None
self._sync_executor_lock = threading.Lock()
# Futures are tracked by durability class so shutdown can give writes
# a bounded FIFO drain, then explicitly report anything abandoned.
self._background_futures: Dict[Future, str] = {}
self._shutting_down = False
self._shutdown_drain_state: Dict[str, Any] = {
"status": "not_started",
"abandoned_writes": 0,
"abandoned_prefetches": 0,
"active_tasks": 0,
}
# -- Registration --------------------------------------------------------
@@ -524,7 +504,7 @@ class MemoryManager:
parts = []
for provider in self._providers:
try:
result = self._prefetch_provider(provider, clean_query, session_id=session_id)
result = provider.prefetch(clean_query, session_id=session_id)
if result and result.strip():
parts.append(result)
except Exception as e:
@@ -534,56 +514,6 @@ class MemoryManager:
)
return "\n\n".join(parts)
def _prefetch_provider(
self, provider: MemoryProvider, query: str, *, session_id: str = ""
) -> str:
if provider.name == "builtin":
return provider.prefetch(query, session_id=session_id)
result_box: Dict[str, str] = {}
error_box: Dict[str, Exception] = {}
def _run() -> None:
try:
result_box["value"] = provider.prefetch(query, session_id=session_id) or ""
except Exception as exc: # pragma: no cover - re-raised by caller
error_box["value"] = exc
thread = threading.Thread(
target=_run,
daemon=True,
name=f"memory-prefetch-{provider.name}",
)
with self._external_prefetch_lock:
existing = self._external_prefetch_threads.get(provider.name)
if existing is not None:
if existing.is_alive():
logger.debug(
"Memory provider '%s' prefetch is still running; skipping this turn",
provider.name,
)
return ""
self._external_prefetch_threads.pop(provider.name, None)
self._external_prefetch_threads[provider.name] = thread
thread.start()
thread.join(self._external_prefetch_timeout)
if thread.is_alive():
logger.warning(
"Memory provider '%s' prefetch timed out after %.1fs; skipping it until "
"the stuck call returns",
provider.name,
self._external_prefetch_timeout,
)
return ""
with self._external_prefetch_lock:
if self._external_prefetch_threads.get(provider.name) is thread:
self._external_prefetch_threads.pop(provider.name, None)
if error_box:
raise error_box["value"]
return result_box.get("value", "")
def queue_prefetch_all(self, query: str, *, session_id: str = "") -> None:
"""Queue background prefetch on all providers for the next turn.
@@ -609,7 +539,7 @@ class MemoryManager:
provider.name, e,
)
self._submit_background(_run, kind="prefetch")
self._submit_background(_run)
# -- Sync ----------------------------------------------------------------
@@ -685,57 +615,46 @@ class MemoryManager:
# -- Background dispatch -------------------------------------------------
def _submit_background(self, fn, *, kind: str = "write") -> None:
"""Queue ``fn`` on the serialized worker and track its durability class."""
def _submit_background(self, fn) -> None:
"""Run ``fn`` on the manager's background worker.
The executor is created lazily and shared across calls. If the
executor can't be created or has already been shut down, ``fn``
runs inline as a last-resort fallback — losing the async benefit
but never losing the write itself. ``fn`` must do its own
per-provider error handling; this wrapper only guards executor
plumbing.
"""
executor = self._get_sync_executor()
if executor is None:
if self._shutting_down:
logger.warning("Memory manager is shutting down; rejecting late %s task", kind)
return
# Creation failure outside shutdown: preserve the historical
# fail-safe behavior and run the operation inline.
# Executor unavailable (shut down / creation failed) — run
# inline rather than drop the work. Slow, but correct.
try:
fn()
except Exception as e: # pragma: no cover - fn guards internally
logger.debug("Inline memory background task failed: %s", e)
return
try:
# Make submit+tracking atomic with the shutdown snapshot. The
# callback is attached after releasing the lock because an already
# completed future invokes callbacks synchronously.
with self._sync_executor_lock:
if self._shutting_down:
logger.warning("Memory manager is shutting down; rejecting late %s task", kind)
return
future = executor.submit(fn)
self._background_futures[future] = kind
future.add_done_callback(self._forget_background_future)
executor.submit(fn)
except RuntimeError:
if self._shutting_down:
logger.warning("Memory manager shut down during %s submission; task rejected", kind)
return
# Executor was shut down between the get and the submit
# (teardown race). Fall back to inline.
try:
fn()
except Exception as e: # pragma: no cover - fn guards internally
logger.debug("Inline memory background task failed: %s", e)
def _forget_background_future(self, future: Future) -> None:
with self._sync_executor_lock:
self._background_futures.pop(future, None)
def _get_sync_executor(self) -> Optional[ThreadPoolExecutor]:
"""Lazily create the single-worker background executor."""
if self._shutting_down:
return None
if self._sync_executor is not None:
return self._sync_executor
with self._sync_executor_lock:
if self._shutting_down:
return None
if self._sync_executor is None:
try:
# Daemon workers (see tools.daemon_pool): a provider wedged
# on a network call must never block interpreter exit.
# on a network call must never block interpreter exit —
# stdlib ThreadPoolExecutor's atexit hook would join it
# unconditionally even after shutdown(wait=False).
from tools.daemon_pool import DaemonThreadPoolExecutor
self._sync_executor = DaemonThreadPoolExecutor(
max_workers=1,
@@ -1150,66 +1069,51 @@ class MemoryManager:
provider.name, e,
)
@property
def shutdown_drain_state(self) -> Dict[str, Any]:
"""Snapshot of the most recent bounded shutdown drain outcome."""
with self._sync_executor_lock:
return dict(self._shutdown_drain_state)
def _drain_sync_executor(self) -> None:
"""Give queued FIFO work a bounded chance, then abandon explicitly."""
"""Shut down the background executor, waiting briefly for drain.
Bounded by ``_SYNC_DRAIN_TIMEOUT_S``: a wedged provider must never
hang process/session teardown. We stop accepting new work and
cancel anything still queued, then wait at most the drain timeout
for the currently-running task on a watcher thread. The worker is
daemon, so an over-running task dies with the interpreter.
"""
with self._sync_executor_lock:
self._shutting_down = True
executor = self._sync_executor
self._sync_executor = None
tracked = dict(self._background_futures)
self._shutdown_drain_state = {
"status": "draining" if executor is not None else "drained",
"abandoned_writes": 0,
"abandoned_prefetches": 0,
"active_tasks": sum(not future.done() for future in tracked),
}
if executor is None:
return
# shutdown(wait=False) closes submission without touching the FIFO.
# Waiting on the tracked futures lets the real single-worker executor
# run every queued write/boundary task in order up to the deadline.
executor.shutdown(wait=False, cancel_futures=False)
_, pending = wait(tuple(tracked), timeout=_SYNC_DRAIN_TIMEOUT_S)
if not pending:
with self._sync_executor_lock:
self._shutdown_drain_state.update(status="drained", active_tasks=0)
try:
# Stop accepting new work and drop anything still queued, but
# do NOT block here — cancel_futures cancels not-yet-started
# tasks; the in-flight one keeps running on its daemon thread.
executor.shutdown(wait=False, cancel_futures=True)
except TypeError:
# Older Python without cancel_futures kwarg.
try:
executor.shutdown(wait=False)
except Exception as e: # pragma: no cover
logger.debug("Memory sync executor shutdown failed: %s", e)
return
abandoned_writes = 0
abandoned_prefetches = 0
active_tasks = 0
for future in pending:
kind = tracked[future]
if future.cancel():
if kind == "prefetch":
abandoned_prefetches += 1
else:
abandoned_writes += 1
else:
active_tasks += 1
with self._sync_executor_lock:
self._shutdown_drain_state.update(
status="timed_out",
abandoned_writes=abandoned_writes,
abandoned_prefetches=abandoned_prefetches,
active_tasks=active_tasks,
)
logger.warning(
"Memory shutdown drain timed out after %.2fs; abandoning %d queued "
"memory write(s) and %d queued prefetch(es); %d active task(s) remain detached",
_SYNC_DRAIN_TIMEOUT_S,
abandoned_writes,
abandoned_prefetches,
active_tasks,
except Exception as e: # pragma: no cover
logger.debug("Memory sync executor shutdown failed: %s", e)
return
# Give an in-flight sync a bounded chance to finish on a watcher
# thread so we don't block the caller past the drain timeout.
drainer = threading.Thread(
target=lambda: self._bounded_executor_wait(executor),
daemon=True,
name="mem-sync-drain",
)
drainer.start()
drainer.join(timeout=_SYNC_DRAIN_TIMEOUT_S)
@staticmethod
def _bounded_executor_wait(executor: ThreadPoolExecutor) -> None:
try:
executor.shutdown(wait=True)
except Exception as e: # pragma: no cover
logger.debug("Memory sync executor drain wait failed: %s", e)
def initialize_all(self, session_id: str, **kwargs) -> None:
"""Initialize all providers.
+76 -239
View File
@@ -14,7 +14,6 @@ from concurrent.futures import ThreadPoolExecutor
from typing import Any
from agent.auxiliary_client import call_llm
from agent.message_content import flatten_message_text
from agent.transports import get_transport
logger = logging.getLogger(__name__)
@@ -120,54 +119,11 @@ _REFERENCE_SYSTEM_PROMPT = (
def _slot_label(slot: dict[str, Any]) -> str:
label = f"{(slot.get('provider') or '').strip()}:{(slot.get('model') or '').strip()}"
effort = str(slot.get("reasoning_effort") or "").strip()
return f"{label}[reasoning={effort}]" if effort else label
def _slot_label(slot: dict[str, str]) -> str:
return f"{(slot.get('provider') or '').strip()}:{(slot.get('model') or '').strip()}"
def _slot_reasoning_config(slot: dict[str, Any]) -> dict[str, Any] | None:
"""Translate optional per-MoA-slot reasoning_effort into runtime config."""
effort = slot.get("reasoning_effort")
try:
from hermes_constants import parse_reasoning_effort
return parse_reasoning_effort(effort)
except Exception: # pragma: no cover - defensive; bad config must not break MoA
return None
def _aggregator_reasoning_config(aggregator: dict[str, Any]) -> dict[str, Any] | None:
"""Resolve the aggregator's reasoning config: slot > per-model > global.
The aggregator is MoA's ACTING model, so when its slot doesn't pin a
reasoning_effort it must resolve exactly like any other acting model:
through the shared chokepoint (``resolve_reasoning_config``), which
applies ``agent.reasoning_overrides`` for the slot's model first, then
the global ``agent.reasoning_effort``. Without this the main loop's
reasoning gates (keyed to the virtual ``moa://local`` identity) never
fire, so the aggregator silently ran at the backend default (#64187).
Reference advisors intentionally do NOT get this fallback: they are side
calls (like auxiliary tasks), and inheriting a global ``xhigh`` into every
advisor fan-out would silently multiply cost. Their depth is slot-or-
provider-default only.
"""
cfg = _slot_reasoning_config(aggregator)
if cfg is not None:
return cfg
try:
from hermes_cli.config import load_config
from hermes_constants import resolve_reasoning_config
return resolve_reasoning_config(
load_config() or {}, str(aggregator.get("model") or "")
)
except Exception: # pragma: no cover - defensive; bad config must not break MoA
return None
def _slot_runtime(slot: dict[str, Any]) -> dict[str, Any]:
def _slot_runtime(slot: dict[str, str]) -> dict[str, Any]:
"""Resolve a reference/aggregator slot to real runtime call kwargs.
A MoA slot is just a model selection — it must be called the same way any
@@ -319,7 +275,6 @@ def _run_reference(
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
reasoning_config=_slot_reasoning_config(slot),
**runtime,
)
usage = CanonicalUsage()
@@ -404,12 +359,6 @@ def _run_references_parallel(
results: list[tuple[str, str, Any] | None] = [None] * len(reference_models)
futures = {}
workers = min(_MAX_REFERENCE_WORKERS, len(reference_models))
# Reference slots run on bare executor threads, which start with an empty
# contextvars.Context — propagate the parent turn's context (approval
# callbacks + the Nous Portal conversation tag) into each worker so
# advisor calls attribute to the same conversation as the acting turn.
from tools.thread_context import propagate_context_to_thread
with ThreadPoolExecutor(max_workers=workers) as executor:
for idx, slot in enumerate(reference_models):
if slot.get("provider") == "moa":
@@ -421,7 +370,7 @@ def _run_references_parallel(
continue
futures[
executor.submit(
propagate_context_to_thread(_run_reference),
_run_reference,
slot,
ref_messages,
temperature=temperature,
@@ -521,52 +470,13 @@ def _reference_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
for msg in messages:
role = msg.get("role")
content = msg.get("content")
# Flatten structured content (lists of parts) to visible text. Content
# arrives as a list — not a string — in two common cases:
# 1. Anthropic prompt-cache decoration: conversation_loop runs
# apply_anthropic_cache_control BEFORE the MoA facade, converting
# string content to [{"type": "text", "text": ..., "cache_control":
# ...}]. A str-only read here flattened the user's ENTIRE prompt to
# "" — Claude references then 400'd ("messages: at least one
# message is required") while tolerant models answered "no user
# request is present".
# 2. Multimodal turns (pasted image → text + image_url parts) and
# multimodal tool results (screenshots).
# flatten_message_text extracts the text parts and skips image parts,
# and returns strings unchanged — so a decorated and an undecorated
# transcript produce a byte-identical advisory view (which keeps the
# advisory prefix stable across iterations for advisor prompt caching).
text = flatten_message_text(content)
text = content if isinstance(content, str) else ""
if role == "system":
continue
if role == "user":
if not text.strip() and isinstance(content, list) and content:
# Structured content with no extractable text (e.g. an
# image-only turn). Emitting an empty user message would be
# dropped/rejected by strict providers (Anthropic 400s on
# empty text blocks — the original "closed" preset failure
# mode), and silently skipping the turn would break
# user/assistant alternation in the advisory view. Substitute
# a placeholder so the reference knows a non-text turn
# happened. Only structured content qualifies — an empty or
# whitespace-only STRING turn carries nothing and is dropped
# below instead.
text = "[user sent non-text content (e.g. an image attachment)]"
if not text.strip():
# Genuinely empty user turn (content="" / None). It carries
# nothing advisory, and strict providers (Kimi/Moonshot, ZAI,
# and others that enforce non-empty user content) reject it
# with 400 "message ... with role 'user' must not be empty" —
# the same way the assistant branch below drops turns with no
# parts. Lenient providers (DeepSeek) accept the empty turn,
# which is why a MoA fan-out would fail on one reference and
# pass on another for the identical rendered view. The
# advisory view is already not strictly alternating (adjacent
# assistant turns occur in every tool loop), so dropping a
# contentless turn is safe.
continue
last_user_content = text
if text.strip():
last_user_content = text
rendered.append({"role": "user", "content": text})
elif role == "assistant":
parts: list[str] = []
@@ -607,10 +517,8 @@ def _reference_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
if last_user_content is not None:
return [{"role": "user", "content": last_user_content}]
for msg in reversed(messages):
if msg.get("role") == "user":
fallback_text = flatten_message_text(msg.get("content"))
if fallback_text.strip():
return [{"role": "user", "content": fallback_text}]
if msg.get("role") == "user" and isinstance(msg.get("content"), str):
return [{"role": "user", "content": msg["content"]}]
return rendered
@@ -730,7 +638,6 @@ def aggregate_moa_context(
messages=agg_messages,
temperature=aggregator_temperature,
max_tokens=max_tokens,
reasoning_config=_aggregator_reasoning_config(aggregator),
**agg_runtime,
)
synthesis = _extract_text(response)
@@ -764,28 +671,13 @@ def _attach_reference_guidance(agg_messages: list[dict[str, Any]], guidance: str
Appending at the very end keeps the ``[system][task][tool-history]`` prefix
stable and cache-reusable (only the new block re-prefills), and gives the
aggregator the references with recency. Merge into the last message only when
it is already a trailing ``user`` turn (plain chat — still at the end).
A trailing user turn's content may be a STRING or a LIST of content parts —
Anthropic prompt-cache decoration (which runs before the MoA facade)
converts string content to ``[{"type": "text", ..., "cache_control": ...}]``,
and multimodal turns are lists natively. Both shapes are merged in place:
appending a new text part AFTER the cache_control-marked part keeps the
cached prefix byte-stable (the marker still terminates it) while the
turn-varying guidance rides outside the cached span. Appending a SEPARATE
user message here instead would produce two consecutive user turns —
strict providers reject that.
it is already a trailing string ``user`` turn (plain chat — still at the end).
"""
last = agg_messages[-1] if agg_messages else None
if last is not None and last.get("role") == "user":
last_content = last.get("content")
if isinstance(last_content, str):
last["content"] = last_content + "\n\n" + guidance
return
if isinstance(last_content, list):
last["content"] = [*last_content, {"type": "text", "text": "\n\n" + guidance}]
return
agg_messages.append({"role": "user", "content": guidance})
if last is not None and last.get("role") == "user" and isinstance(last.get("content"), str):
last["content"] = last["content"] + "\n\n" + guidance
else:
agg_messages.append({"role": "user", "content": guidance})
class MoAChatCompletions:
@@ -905,114 +797,7 @@ class MoAChatCompletions:
except Exception as exc: # pragma: no cover - display must never break the turn
logger.debug("MoA reference_callback failed for %s: %s", event, exc)
def prepare(self, messages: list[dict[str, Any]]) -> dict[str, Any]:
"""Run the advisor fan-out and return the exact aggregator request.
The normal agent loop needs to measure this augmented prompt before its
compression gate. ``create()`` also uses this method for direct callers;
when the loop supplies the returned private object back to ``create()``,
the advisor fan-out is not repeated.
"""
return self.create(messages=messages, _moa_prepare_only=True)
def rebase_prepared_request(
self, prepared: dict[str, Any], messages: list[dict[str, Any]]
) -> dict[str, Any]:
"""Apply already-generated advisor guidance to a rebuilt API transcript.
Context compression changes the persisted transcript but not the
ephemeral advisor result. Reusing the guidance avoids a second costly
fan-out while keeping the aggregator request aligned with the compacted
history.
"""
guidance = prepared.get("guidance")
agg_messages = [dict(message) for message in messages]
if guidance:
_attach_reference_guidance(agg_messages, str(guidance))
return {**prepared, "messages": agg_messages}
def _call_prepared_aggregator(
self, prepared: dict[str, Any], api_kwargs: dict[str, Any]
) -> Any:
"""Send an already prepared MoA aggregator request exactly once."""
agg_messages = prepared["messages"]
aggregator = prepared["aggregator"]
aggregator_temperature = prepared["aggregator_temperature"]
if aggregator.get("provider") == "moa":
raise RuntimeError("MoA aggregator cannot be another MoA preset")
agg_kwargs = dict(api_kwargs)
max_tokens: Any = agg_kwargs.get("max_tokens")
tools: Any = agg_kwargs.get("tools")
extra_body: Any = agg_kwargs.get("extra_body")
# Record the exact aggregator INPUT (incl. the injected reference
# context) into the pending trace so a trace captures what the
# aggregator actually saw, not a reconstruction.
if self._pending_trace is not None:
self._pending_trace["aggregator_input_messages"] = agg_messages
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
# The aggregator is the acting model. Resolve its slot to the provider's
# real runtime (base_url/api_key/api_mode) and call it through the same
# request-building path any model uses — so per-model wire-format
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
# temperature) applies identically to it. MoA imposes no output cap:
# max_tokens is passed through from the caller (normally None → omitted
# → the model's real maximum). The preset's old hardcoded 4096 default
# is gone — it truncated long syntheses.
# When the agent's streaming consumer calls us with stream=True, run the
# references first (above) and then return the aggregator's RAW token
# stream so the acting model's output reaches the user live. The consumer
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
# back to a non-streaming retry on error. The non-streaming path
# (stream=False) is unchanged — no stream/stream_options/timeout are
# forwarded, so its behavior is byte-for-byte identical to before.
stream = bool(api_kwargs.get("stream"))
stream_kwargs: dict[str, Any] = {}
if stream:
stream_kwargs["stream"] = True
stream_kwargs["stream_options"] = (
api_kwargs.get("stream_options") or {"include_usage": True}
)
# Forward the consumer's per-request (stream read) timeout so it
# actually governs the aggregator stream, not just call_llm's default.
if api_kwargs.get("timeout") is not None:
stream_kwargs["timeout"] = api_kwargs["timeout"]
_agg_response = call_llm(
task="moa_aggregator",
messages=agg_messages,
temperature=aggregator_temperature,
max_tokens=max_tokens,
tools=tools,
extra_body=extra_body,
# Prepared requests must retain the acting aggregator's reasoning
# policy exactly as the direct create() path does (#64187).
reasoning_config=_aggregator_reasoning_config(aggregator),
**stream_kwargs,
**_slot_runtime(aggregator),
)
# Non-streaming path (quiet mode / eval / subagents): the aggregator
# output is available inline, so capture it into the pending trace now.
# Streaming path: the aggregator's raw token stream is returned to the
# consumer live and its acting output lands as the turn's assistant
# message; the trace marks it streamed and points there.
if self._pending_trace is not None:
if stream:
self._pending_trace["aggregator_streamed"] = True
self._pending_trace["aggregator_output"] = None
else:
self._pending_trace["aggregator_streamed"] = False
try:
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
except Exception: # pragma: no cover - defensive
self._pending_trace["aggregator_output"] = None
return _agg_response
def create(self, **api_kwargs: Any) -> Any:
prepared_request = api_kwargs.pop("_moa_prepared_request", None)
if prepared_request is not None:
if not isinstance(prepared_request, dict):
raise TypeError("_moa_prepared_request must be a dict")
return self._call_prepared_aggregator(prepared_request, api_kwargs)
from hermes_cli.config import load_config
from hermes_cli.moa_config import resolve_moa_preset
@@ -1172,7 +957,6 @@ class MoAChatCompletions:
ref_count=_ref_count,
)
guidance: str | None = None
agg_messages = [dict(m) for m in messages]
if reference_outputs:
joined = "\n\n".join(
@@ -1190,15 +974,68 @@ class MoAChatCompletions:
)
_attach_reference_guidance(agg_messages, guidance)
prepared_request = {
"messages": agg_messages,
"guidance": guidance,
"aggregator": aggregator,
"aggregator_temperature": aggregator_temperature,
}
if api_kwargs.pop("_moa_prepare_only", False):
return prepared_request
return self._call_prepared_aggregator(prepared_request, api_kwargs)
if aggregator.get("provider") == "moa":
raise RuntimeError("MoA aggregator cannot be another MoA preset")
agg_kwargs = dict(api_kwargs)
agg_kwargs["messages"] = agg_messages
# Record the exact aggregator INPUT (incl. the injected reference
# context) into the pending trace so a trace captures what the
# aggregator actually saw, not a reconstruction.
if self._pending_trace is not None:
self._pending_trace["aggregator_input_messages"] = agg_messages
self._pending_trace["aggregator_label"] = _slot_label(aggregator)
# The aggregator is the acting model. Resolve its slot to the provider's
# real runtime (base_url/api_key/api_mode) and call it through the same
# request-building path any model uses — so per-model wire-format
# handling (anthropic_messages, max_completion_tokens, fixed/forbidden
# temperature) applies identically to it. MoA imposes no output cap:
# max_tokens is passed through from the caller (normally None → omitted
# → the model's real maximum). The preset's old hardcoded 4096 default
# is gone — it truncated long syntheses.
# When the agent's streaming consumer calls us with stream=True, run the
# references first (above) and then return the aggregator's RAW token
# stream so the acting model's output reaches the user live. The consumer
# reassembles chunks + tool_calls, runs stale-stream detection, and falls
# back to a non-streaming retry on error. The non-streaming path
# (stream=False) is unchanged — no stream/stream_options/timeout are
# forwarded, so its behavior is byte-for-byte identical to before.
stream = bool(api_kwargs.get("stream"))
stream_kwargs: dict[str, Any] = {}
if stream:
stream_kwargs["stream"] = True
stream_kwargs["stream_options"] = (
api_kwargs.get("stream_options") or {"include_usage": True}
)
# Forward the consumer's per-request (stream read) timeout so it
# actually governs the aggregator stream, not just call_llm's default.
if api_kwargs.get("timeout") is not None:
stream_kwargs["timeout"] = api_kwargs["timeout"]
_agg_response = call_llm(
task="moa_aggregator",
messages=agg_messages,
temperature=aggregator_temperature,
max_tokens=agg_kwargs.get("max_tokens"),
tools=agg_kwargs.get("tools"),
extra_body=agg_kwargs.get("extra_body"),
**stream_kwargs,
**_slot_runtime(aggregator),
)
# Non-streaming path (quiet mode / eval / subagents): the aggregator
# output is available inline, so capture it into the pending trace now.
# Streaming path: the aggregator's raw token stream is returned to the
# consumer live and its acting output lands as the turn's assistant
# message; the trace marks it streamed and points there.
if self._pending_trace is not None:
if stream:
self._pending_trace["aggregator_streamed"] = True
self._pending_trace["aggregator_output"] = None
else:
self._pending_trace["aggregator_streamed"] = False
try:
self._pending_trace["aggregator_output"] = _extract_text(_agg_response)
except Exception: # pragma: no cover - defensive
self._pending_trace["aggregator_output"] = None
return _agg_response
class MoAClient:
+66 -351
View File
@@ -4,8 +4,6 @@ Pure utility functions with no AIAgent dependency. Used by ContextCompressor
and run_agent.py for pre-flight context checks.
"""
import base64
import hashlib
import ipaddress
import json
import logging
@@ -49,7 +47,7 @@ def _resolve_requests_verify() -> bool | str:
# are preserved so the full model name reaches cache lookups and server queries.
_PROVIDER_PREFIXES: frozenset[str] = frozenset({
"openrouter", "nous", "openai-codex", "copilot", "copilot-acp",
"gemini", "ollama-cloud", "zai", "kimi-coding", "kimi-coding-cn", "stepfun", "minimax", "minimax-oauth", "minimax-cn", "anthropic", "deepseek", "deepinfra",
"gemini", "ollama-cloud", "zai", "kimi-coding", "kimi-coding-cn", "stepfun", "minimax", "minimax-oauth", "minimax-cn", "anthropic", "deepseek",
"opencode-zen", "opencode-go", "kilocode", "alibaba", "novita",
"qwen-oauth",
"xiaomi",
@@ -60,7 +58,7 @@ _PROVIDER_PREFIXES: frozenset[str] = frozenset({
# Common aliases
"google", "google-gemini", "google-ai-studio",
"glm", "z-ai", "z.ai", "zhipu", "github", "github-copilot",
"github-models", "kimi", "moonshot", "kimi-cn", "moonshot-cn", "claude", "deep-seek", "deep-infra",
"github-models", "kimi", "moonshot", "kimi-cn", "moonshot-cn", "claude", "deep-seek",
"ollama",
"stepfun", "opencode", "zen", "go", "kilo", "dashscope", "aliyun", "qwen",
"mimo", "xiaomi-mimo",
@@ -215,7 +213,6 @@ DEFAULT_CONTEXT_LENGTHS = {
# OpenRouter-prefixed models resolve via OpenRouter live API or models.dev.
"claude-fable-5": 1000000,
"claude-fable": 1000000,
"claude-sonnet-5": 1000000,
"claude-opus-4-8": 1000000,
"claude-opus-4.8": 1000000,
"claude-opus-4-7": 1000000,
@@ -278,10 +275,8 @@ DEFAULT_CONTEXT_LENGTHS = {
# Qwen — specific model families before the catch-all.
# Official docs: https://help.aliyun.com/zh/model-studio/developer-reference/
"qwen3.6-plus": 1048576, # 1M context (DashScope/Alibaba & OpenRouter)
"qwen3.7-plus": 1048576, # 1M context (DashScope/Alibaba)
"qwen3-coder-plus": 1000000, # 1M context
"qwen3-coder": 262144, # 256K context
"qwen3-max": 262144, # 256K context (qwen3-max-2026-01-23 snapshot, Coding Plan)
"qwen": 131072,
# MiniMax — M3 is 1M context (max output 512K); M2.x series is 204,800.
# Keys use substring matching (longest-first), so "minimax-m3" wins over
@@ -321,23 +316,8 @@ DEFAULT_CONTEXT_LENGTHS = {
"grok-3": 131072, # grok-3, grok-3-mini, grok-3-fast, grok-3-mini-fast
"grok-2": 131072, # grok-2, grok-2-1212, grok-2-latest
"grok": 131072, # catch-all (grok-beta, unknown grok-*)
# Kimi — K3 ships with a 1 Mi context window (1,048,576; verified against
# models.dev and OpenRouter live metadata, matching the endpoint-scoped
# override in _endpoint_scoped_context_length). Longest-key-first substring
# matching ensures "kimi-k3" resolves to 1M while older/unknown Kimi models
# still hit the generic 256K fallback.
"kimi-k3": 1_048_576,
# Kimi
"kimi": 262144,
# Upstage Solar — api.upstage.ai/v1/models does not return context_length,
# so these fallbacks keep token budgeting / compression from probing down
# to the 128k default. Ids are matched longest-first, so dated variants
# (e.g. solar-pro3-250127) resolve via their family prefix.
# Sources: Solar Pro 3 = 128K, Solar Pro 2 = 64K, Solar Mini = 32K,
# Solar Open 2 = 256K.
"solar-open2": 262144, # 256K
"solar-pro3": 131072,
"solar-pro2": 65536,
"solar-mini": 32768,
# Tencent — Hy3 Preview (Hunyuan) with 256K context window.
# OpenRouter live metadata reports 262144 (256 × 1024); align the
# static fallback so cache and offline both agree (issue #22268).
@@ -549,47 +529,13 @@ def _is_known_provider_base_url(base_url: str) -> bool:
return _infer_provider_from_url(base_url) is not None
def _endpoint_scoped_context_length(model: str, base_url: str) -> Optional[int]:
"""Return metadata confirmed only for the Kimi Coding endpoint.
Kimi Coding serves K3 under the bare slug ``k3``, but users may also
configure or select the public-facing aliases ``kimi-k3`` and
``kimi-k3-cot``. Only canonical ``https://api.kimi.com/coding`` endpoints
(legacy Moonshot keys do not serve K3) get the 1 Mi context window.
"""
normalized = _normalize_base_url(base_url)
try:
parsed = urlparse(normalized)
port = parsed.port
except ValueError:
return None
if (
parsed.scheme.lower() == "https"
and (parsed.hostname or "").lower() == "api.kimi.com"
and port in (None, 443)
and parsed.username is None
and parsed.password is None
and parsed.path.rstrip("/") in {"/coding", "/coding/v1"}
and not parsed.query
and not parsed.fragment
and model.strip().lower() in {"k3", "kimi-k3", "kimi-k3-cot"}
):
return 1_048_576
return None
def _skip_persistent_context_cache(base_url: str, provider: str) -> bool:
"""Return True when the on-disk context cache must not short-circuit probing.
LM Studio excludes caching because loaded context is transient — the user
can reload the model with a different context_length at any time.
Codex OAuth excludes caching because its context window is account- and
entitlement-specific metadata supplied by the authenticated /models
endpoint. A fallback value written after a transient probe failure must
not prevent a later live probe from observing an updated allocation.
"""
return (provider or "").strip().lower() in {"lmstudio", "openai-codex"}
"""
return provider == "lmstudio"
def _maybe_cache_local_context_length(
@@ -861,24 +807,6 @@ def _extract_pricing(payload: Dict[str, Any]) -> Dict[str, Any]:
pricing["completion"] = str(float(novita_output) / 10_000 / 1_000_000)
return pricing
# DeepInfra ships pricing under ``metadata.pricing`` with $/MTok values:
# ``input_tokens``, ``output_tokens``, ``cache_read_tokens``. Convert to
# per-token strings so the generic cost machinery (usage_pricing.py)
# consumes them through the same path as OpenRouter / OpenAI.
metadata = payload.get("metadata") if isinstance(payload.get("metadata"), dict) else None
deepinfra_pricing = metadata.get("pricing") if metadata else None
if isinstance(deepinfra_pricing, dict) and any(
k in deepinfra_pricing for k in ("input_tokens", "output_tokens", "cache_read_tokens")
):
result: Dict[str, Any] = {}
if deepinfra_pricing.get("input_tokens") is not None:
result["prompt"] = str(float(deepinfra_pricing["input_tokens"]) / 1_000_000)
if deepinfra_pricing.get("output_tokens") is not None:
result["completion"] = str(float(deepinfra_pricing["output_tokens"]) / 1_000_000)
if deepinfra_pricing.get("cache_read_tokens") is not None:
result["cache_read"] = str(float(deepinfra_pricing["cache_read_tokens"]) / 1_000_000)
return result
alias_map = {
"prompt": ("prompt", "input", "input_cost_per_token", "prompt_token_cost"),
"completion": ("completion", "output", "output_cost_per_token", "completion_token_cost"),
@@ -1925,72 +1853,32 @@ _CODEX_OAUTH_CONTEXT_FALLBACK: Dict[str, int] = {
}
_codex_oauth_context_cache: Dict[str, Tuple[Dict[str, int], float]] = {}
_codex_oauth_context_cache: Dict[str, int] = {}
_codex_oauth_context_cache_time: float = 0.0
_CODEX_OAUTH_CONTEXT_CACHE_TTL = 3600 # 1 hour
def _codex_oauth_token_fingerprint(access_token: str) -> str:
"""Return a non-secret cache key for a Codex OAuth access token."""
return hashlib.sha256(access_token.encode("utf-8")).hexdigest()[:16]
def _fetch_codex_oauth_context_lengths(access_token: str) -> Dict[str, int]:
"""Probe the ChatGPT Codex /models endpoint for per-slug context windows.
Codex OAuth imposes its own context limits that differ from the direct
OpenAI API (e.g. gpt-5.5 is 1.05M on the API, 272K on Codex). The
`context_window` field in each model entry is the authoritative source.
def _extract_chatgpt_account_id(access_token: str) -> Optional[str]:
"""Extract ``chatgpt_account_id`` from the Codex OAuth JWT.
The Codex ``/backend-api/codex/models`` endpoint returns the per-account
catalog only when the ``ChatGPT-Account-Id`` header is present; without
it, the endpoint returns ``{"models":[]}`` (HTTP 200) and the context
probe falls back to the hardcoded defaults — which can be stale or
wrong for the active account's plan. Mirrors the same extraction done
in ``auxiliary_client.py`` for the request path.
Returns ``None`` on any parse error rather than raising, so a bad
token still surfaces as a normal probe failure instead of crashing
the metadata resolver.
Returns a ``{slug: context_window}`` dict. Empty on failure.
"""
try:
parts = access_token.split(".")
if len(parts) < 2:
return None
payload_b64 = parts[1] + "=" * (-len(parts[1]) % 4)
claims = json.loads(base64.urlsafe_b64decode(payload_b64))
if not isinstance(claims, dict):
return None
acct_id = claims.get("https://api.openai.com/auth", {}).get("chatgpt_account_id")
return acct_id if isinstance(acct_id, str) and acct_id else None
except Exception:
return None
def _fetch_codex_oauth_context_lengths_with_source(
access_token: str,
) -> Tuple[Dict[str, int], bool]:
"""Fetch Codex catalogue data and report whether it came from HTTP.
The in-process cache is scoped by token fingerprint because Codex model
availability and context windows can vary by account entitlement. The raw
token is never retained in the cache key. The boolean is false for a
same-token in-process hit, which must not be treated as a fresh provider
confirmation when deciding whether to update persistent state.
"""
global _codex_oauth_context_cache
global _codex_oauth_context_cache, _codex_oauth_context_cache_time
now = time.time()
cache_key = _codex_oauth_token_fingerprint(access_token)
cached = _codex_oauth_context_cache.get(cache_key)
if cached is not None:
cached_models, cached_at = cached
if now - cached_at < _CODEX_OAUTH_CONTEXT_CACHE_TTL:
return cached_models, False
headers = {"Authorization": f"Bearer {access_token}"}
acct_id = _extract_chatgpt_account_id(access_token)
if acct_id:
headers["ChatGPT-Account-Id"] = acct_id
if (
_codex_oauth_context_cache
and now - _codex_oauth_context_cache_time < _CODEX_OAUTH_CONTEXT_CACHE_TTL
):
return _codex_oauth_context_cache
try:
resp = requests.get(
"https://chatgpt.com/backend-api/codex/models?client_version=1.0.0",
headers=headers,
headers={"Authorization": f"Bearer {access_token}"},
timeout=(5, 10),
verify=_resolve_requests_verify(),
)
@@ -1999,11 +1887,11 @@ def _fetch_codex_oauth_context_lengths_with_source(
"Codex /models probe returned HTTP %s; falling back to hardcoded defaults",
resp.status_code,
)
return {}, False
return {}
data = resp.json()
except Exception as exc:
logger.debug("Codex /models probe failed: %s", exc)
return {}, False
return {}
entries = data.get("models", []) if isinstance(data, dict) else []
result: Dict[str, int] = {}
@@ -2016,50 +1904,32 @@ def _fetch_codex_oauth_context_lengths_with_source(
result[slug.strip()] = ctx
if result:
_codex_oauth_context_cache[cache_key] = (result, now)
return result, True
def _fetch_codex_oauth_context_lengths(access_token: str) -> Dict[str, int]:
"""Probe the ChatGPT Codex /models endpoint for per-slug context windows.
Codex OAuth imposes its own context limits that differ from the direct
OpenAI API (e.g. gpt-5.5 is 1.05M on the API, 272K on Codex). The
`context_window` field in each model entry is the authoritative source.
Returns a ``{slug: context_window}`` dict. Empty on failure.
"""
result, _fresh = _fetch_codex_oauth_context_lengths_with_source(access_token)
_codex_oauth_context_cache = result
_codex_oauth_context_cache_time = now
return result
def _resolve_codex_oauth_context_length_with_source(
def _resolve_codex_oauth_context_length(
model: str, access_token: str = ""
) -> Tuple[Optional[int], str]:
) -> Optional[int]:
"""Resolve a Codex OAuth model's real context window.
Prefers a live probe of chatgpt.com/backend-api/codex/models (when we
have a bearer token), then falls back to ``_CODEX_OAUTH_CONTEXT_FALLBACK``.
Returns ``(context_length, source)`` where source is ``"live"`` for a
value returned by a fresh authenticated endpoint probe, ``"memory"`` for
a same-token in-process catalogue hit, or ``"fallback"`` for the static
conservative table. Only ``"live"`` is eligible for persistent writes.
"""
model_bare = _strip_provider_prefix(model).strip()
if not model_bare:
return None, ""
return None
if access_token:
live, fresh_probe = _fetch_codex_oauth_context_lengths_with_source(access_token)
live_source = "live" if fresh_probe else "memory"
live = _fetch_codex_oauth_context_lengths(access_token)
if model_bare in live:
return live[model_bare], live_source
return live[model_bare]
# Case-insensitive match in case casing drifts
model_lower = model_bare.lower()
for slug, ctx in live.items():
if slug.lower() == model_lower:
return ctx, live_source
return ctx
# Fallback: longest-key-first substring match over hardcoded defaults.
model_lower = model_bare.lower()
@@ -2067,19 +1937,9 @@ def _resolve_codex_oauth_context_length_with_source(
_CODEX_OAUTH_CONTEXT_FALLBACK.items(), key=lambda x: len(x[0]), reverse=True
):
if slug in model_lower:
return ctx, "fallback"
return ctx
return None, ""
def _resolve_codex_oauth_context_length(
model: str, access_token: str = ""
) -> Optional[int]:
"""Resolve a Codex OAuth model's context length (compatibility wrapper)."""
context_length, _source = _resolve_codex_oauth_context_length_with_source(
model, access_token=access_token,
)
return context_length
return None
def _resolve_nous_context_length(
@@ -2168,10 +2028,9 @@ def get_model_context_length(
Resolution order:
0. Explicit config override (model.context_length or custom_providers per-model)
0c. Endpoint-scoped metadata for models validated on one multiplexed endpoint
1. Persistent cache (previously discovered via probing). Nous URLs,
LM Studio, and Codex OAuth bypass the cache here so their provider
metadata can be reconciled against the authoritative live source.
1. Persistent cache (previously discovered via probing). Nous URLs
bypass the cache here so step 5b can always reconcile against
the authoritative portal /v1/models response.
1b. AWS Bedrock static table (must precede custom-endpoint probe)
2. Active endpoint metadata (/models for explicit custom endpoints)
3. Local server query (for local endpoints)
@@ -2238,46 +2097,33 @@ def get_model_context_length(
except Exception:
pass # fall through to probing
# Malformed user-provided URLs (for example an unmatched IPv6 bracket)
# make urllib.parse raise. Context resolution should treat those as an
# unknown endpoint rather than crashing before the inference layer can
# report the configuration error itself.
if base_url:
try:
parsed_base_url = urlparse(_normalize_base_url(base_url))
_ = parsed_base_url.port
except ValueError:
base_url = ""
# Normalise provider-prefixed model names (e.g. "local:model-name" →
# "model-name") so cache lookups and server queries use the bare ID that
# local servers actually know about. Ollama "model:tag" colons are preserved.
model = _strip_provider_prefix(model)
# Endpoint-scoped provider metadata. Keep this ahead of the persistent
# cache so a value learned for a multiplexed provider's other endpoint
# cannot override the endpoint where the model was actually validated.
endpoint_context = _endpoint_scoped_context_length(model, base_url)
if endpoint_context is not None:
return endpoint_context
is_bedrock_context = provider == "bedrock" or (
base_url
and base_url_hostname(base_url).startswith("bedrock-runtime.")
and base_url_host_matches(base_url, "amazonaws.com")
)
# 1. Check persistent cache (model+provider)
# LM Studio is excluded — its loaded context length is transient (the
# user can reload the model with a different context_length at any time
# via /api/v1/models/load), so a stale cached value would mask reloads.
# Codex OAuth is excluded because the authenticated /models catalogue is
# account-specific and a fallback must never suppress later revalidation.
if base_url and not _skip_persistent_context_cache(base_url, provider):
cached = get_cached_context_length(model, base_url)
if cached is not None:
# Invalidate stale Codex OAuth cache entries: pre-PR #14935 builds
# resolved gpt-5.x to the direct-API value (e.g. 1.05M) via
# models.dev and persisted it. Codex OAuth caps at 272K for every
# slug, so any cached Codex entry at or above 400K is a leftover
# from the old resolution path. Drop it and fall through to the
# live /models probe in step 5 below.
if provider == "openai-codex" and cached >= 400_000:
logger.info(
"Dropping stale Codex cache entry %s@%s -> %s (pre-fix value); "
"re-resolving via live /models probe",
model, base_url, f"{cached:,}",
)
_invalidate_cached_context_length(model, base_url)
# Invalidate stale 32k cache entries for Kimi-family models.
if cached <= 32768 and _model_name_suggests_kimi(model):
elif cached <= 32768 and _model_name_suggests_kimi(model):
logger.info(
"Dropping stale Kimi cache entry %s@%s -> %s (OpenRouter underreport); "
"re-resolving via hardcoded defaults",
@@ -2324,30 +2170,6 @@ def get_model_context_length(
model, base_url,
)
# Fall through; step 5b reconciles and overwrites if portal responds.
# Invalidate stale Bedrock entries seeded before the Claude 4.6+
# long-context table was corrected to 1M. The static table is a
# FLOOR, not an override: probe-derived cache entries (step 1b)
# may legitimately exceed the table (real window read from
# Bedrock's length-validation error), so only under-reporting
# entries are dropped — never a cached value above the table.
elif is_bedrock_context:
try:
from agent.bedrock_adapter import get_bedrock_context_length
bedrock_ctx = get_bedrock_context_length(model)
if cached < bedrock_ctx:
logger.info(
"Dropping stale Bedrock cache entry %s@%s -> %s; "
"using static Bedrock table value %s",
model,
base_url,
f"{cached:,}",
f"{bedrock_ctx:,}",
)
_invalidate_cached_context_length(model, base_url)
return bedrock_ctx
except ImportError:
pass
return cached
else:
if is_local_endpoint(base_url):
return _reconcile_local_cached_context_length(
@@ -2358,50 +2180,22 @@ def get_model_context_length(
# 1b. AWS Bedrock — use static context length table.
# Bedrock's ListFoundationModels API doesn't expose context window sizes,
# so we maintain a curated table in bedrock_adapter.py that reflects
# Bedrock-hosted model limits (e.g. older Claude 4 at 200K; Claude
# Opus/Sonnet 4.6+ at 1M). This must run BEFORE the custom-endpoint probe at
# AWS-imposed limits (e.g. 200K for Claude models vs 1M on the native
# Anthropic API). This must run BEFORE the custom-endpoint probe at
# step 2 — bedrock-runtime.<region>.amazonaws.com is not in
# _URL_TO_PROVIDER, so it would otherwise be treated as a custom endpoint,
# fail the /models probe (Bedrock doesn't expose that shape), and fall
# back to the 128K default before reaching the original step 4b branch.
if is_bedrock_context:
if provider == "bedrock" or (
base_url
and base_url_hostname(base_url).startswith("bedrock-runtime.")
and base_url_host_matches(base_url, "amazonaws.com")
):
try:
from agent.bedrock_adapter import (
get_bedrock_context_length,
resolve_bedrock_region,
)
from agent.bedrock_adapter import get_bedrock_context_length
return get_bedrock_context_length(model)
except ImportError:
pass # boto3 not installed — fall through to generic resolution
else:
# Bedrock does not expose the context window via any metadata API,
# so get_bedrock_context_length() probes the live endpoint (one
# fast, pre-inference length rejection) to read the real window.
# Cache the probe result per model so we pay that cost once, not
# every turn — keyed by base_url when present, else a synthetic
# bedrock:// key so display/offline paths share the entry.
cache_key_url = base_url or "bedrock://"
cached = get_cached_context_length(model, cache_key_url)
if cached is not None:
return cached
# Resolve region from the base_url host first, then the standard
# AWS region chain. An empty region disables probing (table only).
region = ""
if base_url:
_m = re.search(r"bedrock-runtime\.([a-z0-9-]+)\.", base_url)
if _m:
region = _m.group(1)
if not region:
try:
region = resolve_bedrock_region()
except Exception:
region = ""
ctx = get_bedrock_context_length(model, region=region, probe=bool(region))
if ctx and region:
# Only persist probe-derived values (region present); a pure
# table fallback shouldn't poison the cache against a later
# successful probe.
save_context_length(model, cache_key_url, ctx)
return ctx
if provider == "novita" or (base_url and base_url_host_matches(base_url, "api.novita.ai")):
ctx = _resolve_endpoint_context_length(model, base_url or "https://api.novita.ai/openai/v1", api_key=api_key)
@@ -2516,14 +2310,9 @@ def get_model_context_length(
# Codex OAuth enforces lower context limits than the direct OpenAI
# API for the same slug (e.g. gpt-5.5 is 1.05M on the API but 272K
# on Codex). Authoritative source is Codex's own /models endpoint.
codex_ctx, codex_source = _resolve_codex_oauth_context_length_with_source(
model, access_token=api_key or "",
)
codex_ctx = _resolve_codex_oauth_context_length(model, access_token=api_key or "")
if codex_ctx:
# Only a successful authenticated catalogue response is safe to
# persist. The static fallback is deliberately runtime-only so a
# transient OAuth/network failure cannot poison future probes.
if base_url and codex_source == "live":
if base_url:
save_context_length(model, base_url, codex_ctx)
return codex_ctx
if effective_provider == "gmi" and base_url:
@@ -2666,61 +2455,16 @@ async def get_model_context_length_async(
)
def _is_cjk_token_dense_char(ch: str) -> bool:
code = ord(ch)
return (
0x1100 <= code <= 0x11FF # Hangul Jamo
or 0x2E80 <= code <= 0x9FFF # CJK radicals/ideographs
or 0xA960 <= code <= 0xA97F # Hangul Jamo Extended-A
or 0xAC00 <= code <= 0xD7AF # Hangul Syllables
or 0xF900 <= code <= 0xFAFF # CJK compatibility ideographs
or 0xFF00 <= code <= 0xFFEF # Fullwidth forms / halfwidth kana
)
# Same codepoint ranges as _is_cjk_token_dense_char, as a compiled character
# class so dense-char counting runs in C (``len(text) - len(re.sub(...))``)
# instead of a per-char Python loop. MUST stay in sync with
# _is_cjk_token_dense_char.
_CJK_DENSE_RE = re.compile(
"[\u1100-\u11ff" # Hangul Jamo
"\u2e80-\u9fff" # CJK radicals/ideographs
"\ua960-\ua97f" # Hangul Jamo Extended-A
"\uac00-\ud7af" # Hangul Syllables
"\uf900-\ufaff" # CJK compatibility ideographs
"\uff00-\uffef]" # Fullwidth forms / halfwidth kana
)
def estimate_tokens_rough(text: str) -> int:
"""Rough token estimate for pre-flight checks.
"""Rough token estimate (~4 chars/token) for pre-flight checks.
Uses ceiling division so short texts (1-3 chars) never estimate as
0 tokens, which would cause the compressor and pre-flight checks to
systematically undercount when many short tool results are present.
CJK/Hangul/Kana text is much denser than English under common LLM
tokenizers, so count those codepoints as roughly one token each instead
of applying the English-centric ~4 chars/token rule.
Perf: this runs on every message in every preflight/compaction walk,
including MB-scale tool outputs, so the common all-ASCII case must stay
O(1). ``str.isascii()`` is a flag check on CPython's compact unicode
representation (no scan), and the CJK counting itself is a single
C-level ``re.findall`` rather than a per-character Python loop.
"""
if not text:
return 0
text = str(text)
if text.isascii():
# O(1) fast path — ASCII text cannot contain token-dense CJK chars.
return (len(text) + 3) // 4
dense = len(text) - len(_CJK_DENSE_RE.sub("", text))
if not dense:
# Non-ASCII but no CJK (accents, Cyrillic, emoji, ...): keep the
# classic ~4 chars/token rule.
return (len(text) + 3) // 4
sparse = len(text) - dense
return dense + ((sparse + 3) // 4)
return (len(text) + 3) // 4
def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
@@ -2732,12 +2476,12 @@ def estimate_messages_tokens_rough(messages: List[Dict[str, Any]]) -> int:
estimated at ~250K tokens and trigger premature context compression.
"""
_IMAGE_TOKEN_COST = 1500
text_tokens = 0
total_chars = 0
image_tokens = 0
for msg in messages:
text_tokens += _estimate_message_tokens_without_images(msg)
total_chars += _estimate_message_chars(msg)
image_tokens += _count_image_tokens(msg, _IMAGE_TOKEN_COST)
return text_tokens + image_tokens
return ((total_chars + 3) // 4) + image_tokens
def _count_image_tokens(msg: Dict[str, Any], cost_per_image: int) -> int:
@@ -2799,35 +2543,6 @@ def _estimate_message_chars(msg: Dict[str, Any]) -> int:
return len(str(shadow))
def _estimate_message_tokens_without_images(msg: Dict[str, Any]) -> int:
"""Token estimate for a message shadow with image payloads stripped."""
if not isinstance(msg, dict):
return estimate_tokens_rough(str(msg))
shadow: Dict[str, Any] = {}
for k, v in msg.items():
if k == "_anthropic_content_blocks":
continue
if k == "content":
if isinstance(v, list):
cleaned = []
for part in v:
if isinstance(part, dict):
if part.get("type") in {"image", "image_url", "input_image"}:
cleaned.append({"type": part.get("type"), "image": "[stripped]"})
else:
cleaned.append(part)
else:
cleaned.append(part)
shadow[k] = cleaned
elif isinstance(v, dict) and v.get("_multimodal"):
shadow[k] = v.get("text_summary", "")
else:
shadow[k] = v
else:
shadow[k] = v
return estimate_tokens_rough(str(shadow))
def estimate_request_tokens_rough(
messages: List[Dict[str, Any]],
*,
@@ -2844,7 +2559,7 @@ def estimate_request_tokens_rough(
"""
total = 0
if system_prompt:
total += estimate_tokens_rough(system_prompt)
total += (len(system_prompt) + 3) // 4
if messages:
total += estimate_messages_tokens_rough(messages)
if tools:
+2 -33
View File
@@ -15,9 +15,6 @@ and MoonshotAI/kimi-cli#1595:
2. When ``anyOf`` is used, ``type`` must be on the ``anyOf`` children, not
the parent. Presence of both causes "type should be defined in anyOf
items instead of the parent schema".
3. Every object schema must carry a ``required`` array, even an empty one.
Standard JSON Schema allows omitting it; Moonshot 400s with
"required must be an array".
The ``#/definitions/...`` → ``#/$defs/...`` rewrite for draft-07 refs is
handled separately in ``tools/mcp_tool._normalize_mcp_input_schema`` so it
@@ -133,32 +130,9 @@ def _repair_schema(node: Any, is_schema: bool = True) -> Any:
else:
repaired.pop("enum")
# Rule 4: object schemas must carry a `required` array, even when empty.
if repaired.get("type") == "object":
repaired = _ensure_required_array(repaired)
return repaired
def _ensure_required_array(node: Dict[str, Any]) -> Dict[str, Any]:
"""Guarantee an object schema carries a ``required`` array (Moonshot rule).
Standard JSON Schema lets you omit ``required`` when nothing is required;
Moonshot 400s on that ("required must be an array"). Ensure the key is a
list. When ``properties`` is known, prune ``required`` entries that don't
name a real property — defensive against dangling names, which Moonshot
also rejects. Mutates and returns ``node``.
"""
props = node.get("properties")
req = node.get("required")
if isinstance(req, list):
if isinstance(props, dict):
node["required"] = [r for r in req if r in props]
else:
node["required"] = []
return node
def _fill_missing_type(node: Dict[str, Any]) -> Dict[str, Any]:
"""Infer a reasonable ``type`` if this schema node has none."""
node_type = node.get("type")
@@ -200,18 +174,17 @@ def sanitize_moonshot_tool_parameters(parameters: Any) -> Dict[str, Any]:
applied. Input is not mutated.
"""
if not isinstance(parameters, dict):
return {"type": "object", "properties": {}, "required": []}
return {"type": "object", "properties": {}}
repaired = _repair_schema(copy.deepcopy(parameters), is_schema=True)
if not isinstance(repaired, dict):
return {"type": "object", "properties": {}, "required": []}
return {"type": "object", "properties": {}}
# Top-level must be an object schema
if repaired.get("type") != "object":
repaired["type"] = "object"
if "properties" not in repaired:
repaired["properties"] = {}
_ensure_required_array(repaired)
return repaired
@@ -259,10 +232,6 @@ def is_moonshot_model(model: str | None) -> bool:
tail = bare.rsplit("/", 1)[-1]
if tail.startswith("kimi-") or tail == "kimi":
return True
# Kimi Coding Plan serves K3 under the bare slug ``k3`` (plus dated /
# suffixed variants like ``k3.1`` or ``k3-turbo``).
if tail == "k3" or tail.startswith(("k3.", "k3-")):
return True
# Vendor-prefixed forms commonly used on aggregators
if "moonshot" in bare or "/kimi" in bare or bare.startswith("kimi"):
return True
+3 -83
View File
@@ -31,55 +31,7 @@ version can change at runtime (editable installs, hot-reload tooling), and
from __future__ import annotations
from contextvars import ContextVar
from typing import List, Optional
# ── Ambient conversation context ─────────────────────────────────────────────
#
# The main agent loop knows its ``session_id``; the dozens of auxiliary call
# sites (compression, title generation, vision, web_extract, session_search,
# MoA reference/aggregator slots, curator, kanban helpers, ...) do not — they
# funnel through ``agent.auxiliary_client.call_llm`` which has no session
# handle. Rather than threading a ``session_id`` parameter through every one
# of those call sites (and every future one), the agent loop publishes the
# active conversation id here and ``nous_portal_tags()`` picks it up as a
# fallback whenever no explicit ``session_id`` is passed.
#
# ContextVar (not a module global) so concurrent agents in one process —
# gateway sessions, delegate_task subagents, batch runners — never see each
# other's conversation id. Worker threads spawned via
# ``tools.thread_context.propagate_context_to_thread`` (background review,
# MoA fan-out, tool executor) inherit it through the copied Context; bare
# threads (title generator) capture it explicitly at spawn time.
_conversation_id: ContextVar[Optional[str]] = ContextVar(
"nous_portal_conversation_id", default=None
)
def set_conversation_context(conversation_id: Optional[str]):
"""Publish the active conversation id for ambient Portal tagging.
Called by the agent loop at turn entry with the conversation's stable
id (the session-lineage ROOT id, so the tag survives context-compression
session rotation). Pass ``None`` to clear. Returns the ContextVar token
so callers can ``reset_conversation_context(token)`` on turn exit.
"""
return _conversation_id.set(conversation_id or None)
def reset_conversation_context(token) -> None:
"""Restore the previous conversation context (pair with ``set_...``)."""
try:
_conversation_id.reset(token)
except Exception:
# Token from another Context (e.g. reset on a different thread) —
# fall back to clearing rather than raising in cleanup paths.
_conversation_id.set(None)
def get_conversation_context() -> Optional[str]:
"""Return the ambient conversation id, or ``None`` when unset."""
return _conversation_id.get()
from typing import List
def _hermes_version() -> str:
@@ -103,42 +55,10 @@ def hermes_client_tag() -> str:
return f"client=hermes-client-v{_hermes_version()}"
def conversation_tag(session_id: str) -> str:
"""Return the ``conversation=...`` tag for a Hermes session/conversation.
Format: ``conversation=<session_id>``. ``session_id`` is the canonical
Hermes conversation identifier (``AIAgent.session_id``) — the same value
used for ``~/.hermes/sessions/`` storage, session logs, and lineage.
Unlike the product/client tags this is high-cardinality (one value per
conversation), so it is only appended when a session id is actually
available — never as part of the always-on base tag set.
"""
return f"conversation={session_id}"
def nous_portal_tags(session_id: str | None = None) -> List[str]:
def nous_portal_tags() -> List[str]:
"""Return the canonical list of Nous Portal product tags.
Always returns a fresh list so callers can mutate it freely
(e.g. ``merged_extra.setdefault("tags", []).extend(nous_portal_tags())``).
When ``session_id`` is provided, a ``conversation=<session_id>`` tag is
appended so Portal usage can be attributed to a specific Hermes
conversation. When it is omitted, the ambient conversation context
(``set_conversation_context``, published by the agent loop at turn
entry) is used instead — this is how auxiliary calls (compression,
titles, vision, MoA slots, ...) inherit the conversation tag without
per-call-site plumbing. Callers outside any conversation (e.g. the
auxiliary client's import-time base tags) get the canonical two-tag set.
"""
tags = ["product=hermes-agent", hermes_client_tag()]
# Ambient context first: the agent loop publishes the lineage ROOT id
# (stable across context-compression rotation and delegate subagent
# trees), which is the better conversation key than a per-segment
# session_id passed explicitly. The explicit argument remains as a
# fallback for callers running outside any agent turn.
effective = get_conversation_context() or session_id
if effective:
tags.append(conversation_tag(effective))
return tags
return ["product=hermes-agent", hermes_client_tag()]
+22 -105
View File
@@ -58,14 +58,6 @@ def _scan_context_content(content: str, filename: str) -> str:
BLOCKED at this layer because the file would otherwise enter the
system prompt verbatim and the user has no chance to intervene.
"""
# Editors (Windows Notepad, PowerShell Out-File without -Encoding
# utf8NoBOM, some VS Code profiles) prefix a UTF-8 BOM as an encoding
# artifact, not a prompt injection. Strip a leading U+FEFF silently so a
# context file (SOUL.md, AGENTS.md, ...) is not blocked wholesale; BOMs
# elsewhere in the content remain subject to the threat scan below.
if content.startswith("\ufeff"):
content = content[1:]
findings = _scan_for_threats(content, scope="context")
if findings:
logger.warning("Context file %s blocked: %s", filename, ", ".join(findings))
@@ -122,7 +114,6 @@ def _strip_yaml_frontmatter(content: str) -> str:
strip it so only the human-readable markdown body is injected into the
system prompt.
"""
content = content.lstrip("\ufeff") # tolerate UTF-8 BOM (Windows editors)
if content.startswith("---"):
end = content.find("\n---", 3)
if end != -1:
@@ -266,10 +257,6 @@ KANBAN_GUIDANCE = (
"- **Deliverables.** Files a human wants go in "
"`kanban_complete(artifacts=[<absolute paths>])` (top-level param; paths in "
"`metadata` are NOT uploaded). Files must exist at completion.\n"
"- **Attachments.** Attach real downloadable artifacts instead of pasting "
"links in comments: `kanban_attach` (base64) or `kanban_attach_url` "
"(server-side public http(s) fetch); 25 MB cap, `kanban_attachments` "
"lists them. Workers may only attach to their own task.\n"
"- **Created cards.** List ids in `kanban_complete(created_cards=[...])` "
"ONLY when captured from a successful `kanban_create` return — never invent "
"or paste ids; the kernel rejects the completion on any phantom id.\n"
@@ -557,29 +544,6 @@ def computer_use_guidance(platform_name: Optional[str] = None) -> str:
"4. After any state-changing action, re-capture to verify. You can "
"pass `capture_after=true` to get the follow-up screenshot in one "
"round-trip.\n\n"
"## Verify → escalate ladder (background-first, NOT background-only)\n"
"Background delivery is the DEFAULT and the co-work path, but it is "
"the first rung, not the only one. Read each action's structured "
"result and climb only when the driver tells you to:\n"
"- `effect: 'confirmed'` + `verified: true` — the driver read the "
"result back. Done.\n"
"- `effect: 'unverifiable'` — the input was delivered but the driver "
"can't confirm it. Re-capture and check the screenshot/tree yourself "
"before deciding it worked.\n"
"- `effect: 'suspected_noop'`, `code: 'background_unavailable'`, or an "
"`escalation.recommended` field — the action did NOT land. Follow "
"`escalation.recommended`:\n"
" - `'px'` → re-issue addressing the target by `coordinate=[x,y]` "
"read off the screenshot instead of `element`.\n"
" - `'foreground'` (or a pixel click still didn't land) → re-issue "
"the SAME action with `delivery_mode='foreground'`. This briefly "
"raises the window; it needs its own approval and is only appropriate "
"when the user isn't actively working. Common for Electron/Chromium "
"consent dialogs, DirectInput games, and raw-input canvases.\n"
"- Escalate to foreground as a REACTION to a returned signal, never "
"as a prediction from the app being Electron/Chromium/GTK. Do not "
"silently retry the same rung expecting a different result, and do "
"not conclude 'cua-driver can't drive this app' — climb the ladder.\n\n"
"## Background mode rules\n"
"- Do NOT use `raise_window=true` on `focus_app` unless the user "
"explicitly asked you to bring a window to front. Input routing to "
@@ -695,7 +659,19 @@ PLATFORM_HINTS = {
"Standard Markdown is automatically converted to Telegram formatting. "
"Supported: **bold**, *italic*, ~~strikethrough~~, ||spoiler||, "
"`inline code`, ```code blocks```, [links](url), and ## headers. "
"Prefer bullet lists and labeled key:value pairs for structured data. "
"Telegram now supports rich Markdown, so lean into it: whenever it "
"makes the answer clearer or easier to scan, actively reach for real "
"Markdown tables (pipe `| col | col |` syntax), bullet and numbered "
"lists, task lists (`- [ ]` / `- [x]`), headings, nested blockquotes, "
"collapsible details, footnotes/references, math/formulas (`$...$`, "
"`$$...$$`), underline, subscript/superscript, marked (highlighted) "
"text, and anchors. Default to structured formatting over dense "
"paragraphs for any comparison, set of steps, key/value summary, or "
"tabular data. Prefer real Markdown tables and task lists over "
"hand-built bullet substitutes when presenting structured data; these "
"degrade gracefully (tables become readable bullet groups) when rich "
"rendering is unavailable, but advanced constructs like math and "
"collapsible details may render as plain source text in that case. "
"You can send media files natively: to deliver a file to the user, "
"include MEDIA:/absolute/path/to/file in your response. Images "
"(.png, .jpg, .webp) appear as photos, audio (.ogg) sends as voice "
@@ -805,19 +781,8 @@ PLATFORM_HINTS = {
),
"matrix": (
"You are in a Matrix room communicating with your user. "
"The adapter converts your Markdown to HTML for rich display — bold, "
"italic, inline code, fenced code blocks, headings, bullet and "
"numbered lists, blockquotes, and links all render.\n\n"
"Do NOT use Markdown tables: many popular Matrix clients (Element X, "
"Beeper, most mobile apps) do not render HTML tables, so the cells "
"collapse into one continuous run of text. Present tabular data as "
"labeled '**Label:** value' lines or bullet lists instead.\n\n"
"Avoid ||spoiler|| tags, ~~strikethrough~~, and checkboxes "
"(- [ ] / - [x]) — they are not converted and appear as literal "
"characters.\n\n"
"LINKS: prefer [descriptive link text](url) over bare URLs. When "
"referencing something with an associated URL (events, sources, "
"people), make the name a clickable link.\n\n"
"Matrix renders Markdown — bold, italic, code blocks, and links work; "
"the adapter converts your Markdown to HTML for rich display. "
"You can send media files natively: include MEDIA:/absolute/path/to/file "
"in your response. Images (.jpg, .png, .webp) are sent as inline photos, "
"audio (.ogg, .mp3) as voice/audio messages, video (.mp4) inline, "
@@ -900,27 +865,6 @@ PLATFORM_HINTS = {
),
}
# Telegram rich-messages extension — only injected when the user has opted in
# to ``platforms.telegram.extra.rich_messages: true``. The base
# PLATFORM_HINTS["telegram"] covers MarkdownV2-compatible constructs; this
# extension adds the Bot API 10.1 rich-Markdown guidance (tables, task lists,
# collapsible details, math, etc.).
TELEGRAM_RICH_MESSAGES_HINT = (
"Telegram now supports rich Markdown, so lean into it: whenever it "
"makes the answer clearer or easier to scan, actively reach for real "
"Markdown tables (pipe `| col | col |` syntax), bullet and numbered "
"lists, task lists (`- [ ]` / `- [x]`), headings, nested blockquotes, "
"collapsible details, footnotes/references, math/formulas (`$...$`, "
"`$$...$$`), underline, subscript/superscript, marked (highlighted) "
"text, and anchors. Default to structured formatting over dense "
"paragraphs for any comparison, set of steps, key/value summary, or "
"tabular data. Prefer real Markdown tables and task lists over "
"hand-built bullet substitutes when presenting structured data; these "
"degrade gracefully (tables become readable bullet groups) when rich "
"rendering is unavailable, but advanced constructs like math and "
"collapsible details may render as plain source text in that case. "
)
# ---------------------------------------------------------------------------
# Environment hints — execution-environment awareness for the agent.
# Unlike PLATFORM_HINTS (which describe the messaging channel), these describe
@@ -2004,7 +1948,6 @@ def build_context_files_prompt(
cwd: Optional[str] = None,
skip_soul: bool = False,
context_length: Optional[int] = None,
allow_install_tree_fallback: bool = False,
) -> str:
"""Discover and load context files for the system prompt.
@@ -2026,43 +1969,17 @@ def build_context_files_prompt(
"""
if cwd is None:
cwd = os.getcwd()
cwd_is_fallback = True
else:
cwd_is_fallback = False
cwd_path = Path(cwd).resolve()
sections = []
# Never let a FALLBACK-picked directory inside the Hermes install/source
# tree gain system-prompt authority. A backend that self-spawns into that
# tree (the desktop app default) would otherwise load this repo's
# contributor AGENTS.md as authoritative project context (#64590). An
# explicitly configured cwd is honored verbatim — the Hermes tree is a
# legitimate workspace when the user deliberately points a session at it —
# and CLI-style surfaces pass allow_install_tree_fallback=True because
# their launch dir IS the user's shell cwd (developing Hermes in-tree).
from agent.runtime_cwd import _is_install_tree
if (
cwd_is_fallback
and not allow_install_tree_fallback
and _is_install_tree(cwd_path)
):
logger.warning(
"skipping project-context discovery: working-directory resolution "
"fell back to the Hermes install tree (%s) — set terminal.cwd to "
"your project directory",
cwd_path,
)
project_context = ""
else:
# Priority-based project context: first match wins
project_context = (
_load_hermes_md(cwd_path, context_length)
or _load_agents_md(cwd_path, context_length)
or _load_claude_md(cwd_path, context_length)
or _load_cursorrules(cwd_path, context_length)
)
# Priority-based project context: first match wins
project_context = (
_load_hermes_md(cwd_path, context_length)
or _load_agents_md(cwd_path, context_length)
or _load_claude_md(cwd_path, context_length)
or _load_cursorrules(cwd_path, context_length)
)
if project_context:
sections.append(project_context)
-2
View File
@@ -102,7 +102,6 @@ _REASONING_STALE_TIMEOUT_FLOORS: tuple[tuple[str, int], ...] = (
# ``claude-opus-4`` so non-thinking Claude 3.x or future
# non-reasoning Claude variants don't match.
("claude-opus-4", 240),
("claude-sonnet-5", 180),
("claude-sonnet-4.5", 180),
("claude-sonnet-4.6", 180),
# xAI Grok reasoning variants. Explicit reasoning-only keys
@@ -112,7 +111,6 @@ _REASONING_STALE_TIMEOUT_FLOORS: tuple[tuple[str, int], ...] = (
# non-reasoning pairs.
("grok-4-fast-reasoning", 300),
("grok-4.20-reasoning", 300),
("grok-4.5", 300),
("grok-4-fast-non-reasoning", 180),
)
-61
View File
@@ -11,7 +11,6 @@ import logging
import os
import re
import shlex
from urllib.parse import unquote_plus
logger = logging.getLogger(__name__)
@@ -286,22 +285,6 @@ _URL_USERINFO_RE = re.compile(
r"(https?|wss?|ftp)://([^/\s:@]+):([^/\s@]+)@",
)
# Strict provider-egress URL redaction accepts more URL-reference forms than
# the display/log helpers above. Parameter delimiters stay in capture groups so
# redaction preserves the original query/fragment layout byte-for-byte, while
# the key is decoded separately for classification. Values stop at query or
# fragment pair separators; both ``&`` and ``;`` are valid in deployed URLs.
_STRICT_URL_PARAM_RE = re.compile(
r"([?#&;])([A-Za-z0-9_.~+%\-]+)=([^#&;\s\"'<>]*)"
)
# Match userinfo in both absolute (``scheme://user:pass@host``) and
# network-path (``//user:pass@host``) references. The authority boundary stops
# at path/query/fragment delimiters so an ``@`` elsewhere in a URL is ignored.
_STRICT_URL_USERINFO_RE = re.compile(
r"((?:[A-Za-z][A-Za-z0-9+.-]*:)?//)([^/\s?#@]+)@"
)
# HTTP access logs often use a relative request target rather than a full URL:
# `"POST /webhook?password=... HTTP/1.1"`. The full-URL redactor above only
# sees strings containing `://`, so handle request-target query strings too.
@@ -428,41 +411,6 @@ def _redact_url_userinfo(text: str) -> str:
)
def _canonical_url_param_name(name: str) -> str:
"""Decode a URL parameter name for bounded, case-insensitive matching."""
decoded = name
for _ in range(3):
next_value = unquote_plus(decoded)
if next_value == decoded:
break
decoded = next_value
return decoded.casefold().replace("-", "_")
def _redact_strict_url_credentials(text: str) -> str:
"""Redact credentials from absolute, relative, and network URL references.
This is intentionally stricter than display/log redaction and is used only
at explicit secret-egress boundaries. It preserves original keys,
separators, public parameters, hosts, and paths while masking sensitive
values and URL userinfo.
"""
def _redact_param(match: re.Match) -> str:
if _canonical_url_param_name(match.group(2)) not in _SENSITIVE_QUERY_PARAMS:
return match.group(0)
return f"{match.group(1)}{match.group(2)}=***"
def _redact_userinfo(match: re.Match) -> str:
userinfo = match.group(2)
if ":" in userinfo:
username, _, _password = userinfo.partition(":")
return f"{match.group(1)}{username}:***@"
return f"{match.group(1)}***@"
text = _STRICT_URL_PARAM_RE.sub(_redact_param, text)
return _STRICT_URL_USERINFO_RE.sub(_redact_userinfo, text)
def redact_cdp_url(value: object) -> str:
"""Mask secrets in a CDP/browser endpoint URL before it is logged.
@@ -546,7 +494,6 @@ def redact_sensitive_text(
force: bool = False,
code_file: bool = False,
file_read: bool = False,
redact_url_credentials: bool = False,
) -> str:
"""Apply all redaction patterns to a block of text.
@@ -555,11 +502,6 @@ def redact_sensitive_text(
Set force=True for safety boundaries that must never return raw secrets
regardless of the user's global logging redaction preference.
Set redact_url_credentials=True at non-navigation egress boundaries to
additionally redact credential-named query parameters and ``user:pass@``
URL userinfo. The default remains False because actionable OAuth callback,
magic-link, and pre-signed URLs must survive ordinary tool flows unchanged.
Set code_file=True to skip the ENV-assignment and JSON-field regex
patterns when the text is known to be source code (e.g. MAX_TOKENS=***
constants, "apiKey": "test" fixtures). Prefix patterns, auth headers,
@@ -724,9 +666,6 @@ def redact_sensitive_text(
# string), so masking it can't break a skill. The ``user:pass@`` form is
# left to pass through per #34029.
if redact_url_credentials:
text = _redact_strict_url_credentials(text)
# Form-urlencoded bodies (only triggers on clean k=v&k=v inputs).
if "&" in text and "=" in text:
text = _redact_form_body(text)
+5 -71
View File
@@ -20,10 +20,6 @@ from __future__ import annotations
import logging
from typing import Any, Dict, List
from agent.tool_dispatch_helpers import make_tool_result_message
from agent.tool_result_classification import tool_may_have_side_effect
from agent.turn_context import drop_stale_api_content
logger = logging.getLogger(__name__)
@@ -68,40 +64,8 @@ def strip_interrupted_tool_tails(
is_interrupted_tool_result(m.get("content", ""))
for m in tool_results
):
calls = msg.get("tool_calls") or []
if any(
tool_may_have_side_effect(
str((call.get("function") or {}).get("name") or "")
)
for call in calls
):
call_names = {
str(call.get("id") or call.get("call_id") or ""): str(
(call.get("function") or {}).get("name") or ""
)
for call in calls
}
cleaned.append(msg)
for tool_result in tool_results:
if not is_interrupted_tool_result(tool_result.get("content", "")):
cleaned.append(tool_result)
continue
recovered = dict(tool_result)
name = call_names.get(str(tool_result.get("tool_call_id") or ""), "")
recovered["effect_disposition"] = (
"unknown" if tool_may_have_side_effect(name) else "none"
)
recovered["content"] = (
"[Orphan recovery: interrupted side-effecting tool may have "
"executed; its effect is UNKNOWN. Inspect state before retrying.]"
if recovered["effect_disposition"] == "unknown"
else "[Orphan recovery: interrupted read-only tool did not complete.]"
)
cleaned.append(recovered)
i = j
continue
logger.debug(
"Stripping interrupted read-only assistant→tool replay block "
"Stripping interrupted assistant→tool replay block "
"(indices %d–%d, tool_results=%d)",
i, j - 1, len(tool_results),
)
@@ -152,36 +116,11 @@ def strip_dangling_tool_call_tail(
):
return agent_history
tool_calls = last.get("tool_calls") or []
if any(
tool_may_have_side_effect(
str((call.get("function") or {}).get("name") or "")
)
for call in tool_calls
):
recovered = list(agent_history)
for call in tool_calls:
function = call.get("function") or {}
name = str(function.get("name") or "unknown")
call_id = str(call.get("id") or call.get("call_id") or "")
disposition = "unknown" if tool_may_have_side_effect(name) else "none"
content = (
"[Orphan recovery: this tool may have executed before Hermes stopped; "
"its effect is UNKNOWN. Inspect current state before retrying.]"
if disposition == "unknown"
else "[Orphan recovery: this read-only tool did not complete and had no effect.]"
)
recovered.append(make_tool_result_message(
name, content, call_id, effect_disposition=disposition,
))
logger.warning(
"Recovered dangling side-effecting tool call(s) as UNKNOWN instead of erasing them"
)
return recovered
logger.debug(
"Stripping dangling unanswered read-only assistant(tool_calls) tail (%d call(s))",
len(tool_calls),
"Stripping dangling unanswered assistant(tool_calls) tail "
"(%d call(s)) — process likely killed mid-tool-call by a "
"restart/shutdown command (#49201)",
len(last.get("tool_calls") or []),
)
return agent_history[:-1]
@@ -312,11 +251,6 @@ def strip_stale_dangerous_confirmations(
)
redacted = dict(msg)
redacted["content"] = _EXPIRED_CONFIRMATION_SENTINEL
# Drop the api_content sidecar: it carries the exact bytes
# previously sent — i.e. the dangerous confirmation this
# redaction exists to expire. Replaying it verbatim would
# undo the redaction on the wire.
drop_stale_api_content(redacted)
cleaned.append(redacted)
continue
cleaned.append(msg)
+5 -43
View File
@@ -10,36 +10,15 @@ Multi-session gateways can pin a logical cwd via the `_SESSION_CWD`
contextvar; CLI/cron fall through to `TERMINAL_CWD`/launch cwd.
"""
import logging
import os
from contextvars import ContextVar, Token
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
_UNSET: Any = object()
_SESSION_CWD: ContextVar = ContextVar("HERMES_SESSION_CWD", default=_UNSET)
# The Python package/source root (this file lives at <root>/agent/runtime_cwd.py).
# When a backend is launched from, or self-spawns into, this tree (the desktop
# app default), an os.getcwd() fallback would inject this repo's contributor
# AGENTS.md as authoritative project context. Context discovery must never
# resolve here.
_PACKAGE_ROOT = Path(__file__).resolve().parent.parent
def _is_install_tree(p: Path) -> bool:
# True only when p IS the package root or sits inside it. Ancestors of the
# package root (a user home that happens to contain the checkout, a --user
# site-packages parent) are legitimate workspaces and must not be blocked.
try:
p = p.resolve()
except Exception:
return False
return p == _PACKAGE_ROOT or _PACKAGE_ROOT in p.parents
def set_session_cwd(cwd: str | None) -> Token:
"""Pin the logical cwd for the current context."""
@@ -63,38 +42,21 @@ def resolve_agent_cwd() -> Path:
p = Path(override).expanduser()
if p.is_dir():
return p
logger.warning("configured working directory does not exist: %s", override)
raw = os.environ.get("TERMINAL_CWD", "").strip()
if raw:
p = Path(raw).expanduser()
if p.is_dir():
return p
logger.warning("TERMINAL_CWD does not exist: %s", raw)
return Path(os.getcwd())
def resolve_context_cwd() -> Path | None:
# None means "no configured cwd": build_context_files_prompt then falls back
# to the launch dir (os.getcwd()), correct for a local CLI launched inside a
# real project. A configured path is validated here (previously it was passed
# through unchecked, diverging from resolve_agent_cwd). An explicitly
# configured path is otherwise honored verbatim — including the Hermes
# source tree itself, which is a legitimate workspace when the user is
# developing Hermes (per-surface policy for fallback-picked directories
# lives in build_context_files_prompt; see #64590).
# to the launch dir (os.getcwd()) — correct for the local CLI. The gateway
# avoids slurping its install dir by setting TERMINAL_CWD (see system_prompt.py)
# or, per session, the _SESSION_CWD contextvar above.
override = _session_cwd_override()
if override:
p = Path(override).expanduser()
if not p.is_dir():
logger.warning("configured working directory does not exist: %s", override)
else:
return p
return None
return Path(override).expanduser()
raw = os.environ.get("TERMINAL_CWD", "").strip()
if raw:
p = Path(raw).expanduser()
if not p.is_dir():
logger.warning("TERMINAL_CWD does not exist: %s", raw)
else:
return p
return None
return Path(raw).expanduser() if raw else None
+5 -35
View File
@@ -127,16 +127,10 @@ def get_secret(name: str, default: Optional[str] = None) -> Optional[str]:
1. Genuinely-global vars (``_is_global_env``) always read ``os.environ`` —
they are deployment settings, not profile secrets.
2. When a secret scope is installed (multiplexed turn), read from it. Under
multiplexing the scope is authoritative — an absent key returns
``default`` and we do NOT fall through to ``os.environ``, because in a
multiplexer ``os.environ`` may hold another profile's value. When
multiplexing is OFF, a scope miss falls through to ``os.environ``:
single-profile deployments legitimately provide credentials via the
process environment (systemd ``Environment=``, secret-manager wrappers
like ``pass-cli run`` / ``op run``, plain shell exports) rather than
``<home>/.env``, and the scope — installed unconditionally around e.g.
every cron job — must stay a ``.env`` overlay, not a blindfold.
2. When a secret scope is installed (multiplexed turn), read from it; an
absent key returns ``default``. The scope is authoritative — we do NOT
fall through to ``os.environ``, because in a multiplexer ``os.environ``
may hold another profile's value.
3. No scope installed:
- multiplex INACTIVE (default deployment): read ``os.environ`` —
identical to the legacy ``os.getenv`` behavior every caller had before.
@@ -150,17 +144,6 @@ def get_secret(name: str, default: Optional[str] = None) -> Optional[str]:
scope = _SECRET_SCOPE.get()
if scope is not None:
val = scope.get(name)
if val is not None:
return val
if _MULTIPLEX_ACTIVE:
return default
# Multiplex off: the scope is an overlay over the process environment,
# not an isolation boundary — there is no other profile to leak from.
# Without this fallthrough, credentials injected only into the process
# environment vanish inside any set_secret_scope(...) block (the cron
# scheduler installs one around every job), so cron jobs send a
# placeholder API key and 401 while interactive turns keep working.
val = os.environ.get(name)
return val if val is not None else default
if _MULTIPLEX_ACTIVE:
@@ -218,18 +201,5 @@ def build_profile_secret_scope(hermes_home: Path) -> Dict[str, str]:
global vars are intentionally NOT copied in — ``get_secret`` reads those
from ``os.environ`` directly, so the scope holds only profile secrets.
"""
home = Path(hermes_home)
secrets = load_env_file(home / ".env")
return load_env_file(Path(hermes_home) / ".env")
try:
from hermes_cli.env_loader import get_secret_source_values
external_secrets = get_secret_source_values(home)
except Exception:
external_secrets = {}
for key, value in external_secrets.items():
if _is_global_env(key):
continue
secrets[key] = value
return secrets
-39
View File
@@ -190,45 +190,6 @@ class SecretSource(ABC):
"""
return {}
def remediation(self, kind: Optional["ErrorKind"], cfg: dict) -> str:
"""One-line, actionable next step for a failed fetch.
Called by the startup status printer (and ``hermes secrets ...
status``) right after a fetch error is surfaced, so the user sees
*what to run* next to fix it — not just what broke. Sources
should override this to point at their own CLI verbs (e.g.
``hermes secrets bitwarden token`` for AUTH_FAILED). Return an
empty string to suppress the hint.
Must never raise and must not perform I/O — it's a pure
kind→string mapping on the startup path.
"""
generic = {
ErrorKind.NOT_CONFIGURED: (
f"Run `hermes secrets {self.name} setup` to finish configuration."
),
ErrorKind.BINARY_MISSING: (
f"Run `hermes secrets {self.name} setup` to install the helper CLI."
),
ErrorKind.AUTH_FAILED: (
f"Credentials rejected — run `hermes secrets {self.name} setup` "
"to re-authenticate."
),
ErrorKind.AUTH_EXPIRED: (
f"Credentials expired — run `hermes secrets {self.name} setup` "
"to re-authenticate."
),
ErrorKind.NETWORK: (
"Network problem reaching the secrets backend — check "
"connectivity and retry."
),
ErrorKind.TIMEOUT: (
f"Backend was slow — raise secrets.{self.name}.timeout_seconds "
"if this recurs."
),
}
return generic.get(kind, "") if kind is not None else ""
# ---------------------------------------------------------------------------
# Shared helpers — use these instead of hand-rolling per backend
+18 -328
View File
@@ -29,13 +29,11 @@ is easier to lazy-install than a wheels-with-Rust-extension dependency.
from __future__ import annotations
import base64
import hashlib
import json
import logging
import os
import platform
import re
import shutil
import stat
import subprocess
@@ -47,10 +45,6 @@ import zipfile
from pathlib import Path
from typing import Dict, List, Optional, Tuple
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from agent.secret_sources._cache import (
CachedFetch as _CachedFetch,
DiskCache,
@@ -97,9 +91,6 @@ _CACHE: Dict[_CacheKey, _CachedFetch] = {}
# accidentally commit BSM-sourced secrets. The atomic-write/0600/TTL mechanics
# live in agent.secret_sources._cache.DiskCache, shared with the other backends.
_DISK_CACHE_BASENAME = "bws_cache.json"
_ENCRYPTED_CACHE_BASENAME = "bws_cache.enc.json"
_ENCRYPTED_CACHE_VERSION = 1
_ENCRYPTED_CACHE_INFO = b"hermes-bws-encrypted-cache-v1"
def _cache_key_str(cache_key: _CacheKey) -> str:
@@ -122,13 +113,6 @@ def _disk_cache_path(home_path: Optional[Path] = None) -> Path:
return _DISK_CACHE.path(home_path)
def _encrypted_disk_cache_path(home_path: Optional[Path] = None) -> Path:
"""Return the encrypted disk cache path under hermes_home/cache/."""
from agent.secret_sources._cache import resolve_cache_home
return resolve_cache_home(home_path) / "cache" / _ENCRYPTED_CACHE_BASENAME
# ---------------------------------------------------------------------------
# Binary discovery + lazy install
# ---------------------------------------------------------------------------
@@ -364,134 +348,6 @@ def _token_fingerprint(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()[:16]
def _b64e(raw: bytes) -> str:
return base64.b64encode(raw).decode("ascii")
def _b64d(text: str) -> bytes:
return base64.b64decode(text.encode("ascii"), validate=True)
def _derive_encrypted_cache_key(access_token: str, salt: bytes) -> bytes:
"""Derive the local cache encryption key from the bootstrap BWS token."""
return HKDF(
algorithm=hashes.SHA256(),
length=32,
salt=salt,
info=_ENCRYPTED_CACHE_INFO,
).derive(access_token.encode("utf-8"))
def _write_encrypted_disk_cache(
*,
cache_key: _CacheKey,
access_token: str,
entry: _CachedFetch,
home_path: Optional[Path] = None,
) -> None:
"""Persist an encrypted last-good cache entry atomically.
Best-effort by design: cache write failure must never block a fresh BWS
fetch. The raw BWS access token is not stored; it only derives the AES key.
"""
path = _encrypted_disk_cache_path(home_path)
try:
cache_dir = path.parent
cache_dir.mkdir(parents=True, exist_ok=True)
try:
os.chmod(cache_dir, 0o700)
except OSError:
pass
salt = os.urandom(16)
nonce = os.urandom(12)
serialized_key = _cache_key_str(cache_key)
key = _derive_encrypted_cache_key(access_token, salt)
plaintext = json.dumps(
{"secrets": entry.secrets, "fetched_at": entry.fetched_at},
separators=(",", ":"),
).encode("utf-8")
ciphertext = AESGCM(key).encrypt(
nonce, plaintext, serialized_key.encode("utf-8")
)
payload = {
"version": _ENCRYPTED_CACHE_VERSION,
"key": serialized_key,
"salt": _b64e(salt),
"nonce": _b64e(nonce),
"ciphertext": _b64e(ciphertext),
}
fd, tmp = tempfile.mkstemp(
prefix=".bws_cache_enc_", suffix=".tmp", dir=str(cache_dir)
)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
json.dump(payload, f)
os.chmod(tmp, 0o600)
os.replace(tmp, path)
# A successful encrypted write completes migration; remove the
# legacy plaintext cache so stale secrets cannot remain on disk.
try:
_disk_cache_path(home_path).unlink()
except FileNotFoundError:
pass
except OSError:
pass
except BaseException:
try:
os.unlink(tmp)
except OSError:
pass
raise
except Exception: # noqa: BLE001 — best-effort cache only
return
def _read_encrypted_disk_cache(
*,
cache_key: _CacheKey,
access_token: str,
max_age_seconds: float,
home_path: Optional[Path] = None,
) -> Optional[_CachedFetch]:
"""Return a decrypted encrypted-cache entry if it matches and is in-window."""
if max_age_seconds <= 0:
return None
path = _encrypted_disk_cache_path(home_path)
try:
payload = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(payload, dict):
return None
serialized_key = _cache_key_str(cache_key)
if payload.get("version") != _ENCRYPTED_CACHE_VERSION:
return None
if payload.get("key") != serialized_key:
return None
salt = _b64d(str(payload.get("salt", "")))
nonce = _b64d(str(payload.get("nonce", "")))
ciphertext = _b64d(str(payload.get("ciphertext", "")))
key = _derive_encrypted_cache_key(access_token, salt)
raw = AESGCM(key).decrypt(
nonce, ciphertext, serialized_key.encode("utf-8")
)
inner = json.loads(raw.decode("utf-8"))
if not isinstance(inner, dict):
return None
secrets = inner.get("secrets")
inner_fetched_at = inner.get("fetched_at")
if not isinstance(secrets, dict) or not isinstance(inner_fetched_at, (int, float)):
return None
entry_age = time.time() - float(inner_fetched_at)
if entry_age < 0 or entry_age > max_age_seconds:
return None
typed = {
k: v for k, v in secrets.items()
if isinstance(k, str) and isinstance(v, str)
}
return _CachedFetch(secrets=typed, fetched_at=float(inner_fetched_at))
except Exception: # noqa: BLE001 — cache miss on parse/decrypt/I/O errors
return None
def fetch_bitwarden_secrets(
*,
access_token: str,
@@ -501,8 +357,6 @@ def fetch_bitwarden_secrets(
use_cache: bool = True,
server_url: str = "",
home_path: Optional[Path] = None,
encrypted_cache_enabled: bool = False,
encrypted_cache_max_stale_seconds: float = 0,
) -> Tuple[Dict[str, str], List[str]]:
"""Pull the secrets for ``project_id`` from Bitwarden Secrets Manager.
@@ -514,13 +368,12 @@ def fetch_bitwarden_secrets(
(``https://vault.bitwarden.com``, US Cloud). This is plumbed into
the subprocess as ``BWS_SERVER_URL``.
``cache_ttl_seconds`` controls the normal fresh cache. When
``encrypted_cache_enabled`` is true, fresh cache entries are written as
AES-GCM encrypted JSON instead of plaintext, and a last-good encrypted
entry may be used after NETWORK/TIMEOUT failures for up to
``encrypted_cache_max_stale_seconds``. This stale fallback is separate
from the fresh-cache TTL so operators can set ``cache_ttl_seconds: 0``
while still keeping an encrypted break-glass cache for offline startup.
Caching is a two-layer LRU: an in-process dict (for hot-reload paths
inside one process) and a disk-persisted JSON file under
``<hermes_home>/cache/bws_cache.json`` (for back-to-back CLI invocations).
Both share the same TTL. Pass ``home_path`` so disk cache lookups find
the right directory in tests / non-standard installs; otherwise we fall
back to ``$HERMES_HOME`` / ``~/.hermes``.
Raises :class:`RuntimeError` for fatal conditions (missing binary,
auth failure, unparseable output). Callers in the env_loader path
@@ -533,20 +386,12 @@ def fetch_bitwarden_secrets(
raise RuntimeError("Bitwarden project_id is empty")
cache_key = (_token_fingerprint(access_token), project_id, server_url or "")
if use_cache and cache_ttl_seconds > 0:
if use_cache:
cached = _CACHE.get(cache_key)
if cached and cached.is_fresh(cache_ttl_seconds):
return cached.secrets, []
# L2: disk cache. ~5ms on cache hit vs ~380ms for `bws secret list`.
if encrypted_cache_enabled:
disk_cached = _read_encrypted_disk_cache(
cache_key=cache_key,
access_token=access_token,
max_age_seconds=cache_ttl_seconds,
home_path=home_path,
)
else:
disk_cached = _DISK_CACHE.read(cache_key, cache_ttl_seconds, home_path)
disk_cached = _DISK_CACHE.read(cache_key, cache_ttl_seconds, home_path)
if disk_cached is not None:
# Promote into in-process cache so subsequent fetches in the
# same process skip the disk read too.
@@ -562,107 +407,14 @@ def fetch_bitwarden_secrets(
"`hermes secrets bitwarden setup`."
)
try:
secrets, warnings = _run_bws_list(bws, access_token, project_id, server_url)
except RuntimeError as exc:
# Live fetch failed. Fall back to a stale disk cache ONLY for
# transport-level failures (network down, DNS error, transient BWS
# outage / timeout) — never for AUTH_FAILED or a malformed-output
# INTERNAL error, where serving old secrets would mask a real
# config/credential problem the caller needs to see. Without this
# fallback a fleet of bots sharing one BWS project all stop working
# on a single network blip.
#
# Two fallback tiers share the transport-only gate:
# * encrypted cache (opt-in) — AES-GCM payload keyed off the
# bootstrap token, with its own max_stale_seconds window. When
# enabled it is the ONLY fallback consulted: the whole point is
# that the at-rest payload is never plaintext, so we don't
# quietly serve the plaintext file alongside it.
# * plaintext disk cache (default) — the ordinary DiskCache file.
# `cache_ttl_seconds <= 0` means the caller opted out of caching
# entirely (DiskCache.read/write both short-circuit on it) —
# honor that on the fallback path too. `ttl_seconds=inf` on the
# read bypasses freshness (we explicitly want a stale hit); the
# caller's real TTL gates whether we even attempt the read.
kind = _classify_bws_error(str(exc))
if use_cache and kind in (ErrorKind.NETWORK, ErrorKind.TIMEOUT):
if encrypted_cache_enabled:
stale = _read_encrypted_disk_cache(
cache_key=cache_key,
access_token=access_token,
max_age_seconds=encrypted_cache_max_stale_seconds,
home_path=home_path,
)
if stale is not None:
age = max(0.0, time.time() - stale.fetched_at)
_CACHE[cache_key] = stale
return stale.secrets, [
f"bws live fetch failed ({exc}); falling back to "
f"stale ENCRYPTED disk cache ({int(age)}s old)"
]
elif cache_ttl_seconds > 0:
stale = _DISK_CACHE.read(cache_key, float("inf"), home_path)
if stale is not None:
age = max(0.0, time.time() - stale.fetched_at)
_CACHE[cache_key] = stale
return stale.secrets, [
f"bws live fetch failed ({exc}); "
f"falling back to stale disk cache ({int(age)}s old)"
]
raise
secrets, warnings = _run_bws_list(bws, access_token, project_id, server_url)
entry = _CachedFetch(secrets=secrets, fetched_at=time.time())
_CACHE[cache_key] = entry
if use_cache:
if cache_ttl_seconds > 0:
_CACHE[cache_key] = entry
if encrypted_cache_enabled:
# Encryption is the storage policy; max_stale_seconds only controls
# whether an outage may consume the last-good entry. Never fall
# back to the plaintext cache just because stale fallback is off.
_write_encrypted_disk_cache(
cache_key=cache_key,
access_token=access_token,
entry=entry,
home_path=home_path,
)
elif cache_ttl_seconds > 0:
_DISK_CACHE.write(cache_key, entry, cache_ttl_seconds, home_path)
_DISK_CACHE.write(cache_key, entry, cache_ttl_seconds, home_path)
return secrets, warnings
def _summarize_bws_stderr(raw: str) -> str:
"""Reduce a bws (Rust color-eyre) error dump to its cause line(s).
bws failures look like::
Error:
0: Received error message from server: [400 Bad Request] {"error":"invalid_client"}
Location:
crates/bws/src/main.rs:108
...
Everything from ``Location:`` on is diagnostic noise for a Hermes
user. Keep the numbered cause lines (joined), drop the rest, and
fall back to the stripped raw text when the shape is unrecognized.
"""
text = raw.replace("\x1b", "").strip()
if not text:
return text
causes: List[str] = []
for line in text.splitlines():
stripped = line.strip()
if stripped.startswith(("Location:", "Backtrace omitted", "Run with ")):
break
if stripped in ("", "Error:"):
continue
# Cause lines are numbered "0: ...", "1: ..." — strip the index.
stripped = re.sub(r"^\d+:\s*", "", stripped)
if stripped:
causes.append(stripped)
return "; ".join(causes) if causes else text
def _run_bws_list(
bws: Path, access_token: str, project_id: str, server_url: str = ""
) -> Tuple[Dict[str, str], List[str]]:
@@ -696,11 +448,9 @@ def _run_bws_list(
raise RuntimeError(f"failed to invoke bws: {exc}") from exc
if proc.returncode != 0:
# bws writes auth/network errors to stderr as a Rust error-report
# dump (color-eyre): an "Error:" header, indented cause lines, then
# "Location:" / "Backtrace omitted" noise. Strip ANSI and boil it
# down to the meaningful cause line(s) before surfacing.
err = _summarize_bws_stderr(proc.stderr or proc.stdout or "")
# bws writes auth/network errors to stderr in plain English.
# Strip ANSI just in case and surface the first 200 chars.
err = (proc.stderr or proc.stdout or "").strip().replace("\x1b", "")
raise RuntimeError(
f"bws exited {proc.returncode}: {err[:200]}"
)
@@ -752,8 +502,6 @@ def apply_bitwarden_secrets(
auto_install: bool = True,
server_url: str = "",
home_path: Optional[Path] = None,
encrypted_cache_enabled: bool = False,
encrypted_cache_max_stale_seconds: float = 0,
) -> FetchResult:
"""Pull secrets from BSM and set them on ``os.environ``.
@@ -805,8 +553,6 @@ def apply_bitwarden_secrets(
cache_ttl_seconds=cache_ttl_seconds,
server_url=server_url,
home_path=home_path,
encrypted_cache_enabled=encrypted_cache_enabled,
encrypted_cache_max_stale_seconds=encrypted_cache_max_stale_seconds,
)
except RuntimeError as exc:
result.error = str(exc)
@@ -876,16 +622,9 @@ class BitwardenSource(SecretSource):
},
"project_id": {"description": "BSM project UUID", "default": ""},
"cache_ttl_seconds": {
"description": "Fresh disk+memory cache TTL; 0 disables fresh-cache reuse",
"description": "Disk+memory cache TTL; 0 disables",
"default": 300,
},
"encrypted_cache": {
"description": "Encrypted last-good cache for network/timeout fallback",
"default": {
"enabled": False,
"max_stale_seconds": 0,
},
},
"override_existing": {
"description": "BSM values overwrite .env/shell values",
"default": True,
@@ -939,14 +678,6 @@ class BitwardenSource(SecretSource):
except (TypeError, ValueError):
ttl = 300.0
encrypted_cfg = cfg.get("encrypted_cache")
encrypted_cfg = encrypted_cfg if isinstance(encrypted_cfg, dict) else {}
encrypted_enabled = bool(encrypted_cfg.get("enabled", False))
try:
encrypted_max_stale = float(encrypted_cfg.get("max_stale_seconds", 0))
except (TypeError, ValueError):
encrypted_max_stale = 0.0
try:
secrets, warnings = fetch_bitwarden_secrets(
access_token=access_token,
@@ -955,36 +686,16 @@ class BitwardenSource(SecretSource):
cache_ttl_seconds=ttl,
server_url=str(cfg.get("server_url", "") or "").strip(),
home_path=home_path,
encrypted_cache_enabled=encrypted_enabled,
encrypted_cache_max_stale_seconds=encrypted_max_stale,
)
except RuntimeError as exc:
result.error = str(exc)
result.error_kind = _classify_bws_error(str(exc))
if result.error_kind == ErrorKind.AUTH_FAILED:
# Translate the raw OAuth reject into what it actually means
# for the user before the mechanics.
result.error = (
"Bitwarden rejected the machine-account access token "
f"({access_token_env}) — it was likely revoked, expired, "
f"or belongs to another region. ({result.error})"
)
return result
result.secrets = secrets
result.warnings.extend(warnings)
return result
def remediation(self, kind, cfg: dict) -> str:
if kind in (ErrorKind.AUTH_FAILED, ErrorKind.AUTH_EXPIRED):
return (
"Run `hermes secrets bitwarden token` to paste a fresh access "
"token (create one in the Bitwarden web app: Secrets Manager → "
"Machine accounts → Access tokens). Wrong region? Re-run "
"`hermes secrets bitwarden setup` and pick EU/self-hosted."
)
return super().remediation(kind, cfg)
def _classify_bws_error(message: str) -> ErrorKind:
"""Best-effort mapping of bws failure text onto the shared taxonomy."""
@@ -994,13 +705,7 @@ def _classify_bws_error(message: str) -> ErrorKind:
if "binary not available" in lowered or "failed to invoke" in lowered:
return ErrorKind.BINARY_MISSING
if any(tok in lowered for tok in ("unauthorized", "invalid token",
"access token", "401", "403",
# The BSM identity endpoint rejects a
# revoked/expired/deleted machine-account
# token with an OAuth-style
# `[400 Bad Request] {"error":"invalid_client"}`.
"invalid_client", "invalid_grant",
"400 bad request")):
"access token", "401", "403")):
return ErrorKind.AUTH_FAILED
if any(tok in lowered for tok in ("network", "connection", "resolve",
"download", "dns")):
@@ -1013,22 +718,6 @@ def _classify_bws_error(message: str) -> ErrorKind:
# ---------------------------------------------------------------------------
def clear_caches(home_path: Optional[Path] = None) -> None:
"""Drop in-process AND disk caches (plaintext and encrypted).
Used after a token rotation (`hermes secrets bitwarden token`) so the
next startup fetches fresh with the new credential instead of serving
a pull cached under the old token's fingerprint. The encrypted cache
is keyed off the old token too, so it must go as well.
"""
_CACHE.clear()
_DISK_CACHE.clear(home_path)
try:
_encrypted_disk_cache_path(home_path).unlink()
except (FileNotFoundError, OSError):
pass
def _reset_cache_for_tests(home_path: Optional[Path] = None) -> None:
"""Clear in-process AND disk caches.
@@ -1036,4 +725,5 @@ def _reset_cache_for_tests(home_path: Optional[Path] = None) -> None:
Without it we fall back to the same default resolution as the cache
writer itself.
"""
clear_caches(home_path)
_CACHE.clear()
_DISK_CACHE.clear(home_path)
-488
View File
@@ -1,488 +0,0 @@
"""``command`` secret source — resolve secrets via a user-configured helper.
Ports the security semantics of the desktop app's TypeScript
``CommandSecretsProvider`` (hermes-desktop ``src/main/secrets/commandProvider.ts``)
to the Python agent. The helper command (e.g. ``keepassxc-cli``,
``secret-tool``, or a script that cats a tmpfs env file) comes from
``secrets.command`` in ``config.yaml`` — NEVER from ``.env``, which holds
only secret values.
Security model (mirrors the TS provider line-for-line where it matters):
* The command string is the USER'S OWN configuration (same trust level as
the ``.env`` file they control), so it is run via ``/bin/sh -c <command>``.
* The requested key is passed to the child ONLY via the ``HERMES_SECRET_KEY``
environment variable — it is NEVER interpolated into the shell string, so
a hostile key name (e.g. ``"; rm -rf ~``) is inert data, not code.
* Hard timeout (default 3s) + output cap (default 1 MiB); any failure
(non-zero exit, timeout, spawn failure, oversized output) degrades to
"no value" rather than raising.
* Failures log ONLY structured fields (exit code / signal / errno) to
stderr — never the command string, the helper's stderr, or any secret
value. The helper's stderr is captured via a pipe and DISCARDED so its
diagnostics (which can carry secret material) never reach our stderr.
* The startup/apply path runs the helper exactly ONCE (with an empty
``HERMES_SECRET_KEY``) — it is never called per-key in a loop, so a
helper that blocks (e.g. on a vault unlock prompt) can't be spawned
dozens of times.
* PLATFORM: the provider is POSIX-only (needs ``/bin/sh``). On Windows it
degrades to an empty result with a warning; Windows users stay on the
default ``env`` provider.
"""
from __future__ import annotations
import os
import platform
import re
import signal as _signal
import subprocess
import sys
from pathlib import Path
from typing import Dict, Optional
# Reuse the exact result shape the bitwarden source returns so
# hermes_cli.env_loader can consume both providers identically.
from agent.secret_sources.base import ErrorKind, SecretSource
from agent.secret_sources.bitwarden import FetchResult
__all__ = [
"FetchResult",
"apply_command_secrets",
"get_command_secret",
"list_command_secrets",
"parse_secret_output",
"unquote_dotenv_value",
]
# Hard cap so a hung helper can never wedge startup. Kept deliberately
# TIGHT (3s) — a configured helper MUST be fast and NON-INTERACTIVE
# (e.g. `keepassxc-cli` against an already-unlocked DB, `secret-tool
# lookup`, or `cat`-ing a tmpfs env file), NOT something that prompts
# for a touch/PIN.
_COMMAND_TIMEOUT_SECONDS = 3.0
# Defensive cap on helper output (1 MiB) — a misbehaving command can't OOM us.
_MAX_OUTPUT_BYTES = 1024 * 1024
# A line is treated as a KEY=VALUE pair only when it matches an env-key
# shape before the '='. Anchored; `.` does not cross newlines, so a
# multi-line blob never matches as a single "env-shaped" value.
_ENV_LINE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)=(.*)$")
def _is_windows() -> bool:
return os.name == "nt" or platform.system() == "Windows"
def unquote_dotenv_value(raw: str) -> str:
"""Strip a single layer of matching surrounding quotes from a dotenv value.
Requires length >= 2 so a lone quote (``"``) is left intact rather than
collapsing to empty, and ``""``/``''`` correctly yield an empty string.
Shared by the single-key parser and the list path so both unquote
identically.
"""
t = raw.strip()
if len(t) >= 2 and (
(t.startswith('"') and t.endswith('"'))
or (t.startswith("'") and t.endswith("'"))
):
return t[1:-1]
return t
def parse_secret_output(stdout: str, wanted_key: str) -> Optional[str]:
"""Parse a secret-fetch helper's stdout. Supports BOTH shapes:
* a bare value (single secret): the whole trimmed stdout is the value.
* a dotenv blob (KEY=VALUE lines): parse them and return the entry for
``wanted_key``.
Mirrors the TS ``parseSecretOutput`` exactly, including the cross-key
misroute guard and the base64-padding disambiguation.
"""
text = stdout.replace("\r\n", "\n")
lines = text.split("\n")
# 1. Exact dotenv match wins: scan for a `wanted_key=...` line. This
# is deterministic and never returns another key's value.
dotenv_lines = [
line
for line in (raw.strip() for raw in lines)
if line and not line.startswith("#") and _ENV_LINE.match(line)
]
for line in dotenv_lines:
m = _ENV_LINE.match(line)
assert m is not None # filtered above
if m.group(1) == wanted_key:
value = unquote_dotenv_value(m.group(2))
# Whitespace-only (e.g. a quoted `K=" "` placeholder) is "no
# value": it would otherwise flow into an Authorization header
# → guaranteed 401.
return value if value.strip() != "" else None
# 2. The output is a multi-key dotenv dump that does NOT contain the
# wanted key → None, rather than mis-returning an unrelated line as
# a bare value. Only >=2 env-shaped lines count as a dump: a SINGLE
# non-matching env-shaped line falls through to the bare-value
# branch, because a bare secret can itself match the KEY=VALUE shape
# (e.g. base64 with '=' padding, "dGVzdA==") and must not be
# misclassified as a dump.
if len(dotenv_lines) > 1:
return None
# 3. Otherwise treat the whole output as a single bare value (a per-key
# helper that printed just the secret). Trim first so whitespace-only
# output (a ' '/'\t' placeholder entry) resolves to None, never a "key".
value = text.strip()
if value == "":
return None
# SECURITY (S2): a single env-shaped line for a DIFFERENT key must not
# be returned as the wanted secret. A sloppy helper (e.g. `head -1
# env-file`, or a grep that matched the wrong line) emitting
# `OTHER_KEY=realvalue` would otherwise flow — key name, '=' and the
# OTHER key's value — into an Authorization header sent to the WANTED
# key's endpoint: cross-provider credential leakage, not just a 401.
# Disambiguation from a bare base64 secret: base64 padding only ever
# produces an env-shaped line whose "value" part is empty or all '='
# (`dGVzdA==` → key `dGVzdA`, value `=`), so a non-trivial value part
# after a non-matching key means a misrouted dotenv entry → None.
env_shaped = _ENV_LINE.match(value)
if (
env_shaped
and env_shaped.group(1) != wanted_key
and re.fullmatch(r"=*", env_shaped.group(2).strip()) is None
):
return None
return value
def _run_helper(
command: str,
secret_key: str,
timeout_seconds: float,
max_output_bytes: int,
) -> Optional[str]:
"""Run the helper via ``/bin/sh -c`` and return its stdout, or None.
The key is passed as DATA via ``HERMES_SECRET_KEY`` — never interpolated
into the command string. Both stdout and stderr are captured via pipes
(never inherited); stderr is discarded. Any failure logs structured
fields only and returns None — never raises.
"""
if _is_windows():
print(
"[secrets:command] the 'command' provider is POSIX-only "
"(needs /bin/sh); resolving no value on Windows",
file=sys.stderr,
)
return None
env = os.environ.copy()
env["HERMES_SECRET_KEY"] = secret_key
try:
proc = subprocess.Popen( # noqa: S602 — command is the user's own config
["/bin/sh", "-c", command],
env=env,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE, # captured and DISCARDED — never inherited
start_new_session=True, # so the hard timeout can kill the whole group
)
except OSError as exc:
print(
f"[secrets:command] helper failed to spawn; resolving no value: "
f"errno={exc.errno}",
file=sys.stderr,
)
return None
try:
stdout_bytes, _stderr_discarded = proc.communicate(timeout=timeout_seconds)
except subprocess.TimeoutExpired:
# Hard timeout: kill the whole process group (a helper script may
# have forked children that would otherwise keep the pipe open).
# POSIX-only by construction: _run_helper early-returns on Windows
# before ever spawning, so this line can't execute there.
try:
os.killpg(os.getpgid(proc.pid), _signal.SIGKILL) # windows-footgun: ok
except (ProcessLookupError, PermissionError, OSError):
proc.kill()
try:
proc.communicate(timeout=1.0)
except (subprocess.TimeoutExpired, ValueError, OSError):
pass
print(
f"[secrets:command] helper timed out after {timeout_seconds:g}s; "
f"resolving no value",
file=sys.stderr,
)
return None
if proc.returncode != 0:
# Structured fields ONLY — never the command string or the helper's
# stderr (either can carry secret material).
if proc.returncode < 0:
try:
sig = _signal.Signals(-proc.returncode).name
except ValueError:
sig = str(-proc.returncode)
code, signame = "?", sig
else:
code, signame = str(proc.returncode), "none"
print(
f"[secrets:command] helper failed; resolving no value: "
f"code={code} signal={signame}",
file=sys.stderr,
)
return None
if len(stdout_bytes) > max_output_bytes:
print(
f"[secrets:command] helper output exceeded the "
f"{max_output_bytes}-byte cap; resolving no value",
file=sys.stderr,
)
return None
return stdout_bytes.decode("utf-8", errors="replace")
def _parse_dotenv_map(stdout: str) -> Dict[str, str]:
"""Parse a KEY=VALUE blob into a map (the list/enumerate path).
Mirrors the TS ``list()``: only env-shaped lines contribute; comments
and non-matching lines are skipped. A bare-value helper yields ``{}``
— per-key resolution via :func:`get_command_secret` still works.
"""
out: Dict[str, str] = {}
for raw in stdout.replace("\r\n", "\n").split("\n"):
line = raw.strip()
if not line or line.startswith("#"):
continue
m = _ENV_LINE.match(line)
if not m:
continue
out[m.group(1)] = unquote_dotenv_value(m.group(2))
return out
def get_command_secret(
*,
command: str,
key: str,
timeout_seconds: float = _COMMAND_TIMEOUT_SECONDS,
max_output_bytes: int = _MAX_OUTPUT_BYTES,
) -> Optional[str]:
"""Resolve a single secret by running the helper with the key in
``HERMES_SECRET_KEY``. Returns None on any failure — never raises."""
command = (command or "").strip()
if not command:
return None
stdout = _run_helper(command, key, timeout_seconds, max_output_bytes)
if stdout is None:
return None
return parse_secret_output(stdout, key)
def list_command_secrets(
*,
command: str,
timeout_seconds: float = _COMMAND_TIMEOUT_SECONDS,
max_output_bytes: int = _MAX_OUTPUT_BYTES,
) -> Dict[str, str]:
"""Enumerate secrets by running the helper ONCE with an empty key.
Returns the dotenv map ONLY when the helper emits a KEY=VALUE blob;
a bare-value helper returns ``{}``. Never raises.
"""
command = (command or "").strip()
if not command:
return {}
stdout = _run_helper(command, "", timeout_seconds, max_output_bytes)
if stdout is None:
return {}
return _parse_dotenv_map(stdout)
# ---------------------------------------------------------------------------
# Public entry point — called from hermes_cli.env_loader
# ---------------------------------------------------------------------------
def apply_command_secrets(
*,
command: str,
override_existing: bool = False,
timeout_seconds: float = _COMMAND_TIMEOUT_SECONDS,
max_output_bytes: int = _MAX_OUTPUT_BYTES,
home_path: Optional[Path] = None,
) -> FetchResult:
"""Run the helper once at startup and set its KEY=VALUE output on
``os.environ``.
LEGACY shim retained for API symmetry with ``apply_bitwarden_secrets``;
the startup path goes through :class:`CommandSource` + the registry
orchestrator instead (which owns precedence and the environ writes).
"""
result = FetchResult()
command = (command or "").strip()
if not command:
result.error = (
"secrets.command.enabled is true but secrets.command.command is "
"empty. Set the helper command in config.yaml."
)
return result
if _is_windows():
result.warnings.append(
"the 'command' secret source is POSIX-only (needs /bin/sh); "
"skipping on Windows"
)
return result
# The list/enumerate path: run the helper exactly ONCE with an empty
# HERMES_SECRET_KEY and parse its stdout as a dotenv blob.
stdout = _run_helper(command, "", timeout_seconds, max_output_bytes)
if stdout is None:
# _run_helper already logged structured fields to stderr.
result.warnings.append(
"helper command failed at startup; no secrets applied "
"(process env / .env values remain in effect)"
)
return result
secrets = _parse_dotenv_map(stdout)
result.secrets = secrets
if not secrets:
result.warnings.append(
"helper output was not a KEY=VALUE map; nothing applied at "
"startup (a bare-value helper still resolves single keys on demand)"
)
return result
for key, value in secrets.items():
if value.strip() == "":
# Whitespace-only placeholder entries are "no value" — applying
# them would flow into an Authorization header → guaranteed 401.
result.skipped.append(key)
continue
if not override_existing and os.environ.get(key):
# Process env / .env win — same precedence as bitwarden.
result.skipped.append(key)
continue
os.environ[key] = value
result.applied.append(key)
return result
# ---------------------------------------------------------------------------
# SecretSource adapter — the registry-facing wrapper around this module.
# ---------------------------------------------------------------------------
class CommandSource(SecretSource):
"""User-configured helper command as a registered secret source.
Composes with the other sources (Bitwarden, 1Password, plugins) through
the ``apply_all()`` orchestrator — enable any combination simultaneously;
there is deliberately NO single-provider selector. ``fetch()`` only
fetches: precedence, ``override_existing`` semantics, conflict warnings,
and the ``os.environ`` writes are the orchestrator's job.
Bulk shape: the helper enumerates a KEY=VALUE blob in one run. Config::
secrets:
command:
enabled: true
command: "cat /run/user/1000/hermes-secrets.env"
# or per-vault CLIs: keepassxc-cli / secret-tool / pass / gpg —
# anything fast and NON-interactive.
"""
name = "command"
label = "Command helper"
shape = "bulk"
def config_schema(self) -> dict:
return {
"enabled": {"description": "Master switch", "default": False},
"command": {
"description": "Helper run via /bin/sh -c; must print a "
"KEY=VALUE blob on stdout",
"default": "",
},
"helper_timeout_seconds": {
"description": "Hard timeout for one helper run",
"default": _COMMAND_TIMEOUT_SECONDS,
},
"override_existing": {
"description": "Helper values overwrite .env/shell values",
"default": False,
},
}
def fetch(self, cfg: dict, home_path: Path) -> FetchResult:
cfg = cfg if isinstance(cfg, dict) else {}
result = FetchResult()
command = str(cfg.get("command") or "").strip()
if not command:
result.error = (
"secrets.command.enabled is true but secrets.command.command "
"is empty. Set the helper command in config.yaml."
)
result.error_kind = ErrorKind.NOT_CONFIGURED
return result
if _is_windows():
result.error = (
"the 'command' secret source is POSIX-only (needs /bin/sh); "
"skipping on Windows"
)
result.error_kind = ErrorKind.NOT_CONFIGURED
return result
try:
timeout = float(cfg.get("helper_timeout_seconds",
_COMMAND_TIMEOUT_SECONDS))
except (TypeError, ValueError):
timeout = _COMMAND_TIMEOUT_SECONDS
stdout = _run_helper(command, "", timeout, _MAX_OUTPUT_BYTES)
if stdout is None:
# _run_helper already logged structured fields to stderr.
result.error = (
"helper command failed (see structured fields above); "
"no secrets applied"
)
result.error_kind = ErrorKind.INTERNAL
return result
secrets = _parse_dotenv_map(stdout)
if not secrets:
result.warnings.append(
"helper output was not a KEY=VALUE map; nothing to apply"
)
return result
result.secrets = secrets
return result
def remediation(self, kind, cfg: dict) -> str:
if kind == ErrorKind.NOT_CONFIGURED:
return (
"Set secrets.command.command in config.yaml to a fast, "
"non-interactive helper that prints KEY=VALUE lines."
)
if kind == ErrorKind.INTERNAL:
return (
"Run the helper manually in a shell to see its real error — "
"Hermes discards helper stderr so diagnostics can't leak "
"secret material."
)
return super().remediation(kind, cfg)
+7 -41
View File
@@ -98,9 +98,6 @@ _OP_ENV_ALLOWLIST = (
"OP_ACCOUNT",
"OP_CONNECT_HOST",
"OP_CONNECT_TOKEN",
# Lets a user skip op's desktop-app integration probe (which can hang with
# no timeout on a wedged desktop container) and go straight to token auth.
"OP_LOAD_DESKTOP_APP_SETTINGS",
)
@@ -175,19 +172,16 @@ def _validate_references(
def _auth_fingerprint(token_env: str) -> str:
"""SHA-256 prefix over the auth material `op` would use.
Folds in the service-account token, ``OP_ACCOUNT``, the 1Password Connect
``OP_CONNECT_HOST``/``OP_CONNECT_TOKEN``, and *all* ``OP_SESSION_*`` vars
(the names `op` actually exports for interactive sessions —
``OP_SESSION_<account_shorthand>``). Signing out and into a different
identity therefore changes the cache key, so a value cached under a
previous identity is never served under a new one. Never logged or
Folds in the service-account token, ``OP_ACCOUNT``, and *all*
``OP_SESSION_*`` vars (the names `op` actually exports for interactive
sessions — ``OP_SESSION_<account_shorthand>``). Signing out and into a
different identity therefore changes the cache key, so a value cached under
a previous identity is never served under a new one. Never logged or
displayed; the raw token never leaves this hash.
"""
parts: List[str] = [
f"token={os.environ.get(token_env, '')}",
f"account={os.environ.get('OP_ACCOUNT', '')}",
f"connect_host={os.environ.get('OP_CONNECT_HOST', '')}",
f"connect_token={os.environ.get('OP_CONNECT_TOKEN', '')}",
]
for key in sorted(os.environ):
if key.startswith("OP_SESSION_"):
@@ -613,24 +607,6 @@ class OnePasswordSource(SecretSource):
result.warnings.extend(fetch_warnings)
return result
def remediation(self, kind, cfg: dict) -> str:
if kind in (ErrorKind.AUTH_FAILED, ErrorKind.AUTH_EXPIRED):
token_env = _DEFAULT_TOKEN_ENV
if isinstance(cfg, dict):
token_env = str(cfg.get("service_account_token_env") or token_env)
return (
"Run `hermes secrets onepassword token` to paste a fresh "
f"service-account token ({token_env}), or `op signin` for an "
"interactive session."
)
if kind == ErrorKind.BINARY_MISSING:
return (
"Install the 1Password CLI "
"(https://developer.1password.com/docs/cli/get-started/) or "
"set secrets.onepassword.binary_path."
)
return super().remediation(kind, cfg)
def _classify_op_error(message: str) -> ErrorKind:
"""Best-effort mapping of op failure text onto the shared taxonomy."""
@@ -657,21 +633,11 @@ def _classify_op_error(message: str) -> ErrorKind:
# ---------------------------------------------------------------------------
def clear_caches(home_path: Optional[Path] = None) -> None:
"""Drop in-process AND disk caches.
Used after a token rotation (`hermes secrets onepassword token`) so
the next startup resolves fresh with the new credential instead of
serving values cached under the old token's fingerprint.
"""
_CACHE.clear()
_DISK_CACHE.clear(home_path)
def _reset_cache_for_tests(home_path: Optional[Path] = None) -> None:
"""Clear in-process AND disk caches.
Tests can pass ``home_path`` to scope the disk cleanup to a tmpdir.
Without it we fall back to the same default resolution as the writer.
"""
clear_caches(home_path)
_CACHE.clear()
_DISK_CACHE.clear(home_path)
+10 -100
View File
@@ -29,7 +29,6 @@ from __future__ import annotations
import concurrent.futures
import logging
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, List, Optional
@@ -175,13 +174,6 @@ def _ensure_builtin_sources() -> None:
except Exception: # noqa: BLE001 — never block startup
logger.warning("Failed to register bundled 1Password secret source",
exc_info=True)
try:
from agent.secret_sources.command import CommandSource
register_source(CommandSource())
except Exception: # noqa: BLE001 — never block startup
logger.warning("Failed to register bundled command secret source",
exc_info=True)
def _reset_registry_for_tests() -> None:
@@ -283,43 +275,6 @@ def _ordered_enabled_sources(secrets_cfg: dict) -> List[SecretSource]:
return enabled
def _active_profile_name(home_path: Optional[Path]) -> str:
"""Best-effort active profile name for profile-scoped secret aliases.
A named profile's HERMES_HOME is ``~/.hermes/profiles/<name>``; the
default profile (``~/.hermes``) returns "".
"""
if home_path is not None:
resolved = Path(home_path)
if resolved.parent.name == "profiles" and resolved.name:
return resolved.name
for env_name in ("HERMES_PROFILE_NAME", "HERMES_PROFILE"):
value = os.environ.get(env_name, "").strip()
if value and value != "default":
return value
return ""
# Only credential-shaped names get auto-aliased — a random profile-suffixed
# var should not silently hydrate an unsuffixed name.
_ALIAS_SUFFIXES = ("_API_KEY", "_TOKEN", "_SECRET", "_KEY", "_PASSWORD")
def _profile_alias_target(var: str, profile: str) -> Optional[str]:
"""Map ``FOO_<PROFILE>`` to ``FOO`` for the active profile when safe."""
if not profile:
return None
suffix = "_" + profile.replace("-", "_").upper()
if not var.endswith(suffix):
return None
alias = var[: -len(suffix)]
if not alias or not is_valid_env_name(alias):
return None
if not any(alias.endswith(s) for s in _ALIAS_SUFFIXES):
return None
return alias
def apply_all(secrets_cfg: dict, home_path: Path,
environ: Optional[Dict[str, str]] = None) -> ApplyReport:
"""Fetch from every enabled source and apply the merged result to env.
@@ -328,24 +283,14 @@ def apply_all(secrets_cfg: dict, home_path: Path,
Precedence per env var (most-specific intent wins):
1. ``secrets.preserve_existing`` names — a pre-existing env value always
wins for these, even against a source with ``override_existing: true``
(escape hatch for profile-local platform secrets, #58073).
2. Pre-existing env (.env / shell) — unless the winning source has
1. Pre-existing env (.env / shell) — unless the winning source has
``override_existing: true``.
3. Mapped sources, in configured order.
4. Bulk sources, in configured order.
2. Mapped sources, in configured order.
3. Bulk sources, in configured order.
First claim wins. A later source that also carries the var gets a
``skipped_claimed`` entry and a conflict warning — never a silent
clobber, and ``override_existing`` never applies across sources.
Profile aliasing (#51447): when running under a named profile, an applied
var ``FOO_<PROFILE>`` (credential-shaped suffixes only) also hydrates the
canonical ``FOO`` so platform adapters and plugins that read fixed env
names see the profile's value. The alias obeys the same protected /
preserve / claimed / override guards and is disabled with
``secrets.profile_alias: false``.
"""
import os as _os
@@ -357,14 +302,6 @@ def apply_all(secrets_cfg: dict, home_path: Path,
if not enabled:
return report
preserve_raw = secrets_cfg.get("preserve_existing")
preserve: frozenset = frozenset(
n.strip() for n in preserve_raw if isinstance(n, str) and n.strip()
) if isinstance(preserve_raw, list) else frozenset()
alias_enabled = bool(secrets_cfg.get("profile_alias", True))
profile = _active_profile_name(home_path) if alias_enabled else ""
# Mapped sources outrank bulk sources regardless of list order:
# an explicit VAR→ref binding is stronger intent than a project dump.
ordered = ([s for s in enabled if s.shape == "mapped"]
@@ -384,15 +321,6 @@ def apply_all(secrets_cfg: dict, home_path: Path,
except Exception: # noqa: BLE001
pass
# Every var any source supplies directly — an alias never shadows a
# var that some source will (or tried to) claim by its real name.
supplied_directly: set = set()
for _, _, result in fetches:
if result.ok:
supplied_directly.update(
v for v in result.secrets if isinstance(v, str)
)
# Apply phase — sequential, first-wins, fully attributed.
claimed: Dict[str, str] = {} # var → source name that won it
for source, cfg, result in fetches:
@@ -408,14 +336,15 @@ def apply_all(secrets_cfg: dict, home_path: Path,
except Exception: # noqa: BLE001
override = False
def _try_apply(var: str, value: str, *, is_alias: bool = False) -> bool:
"""Apply one var through the shared guard chain. True = applied."""
for var, value in result.secrets.items():
if not isinstance(var, str) or not isinstance(value, str):
continue
if not is_valid_env_name(var):
sr.skipped_invalid.append(var)
return False
continue
if var in protected:
sr.skipped_protected.append(var)
return False
continue
if var in claimed:
sr.skipped_claimed.append(var)
report.conflicts.append(
@@ -423,14 +352,11 @@ def apply_all(secrets_cfg: dict, home_path: Path,
f"{source.name} also supplies it (first source wins — "
"remove one binding or reorder secrets.sources)"
)
return False
continue
existed = bool(env.get(var))
if existed and var in preserve:
sr.skipped_existing.append(var)
return False
if existed and not override:
sr.skipped_existing.append(var)
return False
continue
env[var] = value
claimed[var] = source.name
sr.applied.append(var)
@@ -440,21 +366,5 @@ def apply_all(secrets_cfg: dict, home_path: Path,
shape=source.shape,
overrode_env=existed,
)
return True
for var, value in result.secrets.items():
if not isinstance(var, str) or not isinstance(value, str):
continue
applied = _try_apply(var, value)
if not applied or not profile:
continue
alias = _profile_alias_target(var, profile)
if alias and alias not in supplied_directly and alias not in claimed:
if _try_apply(alias, value, is_alias=True):
result.warnings.append(
f"applied profile-scoped {var} as {alias} "
f"(active profile {profile!r})"
)
return report
+1 -27
View File
@@ -329,7 +329,6 @@ def scan_skill_commands() -> Dict[str, Dict[str, Any]]:
try:
from tools.skills_tool import SKILLS_DIR, _parse_frontmatter, skill_matches_platform, skill_matches_environment, _get_disabled_skill_names
from agent.skill_utils import get_external_skills_dirs, iter_skill_index_files
from hermes_cli.commands import resolve_command
disabled = _get_disabled_skill_names()
seen_names: set = set()
@@ -375,32 +374,7 @@ def scan_skill_commands() -> Dict[str, Dict[str, Any]]:
cmd_name = _SKILL_MULTI_HYPHEN.sub('-', cmd_name).strip('-')
if not cmd_name:
continue
# Skip if this skill's auto-generated /command collides
# with a core Hermes slash command (name or alias). The
# skill remains fully loadable via /skill <name>.
# Uses resolve_command() so aliases and case variants are
# covered without maintaining a separate cache.
if resolve_command(cmd_name) is not None:
logger.warning(
"Skill %r generates slash command '/%s' which "
"collides with a core Hermes command; skipping "
"auto-registration. Use '/skill %s' instead.",
name, cmd_name, name,
)
continue
# Dedup on the resolved slug, not just the raw name: two
# distinct frontmatter names can normalize to the same
# slug (e.g. "git_helper" vs "git-helper"). First-wins
# preserves local-before-external precedence.
cmd_key = f"/{cmd_name}"
if cmd_key in _skill_commands:
logger.warning(
"Skill %r maps to slash command %s already claimed "
"by %r; keeping the first and skipping this one.",
name, cmd_key, _skill_commands[cmd_key]["name"],
)
continue
_skill_commands[cmd_key] = {
_skill_commands[f"/{cmd_name}"] = {
"name": name,
"description": description or f"Invoke the {name} skill",
"skill_md_path": str(skill_md),
-12
View File
@@ -126,22 +126,10 @@ def parse_frontmatter(content: str) -> Tuple[Dict[str, Any], str]:
Uses yaml with CSafeLoader for full YAML support (nested metadata, lists)
with a fallback to simple key:value splitting for robustness.
A single leading UTF-8 BOM (U+FEFF) is stripped before parsing. Windows
GUI editors (Notepad, PowerShell ``>``) prepend one when saving a SKILL.md
as UTF-8, and ``read_text(encoding="utf-8")`` preserves it (only
``utf-8-sig`` strips it). Left in place, the BOM defeats the ``---`` fence
check below and the whole frontmatter is silently discarded — name,
description, ``platforms`` gating, env-var setup, and conditional
activation all vanish. See CONTRIBUTING.md "File encoding".
Returns:
(frontmatter_dict, remaining_body)
"""
frontmatter: Dict[str, Any] = {}
# Strip only a leading BOM; a BOM mid-content is data, not a marker.
if content.startswith("\ufeff"):
content = content[1:]
body = content
if not content.startswith("---"):
-70
View File
@@ -1,70 +0,0 @@
"""Best-effort accessors for the single-writer stream fence (#65991).
The fence itself lives on ``AIAgent`` (``_claim_stream_writer`` /
``_stream_writer_is_current`` in ``run_agent.py``), but the streaming code paths
that use it live in *other* modules — ``chat_completion_helpers`` (chat /
anthropic / bedrock) and ``codex_runtime`` (codex responses). Calling the fence
directly as ``agent._claim_stream_writer()`` from those modules makes them
hard-depend on the method being present on whatever object is passed in as
``agent``.
That coupling is a latent crash: a partially-updated checkout (the streaming
helper module newer than ``run_agent``), a hot-reloaded gateway, a duck-typed
agent, or a test double without the method turns an *additive* safety net into a
fatal ``AttributeError`` that aborts the whole turn. A cron job died exactly
this way with ``'AIAgent' object has no attribute '_claim_stream_writer'``.
The fence is only ever allowed to drop a *provably* superseded stream — never
the sole legitimate writer. So when the guard is unavailable (or raises), the
correct degradation is "no fence": keep streaming. These helpers make the
claim/check best-effort to guarantee that.
"""
from __future__ import annotations
import logging
from typing import Any
logger = logging.getLogger(__name__)
def claim_stream_writer(agent: Any) -> int:
"""Claim the delta sink for the calling stream attempt, best-effort.
Returns the agent's monotonic writer token when the fence is available, or
``0`` when the agent doesn't expose it (or the claim raised). A ``0`` token
pairs with :func:`stream_writer_is_current` always returning ``True``, so a
guard-less agent is simply never fenced instead of crashing the turn.
"""
claim = getattr(agent, "_claim_stream_writer", None)
if callable(claim):
try:
return int(claim())
except Exception:
logger.debug(
"stream single-writer: claim failed; proceeding unfenced",
exc_info=True,
)
return 0
def stream_writer_is_current(agent: Any, token: int) -> bool:
"""True when ``token`` is still the active writer, best-effort.
A falsy token (from a claim that no-oped) or an agent without the fence
means we cannot prove supersession, so the stream is treated as current and
never fenced. This preserves the single-writer invariant's one-way promise:
only a demonstrably stale writer is ever stopped.
"""
if not token:
return True
is_current = getattr(agent, "_stream_writer_is_current", None)
if callable(is_current):
try:
return bool(is_current(token))
except Exception:
logger.debug(
"stream single-writer: is_current check failed; treating as current",
exc_info=True,
)
return True
-507
View File
@@ -1,507 +0,0 @@
"""Surface-agnostic core for the ``/subscription`` TUI screen.
Companion to :mod:`agent.billing_view` — same fail-open philosophy: when not
logged in or the portal is unreachable, return a struct with ``logged_in=False``
and let the surface degrade gracefully (never crash). Money is decimal end-to-end
(server emits decimal strings); we only format for display.
The TUI ``SubscriptionOverlay`` drives the plan change in-terminal (V3): it
previews the effect, then schedules a downgrade / cancellation / resume
(chargeless) or applies an upgrade (charges the card on the subscription). The
portal deep-link (built locally from ``portal_url`` + ``org_id``) remains the
fallback for an upgrade that needs 3DS / was declined.
WS1 dependency: ``GET /api/billing/subscription`` is a NAS endpoint (WS1 Phase A).
Until it ships, the fail-open contract handles 404s — the builder returns
``logged_in=False`` and the surface degrades gracefully.
"""
from __future__ import annotations
import logging
import os
from dataclasses import dataclass
from decimal import Decimal
from typing import Any, Optional
from agent.billing_view import parse_money
logger = logging.getLogger(__name__)
# =============================================================================
# Parsed sub-structures
# =============================================================================
@dataclass(frozen=True)
class CurrentSubscription:
"""The user's active subscription. ``None`` (not this object) = no plan.
When present, ``tier_id`` / ``tier_name`` / ``monthly_credits`` /
``cycle_ends_at`` are always set (NAS guarantees a present ``current`` is a
fully-populated plan). Only ``credits_remaining`` and the cancel/downgrade
fields are optional.
"""
tier_id: Optional[str] = None
tier_name: Optional[str] = None
monthly_credits: Optional[Decimal] = None
credits_remaining: Optional[Decimal] = None
cycle_ends_at: Optional[str] = None # ISO
pending_downgrade_tier_name: Optional[str] = None
pending_downgrade_at: Optional[str] = None # ISO
cancel_at_period_end: bool = False
cancellation_effective_at: Optional[str] = None # ISO
@dataclass(frozen=True)
class SubscriptionTier:
"""A selectable plan in the catalog — one row of the in-terminal tier picker.
Mirrors NAS's ``SubscriptionTierOption``. ``is_current`` marks the active plan
(shown but not selectable); ``is_enabled=False`` is a grandfathered tier the
user is on but that can no longer be selected. ``tier_order`` sorts the picker
and drives the upgrade-vs-downgrade direction hint.
"""
tier_id: str
name: str
tier_order: int = 0
dollars_per_month: Optional[Decimal] = None
monthly_credits: Optional[Decimal] = None
is_current: bool = False
is_enabled: bool = True
@dataclass(frozen=True)
class SubscriptionChangePreview:
"""Parsed ``POST /api/billing/subscription/preview`` — what a change would do.
``effect`` is the disposition the commit would take:
- ``charge_now`` → an upgrade; ``amount_due_now_cents`` is the prorated charge.
- ``scheduled`` → a downgrade / same-price change at ``effective_at`` (period end).
- ``no_op`` → already on the target tier.
- ``blocked`` → the commit would be refused; ``reason`` says why.
"""
effect: str
reason: Optional[str] = None
current_tier_id: Optional[str] = None
current_tier_name: Optional[str] = None
target_tier_id: Optional[str] = None
target_tier_name: Optional[str] = None
monthly_credits_delta: Optional[Decimal] = None
amount_due_now_cents: Optional[int] = None
effective_at: Optional[str] = None # ISO
@dataclass(frozen=True)
class SubscriptionState:
"""Parsed ``GET /api/billing/subscription`` — the overview screen's data.
Fail-open: ``logged_in=False`` (and empty fields) when not logged in or the
portal is unreachable.
"""
logged_in: bool
org_name: Optional[str] = None
org_id: Optional[str] = None # org.id from the NAS response
role: Optional[str] = None # "OWNER" | "ADMIN" | "FINANCE_ADMIN" | "SECURITY_ADMIN" | "MEMBER"
can_change_plan_raw: Optional[bool] = None
context: str = "personal" # "personal" | "team"
current: Optional[CurrentSubscription] = None
tiers: tuple[SubscriptionTier, ...] = () # selectable catalog (picker)
portal_url: Optional[str] = None
# When the fetch failed (vs cleanly not-logged-in), the message for the surface.
error: Optional[str] = None
@property
def is_admin(self) -> bool:
"""Deprecated/display only — a legacy OWNER/ADMIN check.
NOT a capability check; use :attr:`can_change_plan` for gating billing
plan-change actions.
"""
return (self.role or "").upper() in ("OWNER", "ADMIN")
@property
def can_change_plan(self) -> bool:
"""Server capability when supplied; otherwise the legacy role fallback."""
if self.can_change_plan_raw is not None:
return self.can_change_plan_raw
return self.is_admin
# =============================================================================
# Payload parsing
# =============================================================================
def _parse_current(raw: Any) -> Optional[CurrentSubscription]:
# "No plan" is wire-represented as current:null (free personal OR team) —
# the old all-null-object shape is gone. A present current is a real plan,
# so guard on a real tier id and return None otherwise.
if not isinstance(raw, dict):
return None
tier_id = raw.get("tierId") or raw.get("id")
if not tier_id:
return None
return CurrentSubscription(
tier_id=tier_id,
tier_name=raw.get("tierName") or raw.get("name"),
monthly_credits=parse_money(raw.get("monthlyCredits")),
credits_remaining=parse_money(raw.get("creditsRemaining")),
cycle_ends_at=raw.get("cycleEndsAt"),
pending_downgrade_tier_name=raw.get("pendingDowngradeTierName"),
pending_downgrade_at=raw.get("pendingDowngradeAt"),
cancel_at_period_end=bool(raw.get("cancelAtPeriodEnd")),
cancellation_effective_at=raw.get("cancellationEffectiveAt") or None,
)
def _coalesce(*vals: Any) -> Any:
"""First non-``None`` value (preserves a legit ``0``/``0.0``, unlike ``or``).
NAS sends ``0`` for the free tier's ``tierOrder`` / ``dollarsPerMonth``; a plain
``x or default`` would drop those, so coalesce on ``None`` specifically.
"""
for v in vals:
if v is not None:
return v
return None
def _parse_tier(raw: Any) -> Optional[SubscriptionTier]:
"""Map one NAS ``SubscriptionTierOption`` dict into a :class:`SubscriptionTier`."""
if not isinstance(raw, dict):
return None
tier_id = raw.get("tierId") or raw.get("id")
if not tier_id:
return None
return SubscriptionTier(
tier_id=tier_id,
name=raw.get("name") or "",
tier_order=int(_coalesce(raw.get("tierOrder"), 0)),
dollars_per_month=parse_money(raw.get("dollarsPerMonthDisplay")),
monthly_credits=parse_money(raw.get("monthlyCredits")),
is_current=bool(raw.get("isCurrent")),
is_enabled=bool(_coalesce(raw.get("isEnabled"), True)),
)
def subscription_change_preview_from_payload(
payload: dict[str, Any],
) -> SubscriptionChangePreview:
"""Map a raw ``/subscription/preview`` JSON dict into :class:`SubscriptionChangePreview`."""
effect = payload.get("effect")
cents = payload.get("amountDueNowCents")
return SubscriptionChangePreview(
# An unrecognized/missing effect is treated as ``blocked`` — fail safe, never
# charge on a malformed quote.
effect=effect if isinstance(effect, str) else "blocked",
reason=payload.get("reason") or None,
current_tier_id=payload.get("currentTierId"),
current_tier_name=payload.get("currentTierName"),
target_tier_id=payload.get("targetTierId"),
target_tier_name=payload.get("targetTierName"),
monthly_credits_delta=parse_money(payload.get("monthlyCreditsDelta")),
amount_due_now_cents=int(cents) if isinstance(cents, (int, float)) else None,
effective_at=payload.get("effectiveAt") or None,
)
def subscription_state_from_payload(
payload: dict[str, Any], *, portal_url: Optional[str] = None
) -> SubscriptionState:
"""Map a raw ``/api/billing/subscription`` JSON dict into :class:`SubscriptionState`."""
raw_org = payload.get("org")
org: dict[str, Any] = raw_org if isinstance(raw_org, dict) else {}
raw_context = payload.get("context")
context = raw_context if raw_context in ("personal", "team") else "personal"
raw_tiers = payload.get("tiers")
tiers = (
tuple(t for t in (_parse_tier(x) for x in raw_tiers) if t is not None)
if isinstance(raw_tiers, list)
else ()
)
return SubscriptionState(
logged_in=True,
org_name=org.get("name"),
org_id=org.get("id") or None,
role=org.get("role"),
can_change_plan_raw=(
payload.get("canChangePlan")
if isinstance(payload.get("canChangePlan"), bool)
else None
),
context=context,
current=_parse_current(payload.get("current")),
tiers=tiers,
portal_url=portal_url,
)
# =============================================================================
# Fail-open builders (the surface front doors)
# =============================================================================
def build_subscription_state(*, timeout: float = 15.0) -> SubscriptionState:
"""Fetch + parse ``GET /api/billing/subscription``. Fail-open.
Returns ``SubscriptionState(logged_in=False)`` when not logged in. On a
portal/HTTP failure, returns ``logged_in=False`` with ``error`` set so the
surface can show a clear message rather than crashing.
Dev override: when ``HERMES_DEV_SUBSCRIPTION_FIXTURE`` names a fixture state,
``/subscription`` renders from that fixture instead of the real portal — so
every plan/cancel/downgrade/team/not-admin state is testable on both
the CLI and TUI without a live account. Throwaway scaffolding; see
:func:`dev_fixture_subscription_state`.
"""
fixture = dev_fixture_subscription_state()
if fixture is not None:
return fixture
try:
from hermes_cli.nous_billing import (
BillingAuthError,
BillingError,
_absolutize_portal_url,
get_subscription_state,
resolve_portal_base_url,
)
except Exception:
return SubscriptionState(logged_in=False, error="billing client unavailable")
try:
payload = get_subscription_state(timeout=timeout)
except BillingAuthError:
return SubscriptionState(logged_in=False)
except BillingError as exc:
logger.debug("subscription ▸ /state fetch failed (fail-open)", exc_info=True)
return SubscriptionState(logged_in=False, error=str(exc))
except Exception:
logger.debug("subscription ▸ /state unexpected error (fail-open)", exc_info=True)
return SubscriptionState(logged_in=False, error="could not load subscription state")
raw_portal = payload.get("portalUrl") if isinstance(payload, dict) else None
portal_url = _absolutize_portal_url(raw_portal) if raw_portal else None
if not portal_url:
try:
portal_url = resolve_portal_base_url()
except Exception:
portal_url = None
return subscription_state_from_payload(payload, portal_url=portal_url)
def subscription_manage_url(
state: SubscriptionState, tier_id: Optional[str] = None
) -> Optional[str]:
"""Build ``{portal_origin}/manage-subscription?org_id=<id>[&plan=<tier_id>]``.
Mirrors the TUI's ``buildManageUrl`` (``subscription.ts``): the deep-link
target is NAS's OWN ``/manage-subscription`` page (NOT the Stripe Billing
Portal — decided Jun 23), which routes upgrade→Checkout / downgrade→scheduled
internally. ``org_id`` pins the page to the right account in multi-org
situations. Returns ``None`` when no portal URL is resolvable.
``tier_id`` (the stable ``tiers[]`` id, never a name/slug) is appended as
``plan=`` so the portal preselects the picked plan — only for a NEW
subscription / upgrade the user chose. The portal validates it and simply
ignores an unknown tier, so the CLI appends unconditionally when a tier was
picked (parity with the TUI's ``?plan=``).
"""
from urllib.parse import urlencode, urlsplit, urlunsplit
if not state.portal_url:
return None
try:
parts = urlsplit(state.portal_url)
except Exception:
return None
if parts.scheme not in ("http", "https") or not parts.netloc:
return None
from urllib.parse import parse_qsl
# Preserve unrelated portal query params; org_id / plan are contract-owned
# (org_id before plan — insertion order is the emitted query order).
params = dict(parse_qsl(parts.query, keep_blank_values=True))
params.pop("org_id", None)
params.pop("plan", None)
if state.org_id:
params["org_id"] = state.org_id
if tier_id:
params["plan"] = tier_id
query = urlencode(params)
return urlunsplit((parts.scheme, parts.netloc, "/manage-subscription", query, ""))
# =============================================================================
# Shared plan-catalog helpers (consumed by the CLI Free catalog + paid picker)
# =============================================================================
def _format_dollars_grouped(value: Optional[Decimal]) -> str:
"""``$1,000`` / ``$1,234.50`` — the whole-vs-fractional rule of
``billing_view.format_money`` but thousands-grouped, matching the TUI's
``toLocaleString('en-US')``.
The shared ``format_money`` is intentionally ungrouped (and asserted so across
other surfaces), so plan-catalog rows group locally to mirror the TUI.
"""
if value is None:
return "—"
if value == value.to_integral_value():
return f"${format(value.to_integral_value(), ',f')}"
return f"${format(value.quantize(Decimal('0.01')), ',f')}"
def selectable_tiers(state: SubscriptionState) -> list[SubscriptionTier]:
"""Enabled paid tiers other than the current plan, cheapest first.
One derivation shared by the CLI Free catalog and the paid change picker:
``is_enabled and not is_current and tier_order > 0`` (free / no-sub excluded —
dropping to free is a cancellation), sorted by ``tier_order``.
"""
return sorted(
(
t
for t in (state.tiers or ())
if t.is_enabled and not t.is_current and (t.tier_order or 0) > 0
),
key=lambda t: t.tier_order or 0,
)
def format_tier_row(tier: SubscriptionTier) -> str:
"""``name · $X/mo[ · $Y credits/mo]`` — the shared plan-catalog row.
Mirrors the TUI Free rows (``subscriptionOverlay.tsx``): thousands-grouped
money, and the ``$Y credits/mo`` suffix ONLY when monthly credits are present
and > 0 (a ``None`` / zero-credits tier hides it — never ``· — credits/mo`` or
``· $0 credits/mo``).
"""
row = f"{tier.name} · {_format_dollars_grouped(tier.dollars_per_month)}/mo"
mc = tier.monthly_credits
if mc is not None and mc > 0:
row += f" · {_format_dollars_grouped(mc)} credits/mo"
return row
def is_upgrade(state: SubscriptionState, tier_id: str) -> bool:
"""True when ``tier_id`` ranks above the current plan by ``tier_order``.
Prefers the active subscription's tier; falls back to the ``tiers[]``
``is_current`` marker (what the picker derives from), else 0 (free).
"""
orders = {t.tier_id: (t.tier_order or 0) for t in (state.tiers or ())}
cur_id = state.current.tier_id if state.current else None
if cur_id is not None and cur_id in orders:
cur_order = orders[cur_id]
else:
cur_order = next((t.tier_order or 0 for t in (state.tiers or ()) if t.is_current), 0)
return orders.get(tier_id, 0) > cur_order
# =============================================================================
# Dev fixtures (throwaway scaffolding — env-var driven, no live portal)
# =============================================================================
_DEV_FIXTURE_PORTAL = "https://portal.nousresearch.com/billing"
def _dev_current(**over: Any) -> CurrentSubscription:
base: dict[str, Any] = dict(
tier_id="plus",
tier_name="Plus",
monthly_credits=Decimal("1000"),
credits_remaining=Decimal("420"),
cycle_ends_at="2026-07-01",
)
base.update(over)
return CurrentSubscription(**base)
def _dev_tiers(current_id: Optional[str]) -> tuple[SubscriptionTier, ...]:
"""A sample plan catalog for fixtures (marks ``current_id`` as the active tier)."""
specs = (
("free", "Free", 0, "0", "0"),
("plus", "Plus", 1, "20", "1000"),
("super", "Super", 2, "40", "3000"),
("ultra", "Ultra", 3, "80", "7000"),
)
return tuple(
SubscriptionTier(
tier_id=tid,
name=name,
tier_order=order,
dollars_per_month=parse_money(dpm),
monthly_credits=parse_money(mc),
is_current=(tid == current_id),
is_enabled=True,
)
for tid, name, order, dpm, mc in specs
)
def dev_fixture_subscription_state() -> Optional[SubscriptionState]:
"""Return a fixture :class:`SubscriptionState` for ``HERMES_DEV_SUBSCRIPTION_FIXTURE``.
Lets every CLI/TUI subscription state be exercised without a live portal:
free | mid | top | not-admin | downgrade | cancel | team |
logged-out
Returns ``None`` when the env var is unset/empty (the real portal path runs).
Throwaway scaffolding — mirrors ``HERMES_DEV_CREDITS_FIXTURE``.
"""
name = (os.getenv("HERMES_DEV_SUBSCRIPTION_FIXTURE") or "").strip().lower()
if not name:
return None
common = dict(org_name="Acme Inc", org_id="org_acme", role="OWNER", portal_url=_DEV_FIXTURE_PORTAL)
if name in ("logged-out", "logged_out", "loggedout"):
return SubscriptionState(logged_in=False)
if name == "free":
return SubscriptionState(logged_in=True, current=None, tiers=_dev_tiers(None), **common)
if name in ("mid", "mid-tier"):
return SubscriptionState(logged_in=True, current=_dev_current(), tiers=_dev_tiers("plus"), **common)
if name in ("top", "top-tier"):
return SubscriptionState(
logged_in=True,
current=_dev_current(tier_id="ultra", tier_name="Ultra", monthly_credits=Decimal("7000"), credits_remaining=Decimal("5000")),
tiers=_dev_tiers("ultra"),
**common,
)
if name in ("not-admin", "member"):
return SubscriptionState(logged_in=True, current=_dev_current(), tiers=_dev_tiers("plus"), **{**common, "role": "MEMBER"})
if name == "downgrade":
return SubscriptionState(
logged_in=True,
current=_dev_current(tier_id="super", tier_name="Super", monthly_credits=Decimal("3000"), credits_remaining=Decimal("1500"), pending_downgrade_tier_name="Plus", pending_downgrade_at="2026-07-15"),
tiers=_dev_tiers("super"),
**common,
)
if name == "cancel":
return SubscriptionState(
logged_in=True,
current=_dev_current(cancel_at_period_end=True, cancellation_effective_at="2026-07-01"),
tiers=_dev_tiers("plus"),
**common,
)
if name == "team":
return SubscriptionState(logged_in=True, context="team", current=None, org_name="Acme Engineering", org_id="org_eng", role="OWNER", portal_url=_DEV_FIXTURE_PORTAL)
# Unknown name → behave as logged-out so the misconfiguration is visible.
return SubscriptionState(logged_in=False, error=f"unknown HERMES_DEV_SUBSCRIPTION_FIXTURE: {name}")
+5 -28
View File
@@ -40,13 +40,11 @@ from agent.prompt_builder import (
SKILLS_GUIDANCE,
STEER_CHANNEL_NOTE,
TASK_COMPLETION_GUIDANCE,
TELEGRAM_RICH_MESSAGES_HINT,
TOOL_USE_ENFORCEMENT_GUIDANCE,
TOOL_USE_ENFORCEMENT_MODELS,
drain_truncation_warnings,
)
from agent.runtime_cwd import resolve_context_cwd
from hermes_constants import get_hermes_home
from utils import is_truthy_value
@@ -396,7 +394,7 @@ def build_system_prompt_parts(agent: Any, system_message: Optional[str] = None)
if active_profile == "default":
stable_parts.append(
"Active Hermes profile: default. Other profiles (if any) live "
"under " + str(get_hermes_home()) + "/profiles/<name>/. Each profile has its own "
"under ~/.hermes/profiles/<name>/. Each profile has its own "
"skills/, plugins/, cron/, and memories/ that affect a different "
"session than this one. Do not modify another profile's "
"skills/plugins/cron/memories unless the user explicitly directs "
@@ -405,9 +403,9 @@ def build_system_prompt_parts(agent: Any, system_message: Optional[str] = None)
else:
stable_parts.append(
f"Active Hermes profile: {active_profile}. This session reads "
f"and writes {get_hermes_home()}/profiles/{active_profile}/. The default "
f"profile's data lives at {get_hermes_home()}/skills/, {get_hermes_home()}/plugins/, "
f"{get_hermes_home()}/cron/, {get_hermes_home()}/memories/ — those belong to a "
f"and writes ~/.hermes/profiles/{active_profile}/. The default "
f"profile's data lives at ~/.hermes/skills/, ~/.hermes/plugins/, "
f"~/.hermes/cron/, ~/.hermes/memories/ — those belong to a "
f"different session run from a different shell. Do NOT modify "
f"another profile's skills/plugins/cron/memories unless the user "
f"explicitly directs you to. The cross-profile write guard will "
@@ -431,20 +429,6 @@ def build_system_prompt_parts(agent: Any, system_message: Optional[str] = None)
except Exception:
pass
# For Telegram: append the rich-messages extension only when the user has
# opted in to ``platforms.telegram.extra.rich_messages: true``. The base
# hint covers MarkdownV2-compatible constructs; the extension adds Bot API
# 10.1 guidance (tables, task lists, math, collapsible details, etc.).
if platform_key == "telegram" and _default_hint:
try:
from hermes_cli.config import load_config_readonly
_cfg = load_config_readonly()
_tg_extra = ((_cfg.get("platforms") or {}).get("telegram") or {}).get("extra") or {}
if _tg_extra.get("rich_messages"):
_default_hint = _default_hint.rstrip() + " " + TELEGRAM_RICH_MESSAGES_HINT
except Exception:
pass # Config read failure — fall back to base hint only
_effective_hint = _resolve_platform_hint(agent, platform_key, _default_hint)
if platform_key == "tui" and _effective_hint:
_effective_hint = _tui_embedded_pane_clarifier(_effective_hint)
@@ -464,16 +448,9 @@ def build_system_prompt_parts(agent: Any, system_message: Optional[str] = None)
# CLI), None lets build_context_files_prompt fall back to the launch
# dir — the user's real cwd there, but the install dir for the gateway
# daemon, which is why the gateway sets TERMINAL_CWD.
#
# allow_install_tree_fallback: for cli/tui the launch dir IS the
# user's shell cwd, so an in-tree fallback is a deliberate choice
# (developing Hermes). Every other surface (desktop chat panel,
# gateway daemons) self-spawns into the install tree, where the
# fallback would inject this repo's contributor AGENTS.md (#64590).
context_files_prompt = _r.build_context_files_prompt(
cwd=resolve_context_cwd(), skip_soul=_soul_loaded,
context_length=_ctx_len,
allow_install_tree_fallback=agent.platform in ("cli", "tui"))
context_length=_ctx_len)
if context_files_prompt:
context_parts.append(context_files_prompt)
+4 -14
View File
@@ -208,29 +208,19 @@ class StreamingThinkScrubber:
discarded — leaking partial reasoning is worse than a
truncated answer. Otherwise the held-back partial-tag tail is
emitted verbatim (it turned out not to be a real tag prefix).
Always treats the next ``feed()`` as a fresh stream boundary.
Intra-turn retries (thinking-only prefill, empty-response
retry) flush then stream again without calling ``reset()``;
leaving ``_last_emitted_ended_newline`` False made a new
stream's opening ``<think>`` look mid-line and leak into the
visible reply.
"""
if self._in_block:
self._buf = ""
self._in_block = False
# Next feed() is a new stream — start-of-stream is a boundary.
self._last_emitted_ended_newline = True
return ""
tail = self._buf
self._buf = ""
# Same for the non-block path: do NOT derive the boundary flag
# from the flushed tail (e.g. a held-back '<'). End-of-stream
# means the next feed() starts a new model response.
self._last_emitted_ended_newline = True
if not tail:
return ""
return self._strip_orphan_close_tags(tail)
tail = self._strip_orphan_close_tags(tail)
if tail:
self._last_emitted_ended_newline = tail.endswith("\n")
return tail
# ── internal helpers ───────────────────────────────────────────────
+4 -176
View File
@@ -19,12 +19,6 @@ logger = logging.getLogger(__name__)
FailureCallback = Callable[[str, BaseException], None]
TitleCallback = Callable[[str], None]
# Validation callback: () -> bool. Called right before the LLM request in
# generate_title(). Return False to skip — e.g. the user switched models
# after this background thread captured its runtime snapshot, and sending
# the request would reload a model the runtime already evicted (#19027).
RuntimeValidator = Callable[[], bool]
_TITLE_PROMPT = (
"Generate a short, descriptive title (3-7 words) for a conversation that starts with the "
"following exchange. The title should capture the main topic or intent. "
@@ -54,30 +48,12 @@ def _title_language() -> str:
return ""
def _auto_title_enabled() -> bool:
"""Return whether automatic session title generation is enabled."""
try:
# Lazy imports, matching _title_language(): title_generator is imported
# from agent code paths where a module-level hermes_cli import risks
# circularity, and the read-only loader avoids config-migration writes.
from hermes_cli.config import load_config_readonly
from utils import is_truthy_value
config = load_config_readonly()
title_config = (config.get("auxiliary") or {}).get("title_generation") or {}
return is_truthy_value(title_config.get("enabled"), default=True)
except Exception:
logger.debug("Failed to read title_generation.enabled", exc_info=True)
return True
def generate_title(
user_message: str,
assistant_response: str,
timeout: Optional[float] = None,
failure_callback: Optional[FailureCallback] = None,
main_runtime: dict = None,
runtime_validator: Optional[RuntimeValidator] = None,
) -> Optional[str]:
"""Generate a session title from the first exchange.
@@ -89,26 +65,7 @@ def generate_title(
auxiliary call raises — the caller typically wires this to
``AIAgent._emit_auxiliary_failure`` so the user sees a warning instead
of silently accumulating untitled sessions.
``runtime_validator`` is called right before the LLM request. If it
returns False (e.g. the user's model was switched since the background
thread captured its runtime snapshot), the call is skipped silently —
no request is sent, so a stale title request can't reload a model the
runtime already unloaded (#19027).
"""
if not _auto_title_enabled():
logger.debug("Auto-title skipped: auxiliary.title_generation.enabled=false")
return None
if runtime_validator is not None:
try:
if not runtime_validator():
logger.debug("Title generation skipped: runtime validator returned False")
return None
except Exception:
# Fail open: a broken validator must not disable titling.
logger.debug("Title runtime validator raised; proceeding", exc_info=True)
# Truncate long messages to keep the request small
user_snippet = user_message[:500] if user_message else ""
assistant_snippet = assistant_response[:500] if assistant_response else ""
@@ -160,53 +117,6 @@ def generate_title(
return None
def _persist_session_title(session_db, session_id, title):
"""Persist a generated title, recovering from duplicate-title collisions.
The write goes through ``set_auto_title_if_empty`` (predicate + write in
one transaction) so a manual ``/title`` set while LLM generation was in
flight is never overwritten — a plain ``set_session_title`` fallback keeps
older stores working. ``set_session_title`` raises ValueError when the
title would collide with another session (the unique-title index). Rather
than swallow it and leave the session untitled (#50537), append a #N
suffix via get_next_title_in_lineage() when the store supports lineage
dedup; otherwise re-raise so the caller can decide.
Returns the title actually persisted, or None when a concurrent manual
title won the race (nothing was written).
"""
atomic_fn = getattr(session_db, "set_auto_title_if_empty", None)
def _set(t):
if atomic_fn is not None:
if not atomic_fn(session_id, t):
# Predicate failed: a title appeared while generation was in
# flight (manual /title wins), or the session vanished.
logger.debug(
"Skipping auto-generated session title because a title "
"was set while generation was in flight"
)
return None
return t
ok = session_db.set_session_title(session_id, t)
if ok is False:
raise RuntimeError(
f"session {session_id} not found when storing title"
)
return t
try:
return _set(title)
except ValueError:
next_title_fn = getattr(session_db, "get_next_title_in_lineage", None)
if next_title_fn is None:
raise
deduped = next_title_fn(title)
if not deduped or deduped == title:
raise
return _set(deduped)
def auto_title_session(
session_db,
session_id: str,
@@ -215,7 +125,6 @@ def auto_title_session(
failure_callback: Optional[FailureCallback] = None,
main_runtime: dict = None,
title_callback: Optional[TitleCallback] = None,
runtime_validator: Optional[RuntimeValidator] = None,
) -> None:
"""Generate and set a session title if one doesn't already exist.
@@ -224,55 +133,7 @@ def auto_title_session(
- session_db is None
- session already has a title (user-set or previously auto-generated)
- title generation fails
- runtime_validator returns False (model was switched)
Never lets an exception escape: this is a daemon-thread target, and an
escaping exception would spray a raw traceback into the user's terminal
via the default threading excepthook. The canonical trigger is the
post-``hermes update`` stale-module window, where this function's lazy
imports read NEW source from disk while already-cached modules
(``agent.portal_tags`` etc.) are still the OLD version — the resulting
ImportError repeats on every auto-title attempt until the long-running
process restarts.
"""
try:
_auto_title_session(
session_db,
session_id,
user_message,
assistant_response,
failure_callback=failure_callback,
main_runtime=main_runtime,
title_callback=title_callback,
runtime_validator=runtime_validator,
)
except Exception as e:
# WARNING (not debug) so operators see it in agent.log; the message
# names the likely cause so "restart the process" is discoverable.
logger.warning(
"Auto-title failed (harmless; if this started after an update, "
"restart the running Hermes process): %s",
e,
)
logger.debug("Auto-title traceback", exc_info=True)
if failure_callback is not None:
try:
failure_callback("title generation", e)
except Exception:
logger.debug("Auto-title failure_callback raised", exc_info=True)
def _auto_title_session(
session_db,
session_id: str,
user_message: str,
assistant_response: str,
failure_callback: Optional[FailureCallback] = None,
main_runtime: dict = None,
title_callback: Optional[TitleCallback] = None,
runtime_validator: Optional[RuntimeValidator] = None,
) -> None:
"""Body of :func:`auto_title_session` — see its docstring."""
if not session_db or not session_id:
return
@@ -284,43 +145,18 @@ def _auto_title_session(
except Exception:
return
# This runs on a bare daemon thread spawned AFTER the turn's ambient
# conversation context was reset, so publish it here from the session id
# we already hold — the title-generation LLM call then carries the same
# ``conversation=`` Portal tag as the turn it titles. Root-of-lineage for
# consistency with the agent loop (a no-op on first exchange, where
# titling happens, but correct if this ever runs on a continuation).
from agent.aux_accounting import set_accounting_context
from agent.portal_tags import set_conversation_context
conversation_id = session_id
try:
conversation_id = session_db.get_conversation_root(session_id) or session_id
except Exception:
pass
set_conversation_context(conversation_id)
# Same for the accounting context, so the title call's token usage is
# recorded against this session (task='title_generation', #23270).
set_accounting_context(session_db, session_id)
title = generate_title(
user_message,
assistant_response,
failure_callback=failure_callback,
main_runtime=main_runtime,
runtime_validator=runtime_validator,
user_message, assistant_response, failure_callback=failure_callback, main_runtime=main_runtime
)
if not title:
return
try:
persisted = _persist_session_title(session_db, session_id, title)
if persisted is None:
return
logger.debug("Auto-generated session title: %s", persisted)
session_db.set_session_title(session_id, title)
logger.debug("Auto-generated session title: %s", title)
if title_callback is not None:
try:
title_callback(persisted)
title_callback(title)
except Exception:
logger.debug("Auto-title callback failed", exc_info=True)
except Exception as e:
@@ -336,7 +172,6 @@ def maybe_auto_title(
failure_callback: Optional[FailureCallback] = None,
main_runtime: dict = None,
title_callback: Optional[TitleCallback] = None,
runtime_validator: Optional[RuntimeValidator] = None,
) -> None:
"""Fire-and-forget title generation after the first exchange.
@@ -355,12 +190,6 @@ def maybe_auto_title(
if user_msg_count > 2:
return
# Config read comes after the cheap first-exchange guard so the file
# isn't touched on every subsequent turn of a long session.
if not _auto_title_enabled():
logger.debug("Auto-title skipped: auxiliary.title_generation.enabled=false")
return
thread = threading.Thread(
target=auto_title_session,
args=(session_db, session_id, user_message, assistant_response),
@@ -368,7 +197,6 @@ def maybe_auto_title(
"failure_callback": failure_callback,
"main_runtime": main_runtime,
"title_callback": title_callback,
"runtime_validator": runtime_validator,
},
daemon=True,
name="auto-title",
+30 -135
View File
@@ -102,149 +102,54 @@ def _is_mcp_tool_parallel_safe(tool_name: str) -> bool:
return False
def _plan_tool_batch_segments(tool_calls, *, execution_cwd: Optional[Path] = None) -> List[tuple]:
"""Split a tool-call batch into ordered ``(kind, calls)`` segments.
def _should_parallelize_tool_batch(tool_calls) -> bool:
"""Return True when a tool-call batch is safe to run concurrently."""
if len(tool_calls) <= 1:
return False
``kind`` is ``"parallel"`` (a maximal contiguous run of parallel-safe
calls) or ``"sequential"`` (one or more barrier calls that must run
in-order on the sequential path). Segments preserve the model's
original call order exactly — a later call never crosses an earlier
barrier — so tool-result ordering and side-effect boundaries are
identical to fully-sequential execution. The per-call safety rules
are the same ones the old all-or-nothing gate applied to the whole
batch:
tool_names = [tc.function.name for tc in tool_calls]
if any(name in _NEVER_PARALLEL_TOOLS for name in tool_names):
return False
* ``_NEVER_PARALLEL_TOOLS`` (interactive tools) → barrier.
* Unparseable / non-dict arguments → barrier.
* Path-scoped tools (``read_file``/``write_file``/``patch``) join a
parallel run only when their target path does not overlap another
path already reserved in the same run; an overlap closes the run so
the conflicting call starts a NEW run after the first completes.
* Anything not in ``_PARALLEL_SAFE_TOOLS`` and not an opted-in MCP
tool → barrier.
Parallel runs shorter than two calls are demoted to sequential (no
concurrency win, and the sequential executor owns the richer inline
dispatch), and adjacent sequential segments are merged.
"""
segments: list[list] = [] # [kind, calls] pairs, normalized to tuples on return
current: list = []
reserved_paths: list[Path] = []
def _close_parallel() -> None:
nonlocal current, reserved_paths
if current:
segments.append(["parallel", current])
current = []
reserved_paths = []
def _add_sequential(tc) -> None:
_close_parallel()
if segments and segments[-1][0] == "sequential":
segments[-1][1].append(tc)
else:
segments.append(["sequential", [tc]])
for tool_call in tool_calls:
tool_name = tool_call.function.name
if tool_name in _NEVER_PARALLEL_TOOLS:
_add_sequential(tool_call)
continue
try:
function_args = json.loads(tool_call.function.arguments)
except Exception:
_raw = tool_call.function.arguments
logging.debug(
"Could not parse args for %s — treating as sequential barrier; raw=%s",
"Could not parse args for %s — defaulting to sequential; raw=%s",
tool_name,
_raw[:200] if isinstance(_raw, str) else repr(_raw)[:200],
tool_call.function.arguments[:200],
)
_add_sequential(tool_call)
continue
return False
if not isinstance(function_args, dict):
logging.debug(
"Non-dict args for %s (%s) — treating as sequential barrier",
"Non-dict args for %s (%s) — defaulting to sequential",
tool_name,
type(function_args).__name__,
)
_add_sequential(tool_call)
continue
return False
if tool_name in _PATH_SCOPED_TOOLS:
scoped_path = _extract_parallel_scope_path(tool_name, function_args, execution_cwd=execution_cwd)
scoped_path = _extract_parallel_scope_path(tool_name, function_args)
if scoped_path is None:
_add_sequential(tool_call)
continue
return False
if any(_paths_overlap(scoped_path, existing) for existing in reserved_paths):
# Same-subtree conflict inside this run: close it so this
# call starts a fresh run AFTER the conflicting one lands.
_close_parallel()
return False
reserved_paths.append(scoped_path)
current.append(tool_call)
continue
if tool_name in _PARALLEL_SAFE_TOOLS or _is_mcp_tool_parallel_safe(tool_name):
current.append(tool_call)
continue
if tool_name not in _PARALLEL_SAFE_TOOLS:
# Check if it's an MCP tool from a server that opted into parallel calls.
if not _is_mcp_tool_parallel_safe(tool_name):
return False
_add_sequential(tool_call)
_close_parallel()
normalized: list[list] = []
for kind, calls in segments:
if kind == "parallel" and len(calls) < 2:
kind = "sequential"
if normalized and normalized[-1][0] == "sequential" and kind == "sequential":
normalized[-1][1].extend(calls)
else:
normalized.append([kind, calls])
return [(kind, calls) for kind, calls in normalized]
return True
def _should_parallelize_tool_batch(tool_calls) -> bool:
"""Return True when the WHOLE tool-call batch is safe to run concurrently.
Thin view over ``_plan_tool_batch_segments`` kept for callers/tests that
only care about the homogeneous case: True iff the planner produces a
single all-parallel segment.
"""
if len(tool_calls) <= 1:
return False
segments = _plan_tool_batch_segments(tool_calls)
return len(segments) == 1 and segments[0][0] == "parallel"
def _canonical_path(raw_path: str, execution_cwd: Optional[Path] = None) -> Path:
"""Return a canonical, OS-aware path for overlap detection.
Uses ``os.path.realpath`` to resolve symlinks on existing path components
and ``os.path.normcase`` for case-insensitive platforms (Windows).
Falls back to ``Path.cwd()`` when *execution_cwd* is not supplied.
"""
expanded = Path(raw_path).expanduser()
base = execution_cwd if execution_cwd is not None else Path.cwd()
candidate = expanded if expanded.is_absolute() else base / expanded
# realpath resolves symlinks on path components that exist; for
# not-yet-created files it canonicalises as far as possible.
resolved = os.path.normcase(os.path.realpath(os.path.abspath(str(candidate))))
return Path(resolved)
def _extract_parallel_scope_path(
tool_name: str,
function_args: dict,
execution_cwd: Optional[Path] = None,
) -> Optional[Path]:
"""Return the canonical file target for path-scoped tools.
*execution_cwd* should be the working directory that the tool will
actually use at runtime. When omitted the process cwd is used,
which may differ from the tool execution environment on some
platforms (e.g. WSL, sandboxed sub-processes).
"""
def _extract_parallel_scope_path(tool_name: str, function_args: dict) -> Optional[Path]:
"""Return the normalized file target for path-scoped tools."""
if tool_name not in _PATH_SCOPED_TOOLS:
return None
@@ -252,16 +157,16 @@ def _extract_parallel_scope_path(
if not isinstance(raw_path, str) or not raw_path.strip():
return None
return _canonical_path(raw_path, execution_cwd)
expanded = Path(raw_path).expanduser()
if expanded.is_absolute():
return Path(os.path.abspath(str(expanded)))
# Avoid resolve(); the file may not exist yet.
return Path(os.path.abspath(str(Path.cwd() / expanded)))
def _paths_overlap(left: Path, right: Path) -> bool:
"""Return True when two paths may refer to the same subtree.
Both *left* and *right* must already be canonical (as returned by
``_extract_parallel_scope_path`` / ``_canonical_path``) so that
symlink aliases and case differences are already normalised.
"""
"""Return True when two paths may refer to the same subtree."""
left_parts = left.parts
right_parts = right.parts
if not left_parts or not right_parts:
@@ -454,13 +359,7 @@ def _trajectory_normalize_msg(msg: Dict[str, Any]) -> Dict[str, Any]:
return msg
def make_tool_result_message(
name: str,
content: Any,
tool_call_id: str,
*,
effect_disposition: str | None = None,
) -> dict:
def make_tool_result_message(name: str, content: Any, tool_call_id: str) -> dict:
"""Build a tool-result message dict with both the OpenAI-format ``name``
field (required by the wire format and provider adapters) and the internal
``tool_name`` field (written to the session DB messages table).
@@ -495,8 +394,6 @@ def make_tool_result_message(
else:
if risk_metadata is not None:
message["_tool_output_risk"] = risk_metadata
if effect_disposition is not None:
message["effect_disposition"] = effect_disposition
return message
@@ -637,9 +534,7 @@ __all__ = [
"_DESTRUCTIVE_PATTERNS",
"_REDIRECT_OVERWRITE",
"_is_destructive_command",
"_plan_tool_batch_segments",
"_should_parallelize_tool_batch",
"_canonical_path",
"_extract_parallel_scope_path",
"_paths_overlap",
"_is_multimodal_tool_result",
+18 -121
View File
@@ -14,7 +14,6 @@ from __future__ import annotations
import concurrent.futures
import json
from pathlib import Path
import logging
import os
import random
@@ -37,7 +36,6 @@ from agent.tool_dispatch_helpers import (
_is_multimodal_tool_result,
_multimodal_text_summary,
_append_subdir_hint_to_multimodal,
_plan_tool_batch_segments,
make_tool_result_message,
)
from tools.terminal_tool import (
@@ -53,28 +51,6 @@ from tools.budget_config import BudgetConfig, DEFAULT_BUDGET, budget_for_context
logger = logging.getLogger(__name__)
def _ensure_file_checkpoint(
agent,
function_name: str,
function_args: dict,
effective_task_id: str,
) -> None:
"""Checkpoint the same workspace path that the file tool will mutate."""
file_path = function_args.get("path", "")
if not file_path:
return
# File tools resolve relative paths against the task's live/session cwd,
# which can differ from the Hermes process cwd (notably in Docker). Resolve
# through that same path pipeline before asking the checkpoint manager to
# discover the project root.
from tools.file_tools import _resolve_path_for_task
resolved_path = _resolve_path_for_task(file_path, effective_task_id or "default")
work_dir = agent._checkpoint_mgr.get_working_dir_for_path(str(resolved_path))
agent._checkpoint_mgr.ensure_checkpoint(work_dir, f"before {function_name}")
def _budget_for_agent(agent) -> BudgetConfig:
"""Resolve a tool-result BudgetConfig scaled to the agent's context window.
@@ -346,15 +322,11 @@ def _run_agent_tool_execution_middleware(
return result, observed_args
def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0, *, finalize: bool = True) -> None:
def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0) -> None:
"""Execute multiple tool calls concurrently using a thread pool.
Results are collected in the original tool-call order and appended to
messages so the API sees them in the expected sequence.
``finalize=False`` skips the end-of-batch aggregate budget enforcement
and /steer injection — used when this call is one segment of a larger
mixed batch and the segmented dispatcher owns the turn-end work.
"""
tool_calls = assistant_message.tool_calls
num_tools = len(tool_calls)
@@ -371,7 +343,6 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
tc.function.name,
f"[Tool execution cancelled — {tc.function.name} was skipped due to user interrupt]",
tc.id,
effect_disposition="none",
))
_flush_session_db_after_tool_progress(
agent,
@@ -524,12 +495,10 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
# Checkpoint for file-mutating tools
if function_name in {"write_file", "patch"} and agent._checkpoint_mgr.enabled:
try:
_ensure_file_checkpoint(
agent,
function_name,
function_args,
effective_task_id,
)
file_path = function_args.get("path", "")
if file_path:
work_dir = agent._checkpoint_mgr.get_working_dir_for_path(file_path)
agent._checkpoint_mgr.ensure_checkpoint(work_dir, f"before {function_name}")
except Exception:
pass
@@ -858,11 +827,9 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
# deadline snapshot (timed_out_indices, taken from not_done) and this
# loop. Prefer that real result over a fabricated timeout message — the
# tool genuinely succeeded, just slightly late.
effect_disposition = None
if i in timed_out_indices and r is None:
suffix = f"{timeout_s:.1f}s" if timeout_s is not None else "the configured timeout"
function_result = f"Error executing tool '{name}': timed out after {suffix}"
effect_disposition = "unknown"
_emit_terminal_post_tool_call(
agent,
function_name=name,
@@ -909,8 +876,6 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
tool_duration = 0.0
else:
function_name, function_args, function_result, tool_duration, is_error, blocked, middleware_trace = r
if blocked:
effect_disposition = "none"
if not blocked:
function_result = agent._append_guardrail_observation(
@@ -999,12 +964,7 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
# image tool result never poisons canonical session history.
# String results pass through unchanged.
_tool_content = agent._tool_result_content_for_active_model(name, function_result)
tool_message = make_tool_result_message(
name,
_tool_content,
tc.id,
effect_disposition=effect_disposition,
)
tool_message = make_tool_result_message(name, _tool_content, tc.id)
messages.append(tool_message)
risk_metadata = tool_message.get("_tool_output_risk")
if (
@@ -1036,7 +996,7 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
# ── Per-turn aggregate budget enforcement ─────────────────────────
num_tools = len(parsed_calls)
if finalize and num_tools > 0:
if num_tools > 0:
turn_tool_msgs = messages[-num_tools:]
enforce_turn_budget(turn_tool_msgs, env=get_active_env(effective_task_id), config=_tool_budget)
@@ -1044,18 +1004,13 @@ def execute_tool_calls_concurrent(agent, assistant_message, messages: list, effe
# Append any pending user steer text to the last tool result so the
# agent sees it on its next iteration. Runs AFTER budget enforcement
# so the steer marker is never truncated. See steer() for details.
if finalize and num_tools > 0:
if num_tools > 0:
agent._apply_pending_steer_to_tool_results(messages, num_tools)
def execute_tool_calls_sequential(agent, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0, *, finalize: bool = True) -> None:
"""Execute tool calls sequentially (original behavior). Used for single calls or interactive tools.
``finalize=False`` skips the end-of-batch aggregate budget enforcement
and /steer injection — used when this call is one segment of a larger
mixed batch and the segmented dispatcher owns the turn-end work.
"""
def execute_tool_calls_sequential(agent, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0) -> None:
"""Execute tool calls sequentially (original behavior). Used for single calls or interactive tools."""
# Resolve the context-scaled tool-output budget once per turn.
_tool_budget = _budget_for_agent(agent)
for i, tool_call in enumerate(assistant_message.tool_calls, 1):
@@ -1072,7 +1027,6 @@ def execute_tool_calls_sequential(agent, assistant_message, messages: list, effe
skipped_name,
f"[Tool execution cancelled — {skipped_name} was skipped due to user interrupt]",
skipped_tc.id,
effect_disposition="none",
))
_flush_session_db_after_tool_progress(
agent,
@@ -1212,12 +1166,12 @@ def execute_tool_calls_sequential(agent, assistant_message, messages: list, effe
# Checkpoint: snapshot working dir before file-mutating tools
if not _execution_blocked and function_name in {"write_file", "patch"} and agent._checkpoint_mgr.enabled:
try:
_ensure_file_checkpoint(
agent,
function_name,
function_args,
effective_task_id,
)
file_path = function_args.get("path", "")
if file_path:
work_dir = agent._checkpoint_mgr.get_working_dir_for_path(file_path)
agent._checkpoint_mgr.ensure_checkpoint(
work_dir, f"before {function_name}"
)
except Exception:
pass # never block tool execution
@@ -1737,7 +1691,6 @@ def execute_tool_calls_sequential(agent, assistant_message, messages: list, effe
skipped_name,
f"[Tool execution skipped — {skipped_name} was not started. User sent a new message]",
skipped_tc.id,
effect_disposition="none",
))
_flush_session_db_after_tool_progress(
agent,
@@ -1751,75 +1704,19 @@ def execute_tool_calls_sequential(agent, assistant_message, messages: list, effe
# ── Per-turn aggregate budget enforcement ─────────────────────────
num_tools_seq = len(assistant_message.tool_calls)
if finalize and num_tools_seq > 0:
if num_tools_seq > 0:
enforce_turn_budget(messages[-num_tools_seq:], env=get_active_env(effective_task_id), config=_tool_budget)
# ── /steer injection ──────────────────────────────────────────────
# See _execute_tool_calls_parallel for the rationale. Same hook,
# applied to sequential execution as well.
if finalize and num_tools_seq > 0:
if num_tools_seq > 0:
agent._apply_pending_steer_to_tool_results(messages, num_tools_seq)
def execute_tool_calls_segmented(agent, assistant_message, messages: list, effective_task_id: str, api_call_count: int = 0, segments=None) -> None:
"""Execute a mixed tool-call batch as ordered parallel/sequential segments.
``segments`` is the ``(kind, calls)`` plan from
``_plan_tool_batch_segments``: maximal contiguous runs of parallel-safe
calls execute on the concurrent path, barrier calls on the sequential
path, strictly in the model's original call order. Because segments are
contiguous, every tool result is still appended one-per-call in emission
order and no call ever starts before an earlier barrier finishes —
identical ordering and side-effect boundaries to fully-sequential
execution, with I/O parallelism recovered inside the safe runs.
Turn-end work (aggregate budget enforcement + /steer injection) is done
once here for the WHOLE batch; the per-segment executor calls run with
``finalize=False`` so a multi-segment turn cannot multiply the budget or
truncate a steer marker.
Interrupt semantics: each segment executor already checks
``agent._interrupt_requested`` up front and appends a cancelled/skipped
result per call, so an interrupt during segment *k* drains segments
*k+1..n* without executing them while preserving one result per
tool_call_id.
"""
from types import SimpleNamespace
if segments is None:
_active_env = get_active_env(effective_task_id)
_exec_cwd = Path(_active_env.cwd) if _active_env is not None and _active_env.cwd else None
segments = _plan_tool_batch_segments(assistant_message.tool_calls, execution_cwd=_exec_cwd)
for kind, calls in segments:
segment_message = SimpleNamespace(tool_calls=list(calls))
if kind == "parallel":
execute_tool_calls_concurrent(
agent, segment_message, messages, effective_task_id, api_call_count,
finalize=False,
)
else:
execute_tool_calls_sequential(
agent, segment_message, messages, effective_task_id, api_call_count,
finalize=False,
)
# ── Whole-turn finalize (budget + /steer) ─────────────────────────
total_tools = len(assistant_message.tool_calls)
if total_tools > 0:
_tool_budget = _budget_for_agent(agent)
enforce_turn_budget(
messages[-total_tools:],
env=get_active_env(effective_task_id),
config=_tool_budget,
)
agent._apply_pending_steer_to_tool_results(messages, total_tools)
__all__ = [
"execute_tool_calls_concurrent",
"execute_tool_calls_sequential",
"execute_tool_calls_segmented",
]
+1 -5
View File
@@ -472,8 +472,4 @@ def _positive_int(value: Any, default: int) -> int:
def _sha256(value: str) -> str:
# surrogatepass: tool results scraped from the web can carry unpaired
# UTF-16 surrogates (e.g. half of a mathematical-bold pair); a strict
# encode raises and takes down the whole conversation loop. The hash only
# needs deterministic bytes, not valid UTF-8.
return hashlib.sha256(value.encode("utf-8", "surrogatepass")).hexdigest()
return hashlib.sha256(value.encode("utf-8")).hexdigest()

Some files were not shown because too many files have changed in this diff Show More