Commit Graph
288 Commits
Author SHA1 Message Date
Devika Verma 37a09360d1 Sandbox lifetime: a dropped engine closes its sandbox; shutdown closes all; orphans reaped at start 2026-09-21 20:56:08 -07:00
Devika Verma 5ce2a18a92 Packaged app: the sidecar runs as the tool runner, and ships the runner sources
openworker-server sandbox-runner serve|attach loads only the runner package. The spec ships the runner and toolkit sources as data, so the zipapp can be packed from the frozen build.
2026-09-21 20:50:18 -07:00
Devika Verma 7792a417f6 Credential grants: the ssh wrapper lives in the copied home, so OpenShell carries it in
The runner daemon puts that folder first on PATH. Live OpenShell test for grants added; passes on the Linux VM.
2026-09-21 20:40:12 -07:00
Devika Verma 48fecdf4ca Settings: a Sandbox page for the provider, the network allow list and shared credentials
Machine-level, read and written through /v1/settings/sandbox into the machine's config.toml.
Adds the sandbox_network_profile setting; strings in English and Chinese.
2026-09-21 19:54:44 -07:00
Devika Verma f1d3711d78 Sandbox: a failed creation removes its folder, so no copied credential is left behind 2026-09-21 19:15:40 -07:00
Devika Verma c9bbe009a1 Sandbox credential grants: chosen home files copied into the sandbox
A machine-level list (ssh, gh, aws, kube to start); enabled entries are copied into the sandbox's private home and their hosts join the allow list.
ssh and git are pointed at the copy, and ssh tunnels through the proxy; the agent is told what it was given.
2026-09-21 19:14:30 -07:00
Devika Verma 58385ef02b Seatbelt provider: the tool runner inside the macOS sandbox
Files limited to the session's folders, network only through the allow-list proxy, no secrets in the environment.
A folder granted while a session runs restarts the sandbox and tells the agent (both providers).
On macOS the executor finds a timed-out command's children without pgrep.
2026-09-19 18:31:35 -07:00
Devika Verma 02f0b4d4c6 Sandbox provider selection, machine status line and the sandbox setup command
Headless: OpenShell when usable, else direct with a loud warning; an explicit setting refuses.
sandbox_provider is machine-level only, so a repository cannot switch the sandbox off.
openworker machine sandbox status|setup is served but not listed in help yet.
2026-09-19 17:16:25 -07:00
Devika Verma 4181039ac4 OpenShell provider: one sandbox per agent, with a registry and network profiles
Folders are bind-mounted at their real paths; the runner is a read-only mount and the main command.
The policy is rendered from the session's roots, with the owner's uid and Landlock as a hard requirement.
The runner's pipe is the gateway's gRPC exec stream; the registry caps and cleans up sandboxes.
2026-09-19 16:53:35 -07:00
Devika Verma ced3390c07 Workspace tools run inside the tool runner when a session uses one
Definitions stay in the server; a proxy sends only the call (one tool.call per tool).
The packed runner carries the server's own aisuite file and git toolkits.
Parity tests run all ten tools in both modes on twin folders.
2026-09-19 16:06:09 -07:00
Devika Verma e70181f776 Tool runner: JSON-RPC daemon, attach relay, client and the Workspace interface
Direct mode stays the default and is unchanged; the runner serves the same Executor contract.
A cut stream resumes, a restarted runner fails running commands, a request id runs once.
Shell tests now run on both paths; runner-local is a developer provider with no confinement.
2026-09-19 15:40:33 -07:00
Rohit C Prasad 81a01ff901 Reconcile ordinary approval cards by exact tool call identity
Carry exact call identities through approval events and reconcile cards on completion or resolved Inbox records.
2026-09-21 13:11:29 -07:00
Rohit C Prasad cc728bcf7c Fix device provenance and linked worker denial recovery
Exclude device sinks from file provenance and atomically resolve linked worker denials with durable recovery.
2026-09-21 12:47:21 -07:00
Rohit C Prasad 2d6bf292c9 Clarify verified revision handoff to publish branch owner
Require the publish-branch owner to fast-forward a clean checkout to the exact independently verified revision.
2026-09-21 12:24:13 -07:00
Rohit C Prasad 2e7b734ab2 Make team coordination event driven and quiet
Keep routine team updates quiet while preserving actionable wakes for questions, blockers and assignments.
2026-09-21 06:02:41 -07:00
Rohit C Prasad a75ca94f5f Add incremental board reads and immutable team artifacts
Return compact board receipts, page comments incrementally and publish shared versioned file artifacts.
2026-09-20 23:09:47 -07:00
Rohit C Prasad 9beb204159 Fail closed when the approval owner is unavailable
Require human review when the approval-owning session is missing instead of assuming its context.
2026-09-20 19:40:18 -07:00
Rohit C Prasad 3146e65594 Carry explicit approval guidance into team reviews
Persist user-reviewed worker guidance and pass deterministic role, assignment and workspace context to reviews.
2026-09-20 19:31:01 -07:00
Rohit C Prasad 89bdad221f Let leads own repository setup and preserve configured targets
Let leads clone repositories and workers create their own worktrees without replacing missing configured sessions.
2026-09-20 16:59:18 -07:00
Rohit C Prasad eef0122b24 Clarify team approvals and surface every pending request
Surface every pending request and review delegated actions using the original worker permission context.
2026-09-20 15:03:01 -07:00
Rohit C Prasad 4099fee5bc Add structured outcome and staffing proposals
Introduce grouped work and staffing proposals with acceptance criteria and explicit external-action intent.
2026-09-20 11:42:30 -07:00
Rohit C Prasad f3c5977eff Refresh team approvals and streamline background activity and navigation
Refresh cached approval settings, group background updates and open workers beside the lead.
2026-09-20 07:46:14 -07:00
Rohit C Prasad 443e50e15f Harden board-owned image attachments and local retention
Copy authorized image evidence into board-owned storage and preserve it after source-worktree removal.
2026-09-19 23:33:33 -07:00
Rohit C Prasad 4f9e79c751 Synchronize resolved team proposal cards across views
Use exact proposal identities and resolved Inbox records to synchronize cards across viewers.
2026-09-19 22:59:37 -07:00
Rohit C Prasad faa047cc6c Make idle check-ins cancellable and batch team wakes
Cancel idle check-ins on earlier activity, carry their reminders forward and batch ordinary board wakes.
2026-09-19 22:11:49 -07:00
Rohit C Prasad db81e29779 Add scoped task reads and reconcile obsolete approvals
Scope task reads to the team and retire approvals only when the exact action already has a result.
2026-09-19 21:47:56 -07:00
Rohit C Prasad f7fcd0d260 Fix team approval handoffs and self-wake behavior
Show the original worker action, retire linked approvals and yield correctly after scheduling a wake.
2026-09-19 20:47:00 -07:00
Rohit C Prasad b20db26029 Separate task details from worker conversations
Keep task evidence separate from worker transcripts and reuse the composer for worker conversations.
2026-09-19 20:03:11 -07:00
Rohit C Prasad 592eb7a669 Harden team accounting and shared status interfaces
Handle overlapping work and incomplete accounting while exposing consistent status updates across interfaces.
2026-09-19 18:39:03 -07:00
Rohit C Prasad 9e9d7e421e Add team progress and usage projections
Expose team progress, worker status and recorded time/token usage through shared projections.
2026-09-19 18:28:23 -07:00
Rohit C Prasad e0c7b6eeb0 CLI: join and up at the top level; one spelling per command
openworker machine keeps status, keys, logs, service and leave.
join takes a link, or a controller's address to approve a code there (was auth join).
2026-09-18 21:58:37 -07:00
Rohit C Prasad a15fb38c93 CLI: machine logs, status --json, launchd service on macOS; install.sh
The installer uses uv or pipx for the current user, and refuses the PyPI placeholder by name.
2026-09-18 21:35:04 -07:00
Rohit C Prasad a5ed9c8937 CLI: openworker machine commands, real help and version; package publishes as openworker
aisuite comes from PyPI (0.2.0), so the package has no git-URL dependency.
Older top-level spellings (openworker join, up, status) keep working, unlisted.
2026-09-18 16:40:07 -07:00
Rohit C Prasad 2be346d7dd Card gallery (dev only): card states, scenario sessions, server contract check
/#/gallery renders every card state from payload files; scenario sessions boot the real app from JSON.
A contract file keeps the states in step with what the server sends. Not in the production bundle.
2026-09-18 11:54:30 -07:00
Rohit C Prasad 7db596dd80 Engine and server: worker grants, lead decisions, parked gates, reviewer, usage
The human's ticks on the staffing card are the worker's grant; approvals follow the lead's mode.
A lead answers a worker's parked call; team and work-item gates keep their payload in the Inbox.
2026-09-17 16:18:03 -07:00
Rohit C Prasad 80d57667b6 Coworkers: models per coworker, reviewer and lead personas, team store
A coworker lists its models in order; reviewer and lead coworkers ship.
Declared connectors and models are a default for a worker, not a ceiling.
2026-09-17 12:19:11 -07:00
Rohit C Prasad ea021ff6aa Connectors on machines: grant handoff, GitHub and Slack on a box, subscriptions
A connector account moves to a machine, or connects there directly; managed grants move by delegation.
GitHub and Slack reach a box by polling the relay; subscriptions register with the cloud first.
2026-09-06 10:19:23 -07:00
Rohit C Prasad 274c9a582b Remote machines: identity, reverse channel, acceptor, sealed key wallet
A machine joins with its own identity and holds a reverse channel to the controller.
Provider keys and connector accounts deploy sealed to the machine's pinned key.
2026-09-05 08:00:16 -07:00
Devika Verma a46ec42dbe Merge pull request #556 from tyoon10/devin/rekey-space-cursor-like-escape
Escape LIKE wildcards when re-keying board cursors
2026-09-18 23:47:32 +05:30
Devika Verma eee3854cce Merge pull request #555 from tyoon10/devin/python310-exceptiongroup-compat
fix(tests): import ExceptionGroup from backport for Python 3.10 compatibility
2026-09-18 23:44:19 +05:30
Devika Verma d6d1a7c9b9 Merge pull request #554 from tyoon10/devin/bedrock-optional-dependency-tests
fix: make Bedrock verification and tests tolerate missing optional boto3
2026-09-18 23:43:52 +05:30
Devika Verma 8cfe67ca07 Merge pull request #566 from harneet2512/fix/security-readonly-variable-expansion
security: reject shell variable expansion in read-only grants
2026-09-18 23:32:39 +05:30
Rohit Prasad e5cb2e8fbe Merge pull request #172 from Anuj04432/fix/relay-hub-exception-chaining
fix: preserve exception chain in RelayHub.wait_dispatched
2026-09-18 10:49:54 -07:00
Rohit Prasad 1b24796995 Merge pull request #123 from engmohamedsalah/fix/windows-rg-parser
fix: handle Windows drive-letter paths in grep (ripgrep) output
2026-09-18 10:49:02 -07:00
Devika Verma c1e8b8cdf3 Make the time a tool instead of a line in every turn (OPE-192, final fix)
The per-turn `<system-context>` block opened with `Now: YYYY-MM-DD HH:MM` (2c9fb74,
2026-08-20) so that absolute scheduling had a clock. The block is glued onto the last
user message — the task prompt in a tool loop — and a provider's prompt cache is
reusable only up to the first byte that differs, so every minute crossing rewrote
message one and re-processed the whole conversation. 9d6a716 worked around it with a
trailing-message shape, an Anthropic-only relocation of that message, and a
`live_clock` switch that headless runs set off. This commit removes the cause instead
and deletes the workarounds; the shape is the same on every provider again.

- The block carries no clock, on any provider. What is left (folders, skill menu,
  mode notices, memory-off notice, disable countermand) changes only when the user
  changes something, so the outbound history is byte-identical turn to turn.
- `current_time` tool (coworker/clock.py), registered for every persona: local time
  with offset and zone, the same instant in UTC, weekday. The environment block's
  "Today's date" now says it is a session-start snapshot and points at the tool.
- `sleep_for(seconds)` is back beside `sleep_until` (a relative wait needs no clock
  arithmetic; the four lead personas' manifests still told the model to call
  sleep_for, which 2c9fb74 had removed). `sleep_until`'s description points at
  current_time for the date/timezone. Timer wake messages carry their fire time.
- Deleted: `live_clock` config + COWORKER_LIVE_CLOCK, the trailing-message shape in
  `_outbound_messages`, `_relocate_ephemeral_note` in the Anthropic provider. The
  cache-diagnostics instrument (off by default) stays.

Validation runs (one long tool-heavy task, one attempt each on Fable 5.1 and Kimi K3,
compared with the 9d6a716 runs of the same task) are recorded on OPE-192.

Tests: tests/test_clock_tool.py (new), tests/test_context_block_placement.py and
tests/test_cache_breakpoint_placement.py rewritten for the single shape,
tests/test_live_clock.py removed; self-wake, attachments, message-source and
multiroot tests updated. Full suite unchanged against the Windows baseline.
2026-09-17 18:25:47 +05:30
Devika Verma c32e4b1274 Keep the per-turn context block out of the cached prompt prefix (OPE-192)
Every turn the engine builds a `<system-context>` block — folders, skill menu, and a
live clock to the minute (owner ruling 2026-08-20) — and appended it to the last
message with role "user". In a chat that is the newest message. In a tool loop, tool
results carry role "tool", so the last user message is the task prompt: message one.
Every minute crossing changed message one, and a provider's prompt cache is reusable
only up to the first byte that differs, so the whole conversation was re-processed.

Measured on a long tool-heavy task (Fable 5.1): Anthropic's
cache-diagnostics beta reported every miss as `messages_changed`; per-message
fingerprints showed message 0 changing on every miss and nothing else; 7 of 7 minute
crossings missed. On one heavier task, 16 of 105 calls held 95% of the cache writes —
three quarters of that session's cost. A long multi-task Kimi K3 session on Together
had the same signature: 60% of minute crossings missed vs 1.5% otherwise, so the
OPE-186 attribution of that session's cache-miss cost to Together's fleet cache
was mostly wrong.

Three parts:

1. Engine: the block is sent as its OWN trailing user message, opening with
   EPHEMERAL_CONTEXT_OPEN and a line saying it is automatic context. On the
   OpenAI-compatible wire that leaves the prefix through the last tool result
   byte-identical turn to turn. Validated on Kimi K3 via OpenRouter/Together: cache
   misses 9/40 -> 0/29 calls, fresh input 0.76 M -> 0.03 M tokens on the same task.

2. Anthropic provider: `convert_messages` relocates a trailing note onto the last user
   message before it — this provider's previous shape. A trailing note there is worse,
   not better: a cache entry made at a block that is followed by another block in the
   same message does not match a later request where the message ends there (27/27
   misses, 2.3x the cost on the same task). The same property means chat
   sessions on Anthropic have never reused the conversation cache (the note was always
   glued to the newest user message); Anthropic's own pattern is to keep per-turn
   reminders in the transcript, which is a larger change tracked on OPE-192.

3. `live_clock` config (default true; COWORKER_LIVE_CLOCK=0): with it off the block
   omits the clock line and is static, so message one stays stable on Anthropic and the
   trailing note stays stable on the OpenAI path. Headless runs set it off (nothing
   schedules there). Validated on Fable 5.1, same task: full-conversation rewrites 6
   -> 0, reported misses 7 -> 0, cost down 63%, same pass and reply count.

Also in this commit, off by default: a cache-diagnostics instrument in the Anthropic
provider (COWORKER_CACHE_DIAGNOSTICS=1 adds the cache-diagnosis beta and records
Anthropic's miss reason plus a per-message fingerprint of each request on the assistant
message) — it is what found the bug and is not used in normal runs.

Tests: tests/test_context_block_placement.py, tests/test_cache_breakpoint_placement.py,
tests/test_live_clock.py; test_multiroot, test_message_source and test_attachments
updated for the trailing note. Full suite unchanged against the Windows baseline.
2026-09-17 14:26:44 +05:30
Devika Verma 227466901e Raise the model-matrix size guardrail for the Claude 4.5/4.6 rows
The previous commit added Claude Opus 4.6, Opus 4.5 and Sonnet 4.5 to the matrix so
OpenWorker can be measured on those models. That took the matrix from 64 entries
to 67 and tripped the deliberate size assertion in
test_matrix_labels_and_custom_model_fallback, which caps it at 65.

Raised to 70 rather than pruning. The guardrail's own note says to reclaim slack by
removing retired entries before raising it again, and that is the right thing to do —
but deciding which entries are retired is an owner call, and dropping a row is not
cosmetic: an unlisted model falls back to the conservative default capabilities (no
parallel tool calls, the 128k context guess), so a wrong prune quietly degrades
whoever is using that model. The debt is recorded in the test comment for whoever
makes that call.

Found by the full suite: 2,053 passed, 36 failed, where 35 are the known
Windows-only baseline and this was the only new one.
2026-09-16 15:38:11 +05:30
Devika Verma 72e62e87b0 Make the compaction summary reliable, and stop losing the task statement (OPE-189)
The OPE-186 trial (15 long tasks) compacted 64 times and gave a usable summary on 7 of 103
summariser calls. Two independent causes, both fixed here.

1. The summariser was asked last and read the session last. `summarizer_messages`
   sent the ~1,800-character instruction as the system message and then tens of
   thousands of characters of transcript, ending wherever the session happened to
   be — usually on a raw tool result. The model continued that voice instead of
   summarising: 13 of 16 short replies were a next-action sentence ("The output
   got truncated. Let me re-run the remaining checks." — 61 characters standing in
   for 75 turns), each stopping voluntarily with most of its budget unused. The
   transcript is now delimited and the instruction restated after it, in bold,
   ending on the exact first heading to write.

2. The budget was 3,000 tokens, shared with the model's reasoning. On 60 of 103
   calls Kimi K3 spent it thinking and the summary was cut off or never started.
   SUMMARY_MAX_TOKENS is now 16,000 and is a setting: `compaction_summary_max_tokens`
   in config.toml, COWORKER_COMPACTION_SUMMARY_MAX_TOKENS in the environment.

Measured by replaying 20 recorded compactions from that trial against Kimi K3,
scored by the gate below: the old shape at 3,000 tokens gives 4/20 usable; moving
the instruction alone gives 18/20; with the 16,000 budget, 20/20. Disabling
reasoning for the call adds nothing over that, so it is not done.

Also:

* A quality gate before a summary is accepted. "Not empty" let a 61-character
  sentence become the coworker's whole memory of a span. `summary_quality_problem`
  refuses empty, under 400 characters, or missing the section headings, and
  `summarize_span` raises — so the engine's existing single retry runs before the
  no-summary trim fallback.
* The first user message is pinned in the compacted block, whole, capped at 4,000
  tokens. `_cap_user_messages` kept the newest 40, so any session with 41 or more
  user messages silently dropped the opening mandate — the one message that never
  goes stale. Omissions now come from the middle, and the block says so there.
* Per-message clipping is replaced by a token budget filled backwards from the
  newest. Each preserved user message was cut at 600 characters; 8 of the 15 trial
  task prompts are longer, up to 1,424, so after every compaction the agent's
  verbatim record of its job was cut mid-sentence. On video-processing that
  discarded the output path and both field names the task checks. The budget
  is 8% of the compaction trigger, bounded to [2,000, 20,000] — a fraction, so
  lowering the trigger to save tokens is not undone by the block, and 20,000 at
  the default trigger is a generous allowance.

The trigger itself is unchanged: min(80% of the context window, cap), cap 250,000,
overridden by `compaction_cap_tokens`. Lowering the default should follow a working
summariser, not precede it.

Unrelated, included here to keep the tree clean: model-matrix rows for Claude Opus
4.6, Opus 4.5 and Sonnet 4.5. These are added so OpenWorker can be measured on
those models against published evaluations of the same models.

Tests: tests/test_compaction.py, tests/test_compaction_engine.py and
tests/test_compaction_transcript.py cover the new request shape, the gate, the pin,
the budget and the config/env plumbing. Their fake summarisers were returning
strings the gate now refuses, and were updated to real-shaped summaries.
2026-09-16 15:31:06 +05:30
Devika Verma 65ba394393 Bound tool results, steer edits, keep compacted turns readable (OPE-186)
Findings from a long multi-task Kimi K3 session: ten long tasks used 57% of
451 M tokens, tool results were 31% of the final context and whole-file
write_file bodies 14%, and compaction fired once. Three harness changes:

1. Tool-result cap. Every tool result is bounded on arrival at 10,000 bytes,
   head + tail, with a marker naming the spill file that holds the full text
   (coworker/toolresult.py; engine `_record_result`; config
   `tool_result_max_bytes`, env COWORKER_TOOL_RESULT_MAX_BYTES, 0 = off).
   The spill folder is the session's scratch root when there is one, else it
   joins the session's directories read-only so read_file can open it.
   shell.py's own 20k-char tail cut becomes a 2M safety net.

2. Edit steering. write_file, replace_in_file, apply_patch and
   apply_unified_diff describe WHEN to use each (catalog.EDIT_TOOL_GUIDANCE),
   and the Cowork prompt says to edit in place rather than rewrite.

3. Compaction. `compaction_cap_tokens` (config / env
   COWORKER_COMPACTION_CAP_TOKENS) lowers the trigger; each compaction writes
   the verbatim transcript of the compacted turns (no reasoning) to the spill
   folder and the compacted block names the file (CompactionState.transcript_path,
   compaction.render_transcript).

Trial on 15 long tasks x 3 attempts (Kimi K3 via Together, cap 60,000):
33/45 passed vs 54/75 for the unchanged harness on the same tasks, tokens per
attempt 3.30 M -> 1.40 M, cost down 44%. Observed and left for a
follow-up: Kimi's summariser reply lands mostly in reasoning within the
3,000-token summary budget (7 proper summaries in 103 calls), and the trial had
16 timeouts vs 9 in the baseline.

Tests: tests/test_tool_result_cap.py, tests/test_edit_tool_guidance.py,
tests/test_compaction_transcript.py; full suite unchanged against the Windows
baseline.
2026-09-15 15:09:47 +05:30
Devika Verma 4caf7564a8 Replay reasoning on the OpenAI-compatible path (OPE-178)
Reasoning models served through the OpenAI-compatible client (Kimi K3 on
Together / Moonshot / OpenRouter, DeepSeek, GLM, …) return their thinking in
a message field — `reasoning_content`, or `reasoning` via OpenRouter — and
Kimi K3 explicitly requires the complete assistant message, thinking
included, to be passed back on later turns ("preserved thinking history").
The client read the field for display only; the engine strips the display
sidecar before every request, and nothing re-attached it, so every later
turn ran without the model's earlier thinking. The Anthropic and Gemini
clients already replay their formats via `extras`; this brings the
OpenAI-compatible client in line.

- `_reasoning_field` records which field the thinking arrived in;
  `_reasoning_extras` persists `{field, text}` as the `_openai_compat`
  sidecar on both the complete() and stream() paths.
- `replay_reasoning` re-attaches the text under that same field on every
  outbound assistant message that carries the sidecar, before
  `_strip_foreign_sidecars` removes the underscore key. Messages without it
  are untouched, so models that never send reasoning see identical requests.

Tests: tests/test_reasoning_replay.py — capture on complete and stream for
both field names, no-reasoning case, replay shape on the wire (original
field, sidecars stripped, foreign sidecars untouched), and an engine round
trip where the second request carries the first reply's thinking.
2026-09-09 19:19:55 +05:30