Colours, fonts and layout as the UX-059 mock, light and dark: the step
colour tokens, the time axis with gridlines, and a step's details in a
pane beside the list. The details show a model call's tokens, context,
input and output; a tool's arguments, approval wait and how it was
allowed; a check's questions and the call it judged.
The step endpoint now returns a tool call's arguments, its approval
wait, and for a check the step it judged.
- The Steps button shared the Artifacts button's class, so window-drag.spec's
`.topbar-artifacts-btn` locator found two buttons. It has its own class now,
styled the same.
- test_ws_error_persists_notice_and_retry_reruns: the fire-and-forget title call
could take the one scripted outage meant for the chat call. Title calls are
answered apart, as test_server already does for the concurrency test.
- Turns left open are closed when the server holds the engine lock, checked
inside build_app, whose signature stays as it was (test_server stubs it).
- A model step starts with the catalogue's window only; the user's saved size
or Ollama's num_ctx is filled in when the call ends, so recording never
delays the request (a title call raced ahead of the main call in
test_server).
- Header: a pill with the approval mode and, in Auto-approve or Shadow, the
reviewer; a stamp line with machine, sandbox, coworker fingerprint, model
and provider (UX-059 OB-1, OB-12, OB-14, OB-15).
- The Auto-approve line also gives the checks' time and the reviewer.
- A Steps link under each answer opens the panel at that turn.
- Row badges as in the mock: exit code, failed call type, over the context
limit, no usage, auto-approved, shadow checks, Inbox waits, unknown steps.
- Details show what the recording has: request and response ids, attempts,
result size, call id, the context's contents, summary tokens, breaker,
decision-model answers.
- Values in details are words in English and Chinese, not raw codes.
- A session switch opens the newest turn; a narrow rail stacks details.
- Context size shown when the window is not known; a line for steps still
running when the app stopped.
- Turns left open by a crash or restart are closed at start as unknown, with a
recording_gap notice (TD-12), only by the process that holds the engine lock.
- Steps closed without their own end keep their kind's fields as "not known";
interactive tools get an empty decision path.
- A Stop at a card, or one that kills a running command, is interrupted, not
denied or ok.
- The skills menu is the one the turn offered; a skill turned off leaves it.
- Model steps carry the window the engine compacts against (saved size,
Ollama num_ctx), so local models show the context.
- Decision-model checks record each question's answer; a session grant set
aside in Auto-approve is in the decision path.
Found by running the panel on real turns:
- picking another turn closes the open step's details (they showed the
old turn's step, with its start measured from the new turn);
- no empty message box under a model call whose reply was only tool calls;
- a duration between 0 and 100 ms reads "<0.1s", not "0.0s";
- a model call's details show the coworker fingerprint, as in OB-2.
Two races found by running the schema test repeatedly:
- The streamed view ended its model step only after the engine had read
the final piece. The engine could add the reply and end the turn first,
and the turn's end then closed the step as interrupted, without its
usage or finish reason. The step now ends when the final piece
arrives, before the engine sees it.
- A shadow check runs in the background and can outlast the turn's last
event. The turn's end is now recorded with the time of its ending
event but sent once running shadow checks finish (at most 30 s), so
their steps are not closed as interrupted while they run.
Also: a server test's fake provider answers the session title apart, so
the title call can no longer take the turn's only reply.
A new test runs representative engine turns (approval cards, session
grants, a summary, live and shadow auto-approver checks, the explore
helper, context steps, notices and sandbox facts) and validates each
recording against recording-format.schema.json, copied from ocw-context
next to the golden test. It found three differences, now fixed:
- blocked sites are {host, port, ...}, not {site, ...};
- the context step's folders and skills_menu are counts, never null,
and skills_hash is the short code of the session's skill names;
- a step with no session or no turn records null, not an empty string.
Phase 1 screens from UX-059 (OB-1 steps timeline, OB-2 step details,
OB-4 session usage), reading the engine's recording endpoints.
- A Steps button in the session header opens the panel in the right
rail's full-view slot. Turn picker; tiles for Total and every time
bucket that is not zero, plus Other when it is 1% or more; the
auto-approve line; every step of the turn on one time axis, nested
steps (helper calls, a check's model call) under their parent.
- Clicking a row shows its details: status, timing, model and provider,
first reply time, finish reason, tokens split by type (a count not
reported shows as a dash, never 0), the decision path and who decided,
the transcript message, and the exact request and response.
- The Usage tab: input, output and cache share, the at-least and
failed or over-limit lines, by purpose, by model (in, cache, out) and
context now. /usage gains the per-model parts and totals for it.
- The panel refreshes on live step events and at a turn's end.
The Issues tab (OB-3) is Phase 2.
Phase 1 read side (ocw-context docs/observability/phase-1/technical-design.md
§6.3, §7.1, §12).
- coworker/observability/calculator.py: the token pill, the "at least"
marker, failed and over-limit calls, by purpose and model, cache share,
and each turn's time buckets. The shared golden test now also checks
expected-numbers.json: 30 of 30.
- GET /v1/sessions/{id}/turns (turns with stamp, buckets and usage),
/turns/{turn_id}/steps (a light list in canonical order),
/steps/{step_id} (one step in full: content, the exchange rebuilt from
its pieces, the transcript message it points at cut at 4 KB, its audit
rows and child steps) and /usage (the session's numbers and context
fill). A session recorded before Phase 1 answers recorded: false.
- While a turn runs, the recording events go to the session's sockets as
{"type": "step", "events": [...]}, batched every 100 ms.
Not yet: sending an explore helper's steps only when its step ends, and
computing /usage from the store's columns instead of the loaded recording.
Phase 1 engine item E23 (ocw-context
docs/observability/phase-1/technical-design.md §5.5, §8.1, TD-16).
OpenAI Decisions and TypeSafe Jev call their own service, not the
provider router. DecisionReviewer.review takes the check's view and
records the request through ProviderView.record_call: a model step
nested in the auto-approver check, purpose auto_approver, provider
openai_decisions or typesafe_jev, with the exchange, provider_meta and
provider_facts from the HTTP capture, and usage read from the service's
reply (the shared usage vectors for both services pass).
Not yet: the per-question answers in the check's content, and score and
cutoff.
Phase 1 engine items E15a, E16, E5 and E13 (ocw-context
docs/observability/phase-1/technical-design.md §5.6, §5.8, §9).
E15a: every turn's stamp has the machine (this machine, own machine or
cloud, from remote.json; id, name, app version, never 0.0.0), the model
and its provider, and the sandbox (provider, enforcement, profile).
E16: the session build computes the coworker fingerprint from its id,
its instructions and its own skill files; it is on every turn's stamp.
E5: a context step at the turn's first model call: session prompt hash,
per-turn block size and hash, mode note, folders, and on the session's
first turn the memories, memory mode, rules and AGENTS.md counts. A
change of the per-turn block mid-turn is a context_changed notice.
Error, interrupted, reviewer_paused, step_limit and truncated are
zero-length notice steps.
E13: network_note keeps per-site counts, first times and the late flag
(V-04) in a private key the engine takes out of the result before the
model sees it; the tool step gets attrs.sandbox (blocked_known,
blocked_count) and the sites in content.
Known gaps: skills_menu and skills_hash are not read yet, and the
fingerprint hashes the coworker's instructions text rather than its
manifest file.
Phase 1 engine items E12 and E6 (ocw-context
docs/observability/phase-1/technical-design.md §5.3, §5.5, §8.2).
E12: each auto-approver check (pre-consulted batch, single, delegated and
shadow) is its own step, named after the tool and linked to the call by
call_id, with the reviewer's model call nested inside through a view
whose parent is the check. Attrs: mode, result (allow, deny, unsure,
could_not_run, context_changed), reviewer kind and model,
usage_reported, the breaker for live checks, and matched_user for shadow
checks, filled when you answer. The reason goes in content.
Reviewer.review takes an optional view for this.
E6: a running tool can read its own step, turn and the engine's view.
The explore helper tags its child engine's provider with purpose
helper, parented to the explore step on the parent's turn, and records
its tool steps there with decision {helper: true}; the helper's own run
is not recorded as a turn.
Phase 1 engine items E20 and E4 (ocw-context
docs/observability/phase-1/technical-design.md §5.4, §5.7, §8.3).
E20: every human wait (approval card, question, plan, team, items,
connector, tool install, folder) is a wait step from the moment the card
is shown to the answer, linked to its tool call, with kind, attended,
where (session or inbox, reported by the Inbox when the card is added or
moved there) and outcome; who answered goes in content. The summary's
"Retry or trim?" question is a wait nested under the summary step.
E4: a summary or trim is a summary step, written only if one happened,
with its model calls nested inside, reason, tokens_before, and
tokens_after from the next main call. Each tool call that went through
_authorize carries attrs.decision = {path, origin}: permission, grant,
auto_approver, card and resolved stages with ids from fixed lists.
Also: every recordings store now shares one writer thread, held weakly,
instead of one thread per store that never exited.
Phase 1 engine items E2, E22 and E3 (ocw-context
docs/observability/phase-1/technical-design.md §3.3, §5.2, §5.2.1, §6).
E2: the provider router is wrapped once in a RecordingProvider; each
caller holds a tagged view (main in the engine, summary, auto_approver,
title, proxy). Each call is a model step with purpose, provider, model,
tool count and hash, time to first streamed piece, finish reason and
usage. A main call's step points at the reply message's mid; a title
tagged with a turn that already ended is after_turn; an untagged call is
recorded as unattributed. Errors get error.type from the root cause.
E22: requests are captured below the SDKs, at httpx's HTTPTransport,
only while a recorded call is reading on its own thread. Each step keeps
the exact exchange (request as a manifest of SHA-256 pieces, response
body as one piece), provider_meta without content or secrets, and
provider_facts with fixed names.
E3: TokenUsage carries reasoning and whether cache counts were sent, so
a count not sent is recorded as unknown, never 0. Cache writes are read
where sent and taken out of fresh input on OpenAI-style wires; Ollama's
cached prompt count is used; Anthropic streams take the final
message_delta totals. The shared usage vectors pass through each
provider's own code.
Also fixes the store's usage_reported column for usage {"reported": false}.
time.monotonic ticks every 15.6 ms on Windows, so a short tool step was
recorded as 0 or 16 ms. perf_counter is monotonic too and has
millisecond resolution everywhere. Found in the manual test.
Phase 1 engine item E1 (ocw-context docs/observability/phase-1/
technical-design.md §4, §5.3, §10).
- run, retry and resume send turn_started at TURN_START and turn_ended
at TURN_END, INTERRUPTED or ERROR; a turn that ends with none of
these (an abandoned generator, a resume stopped mid-tools) is closed
as interrupted after Stop, else unknown. A retry's start waits for
the first message it adds, so it can carry that message's mid.
- Callers pass trigger= (user, channel, schedule, team_wake); anything
that does not say is recorded as unknown.
- Every call that runs is a tool step from _execute_sync to its raw
result, with its call id, position in the reply, result message mid,
risk class, result size and exit code. A call that never ran is a
zero-length step with its status (error, denied, interrupted,
skipped).
- While recording, UI events carry turn_id and ts_ms, and tool events
their tool_call_id and step_id. Six TOOL_FINISHED events and both
TOOL_STARTED events now carry tool_call_id.
- The server gives every engine it builds its recorder, so real turns
are saved in recordings.db.
Phase 1 of agent observability, engine items E7 and E21
(ocw-context docs/observability/phase-1/technical-design.md v1.0).
E7: coworker/observability/recorder.py is the one sink for the five
recording events. It folds them into turns and steps by recording
format 1.0 and keeps the canonical step order. coworker/observability/
store.py saves them in recordings.db next to coworker.db: pending
writes are merged by id and written by one background thread, so a
turn never waits and nothing is dropped. A daily sweep deletes
recordings older than 30 days in small batches, and deleting a session
deletes its recordings and the exchange pieces only it used.
E21: every message appended or inserted into the engine's history gets
a `mid` (a ULID) through one _add_message() helper. `mid` is a sidecar
like `ts`: stripped before any provider sees it. Older transcripts
without mids keep working.
The shared golden test (30 scenarios, copied into tests/corpora/
observability) passes 30 of 30, folded in memory and read back from
recordings.db.
ConversationStore.save() appended messages[existing:] by count, so a repair
in the middle of the history (TurnEngine._repair_dangling_tool_calls, or
load()'s _repair_tool_pairing) was never written and the message after it was
written twice, again on every reopen-and-save.
* save() appends only when the part of the file already written still matches
the start of the history; otherwise it rewrites the file atomically
(temporary file, then replace).
* The match is checked by message identity against the list last saved or
loaded for the session, falling back to comparing the file's lines. load()
records the list only when its repair changed nothing; delete() forgets it.
* tests/test_conversation_save_mid_change.py: load repair then save and
reload, a mid-history insert, restart mid-approval then a mode notice then a
new turn, and a plain append that stays an append.
The test started an echo in the background and expected a huge wait to
come back with the task exited. The wait also returns on new output, and
the echo's line can arrive a moment before the process exits, so under
load the wait returned "running" (reproduced: 3 of 300 runs on a loaded
machine; 0 of 600 with this change).
Restacked onto the OPE-231..238 stack and OPE-224:
- Test rows keep their hand-written source label as report-only data; the reviewer is never shown it (main had started rendering it into the prompt). The reviewer is tested with what production gives it.
- Rows may carry the engine's fact lines (facts, OPE-235/OPE-237), checked against review_facts.FACT_VOCABULARY.
- The seven plan rows added on main as review-122..128 keep their content and become review-284..290, after this change's rows, so the sealed holdout ids stay stable.
- Six layered rows the new secrets floor sends to a person are labelled human_only and left out of the score (review-066, 077, 091, 122, 159, 218); danger-022 stays out of the legacy corpus as on main.
Jev and OpenAI Decisions now accept the same fact lines the session-model reviewer gets (OPE-235, OPE-237). They are used in code, not added to what the model reads, so the measured answers stay as they were. A plan step the user approved sets 'requested' to 1.0, with the model's own answer kept for the audit log; every other rule still applies, so a stop signal on a planned step gives a card. The other facts are accepted and not used yet.
The agent-made-file check now reads only the engine's own line ('<path> was created|downloaded by the agent ...'); any other text in that slot, such as an eval harness's corpus label, no longer marks every action as running an agent-made file.
Jev now asks 29 questions: wrong target and wrong place asked directly and gated on whether the user named one; 'requested' counts the look-ups a task needs; 'contrary' is split into 'did the user set a limit' x 'does this break it'; a new 'deletes something the user did not name' ask signal; a stop signal on a read under a request that hands the task to a file, message or list is a card instead of a block; and the earlier named-item/place questions stay as a net that turns an allow into a card. The allow line moves from 0.90 to 0.80.
Measured outside the repo on the same rows for the old and new sets: AgentDojo normal calls allowed 65 -> 147 of 263 and attacks let through 1 -> 0; InjecAgent unchanged (0 misses); our 128 sealed rows and 40 fresh blind rows let through the same mistakes as before. The product code reproduces the tested variant's verdicts on all 3,067 saved answers. Decisions is unchanged apart from the card rule.
The limit assumed 4 characters per token, so 120,000 characters was about 36,000 tokens at the 3.3 characters per token measured on Jev traffic, over Jev's documented 32k for the state plus one question. It now assumes 3 characters per token: 90,000 characters. An oversized state still fails closed to a card before any vendor call; a test pins both.
When Jev or Decisions say both "the user asked for this kind of action" (requested >= 0.50) and "this looks wrong" (a stop signal), the answers contradict each other, so the person decides on a card. A stop signal still blocks when requested is below 0.50. Re-scored from saved answers on AgentDojo: wrong blocks on normal calls 64 -> 40 (Decisions) and 70 -> 29 (Jev), with no new allow on any attack row or sealed row.