mirror of
https://github.com/andrewyng/openworker.git
synced 2026-09-03 23:03:22 +00:00
Recover truncated tool calls, and never pass a leaked one off as an answer
Small local models drift off the tool-call format, especially with a large tool schema in play. Two failures followed from that, both of which ended the turn looking like the model had simply stopped mid-sentence. Salvage handled well-formed Qwen/Hermes XML but gave up entirely on a call cut off partway through. It now takes the function name plus every parameter that actually closed. A trailing unterminated `<parameter=…>` is dropped rather than guessed, so a half-written path or file body can never reach a tool; if that leaves a required argument missing, the call fails validation and the model gets a corrective tool error, which is the agent loop working. Worse, a call that never parsed at all was reported as `status: completed` — indistinguishable from the model deciding it was done, leaving the user with narration trailing off into stray closing tags. It now ends on the error path, so the GUI offers Retry. That is the right affordance here: the drift is probabilistic, not deterministic, so the same model usually succeeds on a second attempt. Detection ignores fenced and inline code, so a model *explaining* tool-call syntax is still a real answer, and requires that tools were offered at all. Reported against qwen3.5-9b on LM Studio, 2026-07-26. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude
parent
db93d75bf6
commit
d9fbb0790b
@@ -439,3 +439,39 @@ def test_outbound_replaces_images_for_non_vision_models(tmp_path):
|
||||
assert all(p["type"] != "image_url" for p in parts)
|
||||
assert "not viewable" in parts[-1]["text"]
|
||||
assert engine.messages[-1]["content"][1]["type"] == "image_url" # history untouched
|
||||
|
||||
|
||||
def test_leaked_tool_call_ends_the_turn_as_a_retriable_error(tmp_path):
|
||||
"""A tool call the endpoint couldn't parse must not pass as an answer. Ending "completed"
|
||||
made a half-written call indistinguishable from the model deciding it was done — the user
|
||||
saw narration trailing off into stray tags (owner report 2026-07-26, qwen3.5-9b on LM
|
||||
Studio). It ends on the error path so the GUI offers Retry; the drift is probabilistic, so
|
||||
retrying the same model usually works."""
|
||||
leaked = "Let me read the key files.\n<tool_call>\n<function=nope_not_a_tool>\n<parameter="
|
||||
engine, _ = _engine(tmp_path, [_text_turn(leaked)])
|
||||
events = _collect(engine, "explore the codebase")
|
||||
|
||||
assert EventType.ERROR in _types(events)
|
||||
assert EventType.TURN_END not in _types(events)
|
||||
err = next(ev for ev in events if ev.type == EventType.ERROR)
|
||||
assert err.data["error_type"] == "UnparsedToolCall"
|
||||
assert "couldn't parse" in err.data["error"]
|
||||
# Persisted as an error notice, which is what unlocks retry().
|
||||
assert engine.messages[-1] == {
|
||||
**engine.messages[-1],
|
||||
"role": "notice",
|
||||
"kind": "error",
|
||||
}
|
||||
assert engine._tail_is_retriable_error() is True
|
||||
|
||||
|
||||
def test_ordinary_text_answer_still_completes(tmp_path):
|
||||
"""Guard the other side: prose that merely mentions tool syntax inside code fences is a
|
||||
real answer and must still complete normally."""
|
||||
engine, _ = _engine(
|
||||
tmp_path,
|
||||
[_text_turn("Qwen writes calls like:\n```\n<tool_call><function=x>\n```\nThat's it.")],
|
||||
)
|
||||
events = _collect(engine, "how does qwen format tool calls?")
|
||||
assert EventType.ERROR not in _types(events)
|
||||
assert next(ev for ev in events if ev.type == EventType.TURN_END).data["status"] == "completed"
|
||||
|
||||
Reference in New Issue
Block a user