Anthropic complete() streams internally — the non-streaming path was dead on every Anthropic model

The SDK refuses non-streaming requests whose max_tokens (default 32000
here) could exceed ~10 minutes: a ValueError before any network I/O.
Every consumer of the non-streaming complete() path was dead on Anthropic
models — most notably the auto-approve reviewer, which errored on ALL
rows (found by the 2026-08-31 reviewer eval; fail-closed, so verdicts
fell back to asking a human — safe, but the mode delivered nothing on
Claude session models).

stream + get_final_message() returns the identical Message shape as
create() on both the plain and beta-fallback paths; both live-probed OK
(claude-sonnet-4-6, claude-fable-5), and the Sonnet 4.6 eval rerun then
passed every ship gate.
This commit is contained in:
Devika Verma
2026-08-31 08:06:14 +05:30
parent 4a42d8ffc9
commit df919edd3b
+12 -3
View File
@@ -470,14 +470,23 @@ class AnthropicProvider(ProviderClient):
model=model, messages=messages, tools=tools, settings=settings
)
client = self._ensure_client()
# Stream-and-accumulate, not a plain create: the SDK REFUSES non-streaming
# requests whose max_tokens could exceed ~10 minutes (ValueError before any
# network I/O). With DEFAULT_MAX_TOKENS=32000 that killed every consumer of
# the non-streaming path — the auto-approve reviewer errored on ALL rows
# for every Anthropic model (found by the 2026-08-31 eval run; fail-closed,
# so verdicts fell back to asking a human). get_final_message() returns the
# same Message shape create() would.
if _needs_refusal_fallback(model):
response = client.beta.messages.create(
with client.beta.messages.stream(
**kwargs,
betas=[_FALLBACK_BETA],
fallbacks=[{"model": _FALLBACK_MODEL}],
)
) as stream:
response = stream.get_final_message()
else:
response = client.messages.create(**kwargs)
with client.messages.stream(**kwargs) as stream:
response = stream.get_final_message()
text_parts: list[str] = []
tool_calls: list[ToolCall] = []