[bug] Tool results are fed back as the assistant's own prose and the tool_use turn is dropped — the format the model then imitated in #395

[serge-ivo]

The platform hands tool results back as the ASSISTANT's own prose

agent-think.ts:831, once a round of tools has run:

aiMessages.push({ role: "assistant", content: `I called tools:\n${toolResults.join("\n")}` });
aiMessages.push({ role: "user", content: `Continue based on the tool results above. …` });

Two things are wrong with that, and the second one is structural.

1. The model's tool_use turn is never added to the conversation at all. runAnthropic reads
tool_use blocks out of the response (user-ai.ts:186) and converts them to a Workers-AI-shaped
tool_calls array — and nothing puts them back. The next request carries no tool_use and no
tool_result. normalizeForAnthropic (:68) has no concept of either: it maps every role to
user/assistant and merges consecutive same-role turns.

So the exchange the provider sees is: request with tools → response with tool_use →
a request in which that never happened, plus an assistant paragraph narrating results.

2. Ground truth is stored in the one role that means "the model's own words". A tool_result
block is structurally the platform's. An assistant message is structurally the model's. After the
merge in normalizeForAnthropic they are one paragraph, and on the next turn nothing distinguishes
a real tool output from something the model asserted.

This is the likely cause of #395, not a neighbour of it

In #395 the model wrote its own <tool_call> / <tool_response> pairs into the reply, invented
their contents (a repo remote that contradicted the real one, three GitHub issues from a tool that
never ran), and the platform passed it through as fetched fact.

Look at what this loop teaches it, every single turn: tool results appear in the transcript as
assistant prose that begins "I called tools:". The model reproduced the format it was shown. It
had no structural signal that a result is not something an assistant writes, because in this
conversation it is exactly that.

Fixing #395 by stripping the markup treats the output. This is the input.

A third, smaller loss

Dropping the tool_use turn also drops the ARGUMENTS. The follow-up context reads
[repo_read_file]: <contents> with no path. Call one tool twice in a round with different arguments
— two files, two repos, two issue numbers — and the model cannot tell which result belongs to which
call. It has to infer from the content, and when it cannot, it guesses.

Suggested fix

Use the protocol the provider already offers, on the path that already sends tools:

1. Append the assistant turn with its tool_use blocks (ids intact), exactly as returned.
2. Append a user turn of tool_result blocks, each carrying tool_use_id and the output,
   with is_error set on a failed call.
3. Teach normalizeForAnthropic about both so the merge cannot flatten them into prose — its
   merge of consecutive same-role turns is what erases the boundary today.
4. Keep the Workers-AI fallback path on its current prose shape if that provider needs it, but
   branch: the fallback's limitation should not set the protocol for the provider that almost every
   chat actually runs on (claude-sonnet-4-6, per CLAUDE.md).

Then a fabricated <tool_response> in the reply text becomes trivially detectable as well —
anything claiming to be a result outside a tool_result block is one the platform did not produce.

Files: workers/api/src/agent-think.ts:826-833, workers/api/src/lib/user-ai.ts:68-94,183-187.
Related: #395 (the fabrication this makes possible), #396 (the same loop, silently truncated).

[serge-ivo]

Post-#395 measurement: the guard fires, and the rate has not moved. Chess coder (26f71cd8), three consecutive turns this morning:

01:36:51  ⚠️ The agent wrote tool results that no tool produced… Named but never run: repo_read_file
01:38:08  ⚠️ … Named but never run: repo_read_file, repo_read_file

So #395's fix is working exactly as specified — the invention is detected, stripped, and the answer replaced with the platform's record. What it does not do is reduce how often the model tries, which is what this ticket is about.

Two things underneath it, and they are separable:

1. The format (this ticket). Tool results come back as {role:"assistant", content:"I called tools:\n…"}, so the transcript's own convention is that a tool result is assistant prose. The model reproduces the convention it is shown. Until a result arrives as a tool_result block, "write the result yourself" and "read the result" are the same shape.

2. The pressure — #405, filed today. This agent's only repo is a local path the platform marked cloneStatus:"ready" without checking: ~/dev/pas/platform/apps/chess-academy is an empty directory, 0 files, not a git repo. Every read tool therefore returns a non-answer as a success — (no files found at that path), (no git origin remote) — with no diagnosis to relay. Asked about the code, the model has literally nothing true to say and no way to report why.

Worth stating the order plainly, because it affects what to build first: #405 removes the reason, this ticket removes the shape, #395 catches what is left. All three are worth doing, and only the first two reduce the rate.

[serge-ivo]

Shipped in 1550615. Deployed.

The assistant turn now goes back to the model exactly as returned, blocks intact, and is answered by a user turn of tool_result blocks carrying tool_use_id and is_error. So a result is no longer the same shape as a claim: it is a block type an assistant turn cannot legally contain.

Details that mattered more than the diff suggests:

• Ids come from the assistant turn, not the normalized call list. normalizeToolCalls skips a malformed call while its tool_use block remains — and an unanswered tool_use_id 400s the entire request, so sourcing ids from the normalized list would have wedged a chat on exactly the turn where a model emitted one bad call.
• Workers AI keeps the prose shape. The branch is on whether the completion carried blocks, not on a provider enum, so there is nothing to drift when a model or provider is added.
• A constraint the ticket does not mention, which would have broken production: a request whose messages contain tool blocks must also define tools. The two completions that deliberately send none — the final answer and [bug] The model writes its own tool RESULTS and the platform believes them — invented GitHub issues and file contents reported as fetched, raw markup left in the message #395's correction round — are precisely the ones that follow a tool round. They now declare them with tool_choice: "none".

#395 is strengthened, not regressed. Its guard proves the platform never writes result markup into an assistant message; on the structured path that is now true by construction. A source guard asserts the tool_result turn is pushed with role: "user" and never role: "assistant".

Verified: typecheck, full suite green, biome, tokens, ratchet.

Three things asserted against the provider's documented contract rather than a live call — worth watching on the first real chat now that it is deployed: that tool_choice: {type:"none"} is accepted, the tool_result-first ordering requirement, and that echoing contentBlocks verbatim is accepted alongside our system cache-control block. If a chat errors after this deploy, this is the first place to look and I will fix it forward.

Per your own ordering — #405 removes the reason (shipped, 5527cea), this removes the shape, #395 catches what is left. All three are now in. The remaining member of the family is #406: the invented turns already stored are still read as fact, and that one still wants your decision on the existing rows.

[serge-ivo]

Verified delivered in 1550615 (on main, deployed). Read against the four Suggested fix points, in the code as it stands today:

1. The assistant turn goes back with its tool_use blocks, exactly as returned — agent-think.ts:1008, aiMessages.push({ role: "assistant", content: rawResult.contentBlocks }). user-ai.ts:243 carries them out of runAnthropic as contentBlocks, named that rather than content so it cannot be confused with the prose field.
2. Answered by a user turn of real tool_result blocks — agent-think.ts:1010, ids from toolUseIdsOf(rawResult.contentBlocks) and outcomes keyed by id in outcomeById, with is_error set. Ids come from the assistant turn and not from the normalized call list, because normalizeToolCalls skips a malformed call while its tool_use block remains, and an unanswered tool_use_id is a 400 on the whole request rather than a degraded reply.
3. normalizeForAnthropic knows about both — lib/anthropic-tool-turns.ts holds the rules pure: results hoisted to the front of a merge, unmatched halves of a pair dropped, an emptied turn removed. Exactly the two behaviours this issue named (the merge of consecutive same-role turns, and the leading-assistant drop) are what would otherwise bury or orphan a result.
4. The Workers-AI fallback keeps the prose shape, and the branch is on whether the completion carried content blocks at all — self-describing, so there is no provider enum to drift.

The third loss the issue names — dropped ARGUMENTS, so [repo_read_file]: <contents> arrived with no path — is closed by the same change: outcomeById is keyed by tool_use_id, which is the only thing that says which of two calls to the same tool a result answers. anthropic-tool-turns.test.ts covers that case directly.

On the shipping comment's three open questions (that tool_choice: {type:"none"} is accepted, the ordering requirement, and that echoing contentBlocks verbatim coexists with the system cache-control block): I have not re-verified those against a live chat — I have no production session from here. What I can say is that no chat-path regression has been reported in the ~4h since deploy and the API worker has deployed green three times over that code since. If a chat errors, this remains the first place to look.

Closing on the code. The remaining member of the family, #406, is fixed forward in fb1ae10.