Skip to content

fix(codemode): let the model recover from a wrong tool name in one retry - #50

Draft
kevoconnell wants to merge 2 commits into
devfrom
claude/codemode-unknown-tool-hints
Draft

kevoconnell wants to merge 2 commits into
devfrom
claude/codemode-unknown-tool-hints

Conversation

@kevoconnell

@kevoconnell kevoconnell commented Sep 17, 2026

Copy link
Copy Markdown

Issue for this PR

No issue here. It comes from root-causing the Replo agent-health card "CodeMode execute still ~56% errors (repeat, day 6)". Companion PRs in andytown: replohq/andytown#27821 (tool contracts) and replohq/andytown#27824 (stale tool names in prompts, plus a CI lint for them).

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

When the model gets a tool name wrong inside a script, Code Mode tells it to go search. That costs a search call and then the real call, each a full model turn. Over 7 days of production sessions, about 150 of 541 failed scripts were "Unknown tool", and the wrong names were nearly always one step from a real one. Four changes, all about making the first failure carry its own fix:

  1. An unknown tool names the closest real tool, with its signature. tools.bedrock.find_registry_items now answers Did you mean: tools.bedrock.registry_find_items(input: …): Promise<…> plus up to two other close paths. A namespace repeated in the name (bedrock.bedrock_generate_image, 55 of those failures) is dropped before ranking. The ranking is the search tool's own scoring, extracted into rankTools and used by both. Suggestions come from the same search index as search, so a permission-hidden tool still cannot leak.
  2. The host can claim the name first. About 77 of those failures were the model calling one of its regular agent tools through a script (tools.bedrock.set_project_context for the agent tool set-project-context). No search of the MCP catalog can fix that. CodeMode.make takes an optional unknownToolHint(path). opencode supplies it from the tool ids given to the model this turn (ctx.extra.toolIDs, set in session/tools.ts), comparing names with - and _ folded, and answers "'set-project-context' is one of your regular tools, not a Code Mode tool. Call it directly, outside execute." The codemode package stays unaware of host tools, as its AGENTS.md asks.
  3. A call with no arguments sends {}. tools.bedrock.list_sites() failed with "expects exactly one input object". The tool's schema still rejects a missing required field, and a second argument is still an error.
  4. A parse error says where it is. Most parse failures are bracket slips in 1,000 to 2,000 character one-liners, and "',' expected." gives the model nothing to aim at, so it re-emits the whole script. The TypeScript diagnostic already has an offset; the message now ends (line 2, col 41) near: …, in the same format runtime errors use.

And one interpreter bug: String(e) on a caught tool failure returned "[object Object]". The intra-sandbox copy that String() and template literals run their input through kept Date, Map, and the other sandbox values as leaves but not error values, so the symbol brand was gone before coerceToString looked. catch (e) { return String(e) } is the idiom models reach for, so the script "succeeded" and the model received no error text at all: 20 scripts in 18 projects in one week, including failed checkout setups. Error values are now kept as leaves like every other sandbox value, and coerceToString renders them Error: <message> like JavaScript, for String(e), `${e}`, and "" + e.

How did you verify your code works?

  • packages/codemode: bun test 273 pass, bun run typecheck clean, before and after a simplify pass (the pass folded the unknown-tool explanation into resolve, moved the error brand next to the other sandbox value types, and deleted the two per-site brand bypasses). New file test/recoverable-mistakes.test.ts covers each behavior above plus the two guards (a name nothing resembles still points at search; a second argument is still rejected).
  • packages/opencode: bun test test/tool/code-mode.test.ts 40 pass (two new cases: the closest-tool suggestion through the MCP catalog, and the regular-tool hint through ctx.extra.toolIDs), test/tool/task.test.ts and test/tool/registry.test.ts pass, bun run typecheck clean.
  • Not verified: a live sandbox. This needs a prerelease sideloaded onto a box before the version pins in andytown move.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code


Summary by cubic

Fixes Code Mode’s recoverable-mistake handling so a wrong tool name, empty call, parse slip, or caught tool error carries its own correction instead of costing another model turn.

Bug Fixes

  • An unknown tool now suggests the closest real tool with its signature; suggestions still come from the search index, so hidden tools can’t leak.
  • The host can override the suggestion: opencode recognizes regular agent tools called inside a script and tells the model to call them directly.
  • A call with no arguments now sends {}; missing required fields and extra arguments are still rejected.
  • Parse errors now include line, column, and nearby source.
  • Caught tool failures stringify to Error: <message> instead of [object Object].

Migration

  • Needs a prerelease sideloaded onto a sandbox for live verification before the version pins in andytown move.

Written for commit 144ce86. Summary will update on new commits.

Review in cubic

An unknown tool now names the closest real tool with its signature, and
the host can claim the name as one of its own tools, so opencode tells
the model that `tools.bedrock.find_registry_items` is the agent tool
`find-registry-items` and must be called directly. A call with no
arguments sends `{}` instead of failing. A parse error says where it is.
A caught tool failure stringifies to its message instead of
"[object Object]", which was silently discarding the error for the
common `catch (e) { return String(e) }` idiom.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions

Copy link
Copy Markdown

The following comment was made by an LLM, it may be inaccurate:

…nknown tools inside resolve

The error brand now lives with the other sandbox value types, and the
intra-sandbox copy keeps a branded error as a leaf the way it keeps a Date
or a Map, so the two per-site bypasses in String() and template literals go
away. resolve takes the explain function directly instead of being wrapped
in a try/catch that re-threw the same error with new suggestions. The parse
position drops clamps for offsets that cannot occur.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant