Skip to content

fix(acp): explain rejected model selections - #19699

Merged
juliusmarminge merged 5 commits into
pingdotgg:mainfrom
StiensWout:t3code/acp-model-selection-errors
Oct 20, 2026
Merged

juliusmarminge merged 5 commits into
pingdotgg:mainfrom
StiensWout:t3code/acp-model-selection-errors

Conversation

@StiensWout

@StiensWout StiensWout commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

ACP model rejection errors were hidden by generic session-open and turn-start wrappers. Invalid-parameter failures at the model-write boundary now retain a fixed message naming the selected model and explaining how to retry. Native error text stays in the cause; authentication, transport, and other configuration failures retain their handling.

This preserves @StiensWout’s implementation from #19699, resolves the conflicts by merging current main, and consolidates the regression coverage into the existing replay suite. Fixes #18586.

Verification

  • Three real-stdio replay cases cover rejection during open, rejection after a completed turn, and unchanged authentication failure. The original four cases in a one-off file became three parameterized cases in the existing replay fixture suite, removing 83 lines. Removing curated startup-message precedence reproduces the generic error.
  • 89 focused tests passed across orchestration, ACP runtime, Grok selection, and failure/error boundaries. Scoped server typecheck, targeted lint and formatting, and diff checks passed. One existing registry test timed out during concurrent resource-heavy checks, then passed in the complete 46-test sequential rerun without code or timeout changes.
  • One full adversarial review round on the entire diff against main, independently by Claude Opus 5.5 and Codex GPT-6-Sol: no confirmed findings. The implementation was separately assessed by GPT-6.1-Sol after the Astra test pass.
  • In the isolated Linux web app, a local ACP stdio fixture rejected an advertised model with -32602. Verified opening rejection, successful retry with another model, and rejection when switching the same session after a completed response. The earlier completed response remained intact. Captures use the reviewed bda5530; the baseline uses the exact three production files from main ab6be13, with the same fixture, model, prompt and viewport.
Before: generic session error After: rejected model and retry advice
Before: generic ACP session-open error After: selected model rejection with recovery advice

After a completed turn, switching to the rejected model:

Completed response retained above the model-switch rejection

Limits

The live run used a protocol fixture, not fx or Cline with a real account. Resume shares the model configuration and error-delivery paths but was not separately exercised live. Providers returning rejection codes other than -32602 retain existing handling. macOS, Windows, iOS, Android, Electron desktop, and relay/tunnel connections were not verified here.

Prepared in T3 Code using Codex GPT-6-Astra (tests), GPT-6.1-Sol (implementation assessment and integration), GPT-6-Sol and Claude Opus 5.5 (independent reviews).

…ic open failure

When an ACP agent refuses the selected model, the thread showed a generic
failure ("Failed to open acpRegistry provider session ..." on open, or
"The provider could not start this turn" on a model switch), even though
initialize and session/new had succeeded.

The model write in the ACP adapter's session setup now raises a typed
AcpModelSelectionError when the agent answers with invalid params or the
session's model option does not offer the model. Its fixed message names
the selected model; the agent's own reply stays on the cause. Sign-in,
transport, mode and other config failures keep their existing errors.
provider-core's curated cause walk recognizes the error through adapter
wrappers, and a failed session start checks that curated reason before
the wrapper's text.

This does not change which option T3 writes the model to, and does not
make agents that list a non-model selector first (such as fx) work.

Refs pingdotgg#18586

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Oct 15, 2026
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 15, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 84a9af6

Macroscope's review found this PR approvable — This localized ACP error-handling fix replaces generic session-open/turn-start messages with a curated model-rejection explanation while preserving existing handling for authentication and transport failures. The accompanying replay tests cover model rejection during opening and switching without changing product defaults or APIs.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

Only developers with an assigned seat can use this organization's usage-based review budget, and seats here are assigned manually. Ask an admin to assign a seat, or change the review continuation mode in Billing.

Next included review available in 10 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9fb9d8e4-384a-4a14-aeba-2fae1d582097
📥 Commits

Reviewing files that changed from the base of the PR and between bda5530 and 508248a.

📒 Files selected for processing: 4
  • apps/server/src/orchestration-v2/ProviderTurnStartService.ts
  • apps/server/src/orchestration-v2/testkit/OrchestratorReplayFixtures.integration.test.ts
  • packages/provider-acp/src/server/adapter.ts
  • packages/provider-core/src/server/failure.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 55e09f7b-0c3d-457d-b096-883c2dc1e0c1
📥 Commits

Reviewing files that changed from the base of the PR and between 345a1a8 and bda5530.

📒 Files selected for processing: 4
  • apps/server/src/orchestration-v2/ProviderTurnStartService.ts
  • apps/server/src/orchestration-v2/testkit/OrchestratorReplayFixtures.integration.test.ts
  • packages/provider-acp/src/server/adapter.ts
  • packages/provider-core/src/server/failure.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

The ACP adapter classifies invalid-parameter model-selection errors. Provider failure handling preserves known messages, and orchestration uses them when settling provider-start failures. Integration tests cover model-write failures during session opening, first-turn switching, and authentication.

Changes: ACP model-selection failure reporting

Layer / File(s) Summary
Classify and curate model-selection failures
packages/provider-acp/src/server/adapter.ts, packages/provider-core/src/server/failure.ts
The ACP adapter maps request errors with code -32602 to AcpModelSelectionError. Failure extraction distinguishes known domain messages from provider categories, and makeProviderFailure prioritizes explicit and known messages.
Surface and verify failure messages
apps/server/src/orchestration-v2/ProviderTurnStartService.ts, apps/server/src/orchestration-v2/testkit/OrchestratorReplayFixtures.integration.test.ts
Provider-start failure settlement prefers a known message. Replay tests assert outcomes for model-write rejections during opening, first-turn switching, and authentication.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to bda55

The reviewed change is mergeable after normal checks; no unresolved failure in model-rejection reporting was established.

Security architecture risk: ⚪ Minimal · up to bda55

The change improves rejection messages without adding access, privileges, or execution paths. Existing failure handling and message sanitization remain in place. No material security regression was identified in the reviewed changes.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed exposure is the failure explanation associated with an existing provider session or turn. The inspected comparison adds no route, credential capability, tenant selector, or execution authority; the public-entrypoint signal resolves to an internal error declaration.

Trust Boundaries and Controls

  • observed — A provider-controlled invalid-parameter response can trigger classification, but its native text does not become the curated explanation. The adapter constructs that explanation locally, orchestration gives it precedence, and provider-core sanitizes the resulting failure message.

Resilience and Maintainability Implications

  • inferred — The new error mapping preserves the existing failure-containment sequence: local selection is invalidated before configuration, and the requested selection is committed only after configuration succeeds. Startup settlement still returns failures for scheduled retry or records terminal failure according to the existing willRetry branch. Provider-side rollback after partial application remains unproven and is not a new guarantee of this change.
Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely describes the main change: improving error messages for rejected ACP model selections.
Description check Passed The description clearly explains the problem, implementation, scope, verification results, known limits, and agent usage. It references the related issue and provides detailed evidence, although it do…
Full details: Description check

Explanation

The description clearly explains the problem, implementation, scope, verification results, known limits, and agent usage. It references the related issue and provides detailed evidence, although it does not use every template heading or state explicit maintainer approval.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@juliusmarminge

Copy link
Copy Markdown
Member

Taking over this one. I will resolve the conflicts against current main, keep the minimum behavioral coverage, and rebuild the model rejection error path before pushing back here.

@juliusmarminge juliusmarminge self-assigned this Oct 19, 2026
…e/acp-model-selection-errors

# Conflicts:
#	apps/server/src/orchestration-v2/ProviderTurnStartService.ts
#	packages/provider-core/src/server/failure.ts
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 19, 2026 03:57

Dismissing prior approval to re-evaluate bda5530

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 19, 2026
@juliusmarminge

Copy link
Copy Markdown
Member

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

# Conflicts:
#	packages/provider-acp/src/server/adapter.ts
#	packages/provider-core/src/server/failure.ts
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 20, 2026 05:51

Dismissing prior approval to re-evaluate 84a9af6

@juliusmarminge
juliusmarminge merged commit 04ab2fc into pingdotgg:main Oct 20, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ACP agents with multiple category:"model" config options pick the wrong option (session fails to open)

2 participants