Feature hasn't been suggested before.
Describe the enhancement you want to request
Supersedes #7602 (Jan 2026) and the closed PR #20105 (Mar 2026, auto-cleaned). Incorporates lessons learned from #20105 and aligns with the current Effect-based session architecture.
Problem
When a model is unreachable, rate-limited, or times out, the session retry policy (session/retry.ts) retries the same model on the same provider with exponential backoff. After retries are exhausted, halt() is called and the session dies. There is no mechanism to fall back to a different model.
This causes long-running agent workflows to fail on transient provider issues that a different provider would handle fine.
Proposed Behavior
After the retry policy exhausts for a transient error (5xx, 429, network failure, timeout), instead of halting the session:
- Look up the configured fallback model(s) for the current model
- If a fallback exists and is available, switch to it and re-run the LLM stream
- Track which fallback was used for status reporting
- Emit a status event so the UI can show the model switch
- If the fallback also fails, try the next in the chain
- If all fallbacks are exhausted, halt as before
Triggers
- Rate limits (429, Too Many Requests)
- Server errors (500, 502, 503)
- Network failures (ECONNREFUSED, ECONNRESET, ETIMEDOUT, fetch failed)
- Timeouts (header timeout, stream timeout — the primary use case)
- Model unavailable (404, model not found)
Not triggered by:
- Auth errors (401 — different provider has different creds)
- Context overflow (413 — needs compaction, not a model switch)
- Validation/prompt errors
Config Schema
Aligns with the existing provider.models config structure:
Each entry in the fallback array is a "provider/model-id" string. The array defines an ordered fallback chain — tried in sequence until one succeeds.
Architectural Notes
PR #20105 implemented fallback as LLM middleware via wrapLanguageModel. This approach has two issues:
- Interacts badly with the session retry loop — middleware runs inside the AI SDK's own retry, which conflicts with the Effect-based
SessionRetry.policy
- Only 1 fallback — hardcoded Copilot-to-Bedrock mapping, not a configurable chain
The recommended approach is to implement fallback at the session processor layer (processor.ts), after Effect.retry(SessionRetry.policy(...)) exhausts:
Effect.retry(SessionRetry.policy(...))
-> Effect.catch(fallback) // instead of Effect.catch(halt)
-> Effect.catch(halt) // if all fallbacks exhausted
This keeps fallback logic in the Effect-based session pipeline where the error context is richest.
References
Feature hasn't been suggested before.
Describe the enhancement you want to request
Supersedes #7602 (Jan 2026) and the closed PR #20105 (Mar 2026, auto-cleaned). Incorporates lessons learned from #20105 and aligns with the current Effect-based session architecture.
Problem
When a model is unreachable, rate-limited, or times out, the session retry policy (
session/retry.ts) retries the same model on the same provider with exponential backoff. After retries are exhausted,halt()is called and the session dies. There is no mechanism to fall back to a different model.This causes long-running agent workflows to fail on transient provider issues that a different provider would handle fine.
Proposed Behavior
After the retry policy exhausts for a transient error (5xx, 429, network failure, timeout), instead of halting the session:
Triggers
Not triggered by:
Config Schema
Aligns with the existing
provider.modelsconfig structure:{ "provider": { "anthropic": { "models": { "claude-sonnet-4": { "fallback": ["openai/gpt-5", "google/gemini-3-pro"] } } }, "openai": { "models": { "gpt-5": { "fallback": ["anthropic/claude-sonnet-4", "google/gemini-3-pro"] } } } } }Each entry in the
fallbackarray is a"provider/model-id"string. The array defines an ordered fallback chain — tried in sequence until one succeeds.Architectural Notes
PR #20105 implemented fallback as LLM middleware via
wrapLanguageModel. This approach has two issues:SessionRetry.policyThe recommended approach is to implement fallback at the session processor layer (
processor.ts), afterEffect.retry(SessionRetry.policy(...))exhausts:This keeps fallback logic in the Effect-based session pipeline where the error context is richest.
References