Skip to content

[FEATURE]:Add automatic retry with exponential backoff for LLM API timeouts #37412

Description

@wyp2k

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Problem
When an LLM provider's streaming API accepts a connection but never returns data (e.g. server-side stall, queueing), opencode has no automatic retry mechanism. The request hangs until the configured timeout (default 10 minutes) fires, then errors out and stops the session entirely. The user must manually send a message to resume.
This contrasts with Claude Code and Codex, which both retry transient API failures automatically with exponential backoff — users never notice these incidents.
Reproduction

  1. Configure a provider (e.g. volcengine GLM) with timeout: 600000, chunkTimeout: 60000
  2. In a long session (~50K tokens/step, 20+ agentic steps), trigger a stream request
  3. Provider accepts the request but sends no data (or only SSE keepalive pings that reset chunkTimeout without delivering actual content)
  4. Session hangs for the full timeout duration, then errors with TimeoutError: The operation timed out.
  5. Agent loop stops; no retry attempted
    Root Cause
  • No retry / maxRetries option exists in ProviderConfig.options (confirmed via config schema)
  • chunkTimeout is ineffective against providers that send SSE keepalive pings (empty : ping frames) — these reset the chunk timer without delivering data
  • Timeout errors are treated as fatal for the current message, not as retryable failures
    Proposed Solution
    Add provider-level retry configuration:
    "options": {
    "maxRetries": 3,
    "retryDelay": 1000,
    "retryBackoff": "exponential",
    "retryOn": ["timeout", "5xx", "connection-error"]
    }
    Behavior:
  • On timeout / connection error / 5xx, automatically retry the request up to maxRetries times
  • Use exponential backoff (e.g. 1s, 2s, 4s)
  • Show a brief "retrying…" indicator in the TUI
  • If all retries fail, then surface the error to the user
    Additionally, consider:
  • Distinguishing SSE keepalive pings from actual data chunks in chunkTimeout logic (don't reset the timer for empty frames)
  • Treating timeout errors as retryable by default (like the AI SDK's built-in maxRetries which opencode currently doesn't expose)
    Environment
  • opencode v1.17.20
  • Provider: volcengine-plan (GLM-latest), @ai-sdk/openai
  • OS: Windows

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions