feat(threads): opt in to automatic resume after connection loss

[coygeek]

Summary

Add an opt-in setting that continues a task in the same conversation after a confirmed network interruption stops it. Keep chat unchanged during healthy work, and show a compact interruption notice only while needed. Initial automatic recovery supports native Codex using its built-in OpenAI connection.

Problem to solve

A temporary network interruption can stop an active provider turn. Restoring the client connection does not itself continue that task, so the user must notice the interruption and send a continuation manually. Users need an optional way to resume eligible interrupted work without duplicating a task that kept running or finished while the client was disconnected.

Proposed behavior

• Add "Resume after connection loss" beside "Continue threads after restarts" in General settings, and beside the restart control in mobile Maintenance settings. Default it to off for fresh and existing installations. Use the helper text "Automatically resume tasks interrupted by a lost connection when connectivity returns."
• Persist the preference for the selected environment and keep it independent of restart continuation. Disclose that initial support is native Codex with the built-in OpenAI connection. Custom endpoints, proxies, and other providers retain their existing retry behavior.
• Reconcile the task when a client reconnects. Catch up with running or completed work without starting another turn. A quiet task or a lost client connection alone is not evidence that the provider stopped.
• When a supported provider confirms that a connection failure ended its turn, wait for the required provider connection to return, then continue in the same conversation with its existing context and work. Do not replay the original prompt or compete with the provider's own retries.
• Show "Connection lost. Reconnecting…" while the client cannot verify the task state. Show "Connection lost. Waiting to resume…" only for a confirmed interruption with recovery enabled. Show "Task resumed" briefly after matching provider progress confirms recovery, then remove it.
• Let Stop, disabling the setting, newer work, completion, and pending approvals or questions take precedence. If automatic recovery is unsupported or fails, show a concise explanation and preserve manual continuation.

Acceptance criteria

1. The preference defaults to off, persists per environment, and changes independently of restart continuation. When off, the application initiates no recovery turns.
2. With native Codex on its supported built-in OpenAI connection, a confirmed network interruption can recover in the same conversation once the provider connection is usable. Cover Wi-Fi loss, Ethernet loss, and DNS failure. A restored client socket alone must not authorize continuation.
3. Client-only disconnection, task completion while disconnected, and long quiet tool operations create no extra turns. Repeated reconnects, multiple clients, and provider retries cannot create overlapping or duplicate continuations.
4. Recovery waits with backoff while connectivity is unavailable and does not repeatedly create failed turns. Authentication failures, quota limits, and unrelated task errors do not qualify for automatic recovery.
5. Stop, disabling the preference, newer work, completion, and pending user interaction prevent pending recovery from overriding user intent. Recovery must not bypass approvals or questions.
6. Desktop, web, and mobile show no new chat controls during healthy work. Interruption notices are compact, accessible, and do not steal focus. A success notice requires confirmed provider progress and disappears shortly afterward.
7. Settings disclose the initial provider scope. Unsupported confirmed interruptions receive an explanation instead of an implied promise of recovery. Verify supported local and remote client connections without claiming automatic recovery for custom routes or other providers.

Affected area

Environment settings, provider continuation, and thread interruption feedback in web, desktop, and mobile clients.

Non-goals

Persistent execution-status controls, diagnostic panels, activity timestamps, manual status checks, recovery inferred from inactivity, automatic recovery from arbitrary errors, and support for custom provider routes in the initial implementation. Updates, crashes, and machine restarts remain governed by the separate restart setting.

Implementation

Proposed implementation: PR #13744. The PR contains the support boundary, verification results, and before/after images and a simulated-recovery video.

[juliusmarminge]

Triage: #13740

Verdict: Accept
Type: Feature request
Duplicate: No
Already implemented: No
Confidence: High
Suggested labels: enhancement, accepted
Scope: Large. A separate opt-in next to restart continuation, a server-side wait for one Codex connection-failure class, and compact notices on web (desktop uses that UI) and mobile.

What was asked

An opt-in, default off, that continues the same conversation after a confirmed network failure ends a turn. Healthy work stays unchanged. A client reconnect must not start a second turn if the provider is still running or already finished. The first version is native Codex on the built-in OpenAI connection. Stop, a newer user message, completion, turning the setting off, and an open approval or question all win. Auth, quota, and unrelated errors do not qualify.

What the code does today

Restart continuation is the neighboring switch, and it is a different event. continueThreadsAfterServerUpdate defaults to false in packages/contracts/src/settings.ts. It is project-overridable and fanned out in packages/client-runtime/src/state/sharedSettings.ts. The row is Continue threads after restarts in General settings, and Continue after restart on mobile Maintenance. On shutdown, ProviderService.stopAll marks a running turn. On the next boot, reconcileProviderSessions in apps/server/src/serverRuntimeStartup.ts sends one continuation. Codex advertises promptlessTurnContinuation: true, so that send is continuation: true with no prompt. Other providers get the text "Continue where you left off." Updates, crashes, and machine restarts stay on this path. A live network blip while the server process stays up never reaches it.

Codex already retries inside the turn. An error notification with willRetry: true becomes runtime.warning ("Reconnecting... 2/5") and the session stays running (CodexAdapter.ts, CodexSessionRuntime.ts). A non-retrying error, or turn/completed with failed, sets the session to error and clears activeTurnId. Nothing sends a follow-up. The typed connection failures already exist on codexErrorInfo in packages/effect-codex-app-server: httpConnectionFailed, responseStreamConnectionFailed, responseStreamDisconnected, and responseTooManyFailedAttempts ("Reached the retry limit for responses."). T3 does not branch on those variants. The only connection string it treats specially is stderr containing "failed to connect to websocket", which becomes runtime.error and does not end the turn by itself.

A lost client does not stop the provider. The Codex process belongs to the server. Web and mobile already say they are reconnecting to the environment (ChatView.tsx, mobile connection notices). That is the client socket. It is not evidence the turn stopped.

Not a duplicate

• feat(updates): continue active threads across server restarts #9167, fix(server): recover opted-in threads after machine restarts #9803, and fix(server): keep interrupted threads resumable after restarts #10421 (merged) are restart continuation. This request keeps that setting and does not change it.
• feat(chat): continue interrupted turns with one tap #11716 (open) is a manual Continue control after the user interrupts a turn. It sends a "Continue" prompt from the composer. It is not automatic, and it is not a connection failure.
• OpenCode's "connection lost. Reconnecting." is that adapter's own event stream. Leave it alone.
• fix(mobile): recover connections on a Wi-Fi/cellular handoff #12608 and the environment reconnect banners are client-to-server connectivity. fix(desktop): keep remote connections available during idle sleep #7640 is remote connections across sleep. None of them start a provider turn.

No open issue or PR waits for Codex to finish its own retries and then continues the same thread.

Implementation notes

Run this on the server, once per thread. A client that reconnects must not be what sends the turn. Several clients, or a repeated socket, would double the work.

Arm recovery only when all of these are true:

• The setting is on.
• The session is native Codex.
• The turn has completed as failed.
• willRetry is not set. While Codex is still on "Reconnecting... N/5", the turn is alive.
• codexErrorInfo is one of httpConnectionFailed, responseStreamConnectionFailed, responseStreamDisconnected, or responseTooManyFailedAttempts.

Do not arm from stderr text, from a quiet turn, or from the client socket closing. Do not arm for usageLimitExceeded, rateLimitExceeded, unauthorized, serverOverloaded, internalServerError, badRequest, or the policy errors. Those already have their own messages, and this setting must not absorb them.

Persist one marker on the session binding, the same way continueAfterServerUpdate is stored, but a different key. Clear it on success, Stop, a newer user message, completion, disabling the setting, or an open approval or question. If restart continuation is also on and the process dies while this marker is armed, only one of the two paths may send.

Send Codex continuation: true with an empty prompt. Do not replay the original user message, and do not use the restart prompt. Wait between attempts. A failed attempt must not show up as another user turn. If the continuation fails with a non-connection error, stop and leave the existing error. Stop cancels a pending wait. Disabling the setting cancels a pending wait and does not interrupt a turn that already resumed.

Put the key beside continueThreadsAfterServerUpdate: decoding default false, project overrides, shared-settings fan-out, the General row, and the mobile Maintenance row. Helper text should say native Codex on the built-in OpenAI connection. T3 does not currently record whether Codex is pointed at a custom base URL or proxy (config.toml is only mirrored by CodexHomeLayout). Do not claim those routes are excluded unless the session actually records the route. Other providers keep their current behavior and, when this setting is on, get a short explanation instead of a resume.

Notices are thread state from the server, not a second environment banner. "Reconnecting to {environment}" already covers a client that cannot see the server. "Waiting to resume…" only while the marker is armed. "Task resumed" only after later provider progress on that continuation (turn/started or a real item), then it goes away. No new composer control during healthy work. Web covers desktop. Mobile needs the same three states.

Wi-Fi, Ethernet, and DNS are the same signal if they end in those codexErrorInfo variants. One classifier, not three network detectors.

Risks

Treating client reconnect as the trigger will duplicate a turn that never stopped.

Starting a continuation while willRetry is true races Codex's own retries.

responseTooManyFailedAttempts is the retry budget, not a network interface. Keep it in the set only as the terminal failure after those connection errors, not as a reason to resume every exhausted retry.

A probe implemented as a normal user turn will litter the transcript with failed "continue" messages. The acceptance criterion is one quiet continuation after connectivity is back.

The new marker and the restart marker have to stay distinct and mutually exclusive, or a crash during the wait sends two continuations.