Skip to content

fix(client-runtime): slow config snapshots no longer drop working connections - #14518

Merged
t3dotgg merged 3 commits into
pingdotgg:mainfrom
saphid:fix/client-snapshot-timeout
Oct 14, 2026
Merged

t3dotgg merged 3 commits into
pingdotgg:mainfrom
saphid:fix/client-snapshot-timeout

Conversation

@saphid

@saphid saphid commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Model: built with Claude Opus 5.5, Claude Sonnet 5.5 in Claude Code.

Problem

A client gives each connection attempt one fixed setup budget (15 seconds per saved route). It covers resolving credentials, creating the session, opening the socket and receiving the first config snapshot. When a server is slow to build that snapshot (a cold restart with a large database on a loaded machine), the attempt times out with the socket already open and working. The client tears it down and retries against a server that is no faster, and loops. I saw this loop for about 10 minutes. Fixes #14516.

Change

EnvironmentSupervisor keeps the setup budget of 15 seconds per saved route. Each route that reaches the driver's synchronizing stage (the session exists and the socket is opening) adds 75 seconds to the attempt's deadline for the socket to open (still capped at 15 seconds by the RPC session) and the first config snapshot to arrive. A route that fails late in its snapshot therefore does not eat the next route's time. Attempts that never reach synchronizing time out as before. A socket that closes still fails the attempt immediately. The deadline uses monotonic time, so a wall-clock correction cannot cut it short.

The budget is pooled per attempt: 15 seconds for each saved route, plus 75 seconds for each route that reports synchronizing. It is not a separate 90-second cap per route. The trade-off: a socket that opens but never delivers a snapshot is now given up on after up to 90 seconds instead of 15 with one saved route. With two saved routes where the first opens and hangs, the attempt has 105 seconds (2 × 15 + 75), so falling back to the second route takes that much longer. Web, desktop and mobile share this code.

Scope and approval

Fixes triaged bug #14516. One timeout in packages/client-runtime/src/connection/supervisor.ts. No server, contract or UI changes.

Verification

Rebased on main 37de6cbde6, which added multi-route connections (#15467), and later merged with current main.

  • New test "keeps a working socket open while a slow server config snapshot arrives": the snapshot arrives 40 seconds into the attempt and the connection is kept. It fails on main, where the attempt is torn down at 15 s.
  • New test "gives the next route its own snapshot budget after a route fails while synchronizing": the LAN route fails 50 s into its snapshot and T3 Connect needs 40 s. It fails without the per-route budget and passes here.
  • "retries when a session never becomes ready" now checks that the attempt survives 89 s (15 s setup plus 75 s snapshot) and times out at 90 s.
  • vp test run packages/client-runtime/src/connection/supervisor.test.ts: 59 passed. Typecheck, lint and format are clean on the changed files.
  • Real transport check (independent validation, GPT-6.1 Sol, at c511c0d on lxso3): a loopback WebSocket server answered Ping and delayed a valid config snapshot by 20 s, driven by the production RpcSession and EnvironmentSupervisor (driver and auth dependencies bypassed). This branch connected after 20.048 s on one socket. The same fixture against main (91e93a2) gave up and entered backoff at 15.025 s. This is a controlled transport fixture, not a full server or a cold 32.8 GB database; the ~10 minute loop in [Bug]: Client: a slow first config snapshot makes clients drop working connections and reconnect-loop #14516 was not remeasured.
  • 2026-10-14, after merging current main (dff750b) with no conflicts (head f0e3657; main's supervisor changes only add a connectionLost span and a route type): vp test run on supervisor.test.ts, rpc/session.test.ts and connection/routes.test.ts: 97 passed in 3 files. packages/client-runtime typecheck exits 0 (only Effect suggestions in unrelated files). vp lint on the two changed files: clean.
  • Not exercised in a real web, desktop or mobile client against a slow server.

Implemented and tested by Claude Sonnet 5.5 and Claude Opus 5.5 in Claude Code, running inside T3 Code. Reviewed by GPT-6.1 Sol in Codex. Merge with current main and re-verification: Claude Opus 5.5 (Claude Code) via T3 Code; review GPT-6.1 Sol.

🤖 Generated with Claude Code

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Oct 1, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at c511c0d

Macroscope's review found this PR approvable — This is a focused client-runtime timeout bug fix that keeps healthy connections alive during slow configuration snapshots while preserving bounded setup behavior and fallback handling. Targeted tests cover the changed timing paths, with no schema, infrastructure, security, billing, or configuration-default changes.

You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 1, 2026 — with ChatGPT Codex Connector
@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: pingdotgg/t3code/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 137d2009-4d28-420b-badf-ec2bde5b3f7a
📥 Commits

Reviewing files that changed from the base of the PR and between 592d051 and c511c0d.

📒 Files selected for processing (2)
  • packages/client-runtime/src/connection/supervisor.test.ts
  • packages/client-runtime/src/connection/supervisor.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The connection supervisor now calculates a monotonic timeout deadline from a 15-second setup budget per route and a 75-second budget for each route that reaches synchronization. Tests cover delayed readiness, timeout, and route fallback.

Changes: Connection timeout handling

Layer / File(s) Summary
Route-aware connection timeout
packages/client-runtime/src/connection/supervisor.ts, packages/client-runtime/src/connection/supervisor.test.ts
The supervisor tracks routes that reach synchronization and recalculates the timeout deadline using per-route setup and synchronization budgets. Tests cover a 40-second synchronization, timeout after 90 seconds without readiness, and fallback to a relay route after the first route fails.

Priority: ⬆️ High

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to c511c

The extended snapshot wait appears consistent with the intended connection behavior. No actionable merge-blocking risk was established.

Security Architecture Review

Security architecture risk: 🔵 Low · up to c511c

The change gives slow servers more time to deliver their initial configuration, but also lets an unresponsive server hold a connection attempt longer. The inspected identity checks, readiness requirements, socket-open timeout and cleanup paths remain intact. No introduced security vulnerability was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — An already selected endpoint that delays or withholds its initial snapshot can retain that client's environment-connection attempt longer. The changed exposure is connection retention and delayed fallback, not additional identity or service authority. With a fixed finite route list, each route contributes at most one synchronization increment through the inspected sequential driver path.

Trust Boundaries and Controls

  • observed — The extended wait does not remove the initial-configuration environment check or allow synchronization progress alone to expose a connected session. Those controls predate this PR and remain on the inspected establishment path.

Resilience and Maintainability Implications

  • observed — Each attempt creates a fresh counter. Failed route scopes close before subsequent routes are attempted, supervisor attempts run within a scope, and lease references are cleared on exit. Timeout and interruption remain terminal establishment outcomes, preserving the inspected cleanup and failure-containment structure despite the longer deadline.
Pre-merge checks: Passed 5
Passed Passed checks: 5 passed
Check name Status Explanation
Linked Issues check Passed Issue #14516 requires the client to keep a working socket while waiting for the first config snapshot. EnvironmentSupervisor retains the 15-second setup budget until synchronizing, then adds a 75-…
Out of Scope Changes check Passed The current whole-PR summary lists changes only to supervisor.ts and its regression tests in supervisor.test.ts. The timeout change and tests directly support issue #14516. The earlier out-of-scop…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Title check Passed The title clearly and concisely describes the main change: preventing slow configuration snapshots from dropping working client connections.
Description check Passed The description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the issue, implementation, scope, tests, observed results, limitations, and the models…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

Comment @coderabbitai help to get the list of available commands.

@saphid
saphid force-pushed the fix/client-snapshot-timeout branch 2 times, most recently from 8ad513a to 592d051 Compare October 3, 2026 10:48
github-actions Bot and others added 2 commits October 5, 2026 15:47
…budget

The 15 second establishment timeout covered the whole connect attempt,
including the wait for the first server-config snapshot. A cold server
with a large database can take longer than that to answer, so the client
tore down a healthy websocket and retried in a loop. Keep 15 seconds for
resolving credentials and creating the session, then allow 75 more seconds
for the socket to open (already capped at 15) and the snapshot to arrive.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…udget

With saved alternate routes, the first route to reach synchronizing started
the only 75 second snapshot timer. If that route failed late, the next route
inherited what was left and a healthy but slow snapshot was torn down.
Each route that reaches synchronizing now adds its own 75 seconds to the
attempt's deadline, on top of the 15 seconds of setup per route, so the
attempt stays bounded and a late failure no longer eats the next route's time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@saphid
saphid force-pushed the fix/client-snapshot-timeout branch from 592d051 to c511c0d Compare October 5, 2026 10:35
@saphid

saphid commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Review requested

Date (UTC) Reviewer Where
2026-10-06 Julius Discord DM

Logged so this PR shows when a maintainer was asked to review it.

@t3dotgg
t3dotgg merged commit 5bba764 into pingdotgg:main Oct 14, 2026
28 of 29 checks passed
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Oct 14, 2026
## What's Changed
* chore(deps): Effect 4.0.4 and OpenCode 2.0.28, dropping the patches they made unnecessary by @juliusmarminge in pingdotgg/t3code#19213
* fix(server): a Claude 403 no longer tells you to log in again by @shivamhwp in pingdotgg/t3code#15872
* fix(server): persist only changed pull request links by @juliusmarminge in pingdotgg/t3code#19248
* fix(server): delegate_task explains that supervised runtime modes pause the child by @saphid in pingdotgg/t3code#14416
* fix(server): Claude subagent progress no longer re-stores the unchanged prompt by @Gigioxx in pingdotgg/t3code#18375
* fix(server): thread list reads no longer freeze the server on large databases by @t3dotgg in pingdotgg/t3code#19226
* perf(server): streamed events no longer rewrite the whole thread row by @Baker-Harrison in pingdotgg/t3code#18890
* docs: close PRs written by Gemini or Muse Wonder by @t3dotgg in pingdotgg/t3code#19286
* fix: update all no longer runs agents twice or loses background work by @t3dotgg in pingdotgg/t3code#19291
* fix(server): enforce one database owner and guard update rollback by @maria-rcks in pingdotgg/t3code#16102
* perf(server): reuse stored turn item positions by @tannerpolley in pingdotgg/t3code#19230
* perf(server): avoid quadratic resource history replay by @saphid in pingdotgg/t3code#18825
* fix(server): repair ownership UUID generation by @t3dotgg in pingdotgg/t3code#19298
* fix(client-runtime): slow config snapshots no longer drop working connections by @saphid in pingdotgg/t3code#14518
* fix(server): bound activity publishes so stalled responses release the worker by @saphid in pingdotgg/t3code#18830
* perf(shared): cached SQLite statements no longer hold large parameters by @t3dotgg in pingdotgg/t3code#19305
* fix(provider-cursor): Cursor text generation no longer leaks SQLite file handles on every call by @t3dotgg in pingdotgg/t3code#19310
* fix(server): make SQLite checkpoints durable on macOS by @t3dotgg in pingdotgg/t3code#19304
* perf(server): Codex plan streaming no longer commits every token by @t3dotgg in pingdotgg/t3code#19317
* fix(server): large file edits no longer store their full diff twice by @t3dotgg in pingdotgg/t3code#19316
* fix(web): settings save again after a stored value fails decode by @t3dotgg in pingdotgg/t3code#19312
* fix(server): HTML and Markdown attachments no longer fail OpenCode turns by @t3dotgg in pingdotgg/t3code#19319
* fix(server): worktree paths no longer vanish on git before 2.36 by @t3dotgg in pingdotgg/t3code#19315
* fix(acp): custom models now reach the agent instead of failing to open by @t3dotgg in pingdotgg/t3code#19313
* fix(server): Claude no longer shows unavailable when its health check can't run in $HOME by @t3dotgg in pingdotgg/t3code#19320
* fix(server): stalled OAuth token requests no longer block credential calls by @t3dotgg in pingdotgg/t3code#19321
* fix(server): steered Claude turns no longer undercount token usage by @t3dotgg in pingdotgg/t3code#19314
* fix(server): closing a Claude session no longer leaves plugin MCP servers running by @t3dotgg in pingdotgg/t3code#19322
* fix(client-runtime): T3 Connect link timeouts explain the relay 504 by @t3dotgg in pingdotgg/t3code#19329
* fix(web): settle shortcut moves to the next thread like the sidebar by @t3dotgg in pingdotgg/t3code#19328
* fix(acp): Copilot's default agent no longer disappears from the Agent menu by @t3dotgg in pingdotgg/t3code#19330
* perf(server): SQLite checkpoints no longer block the event loop by @t3dotgg in pingdotgg/t3code#19324
* fix(mobile): incompatible environments now say whether to update the app or the host by @t3dotgg in pingdotgg/t3code#19331
* fix(web): live reasoning stays scrolled to the newest text while it streams by @t3dotgg in pingdotgg/t3code#19334
* fix(server): T3 Connect no longer stays down after a network change by @t3dotgg in pingdotgg/t3code#19335
* fix(clients): host images say when the connection lacks file access by @t3dotgg in pingdotgg/t3code#19337
* fix(clients): a resumed subagent no longer moves the later turn's Worked for row by @t3dotgg in pingdotgg/t3code#19338
* fix: Claude and Codex no longer time out behind mise wrappers by @t3dotgg in pingdotgg/t3code#19340
* perf(server): turns no longer re-read merged PRs from GitHub by @t3dotgg in pingdotgg/t3code#19344
* docs(settings): explain that provider update checks also gate the model catalog by @fixfon in pingdotgg/t3code#18208
* fix(web): an unsaved citation comment survives a question or approval by @t3dotgg in pingdotgg/t3code#19350
* fix(server): changing Claude effort no longer fails while background agents run by @t3dotgg in pingdotgg/t3code#19341
* fix(server): service updates no longer fail on full disks or large databases by @t3dotgg in pingdotgg/t3code#19311

## New Contributors
* @Baker-Harrison made their first contribution in pingdotgg/t3code#18890
* @tannerpolley made their first contribution in pingdotgg/t3code#19230

**Full Changelog**: pingdotgg/t3code@v0.0.46-nightly.20261014.3093...v0.0.46-nightly.20261014.3105

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.46-nightly.20261014.3105
github-actions Bot added a commit to davidvanderklay/t3code-flake that referenced this pull request Oct 14, 2026
## What's Changed
* chore(deps): Effect 4.0.4 and OpenCode 2.0.28, dropping the patches they made unnecessary by @juliusmarminge in pingdotgg/t3code#19213
* fix(server): a Claude 403 no longer tells you to log in again by @shivamhwp in pingdotgg/t3code#15872
* fix(server): persist only changed pull request links by @juliusmarminge in pingdotgg/t3code#19248
* fix(server): delegate_task explains that supervised runtime modes pause the child by @saphid in pingdotgg/t3code#14416
* fix(server): Claude subagent progress no longer re-stores the unchanged prompt by @Gigioxx in pingdotgg/t3code#18375
* fix(server): thread list reads no longer freeze the server on large databases by @t3dotgg in pingdotgg/t3code#19226
* perf(server): streamed events no longer rewrite the whole thread row by @Baker-Harrison in pingdotgg/t3code#18890
* docs: close PRs written by Gemini or Muse Wonder by @t3dotgg in pingdotgg/t3code#19286
* fix: update all no longer runs agents twice or loses background work by @t3dotgg in pingdotgg/t3code#19291
* fix(server): enforce one database owner and guard update rollback by @maria-rcks in pingdotgg/t3code#16102
* perf(server): reuse stored turn item positions by @tannerpolley in pingdotgg/t3code#19230
* perf(server): avoid quadratic resource history replay by @saphid in pingdotgg/t3code#18825
* fix(server): repair ownership UUID generation by @t3dotgg in pingdotgg/t3code#19298
* fix(client-runtime): slow config snapshots no longer drop working connections by @saphid in pingdotgg/t3code#14518
* fix(server): bound activity publishes so stalled responses release the worker by @saphid in pingdotgg/t3code#18830
* perf(shared): cached SQLite statements no longer hold large parameters by @t3dotgg in pingdotgg/t3code#19305
* fix(provider-cursor): Cursor text generation no longer leaks SQLite file handles on every call by @t3dotgg in pingdotgg/t3code#19310
* fix(server): make SQLite checkpoints durable on macOS by @t3dotgg in pingdotgg/t3code#19304
* perf(server): Codex plan streaming no longer commits every token by @t3dotgg in pingdotgg/t3code#19317
* fix(server): large file edits no longer store their full diff twice by @t3dotgg in pingdotgg/t3code#19316
* fix(web): settings save again after a stored value fails decode by @t3dotgg in pingdotgg/t3code#19312
* fix(server): HTML and Markdown attachments no longer fail OpenCode turns by @t3dotgg in pingdotgg/t3code#19319
* fix(server): worktree paths no longer vanish on git before 2.36 by @t3dotgg in pingdotgg/t3code#19315
* fix(acp): custom models now reach the agent instead of failing to open by @t3dotgg in pingdotgg/t3code#19313
* fix(server): Claude no longer shows unavailable when its health check can't run in $HOME by @t3dotgg in pingdotgg/t3code#19320
* fix(server): stalled OAuth token requests no longer block credential calls by @t3dotgg in pingdotgg/t3code#19321
* fix(server): steered Claude turns no longer undercount token usage by @t3dotgg in pingdotgg/t3code#19314
* fix(server): closing a Claude session no longer leaves plugin MCP servers running by @t3dotgg in pingdotgg/t3code#19322
* fix(client-runtime): T3 Connect link timeouts explain the relay 504 by @t3dotgg in pingdotgg/t3code#19329
* fix(web): settle shortcut moves to the next thread like the sidebar by @t3dotgg in pingdotgg/t3code#19328
* fix(acp): Copilot's default agent no longer disappears from the Agent menu by @t3dotgg in pingdotgg/t3code#19330
* perf(server): SQLite checkpoints no longer block the event loop by @t3dotgg in pingdotgg/t3code#19324
* fix(mobile): incompatible environments now say whether to update the app or the host by @t3dotgg in pingdotgg/t3code#19331
* fix(web): live reasoning stays scrolled to the newest text while it streams by @t3dotgg in pingdotgg/t3code#19334
* fix(server): T3 Connect no longer stays down after a network change by @t3dotgg in pingdotgg/t3code#19335
* fix(clients): host images say when the connection lacks file access by @t3dotgg in pingdotgg/t3code#19337
* fix(clients): a resumed subagent no longer moves the later turn's Worked for row by @t3dotgg in pingdotgg/t3code#19338
* fix: Claude and Codex no longer time out behind mise wrappers by @t3dotgg in pingdotgg/t3code#19340
* perf(server): turns no longer re-read merged PRs from GitHub by @t3dotgg in pingdotgg/t3code#19344
* docs(settings): explain that provider update checks also gate the model catalog by @fixfon in pingdotgg/t3code#18208
* fix(web): an unsaved citation comment survives a question or approval by @t3dotgg in pingdotgg/t3code#19350
* fix(server): changing Claude effort no longer fails while background agents run by @t3dotgg in pingdotgg/t3code#19341
* fix(server): service updates no longer fail on full disks or large databases by @t3dotgg in pingdotgg/t3code#19311

## New Contributors
* @Baker-Harrison made their first contribution in pingdotgg/t3code#18890
* @tannerpolley made their first contribution in pingdotgg/t3code#19230

**Full Changelog**: pingdotgg/t3code@v0.0.46-nightly.20261014.3093...v0.0.46-nightly.20261014.3105

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.46-nightly.20261014.3105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Client: a slow first config snapshot makes clients drop working connections and reconnect-loop

3 participants