Skip to content

fix(server): block V2 replacements until session cleanup completes - #9806

Closed
shivamhwp wants to merge 447 commits into
pingdotgg:t3code/codex-turn-mappingfrom
shivamhwp:fix/v2-session-release-timeout
Closed

shivamhwp wants to merge 447 commits into
pingdotgg:t3code/codex-turn-mappingfrom
shivamhwp:fix/v2-session-release-timeout

Conversation

@shivamhwp

@shivamhwp shivamhwp commented Sep 4, 2026 •

Copy link
Copy Markdown
Collaborator

V2 marks a provider session stopped after a 30-second cleanup timeout even when the provider is still shutting down. That allows a replacement to start too early and can revoke credentials still held by the old process.

Keep ownership of the pending cleanup until it finishes. A timeout records an error and blocks replacements and new attachments. Close, detach, and provider-wide close retries wait on that same cleanup. Shared credentials survive until the last holder finishes, and an attachment racing with close fails without revoking a peer's credential. Successful cleanup clears the error and block; failed cleanup keeps replacements blocked until the environment restarts.

Targets the Orchestration V2 branch in #2829. Includes main through b70015b6db and the V2 target through a8cc38b95b. The main integrations are separate merge commits; the original cleanup commits remain isolated in the history. The aggregate PR diff now includes main's web, mobile, server, and shared-package updates.

The merge keeps V2's run and projected-item model while carrying forward main's pasted-text attachments, connection controls, notifications, and response streaming settings. Paragraph streaming and missing-worktree fallback are adapted to the V2 execution and launch paths; retired V1 runtime files stay removed. Main advanced to 73b206f4bf during validation; that later worktree-setup feature is not included. The latest integration also includes main's CLI release archives and update/uninstall commands, and its compact-sidebar revert.

Validation on 8023b0a1d8: focused server, web, mobile, and client-runtime tests pass, including session cleanup, V2 replay, paragraph boundaries and completion, and worktree fallback. Typechecks pass for server, web, desktop, mobile, contracts, client-runtime, and shared. Targeted lint, formatting, and the exact unused-code check pass. Follow-up dfdde1f624 passed all GitHub CI jobs after fixing Hermes array compatibility and migrating notification test fixtures to V2. Latest head 96704fdb6b additionally passes 207 focused tests, server/web/desktop typechecks, CLI bundle and help smoke checks, marketing build, targeted lint, formatting, and unused-code checks. All GitHub CI jobs pass on this latest head. Vercel still reports a deployment failure without a log URL, so the PR is not fully green. Macroscope's last correctness review was skipped because its $20.01 estimate exceeded the $15 per-review limit; human review remains needed. No new browser or device verification was performed for this integration; the evidence below predates the merge.

Earlier UI verification

Before: no warning while cleanup is still running.

Before: cleanup times out without warning

After: the PR shows the cleanup timeout and blocks replacement sessions.

After: visible cleanup timeout warning

Recovery: the warning clears automatically after cleanup succeeds.

Recovery: cleanup completed and warning cleared

Captured in a real Chromium client connected to an isolated local server and the real Codex provider. Both the V2 base and PR head used a temporary five-second idle timer and a 120-second cleanup delay. The production 30-second cleanup timeout was unchanged.

  • Before, V2 base 3ed8c4df: the session was marked stopped after the timeout, with no warning while cleanup was still running.
  • After, PR head b0ccd56f: the session stayed in error and the app showed that replacement sessions were blocked until cleanup completed.
  • Recovery: cleanup finished, the stored error cleared, and the warning disappeared without dismissal or a page reload.

Temporary test hooks were removed after capture and are not included in this PR. Original PNGs and the captured session-state evidence are retained with the test artifacts.

Earlier detailed test report. The validation above records the integrated commits.

Implemented and tested by GPT-6 using Codex in T3 Code for shivamhwp.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Sep 4, 2026
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts Outdated
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts Outdated
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts Outdated
@shivamhwp

Copy link
Copy Markdown
Collaborator Author

Note: GPT-6 on behalf of shivam (@shivamhwp).

The first CI run found failures in unchanged V2-base files: Check reports formatting in seven files, Test Server 1 expects 58 migrations but the base registers 59, and Release Smoke fails on an unused expo-notifications@57.0.15 patch. None of those files are changed by this PR. Keeping those fixes separate from session cleanup.

The review fixes are in d68d735a. All 34 focused session-manager and turn-start tests pass locally, along with targeted lint and server typecheck. Waiting for the new CI and review results.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts Outdated
@macroscopeapp

macroscopeapp Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR is a broad product and platform change rather than a contained session-cleanup fix: it adds Forgejo/GitHub routing, reusable authentication, mobile widgets and paste workflows, notifications, and substantial orchestration changes. It also changes product defaults and authentication behavior, so the scope and sensitivity warrant human review.

Not approved because:

  • Per-review cost limit exceeded (workspace setting). Approvability relies on correctness review in order to determine eligibility

Review your spending limits in Billing settings, or comment @macroscope-app review this PR to bypass the limit and review now. You can add or adjust custom eligibility rules. Learn more.

@shivamhwp

Copy link
Copy Markdown
Collaborator Author

Note: GPT-6 on behalf of shivam (@shivamhwp).

Latest head bf93d875 passes Bugbot and Macroscope correctness. All reported review threads are resolved. Locally, 58 focused tests across six session-manager consumer test files pass, as do targeted lint and server typecheck.

CI remains blocked outside this change: seven unchanged files fail formatting, the migration test expects 58 entries instead of the registered 59, and Release Smoke rejects the unused Expo notifications patch. Earlier Test Server 3 logs also show stale replay expectations for the runtime-info prompt suffix. I reproduced the migration failure, the seven formatting failures, and the tool_call_read_only/cursor replay mismatch locally. The relevant adapter, migration, fixture, and release files match the V2 base.

Leaving these base fixes out of the cleanup PR pending maintainer direction. Nothing has been merged.

@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from c61d233 to c86ae9d Compare September 4, 2026 23:41
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts Outdated
Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from f2e4a0d to 3ed8c4d Compare September 5, 2026 00:45
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from 8098b97 to bc10499 Compare September 5, 2026 01:05

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bc10499e4e572b4e7f364ee5d8077acdb6f6ad92. Configure here.

Comment thread apps/server/src/orchestration-v2/ProviderSessionManager.ts
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch 2 times, most recently from b82facd to 2ac9bfe Compare September 5, 2026 04:56
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from b0ccd56 to 38e1a9a Compare September 6, 2026 04:54
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch 4 times, most recently from 8187b17 to ada72ce Compare September 9, 2026 06:39
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from 38e1a9a to e352dd3 Compare September 9, 2026 10:40
@github-actions github-actions Bot added size:XXL 1,000+ changed lines (additions + deletions). size:L 100-499 changed lines (additions + deletions). and removed size:L 100-499 changed lines (additions + deletions). size:XXL 1,000+ changed lines (additions + deletions). labels Sep 9, 2026
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from ed5dc41 to 2b1139a Compare September 10, 2026 19:28
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from 1cb4a21 to a2de137 Compare September 11, 2026 03:49
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch from e03e376 to e39429e Compare September 12, 2026 08:17
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 13, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
…ingdotgg#11518)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 13, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from 20f1f27 to 6bb0952 Compare September 13, 2026 23:48
@shivamhwp
shivamhwp force-pushed the fix/v2-session-release-timeout branch from 6bb0952 to 8023b0a Compare September 14, 2026 04:14
@github-actions github-actions Bot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:L 100-499 changed lines (additions + deletions). labels Sep 14, 2026
@juliusmarminge
juliusmarminge force-pushed the t3code/codex-turn-mapping branch 4 times, most recently from 463d0c6 to e10a1e2 Compare September 15, 2026 06:28
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 15, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
@juliusmarminge

Copy link
Copy Markdown
Member

Stale duplicate of #11492 — the same session-cleanup ownership fix exists on the newer branch. Thanks!

saphid pushed a commit to saphid/t3code that referenced this pull request Sep 15, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 15, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 16, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 16, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 16, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 16, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 16, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
saphid pushed a commit to saphid/t3code that referenced this pull request Sep 17, 2026
…finishes

A V2 provider session released its session-map entry before its scope
cleanup finished, so a replacement open could start a new process while
the old one still owned credentials and resources, and a hung cleanup
silently unblocked reuse.

Releasing sessions now keep a pending-release record until cleanup
actually completes. Replacement opens and runtime thread attachments for
its recorded threads are blocked; repeated close/detach calls join the
same cleanup Deferred instead of racing it; shared MCP credentials are
revoked only when no live session, reservation, or pending release still
holds them; a detach that loses its session to a concurrent close joins
the pending cleanup; and the 30s wait reports an honest error state
without claiming the process stopped.

Ports pingdotgg#9806 onto t3code/codex-turn-mapping, plus
post-review hardening for detach-vs-close and runtime attachment races.

Generated with Devin (swe-2-high, T3 Code via Cursor harness)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants