perf(server): per-thread settlement and PR checks no longer rebuild the whole thread list

[t3dotgg]

On a long-running server with about 5,000 threads and 130 projects, the settlement and PR discovery reactors read the full shell snapshot for every per-thread event. One full read loads every active thread, runs 6 queries, and resolves repository identity for every project. One turn end queued about 3 of these full reads, and the drain worker does not merge duplicates. This came from a long-uptime slowdown report (0.0.40, 9 days of uptime).

Fix

• New helper readSweepSnapshot(snapshots, threadId | null) in ThreadPullRequestReactor.ts.
  • With null, it calls getShellSnapshot() as before. The 1-minute sweeps, merge sweeps, and settings sweeps do not change.
  • With a thread id, it reads getSnapshotSequence, then getThreadShellById, then getProjectShells for the thread's project and the project its saved PR names. Settlement checks that second project.
  • The sequence is read first, so the thread is at least as new as the sequence that guards thread.auto-settle and thread.pull-request.sync. A race can only cause a rejected command. It cannot hide a change.
• ThreadSettlementReactor and ThreadPullRequestReactor use this helper for all sweeps.
• PR discovery now prunes pending backfills only for the thread it read. A one-thread read cannot show which other threads are gone.
• The per-thread settleThread decision no longer opens a span. A full sweep made one span per candidate thread.
• PullRequestSyncReactor does not change here. #13704 already moved it to a query that reads only threads with linked PRs.

Related: #12161 added the per-thread settlement triggers.

Verification

• A real SQLite test in ProjectionSnapshotQuery.test.ts seeds a thread with a linked PR, a latest turn, and a session, plus a thread whose branch PR names another project. It asserts that readSweepSnapshot for each thread returns the same sequence, thread, and project shells as getShellSnapshot. It fails if the helper drops the PR's project or if getThreadShellById drifts from the full snapshot.
• The reactor tests record each shell read as a thread id or as a full read. They send per-thread events (session-set ready, PR sync, turn-diff-completed) and assert a one-thread read. If the reactors go back to the full read, they fail with an assertion, not a timeout.
• A new check sends an event for another thread during a pending backfill retry. It fails if a one-thread read prunes other pending backfills.
• vp test run on Layers/ProjectionSnapshotQuery.test.ts, ThreadPullRequestReactor.test.ts, PullRequestSyncReactor.test.ts, and Layers/OrchestrationReactor.test.ts: 64 passed.
• vp test run ThreadSettlementReactor.test.ts -t ThreadSettlementReactor: 20 passed. The "storage cleanup" tests in this file also fail on unchanged main on this machine. They pass in CI.
• vp lint and vp fmt on the changed files, and vp run --filter t3 typecheck.

Made by Claude Opus 5.5 (1M context) in Claude Code, running in T3 Code.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Improvements
  • Thread-specific pull request and settlement updates now use data scoped to the affected thread and its related projects, while full sweeps continue to use the complete snapshot.
  • Updates check the affected thread when handling pending backfills, and missing threads are looked up before a retry.
  • These changes preserve full-sweep behavior while reducing unnecessary full-snapshot reads for thread-specific updates.

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.5 KiB	13.5 KiB	+35 B (+0.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	7.1 KiB	7.1 KiB	−1 B (−0.0%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.5 KiB	+36 B (+0.5%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	56.2 KiB	56.3 KiB	+44 B (+0.1%)	66.4 KiB	✅
Codex	Live turn messages	9	10	+1 (+11.1%)	21	✅
Claude	Total thread wire	13.5 KiB	13.5 KiB	+22 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	7.1 KiB	7.1 KiB	+4 B (+0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+18 B (+0.3%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	57.0 KiB	57.0 KiB	0 B (0.0%)	66.4 KiB	✅
Claude	Live turn messages	9	9	0 (0.0%)	21	✅

Baseline: 6530de0 · PR result: bebf1ef · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 114.0 KiB
• Claude decoded thread snapshot: 114.7 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: pingdotgg/t3code/.coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 714f841e-1b44-49d9-95bd-530c1e3a76ff

📥 Commits

Reviewing files that changed from the base of the PR and between 0fcc5b0 and bebf1ef.

📒 Files selected for processing (1)

• apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

---

📝 Walkthrough

Walkthrough

Single-thread sweeps in the pull-request and settlement reactors now read thread-scoped snapshots. Full sweeps continue to read full snapshots. The tests track thread-specific reads and verify requests for the expected thread, including a missing-thread backfill case.

Changes

Thread-scoped reactor snapshots

Layer / File(s)	Summary
Read and use thread-scoped snapshots apps/server/src/orchestration/ThreadPullRequestReactor.ts, apps/server/src/orchestration/ThreadSettlementReactor.ts, apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.test.ts	A shared helper reads the requested thread and related projects for single-thread sweeps. The reactors use that snapshot. Pull-request backfill pruning checks only the requested thread during a single-thread sweep. settleThread now uses Effect.fnUntraced. The parity test checks targeted snapshots against shell snapshots.
Verify thread-specific reads apps/server/src/orchestration/ThreadPullRequestReactor.test.ts, apps/server/src/orchestration/ThreadSettlementReactor.test.ts	The test harnesses track full and thread-specific reads and provide thread and project snapshot queries. Tests verify reads for the expected thread, including a missing-thread backfill case.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant Reactor as ThreadPullRequestReactor
  participant SnapshotReader as readSweepSnapshot
  participant Query as ProjectionSnapshotQuery
  Reactor->>SnapshotReader: Request snapshot for threadId
  SnapshotReader->>Query: Read snapshot sequence
  SnapshotReader->>Query: Look up thread by ID
  SnapshotReader->>Query: Load related project shells
  SnapshotReader-->>Reactor: Return targeted snapshot

Loading

Suggested reviewers: bil0000

Merge Risk: ⚪ Minimal · up to bebf1

No actionable merge-blocking risk is identified; merge after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main performance change: per-thread settlement and pull-request checks no longer rebuild the full thread list.
Description check	✅ Passed	The description explains what changed, why it changed, implementation details, verification coverage, and test results. It does not use the template headings exactly and omits the checklist, but the r…

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Approved at bebf1ef

Macroscope's review found this PR approvable — This is a contained server performance refactor that preserves full-sweep behavior and narrows only already thread-scoped reads, with parity and reactor tests covering the changed paths. It introduces no schema, deployment, security, billing, authentication, product-default, or static-analysis-suppression changes.

Notes:

• All code in this push has already been reviewed. Approvability was decided on eligibility alone.

^{You can add or adjust custom eligibility rules. Learn more.}

Dismissing prior approval to re-evaluate 0fcc5b0

Dismissing prior approval to re-evaluate bebf1ef

[macroscopeapp]

This production Effect helper accepts ProjectionSnapshotQueryShape as an explicit service dependency. Could it acquire ProjectionSnapshotQuery.ProjectionSnapshotQuery with yield* instead, and have both reactors and the parity test provide the query through their layers? That keeps service dependencies in the Effect environment rather than passing service instances between production modules.

^{Posted via Macroscope — Effect Service Conventions}

[t3dotgg]

Note

🤖 Claude Opus 5.5 responding on behalf of Theo

Keeping the argument. readSweepSnapshot is a read helper, not a service factory: both reactors already acquire ProjectionSnapshotQuery with yield* in make (ThreadPullRequestReactor.ts:107, ThreadSettlementReactor.ts:83) and pass that instance down. Reading it from the environment inside the helper would leak the requirement into ThreadSettlementReactor.start, which calls runSweep directly for merge events (ThreadSettlementReactor.ts:378), so it would need an extra Effect.provideService to keep its Effect<void, never, Scope> type.