[Bug]: Message sends and queued starts decode every node of the thread first: 83 to 119 ms on 12,000 to 17,000-node threads #18438

What happened

Before a message send, a steer, a queued start and several other commands, the server reads and decodes every execution node of the thread. The command code then uses at most one node, the root node of one run. A plain send uses none.

On my install the three largest threads hold 12,274 to 16,831 nodes. A full node read on them takes 83 to 119 ms (SQL plus decoding every row). Over 7.6 days, 11,121 message sends account for an estimated 548 s of node reads. That estimate multiplies node counts by 6.9 microseconds per node. It is not timed per send.

Diagnosis

readCommandProjection (apps/server/src/orchestration-v2/Orchestrator.ts:930) asks getThreadRecords for the field list "runs", "attempts", "nodes", .... Its comment says command decisions need control records, not historical output, yet "nodes" is in the list. I checked this on main c5d7309152 (2026-10-11). The function is unchanged from b9afba59f4, where I took the measurements.

Timing against a copy of the live database, opened read-only (medians of fresh processes, Node 26.11.1 with its bundled SQLite 3.53.4):

Variation (maximum minus minimum over median) was 1% to 4% for the full read. For the by-id read it reached 33%, which is 0.03 ms of timer noise.

Nine more command types name a node they already know and fit a by-id read. They are prepared-run progress, release, fail and retry, promote-to-steer, queued-run cancel, delegated task request, and delegated completion acknowledge and dispose. About 1,046 of them ran in the same window, an estimated 35 s of node reads.

Steps to reproduce

Not deterministic from a clean install, because it needs a thread with thousands of nodes.

1. Use a thread with more than 10,000 nodes (SELECT COUNT(*) FROM orchestration_v2_projection_nodes WHERE thread_id = ?).
2. Send a message in it and look at the sqlite spans in the server trace, or time SELECT payload_json FROM orchestration_v2_projection_nodes WHERE thread_id = ? plus decoding the rows.
3. The read scales with the node count of the thread.

Version

Source checked on main c5d7309152. Measurements from a Linux nightly install in the 0.0.46-nightly series, 2026-10-10, on code of b9afba59f4.

Environment

Linux x86_64, statev2.sqlite of 15 GB with 1,450 thread rows and no sqlite_stat1 (same condition as #17955). Timings from a Node 26.11.1 harness, not from the server's own runtime.

Evidence

The three rows above. I did not time end-to-end latency per send, and I did not run the server's own SQLite. The estimate of 548 s is from node counts, not from timed commands.

Related issues

• Server commands read every node of a thread (83 to 119 ms on the largest threads) but look up at most one #17939, my Ideas discussion with the proposal and the diff (no replies yet).
• fix(server): speed up long thread message sync #17387 (merged), which sped up long-thread message sync. It does not touch command projections.
• [Bug]: Server never collects SQLite planner statistics, so large databases full-scan and stop answering #17955 (open): without sqlite_stat1 the planner can choose bad plans. This issue is about the rows read, so it does not depend on the plan.
• perf: large threads open from a viewport snapshot, and typing and streaming render less #18220 (open) changes server projection encoding and thread opening; I did not find it changing readCommandProjection.

Fix applied or workaround

None on the machine. I have two commits ready on a branch: a by-id read helper for the commands that name a node, plus tests. It adds about 283 lines and removes 72 in 6 files. About 130 of the added lines are tests. The branch: main...bompus:t3code:perf/command-projection-window. A command's projection would no longer hold stored nodes, so code reading projection.nodes there sees only the nodes the command wrote. I will rebase it and open a pull request only if a maintainer approves this direction.

Filed by

Claude Sonnet 5.5 in Claude Code