Skip to content

Remove shared reader bottlenecks from WTI lookups - #17

Merged
pathscale merged 4 commits into
masterfrom
fix/read-path-scaling
Sep 5, 2026
Merged

pathscale merged 4 commits into
masterfrom
fix/read-path-scaling

Conversation

@pathscale

@pathscale pathscale commented Sep 5, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Publish an immutable chunked route index protected by a local ps-reclaim domain; structural writes copy the root vector and only touched chunks, not the whole BTreeMap.
  • Use per-node RwLocks for concurrent point reads, with bounded generation retries and a canonical topology-lock fallback.
  • Drop reclamation pins before blocking on node locks; retire roots after the even-generation release and amortize registry scans at a 64-root backlog.
  • Add bulk attach_nodes/attach_multi_nodes reconstruction and replace the misleading mutable-looking iter_nodes API with snapshot_nodes -> Vec<Node>.
  • Normalize duplicate search semantics across all search backends and preserve acknowledged inserts across stale topology repairs.

Review resolution

  • B1: full-map publication clone and unbounded hot spin removed.
  • B2: persisted reconstruction API can publish once per bulk load. The corresponding WorkTable caller migration is prepared in WorkTable PR #96 and must merge after this crate.
  • B3: readers never retain a reclamation pin while blocking.
  • B4: two retries followed by the canonical progress path; caller code runs after releasing the structural lock.
  • B5: semantic trap removed through the explicit snapshot_nodes API.
  • M1: writer-interference latency and throughput were validated with the separate local wt-bench harness; that harness is not part of this PR.
  • M3/M4: recursive-lock invariant documented and duplicate search returns the first match for every backend.
  • M5: the private retirement wrapper remains T: Send, Node: Send; it can only destroy a uniquely retired pointer and cannot expose or dereference it.
  • M6: visibility/oracle tests, a Loom weak-memory model, and a manually triggered multi-seed Miri workflow cover publication and reclamation.
  • M7: version policy intentionally unchanged for the coordinated beta release.

The follow-up audits also added a dedicated last-node rekey operation, stale-high/stale-low attachment repair, bounded canonical rebuild on bookkeeping divergence, split/merge hysteresis, miss-before-copy removal, and symmetric recovery tests.

Raw performance

131,072-key pure reads, threads 1/2/4/8 (M lookups/s):

  • Arctic control: 13.644 / 26.843 / 53.410 / 107.62
  • Candidate WTI: 4.1442 / 7.8045 / 15.218 / 28.462
  • Released WTI 0.0.11: 3.7507 / 7.2218 / 6.5732 / 4.4209

At 8 readers the candidate is 6.44x released WTI and scales 6.87x from one reader.

Final local writer-interference run: one million monotonic structural inserts, four stable-key readers, three repetitions (medians):

  • Candidate quiet: 19.626 M reads/s
  • Candidate busy: 17.367 M reads/s (-11.5%)
  • Busy p99: 459 ns; p99.9: 5.25 us
  • Writer: 152.75 ns/insert

Bulk reconstruction after the audit fixes remains linear: 8,000 nodes in 1.072 ms.

Validation

  • 127/127 all-feature library tests plus Loom on the final local commit.
  • Loom publication-order model passes; weakening the final generation Release to Relaxed makes it fail.
  • Reclamation interleaving passes Miri seeds 0..8 against local ps-reclaim; stale-boundary and recovery paths pass seeds 0..4.
  • Strict all-target/all-feature Clippy, formatting, diff check, and packaged-crate compilation pass.
  • Automatic, heavy, and manual Miri workflows run on the final commit before merge.

@pathscale
pathscale marked this pull request as draft September 5, 2026 11:20
@pathscale
pathscale marked this pull request as ready for review September 5, 2026 13:05
@pathscale
pathscale merged commit 1e0e5de into master Sep 5, 2026
4 checks passed
@pathscale
pathscale deleted the fix/read-path-scaling branch September 5, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant