Skip to content

Serve history pages of a resident session from its cached log - #457

Merged
Tryanks merged 1 commit into
mainfrom
cache-session-log-for-paging
Sep 16, 2026
Merged

Tryanks merged 1 commit into
mainfrom
cache-session-log-for-paging

Conversation

@Tryanks

@Tryanks Tryanks commented Sep 16, 2026

Copy link
Copy Markdown
Owner

For a long thread that was only opened (never appended to in this process), every history snapshot and page re-read and re-parsed the whole JSONL, and turn_aligned_start re-folded from record 0 each call. After #454 a 203k-record thread needs 16 pages instead of ~1,000, but each still paid the full read.

Change

  • AppState::event_records (the existing per-session record cache) now holds a SessionLog: the records, the pure fold, and the indices of turn-opening records. turn_aligned_start is a binary search over those indices. Envelope, SESSION_HISTORY_RECORDS and paging semantics are unchanged.
  • One parse per residency: opening a session (TimelineLoadTarget::Active) loads the log on the mailbox in the same turn the snapshot needs it; the background timeline load clones the cached fold instead of re-reading. Background (orchestrate child) loads keep parsing off the mailbox and install the log on completion. record_event_at still lazily loads on first append.
  • A non-resident session is read cold and not retained, so paging it never grows the map.
  • Memory policy: a log exists for live and parked sessions only (bounded by the resident LRU). release_stale_session_logs runs at every residency exit; it queues a store-writer flush barrier and drops the log when the barrier echoes, unless the session became resident again or appended meanwhile (re-armed). A cold read after the drop therefore sees every append the log accepted, so the JSONL lagging the writer can never truncate a reopened thread.
  • Snapshot total_turns now comes from the cached fold rather than the resident timeline, which was still Timeline::default() while the async load was pending.
  • SessionStore::event_reads() counter behind a new tcode-services/test-support feature (resolver 3; dev-dep only, zero cost in production).

Timings (real 25.2 MB / 202,941-record log, snapshot + 16 pages)

before after
debug 11.62 s 0.52 s
release 1.08 s 24 ms
The one-time read moved into select_session (73 ms release, 676 ms debug) and the second parse for the timeline load is gone.

Tests

  • history_pages_of_an_opened_session_parse_the_log_once (old code: one read per page → fails).
  • session_log_follows_residency_and_flushes_before_release (old code never dropped entries → fails).
  • history_paging_bench (#[ignore], TCODE_HISTORY_BENCH_LOG=<path>).
  • parked_readopt_refolds_events_appended_while_parked now appends through record_event (the production writer) instead of writing to the JSONL behind the runtime; the contract it protects is unchanged.

cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo test --workspace --locked, cargo machete green.

A thread that was only opened in this process re-read and re-parsed its
whole JSONL for the initial snapshot and for every history page, and each
window start was found by folding the log from record 0: 25 MB / 203k
records cost ~0.6 s (debug) or ~65 ms (release) per page, for ~16 pages.

The existing per-session record cache (`AppState::event_records`, until now
filled only by in-process appends and never emptied) becomes `SessionLog`:
the records, the pure fold of them, and the indices of the records that
open turns, extended by each append. A session opened for a client loads
its log once on the mailbox, in the same turn as the snapshot that needs
it; the timeline load clones the cached fold instead of parsing and folding
again, and a background load caches its result on completion. Pages align
their start by binary search over the turn-start indices, and the snapshot
reports the fold's turn count instead of a resident timeline that may still
be loading.

Memory policy: a log exists for live and parked sessions only. When a
session leaves residency, a store-writer barrier is queued and the log is
dropped once it echoes, so a later cold read sees every append the log
accepted; appends racing the barrier re-arm it. Paging a non-resident
session reads cold and retains nothing.

Snapshot + 16 pages of the 203k-record thread: debug 11.6 s -> 0.52 s,
release 1.08 s -> 24 ms (`history_paging_bench`, ignored, driven by
TCODE_HISTORY_BENCH_LOG). `SessionStore` counts `read_events` calls under
the new `test-support` feature so a test can prove the single parse.
@Tryanks
Tryanks merged commit 4f81376 into main Sep 16, 2026
6 checks passed
@Tryanks
Tryanks deleted the cache-session-log-for-paging branch September 16, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant