Skip to content

perf(daemon): resume the transcript rescan from a checkpoint - #1314

Merged
edwin-zvs merged 1 commit into
mainfrom
feat/transcript-scan-checkpoint
Aug 28, 2026
Merged

edwin-zvs merged 1 commit into
mainfrom
feat/transcript-scan-checkpoint

Conversation

@edwin-zvs

Copy link
Copy Markdown
Contributor

The problem

The daemon rebuilds several derived session fields from transcripts at
startup — seq counter, message count, last-message/last-error snippets, token
tally (spec 0103), context gauge (spec 0104), fleet token history (spec 0167)
— so records written before a field existed, or that lagged a crash, heal
themselves.

It did that by reading every transcript in full, serially, before binding
the IPC socket
. That is proportional to all history ever recorded.

On my fleet (340 transcripts, 7.0 GB, largest 294 MB) it put ~30s between
launch and the socket existing — on every start:

loaded config  02:12:34.387
listening      02:13:01.626   (+27.2s)
loaded config  15:27:20.937
listening      15:27:50.717   (+29.8s)
loaded config  18:16:51.675
listening      18:17:23.930   (+32.3s)

construct daemon start waits 5s, so it reported a daemon that had failed
while it was in fact still reading:

Error: construct daemon did not become ready within 5s; check the daemon log

Note the auto-start path (ensure_daemon_running) already polls for 60s with
a comment about "reported ~30s slow starts" — this exact problem was hit
before, fixed there, and missed in daemon start.

The fix

Checkpoint the fold. Each session stores its derived state in scan.json next
to the transcript, along with the byte length that state reflects. The next
start stats the file and reads only what was appended past that offset.
Transcripts are append-only, so a recorded length is always a line boundary
and equal lengths mean there is nothing to read.

The checkpoint is a discardable cache, never an authority — missing, stale
past EOF, or written at another version all fall back to a full walk. Deleting
every scan.json on disk costs one slow start and nothing else.

Two details that are easy to get wrong:

  • The model in effect at the resume point is part of the checkpoint. A
    cost sample is attributed to the model in effect at that point in the
    transcript
    (spec 0167). A tail scan starts after the ModelChanged that
    set it, so without persisting it, usage recovered after a resume would be
    silently unattributed where a full walk attributes it correctly.
  • Checkpoints are written during startup only, before any adapter resumes
    and while nothing can append. Sampling a length and a fold state as one
    consistent pair needs that quiet: race it one way and events are skipped
    forever, race it the other and they are counted twice.

Samples are pruned to the retention window as written, so the file cannot
accumulate history no consumer would accept back.

Also raises daemon start's readiness budget to the 60s auto-start already
allows — the first start after this change still pays one full walk to seed
the checkpoints.

Measured, against a copy of that real 7 GB fleet

APFS copy-on-write clones of all 341 real session transcripts (distinct
inodes, so nothing could reach the live data), isolated daemon dirs:

socket live scan read
cold (today's behavior) 32.2s 31,079 ms 7156 MiB
checkpointed 1.2s 24 ms 0 MiB

transcript scan complete sessions=339 up_to_date=336 tail=2 full=1 mib_read=0 elapsed_ms=24

Total checkpoint cost on disk: 0.2 MB across 339 files.

Correctness at scale

Booted checkpointed, snapshotted every summary, deleted all checkpoints,
booted cold again, diffed:

compared 339 sessions x 8 derived fields = 2712 values
RESULT: checkpoint-resumed state is IDENTICAL to a cold full walk

Tests

7 unit tests in session::scan — the central one asserts field-by-field that
a tail scan lands exactly where a full scan would; plus resume-carries-model,
current-checkpoint-reads-nothing, past-EOF rejection, version rejection,
window pruning, missing transcript.

1 end-to-end test through SessionManager::new. "Did not re-read" is asserted
the only way it can be observed from outside: the checkpoint is doctored to a
value the transcript does not support, and the next boot is required to report
it — a boot that rescanned would overwrite it.

Full workspace suite: 2863 passed, 0 failed.

Specs

  • New specs/0211-transcript-scan-checkpoint.md.
  • specs/0167 said "Nothing new is persisted for it" — that free ride existed
    only because the daemon walked every transcript anyway. Updated, along with
    the matching claim in cost_history.rs's module doc.

Binaries

Daemon-side change; the relevant binary is construct:

.claude/worktrees/transcript-scan-checkpoint/target/debug/construct

No recording — daemon startup logic with no user-visible TUI surface.

The daemon rebuilds several derived session fields from transcripts at
startup — seq counter, message count, last-message/last-error snippets,
token tally (spec 0103), context gauge (spec 0104), fleet token history
(spec 0167) — so records written before a field existed, or that lagged a
crash, heal themselves.

It did that by reading every transcript in full, serially, before binding
the IPC socket. That is proportional to all history ever recorded. On a
fleet with 340 transcripts totalling 7 GB it put ~30s between launch and
the socket existing, every single start:

    loaded config  02:12:34.387
    listening      02:13:01.626    (+27.2s)

`construct daemon start` waits 5s, so it reported a daemon that had failed
while it was in fact still reading (#1313).

Checkpoint the fold. Each session stores its derived state in `scan.json`
next to the transcript, with the byte length that state reflects; the next
start `stat`s the file and reads only what was appended past that offset.
Transcripts are append-only, so a recorded length is always a line
boundary and equal lengths mean there is nothing to read.

The checkpoint is a discardable cache, never an authority: missing, stale
past EOF, or written at another version all fall back to a full walk.
Anything carried across the resume point is part of it — including the
model in effect where the scan stopped, without which usage recovered
after a resume attributes differently than a full walk (spec 0167).
Samples are pruned to the retention window as they are written, so the
file cannot accumulate history no consumer would accept back.

Checkpoints are written during startup only, before any adapter resumes
and while nothing can append — sampling a length and a fold state as one
consistent pair needs that quiet, or the race skips events in one
direction and double-counts them in the other.

Also raise `daemon start`'s readiness budget to the 60s the auto-start
path already allows: the first start after this change still pays one
full walk to seed the checkpoints.

Startup now logs what it read, so a slow start is a readable fact:

    transcript scan complete sessions=340 up_to_date=338 tail=2
      full=0 mib_read=3 elapsed_ms=214
@edwin-zvs
edwin-zvs merged commit a365581 into main Aug 28, 2026
1 check passed
@edwin-zvs
edwin-zvs deleted the feat/transcript-scan-checkpoint branch August 28, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant