fix: make session shard export deterministic - #5
Open
aquelejota wants to merge 1 commit into
Open
aquelejota wants to merge 1 commit into
aquelejota wants to merge 1 commit into
Conversation
exportedAt used Date.now(), so the shard payload changed on every export
even when the session itself did not. git saw every shard as modified and
autoPushOnIdle produced a commit touching every session file on each idle.
Derive exportedAt from the session's own time_updated instead, so a repeat
export of an unchanged session is byte-identical and pushes stay incremental.
The existing regression test never caught this: under vitest the dynamic
import("node:sqlite") fails to resolve, sqliteAvailable() reported false
and every session test returned early. Fall back to createRequire so the
builtin loads under test runners and the session tests actually execute.
Refs doomsday616#3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3.
buildShard()wroteexportedAt: Date.now()into every shard, so an unchanged session produced new bytes on every export. Git flagged every shard as modified, and withautoPushOnIdleeach idle created a commit touching every session file. Measured on a real two-machine setup with ~912 shards: every push and every startup pull carried all 912 files.After deriving the timestamp from the session itself, a no-change export is byte-identical and the next push contains only genuinely changed sessions (2 files in our test, the sessions actually in use).
Two changes:
exportedAtnow comes fromsession.time_updated(src/core/sessions.ts), so a repeat export of an unchanged session is deterministic. Existing shards are rewritten once (the field value changes); after that the export is stable. The field is informational only — import readsshard.session.time_updated, notexportedAt— so there is no compatibility concern.The regression test for this already existed but never ran: under vitest the dynamic
import("node:sqlite")fails to resolve ("Failed to load url sqlite"),sqliteAvailable()returnedfalse, and every session test returned early viaif (!available) return.src/core/sqlite.tsnow falls back tocreateRequireto load the builtin, so the session tests actually execute.Verification:
npx vitest run tests/sessions.test.ts— 12 passed; the suite went from ~15ms (all early returns) to ~82ms with the database actually exercised.exportedAtline makeswrites byte-identical shards on a repeat exportfail with different bytes, so the test now guards the behavior.