Rust-mode round-trip rekeys mirrored memories: 245 rows re-inserted under fresh ids, dropping their embeddings and breaking any id-keyed reference

[iceteaSA]

Summary

One rust-mode activation + revert cycle (~6 minutes, on a build carrying the #390 and #391 fixes) deleted and re-inserted 245 of 3262 memory rows under fresh primary keys. Content, supersede pointer values, classification, and status were all preserved byte-for-byte — but the id change silently dropped the embeddings for those rows (they are keyed by memory_id), and would break any other id-keyed reference.

This is distinct from #390 (workspace collision — fixed, confirmed) and #391 (drain rollback — did not recur). No transform-visible error occurred; the mirror round-trip rekeys rows as a side effect of projecting module state back onto the TS store.

Evidence

Fingerprints before flip and after revert, diffed against a pre-flip VACUUM INTO backup:

                     PRE-FLIP     POST-REVERT
memories count       3262     →   3262          identical
supersede ptrs       760      →   760           identical
supersede HASH       277a2afe →   dfb8a613      CHANGED
memory content HASH  c5f97609 →   1fb35e2c      CHANGED
classification HASH  dc6a6536 →   645de167      CHANGED

Counts stable, hashes moved. Row-level diff against the backup:

pointers changed target:      0
importance changed:           0
status changed:               0
content changed:              0
rows only in live (new id):   245
rows only in backup (old id): 245

The 245 "new" rows are content-identical to the 245 "gone" rows (0 diff in either direction via EXCEPT on content). It is a pure rekey:

gone ids:  10-3629   (scattered, original keys)
new ids:   3811-4055 (contiguous, all above the pre-flip MAX id of 3810)
MAX(id):   3810 → 4055  (+245)

Impact: embeddings dropped

Embeddings key on memory_id. Rekeying orphaned then GC'd the old-id embeddings and left the new ids unembedded:

rows missing embeddings:  backup 128  →  live 300   (+172 net; 225 of the rekeyed rows unembedded)
orphaned embedding rows:  0            (old ones were deleted, not left dangling)

So 225 memories became non-recallable by vector search until a re-embed pass. That self-heals via the dreamer's re-embed, but it is silent degradation from an operation that reported success.

What is NOT damaged (checked, to scope it)

• No content loss — 245 rekeyed rows are byte-identical to their originals.
• No pointer retargeting — 0 supersede pairs changed target on the overlap.
• 71 dangling supersede pointers exist, but they are PRE-EXISTING — the identical 71-id set is present in the pre-flip backup. The rekey did not create them (verified: live dangling set == backup dangling set, exactly). Flagging so this isn't misattributed to the rekey; it looks like normal supersede-target archival and is a separate question if it's a concern at all.
• Counts stable across the board — memories/skill/notes/compartments all identical pre/post.

Why it is easy to miss

Every count is stable and all content is preserved, so a count check or a content-only hash reports "clean." Only a hash that incorporates the primary key (id || '>' || superseded_by_memory_id, and id || '|' || content) exposes it — the same paired count+hash instrument that caught #391. A timestamp audit is also blind here: the rekeyed rows carry preserved timestamps.

Mechanism (hypothesis, not source-confirmed)

The mirror projection that syncs module memory state back to the TS store appears to reconstruct rows via delete + insert rather than update-in-place, so the TS memories.id autoincrement advances and every mirrored row lands on a new key. If that is the path, the fix is to preserve the host id across the round-trip (update-in-place keyed by a stable identity, or carry the host id through mirror_identity on the return leg) so id-keyed satellites — embeddings, and anything else FK'd to memories.id — survive.

A repo-wide check of what references memories.id (embeddings confirmed; evidence/mapping tables worth auditing) would size the blast radius beyond embeddings.

Environment

Fork on upstream master (all of #376/#378/#379/#381/#390/#391 present), ck-mc rebuilt at matching generation (1160 tests). Store schema v82, 3262 memories, 760 supersede pointers. Reproduced on a single activate → TS-revert cycle. Pre-flip and post-flip backups retained; the rekey is reversible from the pre-flip backup.

[magic-alfonso]

Thanks @iceteaSA — source-confirmed. Your paired count+hash fingerprint caught exactly what count checks and content-only hashes are structurally unable to see: the mirror preserved every compared payload value while replacing durable row identities.

The reset begins module-side: authority_begin_prepare deletes rows stamped with the current context_store_uuid and authority project, then seed can reinsert them under new module ids. TypeScript received those new ids, but rememberIdentity used INSERT OR IGNORE; the old module backlink still occupied UNIQUE(domain, context_row_id), so the replacement backlink was silently discarded. Mirror pull then applied the old tombstone first, deleted the host row and its FK satellites, and the later insert could no longer find context_row_id, so SQLite allocated a fresh host id. That explains the contiguous ids above the previous maximum and the embedding loss.

The exact source discriminator is the module rows carrying the current store UUID and project, not mapping_origin. Source establishes that subset selection but cannot derive the incident count of 245 without the retained module DB; that count is directly recoverable with the matching-UUID/project query above.

The fix makes stable source identity authoritative: activation replaces stale backlinks, and steady/drain page application prebinds local source identities before tombstones. The round-trip fixture covers the stale-backlink class, a missing backlink, an identity-less module row, supersede pointers, embeddings, and verification mappings. It compares count plus SHA-256 over ordered id||content and id||'>'||superseded_by; restoring stale-backlink behavior or skipping tombstone prebinding makes the invariant red while count remains unchanged.

The blast-radius sweep confirmed more than embeddings: verification mappings cascade away; mutation-log ids, supersede targets, mural manifests, pending Synapse manifests, merge provenance, and measurement/audit ids can become stale. FTS rebuilds correctly under the new rowid, and workspace/session references are project keyed. The 71 pre-existing dangling supersede pointers remain untouched as a separate audit question. No migration or fence movement was needed.