Repository navigation
feat: ship transcript sweep entry point with the engine - #19
Merged
Merged
Conversation
The conversations table and the recall conversation-fallback shipped a while back, but nothing fed them — the table sat at 0 rows, so the verbatim fallback layer effectively did not exist. What kept it broken was structural, not a bug: the only call site of indexSessionTranscripts() lived in one contributor's private harness repo, so fixing the engine reached nobody else. Shipping the sweep next to the engine closes that gap — clone mneme and the ingest is already there, no per-machine path wiring. Log path is overridable via MNEME_SWEEP_LOG (use "-" for stderr only); logs/ is gitignored. Verified on merged main: files=1494 rescanned=4 skipped=669 newMessages=22 in 0.4s. Follow-up to #18 review caveat #1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DottytheHomeless
added a commit
that referenced
this pull request
Aug 3, 2026
…allowing them (#20) Chinatsu flagged the bare `catch {}` while reviewing #19 as a non-blocking note. It is worth a bit more than that: under cron the log file is this script's only observable surface, so a permission error there does not just make debugging confusing — it turns the sweep into a black box that looks idle rather than broken. That is the same shape as a monitor that fails open. A failed write still must not take the sweep down, so: warn once on stderr with the errno and the override hint, set a flag, stay quiet afterwards. Verified with a negative control (log path under a file, so mkdir gives ENOTDIR): warning fires, sweep still exits 0. Normal path emits no warning, and MNEME_SWEEP_LOG="-" stays silent as before. Co-authored-by: 小梦 <xiaomeng@clawgamers.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
DottytheHomeless
added a commit
that referenced
this pull request
Aug 3, 2026
…ing it when it doesn't (#21) Swapping a hook from recallMemories() to recallMemoriesHybrid() looks like a one-line upgrade. On its own it is a no-op, and nothing says so. _embeddingConfig is only assigned inside initMemory(). A caller that imports index.mjs and calls recall directly goes through getDb() and never touches initMemory(), so _embeddingConfig stays null, recallMemoriesHybrid() takes its fallback branch, and returns plain FTS rows. Measured on the live hook: hybrid and FTS return identical rowids in identical order in ~2ms with zero network, while the embedding endpoint answers fine (HTTP 200, 1024d, 221ms). The hybrid path had never run there. Three changes, because one is not enough: 1. hooks/mneme-recall.mjs — the runner moves into this repo (same reasoning as PR #19 for transcript-sweep.mjs: one live copy, next to the engine it imports). It loads .env.local before calling initMemory(), mirroring mcp-server.mjs, which already carries a comment about this exact failure. 2. recallMemoriesHybrid() no longer degrades silently: it warns once per process per reason and tags the returned array with non-enumerable _degradedTo / _degradeReason, so a caller can assert it is really on the hybrid path instead of assuming. The reason is computed after touching getDb(), since sqlite-vec loads lazily and would otherwise be misreported as unloaded on a process's first call. 3. The hook races hybrid against a budget (MNEME_HOOK_HYBRID_TIMEOUT_MS, default 800ms) and serves the FTS rows it already holds if hybrid overruns. UserPromptSubmit is on the synchronous critical path of every prompt. Measured after the fix, per stage: env 2ms, import 7ms, initMemory 37ms, first FTS query 459-473ms, hybrid 230-419ms. The first FTS query, not hybrid, is the largest single cost — and it is a cost the FTS-only version was already paying. Recall quality changes as expected: all 5 sampled queries return different rows than FTS. Co-authored-by: 小梦 <xiaomeng@clawgamers.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #18 — closes the structural gap that review caveat #1 pointed at.
Why
The
conversationstable, its FTS index, and the recall conversation-fallback have all been onmainfor a while. Nothing fed them. The table sat at 0 rows, so the verbatim fallback layer effectively did not exist.What kept that broken wasn't a bug in the engine — it was where the code lived. The only call site of
indexSessionTranscripts()was a script in my private harness repo. So #18 fixing the indexer still reached nobody: another harness would have to know the function exists, write its own runner, and hardcode a path to wherevermnemesits on that machine.Reviewing my own runner turned up two more holes worth naming, because they generalize:
file:///E:/project/mneme/index.mjs— even if someone got the script, it wouldn't run.So the failure mode is one step earlier than "other harnesses can't pick up the fix": I couldn't pick it up on a second machine.
What this does
Adds
scripts/transcript-sweep.mjsto the engine repo. Clone mneme and the ingest is there — relative import, no path wiring.MNEME_SWEEP_LOGoverrides the log path;-means stderr only<repo>/logs/, now gitignored so a clone doesn't dirty its tree on first run.transcript-ingest-state.jsonis safe too (full rescan is idempotent viaINSERT OR IGNORE, just slower)Verified
Against merged
main(44aad8efe):Incremental rescan picks up the live sessions and skips the rest, which is the #18 fix doing its job. Earlier full backfill on this machine: 17,303 messages / 670 sessions in 30s.
Note for reviewers
No new engine behavior here — this is the runner only. The rollout notes (how each harness picks this up, measured baselines, known edges) are in the team KOS card
mneme-transcript-verbatim-rollout.🤖 Generated with Claude Code