Repository navigation
feat(recall): serve recall over HTTP so hooks stop paying node cold start - #17
Merged
Merged
Conversation
…tart The auto-recall hooks shell out to `node index.mjs --recall` on every prompt and every matching tool call. Measured on a warm production-sized DB: in-process recall 6ms the same via CLI 1591ms → 99.6% is node startup, not the query That is boot + DB open + sqlite-vec and tokenizer extension load + embedding config, paid on the critical path before the model sees the prompt. The server already holds all of it warm. - POST /recall on the HTTP transport: plain JSON in, the same payload the `--recall --format json` CLI emits. Deliberately not MCP — a 30-line hook should not have to speak a session-oriented protocol to ask one question. Auth reuses resolveHost, so enforce mode 401s and provenance stays channel-derived: a caller may label WHICH hook it is, never which host. - recallForClients() in index.mjs is now the single implementation; the CLI branch delegates to it. Two copies of "fetch, filter, slice, log" would drift, and this repo has already paid for that once. - Hooks try HTTP first and fall back to spawning the CLI on any failure — server down, timeout, non-200, bad JSON. A latency fix must not become a new single point of failure. Two bugs found while testing this, both worth naming: - Making the stdin handler async broke process lifetime. The safety-net timer was unref()d, correctly, back when the handler was synchronous. Async, the frame returns at the first await with stdin already closed, nothing else references the loop, and node exits 0 before fetch opens its socket — the hook goes silent and no recall happens at all. The timer is ref()d now; every completion path already calls process.exit(0), so the deadline is never actually waited on. - The first version of the new test span the hook with spawnSync while hosting the stub server in the same process. spawnSync blocks that event loop, so the stub could never answer and the test "proved" the HTTP path was broken when the harness was. Uses async spawn now. New recall-endpoint.integration.test.mjs, 13 assertions, wired into CI: the endpoint contract, filter application, CLI/endpoint id-and-order parity, 400 on a missing query, the hook taking the HTTP path (verified with a sentinel the DB cannot produce), and — the one that matters most — the hook still exiting 0 and silent with nothing listening. Full suite 220 green. [prediction] 修复: hook 召回从 ~1.6s 降到本地 round trip;context-builder 以外的 recall_log 来源耗时应从 ~1000ms 掉到两位数 [prediction] 风险: server 挂掉时 hook 变成"先等 800ms 超时再 spawn",比原来 慢一截——用可用性换了延迟,阈值可通过 MNEME_HTTP_TIMEOUT_MS 调 [prediction] 验证: recall_log 里 prompt-recall-hook / tool-recall-hook 的 duration_ms 应显著下降;停掉 18792 后 hook 仍应静默成功 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
起因
自动召回 hook 在每条 prompt 和每次匹配的工具调用上
spawnSync一个新 node 进程跑--recall。在真实规模的库上量了一下:启动开销 = node 冷启 + 开 343MB 库 + 装 sqlite-vec 和分词扩展 + embedding 配置。而且它在关键路径上 ——
UserPromptSubmit跑在模型看到你说的话之前。server 就在那儿跑着,这些全是热的。
改动
POST /recall—— 纯 JSON 进出,payload 跟--recall --format json完全一致。刻意不做成 MCP:一个 30 行的 hook 不该为了问一个问题去讲一套面向 session 的协议。
鉴权复用
resolveHost,所以 enforce 模式照样 401,而且 provenance 仍然是通道派生的 —— 调用方可以标注自己是哪个 hook,但永远不能声明自己是哪个 host。recallForClients()成为唯一实现,CLI 分支改为调它。「取候选 → 过滤 → 切片 → 打点」两份实现一定会漂,这个仓已经为此付过一次学费了。hook 先试 HTTP,任何失败都回落 spawn —— server 挂了、超时、非 200、JSON 坏了。延迟优化不能变成新的单点故障。
测出来两个 bug,都值得点名
① 把 stdin handler 改成 async 会破坏进程生命周期。
那个安全网定时器是
.unref()的 —— 在 handler 还是同步的年代这完全正确。改成 async 之后,第一个await就让同步栈返回,此时 stdin 已经关闭、没有任何东西 ref 事件循环,node 在 fetch 还没来得及开 socket 之前就 exit 0 了 —— hook 静默,召回完全没发生。现在改成 ref 的;所有完成路径本来就显式process.exit(0),所以这个 deadline 永远不会真的被等到。② 新测试第一版是我自己写错的。
我用
spawnSync起 hook,同时把 stub server 放在同一个进程里。spawnSync会阻塞那个事件循环 —— stub 永远没机会应答,于是测试"证明"了 HTTP 路径是坏的,其实坏的是测试台。改成异步spawn。(顺带:stub 一开始只返回 1 条 hit,而 hook 要
minConsensus: 2才注入,所以它安静退出,看起来又像 HTTP 没通。三个不同原因表现出同一个症状——静默 exit 0。)测试
新增
recall-endpoint.integration.test.mjs,13 条断言,已接进 CI:全量 220 绿。
已知代价
server 挂掉时,hook 变成"先等 800ms 超时再 spawn",比原来慢一截 —— 拿延迟换了可用性。阈值走
MNEME_HTTP_TIMEOUT_MS,默认压得很短(健康的 server 是个位数毫秒,超过就说明它不舒服,该直接去 spawn 了)。🤖 Generated with Claude Code