Skip to content

feat(recall): serve recall over HTTP so hooks stop paying node cold start - #17

Merged
MXAntian merged 1 commit into
mainfrom
feat/recall-http-endpoint
Jul 30, 2026
Merged

MXAntian merged 1 commit into
mainfrom
feat/recall-http-endpoint

Conversation

@MXAntian

Copy link
Copy Markdown
Contributor

起因

自动召回 hook 在每条 prompt 和每次匹配的工具调用上 spawnSync 一个新 node 进程跑 --recall。在真实规模的库上量了一下:

进程内 recall        6ms
同一个查询走 CLI   1591ms
→ 99.6% 是进程启动,不是查询

启动开销 = node 冷启 + 开 343MB 库 + 装 sqlite-vec 和分词扩展 + embedding 配置。而且它在关键路径上 —— UserPromptSubmit 跑在模型看到你说的话之前。

server 就在那儿跑着,这些全是热的。

改动

POST /recall —— 纯 JSON 进出,payload 跟 --recall --format json 完全一致。

刻意不做成 MCP:一个 30 行的 hook 不该为了问一个问题去讲一套面向 session 的协议。

鉴权复用 resolveHost,所以 enforce 模式照样 401,而且 provenance 仍然是通道派生的 —— 调用方可以标注自己是哪个 hook,但永远不能声明自己是哪个 host。

recallForClients() 成为唯一实现,CLI 分支改为调它。「取候选 → 过滤 → 切片 → 打点」两份实现一定会漂,这个仓已经为此付过一次学费了。

hook 先试 HTTP,任何失败都回落 spawn —— server 挂了、超时、非 200、JSON 坏了。延迟优化不能变成新的单点故障。

测出来两个 bug,都值得点名

① 把 stdin handler 改成 async 会破坏进程生命周期。

那个安全网定时器是 .unref() 的 —— 在 handler 还是同步的年代这完全正确。改成 async 之后,第一个 await 就让同步栈返回,此时 stdin 已经关闭、没有任何东西 ref 事件循环,node 在 fetch 还没来得及开 socket 之前就 exit 0 了 —— hook 静默,召回完全没发生。现在改成 ref 的;所有完成路径本来就显式 process.exit(0),所以这个 deadline 永远不会真的被等到。

② 新测试第一版是我自己写错的。

我用 spawnSync 起 hook,同时把 stub server 放在同一个进程里。spawnSync 会阻塞那个事件循环 —— stub 永远没机会应答,于是测试"证明"了 HTTP 路径是坏的,其实坏的是测试台。改成异步 spawn。

(顺带:stub 一开始只返回 1 条 hit,而 hook 要 minConsensus: 2 才注入,所以它安静退出,看起来又像 HTTP 没通。三个不同原因表现出同一个症状——静默 exit 0。)

测试

新增 recall-endpoint.integration.test.mjs,13 条断言,已接进 CI:

  • 端点契约 / 过滤真的生效 / 容量信号带回
  • CLI 与端点返回相同 id、相同顺序(防两条路径漂移)
  • 缺 query 返 400 不是 500
  • hook 确实走了 HTTP —— 用一个库里不可能产生的 sentinel 验证,而不是靠私有的 recall_log 打点
  • 最重要的那条:什么都没监听时,hook 仍然 exit 0 且静默

全量 220 绿。

已知代价

server 挂掉时,hook 变成"先等 800ms 超时再 spawn",比原来慢一截 —— 拿延迟换了可用性。阈值走 MNEME_HTTP_TIMEOUT_MS,默认压得很短(健康的 server 是个位数毫秒,超过就说明它不舒服,该直接去 spawn 了)。

🤖 Generated with Claude Code

…tart

The auto-recall hooks shell out to `node index.mjs --recall` on every prompt
and every matching tool call. Measured on a warm production-sized DB:

  in-process recall      6ms
  the same via CLI    1591ms
  → 99.6% is node startup, not the query

That is boot + DB open + sqlite-vec and tokenizer extension load + embedding
config, paid on the critical path before the model sees the prompt. The server
already holds all of it warm.

- POST /recall on the HTTP transport: plain JSON in, the same payload the
  `--recall --format json` CLI emits. Deliberately not MCP — a 30-line hook
  should not have to speak a session-oriented protocol to ask one question.
  Auth reuses resolveHost, so enforce mode 401s and provenance stays
  channel-derived: a caller may label WHICH hook it is, never which host.
- recallForClients() in index.mjs is now the single implementation; the CLI
  branch delegates to it. Two copies of "fetch, filter, slice, log" would
  drift, and this repo has already paid for that once.
- Hooks try HTTP first and fall back to spawning the CLI on any failure —
  server down, timeout, non-200, bad JSON. A latency fix must not become a
  new single point of failure.

Two bugs found while testing this, both worth naming:

- Making the stdin handler async broke process lifetime. The safety-net timer
  was unref()d, correctly, back when the handler was synchronous. Async, the
  frame returns at the first await with stdin already closed, nothing else
  references the loop, and node exits 0 before fetch opens its socket — the
  hook goes silent and no recall happens at all. The timer is ref()d now; every
  completion path already calls process.exit(0), so the deadline is never
  actually waited on.
- The first version of the new test span the hook with spawnSync while hosting
  the stub server in the same process. spawnSync blocks that event loop, so the
  stub could never answer and the test "proved" the HTTP path was broken when
  the harness was. Uses async spawn now.

New recall-endpoint.integration.test.mjs, 13 assertions, wired into CI: the
endpoint contract, filter application, CLI/endpoint id-and-order parity, 400
on a missing query, the hook taking the HTTP path (verified with a sentinel the
DB cannot produce), and — the one that matters most — the hook still exiting 0
and silent with nothing listening. Full suite 220 green.

[prediction] 修复: hook 召回从 ~1.6s 降到本地 round trip;context-builder
  以外的 recall_log 来源耗时应从 ~1000ms 掉到两位数
[prediction] 风险: server 挂掉时 hook 变成"先等 800ms 超时再 spawn",比原来
  慢一截——用可用性换了延迟,阈值可通过 MNEME_HTTP_TIMEOUT_MS 调
[prediction] 验证: recall_log 里 prompt-recall-hook / tool-recall-hook 的
  duration_ms 应显著下降;停掉 18792 后 hook 仍应静默成功

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MXAntian
MXAntian marked this pull request as ready for review July 30, 2026 08:02
@MXAntian
MXAntian merged commit 3e5e120 into main Jul 30, 2026
2 checks passed
@MXAntian
MXAntian deleted the feat/recall-http-endpoint branch July 30, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant