Skip to content

Hermes 0.9.0: import offer, queued writes, capture within server limits (HER-02/07/08) - #53

Merged
Pavel-Tk merged 5 commits into
mainfrom
fix/hermes-import-spool
Oct 11, 2026
Merged

Pavel-Tk merged 5 commits into
mainfrom
fix/hermes-import-spool

Conversation

@Pavel-Tk

Copy link
Copy Markdown
Contributor

Fixes three Hermes findings from the 2026-10-11 customer test, and ports the unmerged Hermes import offer (68c97a1) onto current main.

Changes

  • Import offer (HER-02). Ported 68c97a1 onto main. The offer is put to the agent once per connection (asked_at in import-offer.json), not on every turn until answered. Imports go through substrate_import (preview / start / status / decline) and the spool's memory_import. Tool and system guidance say a direct request for past conversations goes to substrate_import, never memory_remember, and that a failed import must not be reported as done.
  • Queued writes (HER-07). memory_remember / memory_forget return status: "queued" when the write is durable in the spool but the synchronous call could not finish (offline, timeout, 5xx, 429, unknown outcome). The sender is started so the write syncs by itself, and the text says not to retry. Only a write that was neither sent nor queued reports {"error": ...}.
  • Capture within server limits (HER-08).
    • Live capture sends only messages since the last one enqueued for the session (fingerprint-checked; falls back to the window if the history changed).
    • Envelopes are split into parts of at most 128 messages and 200 KiB (contract.capture_turn_parts). History import uses the same splitter.
    • The local validator enforces the server's redaction bounds: 128 items per array or object, depth 8, strings over 8192 bytes.
    • The client refuses requests over 262144 bytes as payload_too_large (was invalid_request at 512 KiB).
    • The spool splits a capture_turn refused as invalid_request / payload_too_large (counter split), trims one oversized message as a last resort (trimmed), and quarantines only otherwise.
    • Resumed history imports judge each part by its own offsets, so a crash after part 1 does not skip part 2.
  • Cross-host: MAX_MESSAGES_PER_ITEM 4096 -> 128 in the Claude and Codex substrate_sync.py copies (byte-identical, as before). Same server bound: their turns over 128 messages were refused too.
  • Versions: substrate-hermes 0.9.0 in plugin.yaml (both), PLUGIN_VERSION, and the release check, matching the repository VERSION. No tag, no release.

Root cause of HER-08

substrate-v5/agent-api/redaction.mjs (redactValue, MAX_KEYS_PER_LEVEL = 128) replaces any array or object with more than 128 items by the string "[TRUNCATED]". A Hermes capture replays the whole session, so from message 129 on, payload.messages became a string. validateHermesLedgerEnvelope then returned invalid_request, importErrorCategory mapped that to rejected, and the plugin quarantined it as a permanent refusal. I confirmed this by running the server's redactValue on a 136-message array: it returns "[TRUNCATED]". The contract still said "<= 4096 messages per turn", which the server never honours.

Tests

  • Full suite: 387 passed, 5 skipped (was 370 passed, 5 skipped before this PR; the skips need a server checkout or node, as before).
  • ruff check ., compileall, check_public_hygiene.py, JSON manifests, and build_release.py --check all pass.
  • New tests:
    • import offer asked once per connection, decision remembered, unanswered offer not repeated, direct import still starts;
    • import routing: tool and system guidance name substrate_import, not memory_remember;
    • offline remember and forget return queued and enqueue a durable write; a write that cannot be enqueued still fails;
    • 150-turn session: every message sent once, in order, each envelope valid and within 128 messages and the byte budget;
    • an oversized or rejected turn is split and delivered, with no quarantine;
    • contract bounds (129 messages refused, strings over 8192 refused, split parts contiguous, deterministic history ids);
    • multi-part history turn resumes part by part;
    • client refuses >256 KiB request as payload_too_large.
  • No live end-to-end run: there is no Hermes binary on this machine.

Risks and notes

  • A turn is no longer re-sent in full each time. If a process restarts, the first turn of each session re-sends the whole window (chunked). Server-side dedupe is by event id, as before.
  • docs/mcp-contract.md still says "<= 4096 messages per turn" in section 5 (a shared doc, left alone here). The effective bound is 128 until the server's redaction is changed (see below).
  • Server option (not made here): raise the redaction bound for payload.messages, or return payload_too_large instead of [TRUNCATED]. The plugin side works with the current server.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b

Ubuntu and others added 5 commits October 11, 2026 22:09
Port of 68c97a1 onto origin/main (e5af0a1): past-conversation import offer,
history.py replay through the spool, substrate_import tool. Versions and
release check bumped to 0.9.0 to match the repository VERSION.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b
Signed-off-by: Pavel-Tk <110731619+Pavel-Tk@users.noreply.github.com>
…pture within server limits

HER-02: the import offer is put to the agent once per connection (recorded in
import-offer.json), and a direct request for past conversations goes to
substrate_import. The tool, the explicit-write tool and the system prompt all
say so: never memory_remember for an import, and say plainly when an import
did not run.

HER-07: memory_remember and memory_forget report status "queued" when the
write is durable in the spool but the synchronous call could not finish
(offline, timeout, 5xx, rate limit, unknown outcome). The spool sender is
started so it syncs by itself, and the message says not to retry. Only a write
that was neither sent nor queued reports an error.

HER-08: the server's redaction keeps arrays and objects of at most 128 items
(substrate-v5 agent-api/redaction.mjs). A capture_turn with more messages
reaches the ledger as payload.messages == "[TRUNCATED]" and is refused as
invalid_request; the plugin quarantined it. Now:
- live capture sends only messages since the last one enqueued for the session
  (fingerprint-checked, falls back to the window when the history changed);
- every envelope is split into parts of at most 128 messages and 200 KiB with
  contiguous offsets (contract.capture_turn_parts), and history import uses the
  same splitter;
- the local validator enforces the server bounds (128 items per array or
  object, depth 8, strings over 8192 bytes refused);
- the client refuses requests over 262144 bytes as payload_too_large, not
  invalid_request;
- the spool splits a capture_turn refused as invalid_request or
  payload_too_large (counter "split"), trims a single oversized message as a
  last resort (counter "trimmed"), and quarantines only otherwise;
- resumed history imports judge each part by its own offsets.

Versions: substrate-hermes 0.9.0 (plugin.yaml, client, release check), matching
the repository VERSION. Claude and Codex sync scripts: MAX_MESSAGES_PER_ITEM
4096 -> 128, the same server bound (their long turns were refused too).

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b
Signed-off-by: Pavel-Tk <110731619+Pavel-Tk@users.noreply.github.com>
The server's redaction bound (128 items per array) makes the effective limit
128 messages per capture_turn, not 4096. Parts of about 200 KiB, and the
/mcp body limit of 262144 bytes.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b
Signed-off-by: Pavel-Tk <110731619+Pavel-Tk@users.noreply.github.com>
The tool now starts the spool sender, so the same write (same event id) is
delivered via memory_import as well. The test asserted the write stayed
pending, which raced with the sender on CI (3.11).

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b
Signed-off-by: Pavel-Tk <110731619+Pavel-Tk@users.noreply.github.com>
…he real profile

The tool now starts the spool sender, so tests that assumed "no sender" raced
with it on CI (3.11): a sender batch could land before an assertion on hits or
pending. Those tests now wait for the spool to settle. The import-offer test
no longer unsets HERMES_HOME, and the process-wide spool is reset per test, so
no test writes into the real ~/.hermes profile.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJwSpwAivFKrfusNkeh96b
Signed-off-by: Pavel-Tk <110731619+Pavel-Tk@users.noreply.github.com>
@Pavel-Tk
Pavel-Tk force-pushed the fix/hermes-import-spool branch from 4d7ac61 to 385785a Compare October 11, 2026 22:11
@Pavel-Tk
Pavel-Tk merged commit af0a37b into main Oct 11, 2026
5 checks passed
@Pavel-Tk
Pavel-Tk deleted the fix/hermes-import-spool branch October 11, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant