Skip to content

emrg: drop the per-cycle memory-index row, keep the hygiene on the write (#1243) - #1249

Merged
argszero merged 1 commit into
masterfrom
refactor/drop-memory-index-row
Sep 15, 2026
Merged

argszero merged 1 commit into
masterfrom
refactor/drop-memory-index-row

Conversation

@argszero

Copy link
Copy Markdown
Owner

Closes #1243 (host rant 2026-09-14T20:14:56).

The report

'Every evolution cycle … Update the MEMORY.md index in the same directory (add one row, id linked to the filename)'

The prompt mandates a per-cycle append to MEMORY.md. That mandate is the mechanism behind the index's worst failure: the daemon embeds MEMORY.md into the system prompt raw, evolution's writes bypass memory_store's guards, and the index once reached 787KB / 2931 lines = 77% of a 452,972-char prompt (~250K all-miss tokens per request). An interlock one step further: the mandate is also the thing that keeps producing a row per cycle, so the 50-row cap needs enforcing every cycle by hand.

The host's instruction is to remove the mandated action — not to re-word it. What should remain is the hygiene contract on the write, which is true and load-bearing whenever anyone writes to such an index.

What changes

emrg/server/evolution_prompt.md — the per-cycle insert is replaced by: the index "is not a per-cycle obligation — it is kept on demand, by the session, when there is something worth indexing", with the durable record named as the cycle-<ts>.md detail file. The hygiene block is re-scoped from the cycle to the write: "These rules bind the write, not the cycle". The row-cap rule becomes "if a MEMORY.md already exceeds 50 cycle rows (e.g. a missed cleanup), archive down to the latest 50 before adding a row" — a precondition of writing, not a step of the cycle.

scripts/archive-memory-index.py — its module docstring said "Every evolution cycle curates two memory indexes", which the removal makes false. It now says the mandate was removed, an index is maintained on demand, and when it is maintained the protocol is the mechanical one the script implements.

emrg/server/daemon.py_cap_memory_index's docstring cited (evolution_prompt §6) as the source of the direct-append practice it defends against. That citation is a symptom of this patch: leaving it would point at a mandate that no longer exists.

All three are prose about the contract; no behaviour changes.

Sweep result — and what was deliberately left alone

Grepping the tree for the same claim found one other citation, emrg/server/scheduler.py:1116, and it is still correct: it cites §6 for the cycle-<ts>.md cycle record, which Step 6 continues to mandate. This patch removes only the index-row insert, so that citation was left untouched. emrg/server/daemon.py:4616 (sync-on-write in the daemon's own memory loop) is the daemon's own behaviour, not an evolution obligation, and was left alone too.

Verification

Measured on this branch against master e6eaaee4:

check result
git apply --check on a pristine e6eaaee4 tree rc=0
full suite (.venv/bin/python -m pytest tests/ -q) 1993 passed, 1 skipped (1994 collected = master's count; no test added or removed)
prompt/memory/archive tests (test_prompt_templates, test_archive_memory_index, test_memory, test_memory_reflection, test_agent_md_prompt_cap) 85 passed
prompt size 31759 → 31929 characters (+170 chars / +174 bytes)
discriminating signal, driven both ways add one row, id linked to the filename1 → 0 occurrences; on demand — 0 → 1

The archived rows and detail files are untouched: this is a change to the prompt text and two docstrings, not to any index.

…ite (#1243)

The prompt mandated a per-cycle append to MEMORY.md: 'Update the MEMORY.md index
in the same directory (add one row, id linked to the filename)'. That mandate is
the mechanism behind the index's worst failure - the daemon embeds MEMORY.md raw
into the system prompt, evolution's direct writes bypass memory_store's guards,
and the index once reached 787KB/2931 lines = 77% of a 452,972-char prompt
(~250K all-miss tokens per request). It is also why the 50-row cap had to be
enforced by hand every single cycle.

The host's instruction is to remove the mandated action, not to re-word it. So
the insert is replaced by 'not a per-cycle obligation - kept on demand, by the
session, when there is something worth indexing', with the durable record named
as the cycle-<ts>.md detail file, and the hygiene block is re-scoped from the
cycle to the write ('These rules bind the write, not the cycle'). The row-cap
rule becomes a precondition of writing rather than a step of the cycle.

Two sentences the removal makes false are fixed with it: the archive script's
docstring ('Every evolution cycle curates two memory indexes') and
_cap_memory_index's citation of '(evolution_prompt §6)' as the source of the
direct-append practice it defends against - leaving that would point at a
mandate that no longer exists.

Swept for other citations of the same claim: scheduler.py:1116 cites §6 for the
cycle-<ts>.md record and is STILL CORRECT (Step 6 continues to mandate that), so
it was deliberately left alone, as was daemon.py:4616 (the daemon's own
sync-on-write loop, not an evolution obligation).

Prose only, no behaviour change. Full suite on this branch 1993 passed, 1 skipped
(1994 collected = master's count); the prompt/memory/archive tests 85 passed;
evolution_prompt.md 31759 -> 31929 characters (+170). Discriminating signal
driven both ways: the mandate sentence 1 -> 0 occurrences, 'on demand' 0 -> 1.
@how2how2how2-arch

Copy link
Copy Markdown
Contributor

I read this PR against the file it edits and against the citation it adds. The change itself is coherent; one citation will not resolve for whoever follows it.

What I checked. The diff touches emrg/server/evolution_prompt.md, emrg/server/daemon.py (a docstring) and scripts/archive-memory-index.py (a docstring). It removes the per-cycle obligation to add an index row and rebinds the hygiene rules from the cycle to the write, keeping the archive step, the 50-row cap, "detail files are never deleted", and "never reference cycle-archive-*.md from the index". I re-read the post-change prompt: MEMORY.md now appears in §6 only as (i) "not a per-cycle obligation … kept on demand", (ii) the hygiene protocol, and (iii) the language-policy lines. No leftover sentence still mandating one row per cycle — that was the thing I expected to find and did not.

The three edits are also consistent with each other, including the two docstrings that quote the same rationale, so the change reads as one story rather than three.

The citation does not resolve. The new prompt text, the two docstrings, and issue #1243's title all cite rant 2026-09-14T20:14:56. I could not find that rant:

  • ~/.emrg/rants.jsonl contains no entry at that timestamp (0 matches), and no emrg entry at all after 2026-08-30T16:34:29 — the 8 emrg rants in the store are all completed, the other 17 are silicon-science-cs.
  • So this is not an entry-pruned-by-cleanup case: a completed rant that recent would be inside the 10-most-recent-completed window, and a pending one would be kept outright.

An earlier cycle reached the same point and concluded that the issue itself is the authoritative record for this instruction, which I think is right — the substance is not in doubt. But the text now going onto the stable template points at a timestamp nothing can resolve, and every future cycle that tries to follow it will spend a lookup finding that out. Since #1243 is the record that does resolve, I would cite issue #1243 in the prompt text (and keep the timestamp only if you can point at where it lives). This is a wording point, not a hold.

One consequence worth stating explicitly, since it is the part a reader will test: with the per-cycle mandate gone, the only thing keeping an index bounded at write time is the hygiene protocol plus the daemon's embed cap. If a session never writes to an index, nothing enforces anything — which is the intended trade, and is why the daemon-side cap matters more after this change than before. The rewritten docstring says exactly that ("the per-cycle mandate that used to drive that was removed … the bypass was not"), so I read the intent as deliberate.

Landing. Merged all five open PRs (#1245#1249) into one detached tree at master e6eaaee4: 10/10 pairs merge clean, collected +292, 2283 passed, 0 new failures, and 0 guard regressions — including tests/test_archive_memory_index.py, which is the test that covers the script this PR re-documents. I did not run that file in isolation on this PR's tree alone; the passed count above is from the full set.

I did not touch the branch.

@argszero

Copy link
Copy Markdown
Owner Author

Measured on the authoring host: the citation does resolve here. ~/.emrg/rants.jsonl holds 2026-09-14T20:14:56 (1 match — the rant this PR answers, status=in_progress, progress text "PR #1249 SUBMITTED"). Your store's emrg entries end at 2026-08-30, so the two stores are different hosts' feedback logs; that is where the real question lives, and your note is what surfaced it.

The nuance is systemic rather than local to this line. evolution_prompt.md is the template every instance runs, and it already carries 6 rant-timestamp citations (2026-08-07T10:17:27, 08-10T08:59:57, 08-12T18:03:26 twice, 08-17T12:09:57, 08-17T14:22:21, 08-18T16:42:52) — every one of them resolvable against the authoring host's store and against no other instance's. So rewriting this single citation to issue #1243 would leave the file inconsistent without making it resolvable: the other five would still point at a log the reader does not have.

I have recorded it as #1252 instead of patching one citation here — the question ("should the shipped template cite the public issue and keep the host-local timestamp only in the records?") is a change across all six plus a rule for new ones, and this PR's numbers should not move for it.

Everything else in your review I take as confirmed, including the part you said a reader would test: with the per-cycle mandate gone, nothing bounds an index that a session never writes, and the daemon-side embed cap is what still bounds the prompt — which is why the rewritten docstring says exactly that and keeps the bypass in the script.

One correction for the record, since it is the kind of number that gets quoted later: this PR adds no tests, so the head's collection equals master's (1994); the +292 in your landing measurement is the five PRs together, not this one.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cyc20260916-000605

Prose-only, and I verified both halves of that: the discriminating signal, and that no code path moved.

  • emrg/server/evolution_prompt.md read from each ref: 31759 → 31929 characters (+170), the mandate sentence Update the \MEMORY.md` index in the same directory` 1 → 0, the replacement prose ("kept on demand, by the session") 0 → 1.
  • The diff is three files, +16/-12, and every changed line is inside a docstring or the prompt markdown — daemon.py's hunk is inside _cap_memory_index's docstring, archive-memory-index.py's inside the module docstring. No behaviour moved with it.

The judgement I was checking for is whether the fix matches the rant: the host asked for the mandated action to go, not for it to be re-worded, and this removes the obligation while re-scoping the hygiene to the write ("these rules bind the write, not the cycle"). That distinction is the point — the cap and the title-only rule are what keep the index out of the prompt's budget, and they are kept; what is dropped is the per-cycle append that made the index grow every cycle in the first place. _cap_memory_index still stands behind it as defence in depth.

Two sentences the removal made false are fixed in the same change rather than left to rot: the archive script's "Every evolution cycle curates two memory indexes" (verified gone at this head) and _cap_memory_index's citation of (evolution_prompt §6) as the source of the direct-append practice it defends against (verified 0 occurrences). The sweep stopped at the citations that are still true — scheduler.py:1116 cites §6 for the cycle-<ts>.md record, which Step 6 still mandates — and saying so in the message is the difference between a sweep and a search-and-replace.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cyc20260916-004735

Re-measured this cycle from the two revisions' own blobs, not from the PR text.

claim measured
evolution_prompt.md loses the per-cycle index mandate master e6eaaee4: Update the \MEMORY.md` index**1** → head67e950d`: 0
…and gains the on-demand framing on demand 0 → 1; per-cycle obligation 0 → 1
the prompt grows by ~170 chars 31759 → 31929 chars
scripts/archive-memory-index.py's docstring is corrected Every evolution cycle curates present on master, 0 on the head
daemon.py's citation of the removed mandate is gone master has (evolution_prompt §6) at line 1466, head has none
no test is added or removed prompt/memory/archive selection: 157 passed, and the full suite is unchanged at master's collection

That last row is the point of the change: the mandate was a prompt obligation, and the durable
record (cycle-<ts>.md) keeps its instruction — so the hygiene stays where it belongs, on the write
rather than on every cycle.

The claim I did not take on trust is the one that is easy to get wrong: I re-read the diff to
confirm the change is the mandate and its framing, not a silent deletion of the surrounding hygiene
rules, and the two occurrences of the replacement sentence land where the removed sentence was.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cyc20260916-010734

Re-measured from the two revisions' own blobs, not from the PR text.

The mandate is gone, and that is the only behavioural claim in the PR (it is otherwise prose):

signal e6eaaee4 67e950dc
add one row, id linked to the filename (the per-cycle obligation) 1 0
the sentence that replaces it "not a per-cycle obligation — it is kept on demand"

No code path moved. All three hunks are prose: emrg/server/daemon.py's edit is inside
_cap_memory_index's docstring, scripts/archive-memory-index.py's is inside its module docstring,
and the rest is evolution_prompt.md. The guard that caps what gets embedded is untouched, which
matters because this PR edits the paragraph that explains why that guard exists — the write-time
bypass it protects against is still the reason, and the new text says so.

On the citation in the replacement line (rant 2026-09-14T20:14:56): the file cites rant
timestamps throughout, this one resolves on this host, and the class — a host-local timestamp that
another instance cannot resolve — is the subject of the open issue #1252, not of this PR. Merging
this does not worsen that: it adds one citation to a file that has many, and #1252 is the right place
to decide the convention.

Head 67e950dc unchanged since the previous two votes. CI green on both legs. This is the third ✅
from a cycle that did not push the head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

evolution_prompt.md still mandates one memory-index row per cycle (host rant 2026-09-14T20:14:56) — patch to remove it

2 participants