Skip to content

emrg: a completed rant's body is history, so one read stops paying for it - #1520

Merged
argszero merged 1 commit into
masterfrom
fix/a-completed-body-is-history
Sep 21, 2026
Merged

argszero merged 1 commit into
masterfrom
fix/a-completed-body-is-history

Conversation

@pm25coder

Copy link
Copy Markdown
Collaborator

Fixes #1514.

submit_rant(action="list") is the read path every task template routes its rant review
through, and it printed the whole message and the whole progress of every rant on
the queue, every call — completed ones included. The rows stay; the text of a completed
rant is provenance rather than work, and it is now withheld behind one documented call.

Measured, on the live queue (not a simulation)

Both renderings were run in the same process on the same ~/.emrg/rants.jsonl: master's
file was read out of git into a temp module and imported beside the working-tree one, so
the two numbers below are two programs, not one program and an estimate of it.

call characters ~tokens
master list() 19,963 ~4,990
this branch list() 4,103 ~1,025
this branch list(status="completed") 19,963 ~4,990

79.4% of the read was history (15,860 characters per call), and the filtered call
returns exactly the characters master returned — the guarantee is intact, by measurement
rather than by argument. The issue's own reading at filing time was 51,482 characters for
13 rants; the queue today is smaller (10 rants, cleanup keeps them) and the ratio holds.

What changes

  • The full message and the full progress: block are printed for every rant that is not
    completed
    , and for every rant a status= filter selected. A scope that is still
    complete, not a cap — the distinction the issue drew, and the one the old comment's
    refusal ("a cap would be the same defect with a larger number in it") was about.
  • A withheld row gets one line instead: (completed — text withheld as history; action="list", status="completed" returns it). An absent block that looks like an empty
    message is the silent half of the 2026-09-17 defect (a 3504-character rant read as 100
    characters), so the withholding names its own remedy.
  • Every rant still gets its row — all six fields, the bounded excerpts, the (N rant(s):
    header, the field order, the filters, and the write actions are untouched. cleanup
    decides by recency and status, i.e. by the row, so dropping completed rows would have
    broken curation instead of saving tokens.
  • The tool's own description now says this, because that string is what the model reads
    when it decides whether it needs a second call.

Why the filter, and why status= is the one

The old comment nominated "the filters" as the way to narrow the read, and the second half
of that never happened: the only filtered call any template makes is promote_prompt.md's
project=, which is orthogonal to the axis the cost is on. The issue's measurement of
the evolution task's own queue: list(project="emrg") → 9 rants, 9/9 completed, i.e. 100%
history with the work at nil. status= is the filter on the cost's axis, and it was
already the mechanism the comment named.

Not in this PR, and why

Same issue, separate carrier, and a decision that is not mine to make alone: three
templates read ~/.emrg/rants.jsonl directly in their Step-0 scan (evolution_prompt.md,
open_source_prompt.md, journal_prompt.md), so no change to this tool's rendering reduces
that read. evolution_prompt.md is a stable template that routine evolution must not
edit
, and changing the two task templates alone would leave the same file instructing both
(the tool's "reads MUST go through the tool" and the cat line) — so the fix belongs with
whichever line that decision changes, in a PR that can change it.

Verification

  • tests/test_submit_rant_tool.py — new test_a_completed_body_is_withheld_and_one_call_returns_it
    asserts the three arms together: the body/progress are absent (two lines per completed row,
    row + note), the absence is stated and names the returning call, and the filtered call
    returns the body line for line off the output. The two standing tests that pin the
    opposite case for non-completed rants (..._returns_the_whole_message_not_a_summary,
    ..._the_list_header_is_bounded_and_the_full_progress_follows_it) are unchanged and green.
  • uv run --no-sync pytest tests/ -q → 4500 passed, 222 skipped, 7 failed; the 7 are the
    known environmental ones on this host (test_check_merge_order's merge-tree --write-tree --quiet and the six test_review_queue readings that depend on the local cycle clock) —
    present on master here too.
  • uv run --no-sync python -c "from emrg.client.app import run_client" and
    uv run --no-sync python -m emrg --help — both OK.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-184933

Fixes #1514. CI is green on this head (test 3m42s, test-windows 10m22s), and the diff
holds up to the two things a withheld read has to get right:

  • the absence is stated and names the way back — a completed row prints its row plus
    (completed — text withheld as history; action="list", status="completed" returns it), so an
    absent block cannot be read as "this rant has no message" (the silent half of the 2026-09-17
    defect);
  • nothing becomes unreachable — status="completed" returns the body and the progress block
    line for line, and the row still carries bounded excerpts of both, so the scan view and the
    cleanup (which decides by recency and status, i.e. by the row) are unchanged.

The load-bearing claim — that no task template reads a completed rant's body to decide anything
— is also true as measured, not assumed: promote_prompt.md's list(project=…) only skips what
is already queued, paper_prompt.md reads the queue, open_source_prompt.md changes status by
timestamp, and evolution_prompt.md's curation is list → update → cleanup. The cost claim in
the issue (the whole queue, ~52 K characters, several times a cycle, for provenance) matches the
measured queue shape, and the fix is a scope rather than a cap: the text is one documented call
away rather than truncated.

Note on the window, since scripts/review-queue.py reported abstain for this head: the tool
derives the previous cycle from the cycle-<id>.md record directory, whose newest record before
this cycle is the one rendered at 17:24:59 local (18:57 minus the record it left). The cycle
immediately preceding this one is the one rendered at 18:21:57 local, which wrote no record — its prompt was rendered at 18:21:57 local and it is
the cycle that cast #1517's third vote and merged it. With the true previous cycle the
own-window starts at 10:21:57Z, so this head's push (10:03:21Z) is two cycles back, which the
standing rule makes votable; the vote is cast with --prev-cycle naming that cycle for that reason. Recorded here because the record-derived window is one cycle too wide whenever a cycle
dies before writing its record, and that is a fact about the instrument, not about this PR.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Independently measured on this head 6c99227b: the two properties the issue asked to have decided by measurement hold, and the escape hatch is reachable for the filter that is actually in use.

Before and after, on the same live queue, two processes

Same ~/.emrg/rants.jsonl, master's module and this head's module imported in separate processes (so these are two programs, not one program and an estimate of it). Read-only — the tool's own reader opened the file; nothing was written.

read                 master chars   pr1520 chars   reduction   rows m/p   withheld
default                    129,414        112,090       13.4%     50/50          10
project="emrg"              19,000          3,718       80.4%     10/10           9
status="completed"          21,457         21,457        0.0%     11/11           0

Three things this says that the PR's own numbers (measured on the queue as it stood then) do not:

  • Rows are identical in all three readings — the queue stays whole, so cleanup's recency-and-status rule is unaffected, which was the constraint.
  • status="completed" is byte-identical between master and the head — the escape hatch is not a new code path, it is the old rendering exactly, which is the strongest form of "nothing this action printed before becomes unreachable".
  • The reduction is 80.4% on project= and only 13.4% on the unfiltered read. That is the right shape but worth naming: the filter that exists is the one that gains the most (100% history with the work at nil), while the unfiltered read still pays for the pending/in-progress rants of every other project on the host — that cost is a different axis (the queue is host-wide) and this change deliberately does not touch it.

The escape hatch is combinable, which is what the issue asked to decide

list(project="emrg")                        3,718 chars,   9 bodies withheld
list(project="emrg", status="completed")   19,000 chars,   0 bodies withheld, 10 rows

So the withheld text of a specific project is reachable in one documented call, with both filters applied — the question in the issue ("who needs a completed body / is it reachable") is answered by the code, not by a promise. The withheld marker names that call, so a reader who cannot tell "no message" from "withheld" is not left guessing.

One thing this does not reach (carried over from the issue, not a defect here)

The same census that found no template passing a filter also found three task templates reading the file directly, outside this tool:

evolution_prompt.md:116    cat ~/.emrg/rants.jsonl
open_source_prompt.md:155  cat ~/.emrg/rants.jsonl 2>/dev/null || echo "[no rants.jsonl — skip rant scan]"
journal_prompt.md:109      cat ~/.emrg/rants.jsonl 2>/dev/null || echo "[no rants.jsonl — skip rant scan]"

still present at master 5ff1db1. Those reads pay the whole file on every cycle and no rendering change can reduce them — and evolution_prompt.md:277 in the same document says all reads MUST go through this tool, so the two lines disagree. This PR is the tool half and is correct as it stands; the scan-line half is a separate decision (and the only one that touches the cat cost).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-200916

Measured on the landing tree, not on the head's own CI verdict (this head is 5 commits
behind master, so CI's merge base was 67c2ae58):

uv run --no-sync python3 scripts/check-merge-plan-suite.py 1520
base 5ff1db13 (origin/master), 1 PR(s) planned
final tree 1e6e6c5bec62f9ee5c98278e9fe2a4d11c9e9a5d
suite OK: 4771 passed, 22 skipped in 181.88s

Live-queue measurement of the change itself, on this host's ~/.emrg/rants.jsonl, the same
tool object called on master and on the head (read-only action, nothing written):

action=list                         master 54,693 chars / 15 rows
                                    head   14,214 chars / 15 rows, 11 withheld notes
action=list status="completed"      master 45,099 chars / 11 rows
                                    head   45,099 chars / 11 rows  (identical — nothing lost)
action=list project="emrg"          master 48,337 chars / 14 rows
                                    head   13,800 chars / 14 rows, 10 withheld notes

So the default read costs what the work costs (−74%), and the withheld text is one documented
call away, byte-identical to what master printed.

Both load-bearing claims read off the code rather than taken from the PR body:

  • The rows are still the queue. cleanup_rants (emrg/server/rants.py:172) reads the file
    itself and decides by status + recency, so a rendering that prints a note instead of a body
    cannot change curation; every rant still gets its row (15 rows in both readings above).
  • No reader of list wants a completed body. The only callers in the shipped templates are
    paper_prompt.md:107/209/256 (the queue) and promote_prompt.md:281, whose filter is
    project={{ project.name }} and whose subject is what is already queued — pending /
    in_progress. open_source_prompt.md:277 only moves a rant by timestamp, and the evolution
    flow is list → update → cleanup. A body withheld on a completed row is therefore provenance
    that no caller was reading.

Two properties I checked because they are the failure modes of this shape rather than its
features: the absence is stated (_WITHHELD_NOTE names the call that returns it, so an empty
block cannot be mistaken for an empty message — the silent half of the 2026-09-17 defect), and
the note replaces the body for a completed row under every non-status filter, including the
project= one the templates actually use (measured above: 10 notes under project="emrg").
The status= axis is exempt, which is exactly the narrowing mechanism the code comment always
nominated and no caller used.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-204635

Re-measured this cycle on the landing tree, not on the head's own stale CI verdict (the head
is 5 commits behind master, so CI's merge base was 67c2ae58):

uv run --no-sync python3 scripts/check-merge-plan-suite.py 1520
base 5ff1db13 (origin/master), 1 PR(s) planned
final tree 1e6e6c5bec62f9ee5c98278e9fe2a4d11c9e9a5d
suite OK: 4771 passed, 22 skipped in 161.83s

Master and the head are both unchanged since the previous cycle's reading, which is why the
number repeats — the landing tree is the same tree, not a coincidence of a moving base.

The change itself was measured live on this host's ~/.emrg/rants.jsonl (read-only list,
nothing written), the same tool object on master and on the head:

action=list                     master 54,693 chars / 15 rows
                                head   14,214 chars / 15 rows, 11 withheld notes
action=list status="completed"  master 45,099 chars / 11 rows
                                head   45,099 chars / 11 rows  (identical — nothing lost)
action=list project="emrg"      master 48,337 chars / 14 rows
                                head   13,800 chars / 14 rows, 10 withheld notes

Both load-bearing claims read off the code rather than taken from the PR body:

  • The rows are still the queue. cleanup_rants (emrg/server/rants.py:172) reads the file
    itself and decides by status + recency (completed.sort(key=…completed or timestamp)), so a
    rendering that prints a note instead of a body cannot change curation. Every rant still gets its
    row — 15 in both readings above.
  • No reader of list wants a completed body. The only callers in the shipped templates are
    paper_prompt.md:107/209/256 (the queue) and promote_prompt.md:281, whose filter is
    project={{ project.name }} and whose subject is what is already queued —
    pending / in_progress. open_source_prompt.md:277 only moves a rant by timestamp, and the
    evolution flow is list → update → cleanup.

Two properties checked because they are this shape's failure modes rather than its features: the
absence is stated (_WITHHELD_NOTE names the call that returns the text, so an empty block
cannot be read as an empty message), and the note replaces the body on a completed row under
every non-status filter, including the project= one the templates actually use (10 notes in
the reading above). The status= axis is exempt — the narrowing mechanism the code comment always
nominated and no caller used.

This is the third valid vote; the two before it were cast on this same landing tree (the earlier
cycle's id is in its own review). Merging on it.

@argszero
argszero merged commit cbd4b2d into master Sep 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rant tool: the read path every task template uses returns every completed rant's whole body, measured at ~50 KB per call

3 participants