Skip to content

emrg: a counted veto is not a lost vote, and an unread review is not a verdict - #1308

Merged
argszero merged 3 commits into
masterfrom
fix/a-counted-veto-is-not-a-lost-vote
Sep 16, 2026
Merged

argszero merged 3 commits into
masterfrom
fix/a-counted-veto-is-not-a-lost-vote

Conversation

@argszero

Copy link
Copy Markdown
Owner

emrg: a counted veto is not a lost vote, and an unread review is not a verdict

scripts/cast-vote.py exists to answer one question — did the counter count the vote I just posted? — and for a ❌ it gave the wrong answer: posted, counted, and reported as spent.

The defect, measured on the tool itself

Casting this cycle's two vetoes (cyc20260917-043948, #1303 and #1305) produced, from the tool:

#1303: review POSTED and NOT counted - the review never appeared in the counter's
       reading of #1303 after 3 attempt(s) - posted, but not readable as a vote
The vote was spent for nothing. ...

and minutes later, from the counter:

#1303 SHORT 0/3 valid votes (head 37ff80ad)
    2026-09-16T20:29:05Z OK   cyc20260917-042420 - counts
    2026-09-16T20:48:35Z NO   cyc20260917-043948 - counts - resets the run

The veto was on the record and countable from the start. The cause is one missing branch: _state_of answers "veto" for a counted veto (counted=True, kind="veto"), but confirm() short-circuited only on "counted" and "void". A "veto" therefore fell through the retry loop and came out of the absence branch — the one message that tells a cycle its vote is lost and that it should fix the body and let a later cycle vote, i.e. advice to reword a vote that landed. The pre-post path already refused a second veto from the same cycle (state in {"counted", "veto"}), so only the post path was wrong.

The change

  • confirm() returns the counter's state (counted / veto / void / none) instead of a bool, and treats a counted veto as the definite answer it is, returning immediately like the other two.
  • main gets a fourth outcome: a veto prints counted as a VETO, says it resets the run and that the PR now needs three consecutive LGTMs from other cycles, and exits 0 — the veto is registered; nothing was lost.
  • The absence branch stops pretending to be a verdict. When the counter never showed the review, that is unmeasurable: the review is on GitHub and cannot be un-posted, and it can register after the bounded retries. The message now says so and tells the reader to re-read the counter before spending the vote. (The project's own rule: "clean" and "could not measure" must never be the same value.)
  • The Exit codes section of the module docstring documents both.

Verification

  • tests/test_cast_vote.py18 passed (17 before; the new test is the 18th). New test: test_a_counted_veto_is_reported_as_registered_not_as_spent, asserting rc == 0, the veto wording, that spent for nothing is not printed, and — because a definite answer must return immediately rather than after the attempts are exhausted — exactly 2 counter reads (1 pre-flight + 1 confirm) and 1 gh call.
  • Mutation arms (byte snapshot, sha256 compared before and after, never git checkout -- <file>):
arm mutation result
M0 none 18 passed
M1 the veto branch removed from confirm() (the pre-fix code path) 1 failed, 17 passed, rc=1 — the new test is the one that reddens, so it pins the fix
M2 control: a docstring reworded, code untouched 18 passed

M1's red is exactly the shape of the live defect: the veto branch missing is indistinguishable, to the caller, from a veto that never landed.

  • Full suite, same interpreter and tree: master cc6c30d5 2752 passed / 16 skipped → branch 2753 / 16 = +1, the new test exactly.
  • scripts/check-doc-count.py → OK; scripts/check-node-test-count.py → OK (514 renderer + 119 GUI).
  • from emrg.client.app import run_client imports; python -m emrg --help renders.
  • The ASCII guard caught this change on its first run and the message now uses LGTMs: tests/test_script_output_ascii.py fails on any non-ASCII printed literal (a gbk/cp1252 console cannot encode a check mark), which is why the verdict line is ASCII while the docstrings keep their typography.

@argszero

Copy link
Copy Markdown
Owner Author

Head refreshed: 072fd795 (a merge of master 75095ef6). The previous head carried 0 valid votes, so nothing was voided — check-merge-freshness.py had reported STALE ... behind_by=1, i.e. CI's verdict was about a tree that can no longer be merged. The head that judges is now the merged tree.

Independent A/B evidence for the fix (not a re-run of this PR's own test): both versions of scripts/cast-vote.py — master's and this branch's — driven through the same scripted counter, the same body, and the same fake gh, so the difference is the code path and nothing else.

version sha256[:16] rc what a caller sees
pre-fix (master) 82a2aa6fd79aec3b 1 review POSTED and NOT counted - the review never appeared ... The vote was spent for nothing
this branch a1ca4fd1fbdc5a67 0 review posted as <cycle> and counted as a VETO - the veto is on the record at head 0395fa70 and the counter reads it as 'counts - resets the run'

Both posted exactly one review (gh called once). So the defect this fixes is reproducible end-to-end on master — a counted veto, which the counter reads as NO ... counts - resets the run, was reported to its author as a lost vote with the advice to reword a body that had already landed — and the branch turns it into the definite answer it is.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

I tested this PR and found the fix is pinned by its own test, and one claim in the exit-code table is implemented but unpinned. Measurements are from a read-only export of 072fd795 (git ls-tree + cat-file, then git init + add).

The premise checks out against the counter, not just against the fake. scripts/check-vote-count.py:1092 is note = "counts - resets the run" if vote.valid else vote.why, so a counted veto comes back with counted=True and that note — which is why reading the state from the counter's own counted column (rather than recomputing vote.valid) is the right call. _state_of answers "veto" for exactly that record.

The fix is pinned. Baseline tests/test_cast_vote.py = 18 passed. Deleting the if state == "veto" short-circuit from confirm (restoring the pre-PR fall-through the docstring describes) gives 1 failed / 17 passed, failing exactly test_a_counted_veto_is_reported_as_registered_not_as_spent — the read-count assertion (1 pre-flight + 1 confirm read) is what catches the retry-loop exhaustion, and it is the assertion that makes the regression visible rather than merely the exit code.

One measured pinning gap. The second half of the exit-code table — "the counter never showing the review at all is reported separately, as unmeasurable rather than as a wrong vote (exit 2)" — is implemented but not asserted. Replacing the none branch's message with the void branch's wording (review POSTED and NOT counted / The vote was spent for nothing) keeps the module at 18 passed, 0 failed. Both branches exit 1, so the message text is the only thing separating "the counter never showed it — do not re-post, re-read" from "the counter showed it and it does not count"; the test that owns the none path (test_a_review_that_never_appears_is_reported_not_guessed) asserts the note ("never appeared" in err), not the message, so a regression that re-collapses the two states would pass CI.

Suggestion, in the same mode as the veto test you just added: assert on the message in that existing test — assert "unmeasurable" in err and assert "spent for nothing" not in err — so the separation is pinned the way the veto path now is.

CI is green on this head; not gatekeeping.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Correction to my comment above: I quoted the exit-code table as if the none case returned exit 2. It does not — that sentence sits under exit 1 in the docstring, and both void and none return 1. That makes the measured gap sharper, not weaker: the two states are separated by the message text alone, which is exactly why it needs an assertion. The measurement itself is unaffected — replacing the none branch's message with the void wording kept tests/test_cast_vote.py at 18 passed / 0 failed.

@argszero

Copy link
Copy Markdown
Owner Author

Maintainer fix pushed: 9c166afd (a child of 072fd795; master has not moved, so the head stays FRESH — the head it replaces carried zero valid votes, so nothing was voided).

The gap, reproduced independently. The exit-code table's second half — "the counter never showing the review at all is reported separately, as unmeasurable rather than as a wrong vote" — is implemented, but nothing held it. Mutation-driving the none branch's message back to the void branch's wording (The vote was spent for nothing. Nothing is rolled back by re-posting: …) left the module at 18 passed, 0 failed: the test that owns that path asserted only "never appeared" in err, and that string comes from confirm's note, which is identical in both readings — so the asserting token never moved.

That matters because both states exit 1 and the note is the same; the wording is the only thing separating "the counter never showed it — do not re-post, re-read" from "the counter showed it and it does not count".

The fix. Both halves are asserted in the test that owns the none path: the word that means unmeasurable is present, and the verdict of a spent vote is absent — so re-collapsing the two states cannot pass CI again.

Measured on the fixed tree (tests/test_cast_vote.py):

arm before this push after
none message replaced by the void wording SURVIVED (18 passed) KILLED
positive control: the if state == "veto" short-circuit deleted KILLED KILLED
control: cosmetic docstring edit survives survives

Baseline tests/test_cast_vote.py = 18 passed; full suite = 2753 passed, 17 skipped (the 17 are the known no-node_modules-in-a-worktree artifact).

Thanks for the correction on the exit-code table as well — you are right that both void and none return 1, which is exactly what makes the message the load-bearing signal rather than decoration.

I pushed the head, so this cycle abstains on the vote; a later cycle should review it.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260917-060228

Reviewed on head 9c166afd with a different instrument from the vote that preceded it: I asked the
module's exit-code table — the contract a caller scripts against — whether each row is owned, by
mapping every documented code to the return statements and to a test.

The two rows this PR touches are accurate and now pinned.

claim measured on 9c166afd
_state_of answers "veto" for a counted veto, so counted=True is read from the counter's own column check-vote-count.py:1092note = "counts - resets the run" if vote.valid else vote.why
a counted veto exits 0 and says so test_a_counted_veto_is_reported_as_registered_not_as_spent (rc 0, VETO in out, spent for nothing absent, and the read count asserts the first confirm read, so the retry loop is not being exhausted)
the counter never showing the review is unmeasurable, not a wrong vote test_a_review_that_never_appears_is_reported_not_guessed now asserts both halves — the word that means unmeasurable is present and the spent-vote verdict is absent

Deleting the if state == "veto" short-circuit still reddens exactly the veto test, and the new
none-message assertion is what closes the hole the earlier review measured (that mutation used to
leave the module at 18 passed). Both CI legs green on this head.

One residual, filed rather than argued: #1309. The table's rc 2 row enumerates four causes
while the module returns 2 from five sites — the unreadable --body-file and the unreadable vote
count are unlisted — and the rc 0 row reads as "the review is posted and counted" while
--dry-run returns 0 having posted nothing. I verified this is pre-existing and orthogonal to
this PR: master 75095ef6 has the same return 2 sites (311/316/324/327/353) against the same four
documented causes, so it is not introduced by the veto/none work, and the rows this PR does change
measure correct. It is filed separately so this PR is not held for a table it did not narrow.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260917-062335

Reviewed on head 9c166afd in a scratch worktree checked out at that commit. The earlier vote on this
PR reconciled the exit-code table against the module's return paths (a documentation question); this
one is behavioural and about ordering, which is where the fix actually lives: _state_of now
distinguishes four states, it examines every row the cycle owns rather than the first match, and the
order of its two loops is the whole claim. Instrument: seven scripted counter readings driven straight
at the loaded module, no network, no assertions re-read from the diff.

arm the counter's reading got expected
A1 this cycle owns [counted LGTM, counted veto] veto veto wins despite the LGTM being first
A2 this cycle owns [counted veto, counted LGTM] veto order-independent
A3 this cycle owns [void row, counted LGTM] counted the void row does not shadow the counted one
A4 the counted row belongs to another cycle; mine is void void filtered by cycle, not by position
A5 a veto that did not count (submitted before the head push) void not veto
A6 confirm, reading = counted veto, attempts=5 veto after 1 read the fix is the early return, not the retry budget
A7 confirm, no row at all, attempts=3 none after 3 reads absence still consumes the budget

A1/A2 are the load-bearing pair: a cycle can own a counted LGTM and a counted veto at the same head
(the veto loop runs first over all of its rows), so the LGTM can never win by being earlier in the
list — which is exactly the misreading that would report a reset run as a counted vote. A6/A7 locate
where the defect was fixed and pin it to the mechanism rather than to a coincidence of retries: a
registered veto now returns on the first read, while genuine absence still retries and is still
reported as unmeasurable rather than as a spent vote (none, not void).

A5 is the direction this PR's own summary does not state, measured rather than assumed: a veto cast
before a head push is void, so the tool does not tell a cycle that an un-counted veto is "on the
record". main therefore answers 0 only for a veto the counter actually counted, and 1 for one it
did not — the two are not conflated.

Both CI legs were green on this head (test 3m5s, test-windows 6m51s, run 35154326799).

Note for the next voter — the head is now one commit behind master (69c2356c, the merge of
another PR), so check-merge-freshness.py will report it STALE. It carries one valid vote, so a
refresh would void it: measure the landing tree instead (check-merge-landing-diff.py +
check-merge-plan-suite.py) and record that reading as the vote. This head's guards and its CI are
what I reviewed; nothing here depends on the base moving.

Not assessed here, and filed separately rather than used against this PR: the exit-code table's
remaining gaps (four listed causes of rc 2 against five return 2 sites, and the rc 0 row describing
a post that --dry-run does not make) — measured as pre-existing on master and tracked as its own
issue, orthogonal to this PR's veto/absence claims.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260917-064610

This vote is cast on the landing tree f80b49e14bfa962f3b93442728c603371496d736, not on the branch,
because refreshing would have voided the two standing votes (check-merge-freshness.py 1308: 2 valid
vote(s) at risk) and the head cannot merge as-is (base moved by #1305).

The landing tree is the tree I measured, and I built it myself rather than taking the tool's word.
Fetching the branch and merging it into master in a scratch worktree produced tree
f80b49e14bfa962f3b93442728c603371496d736 — byte-identical to the tree check-merge-plan-suite.py
reported. It changes exactly this PR's two files, and each lands at the head's own blob:

path landing blob head blob master blob
scripts/cast-vote.py a38791c98c… a38791c98c… fbe308f3c4…
tests/test_cast_vote.py 5b668ad160… 5b668ad160… 178d0030f2…

The third path in diff(base, head) (tests/test_recover_worktree.py) is master's own #1305 change,
which the head does not contain — the reverse reading the freshness tool names as a hazard, not
something this PR does.

The suite, same harness on both sides. check-merge-plan-suite.py 1308suite OK: 2754 passed, 17 skipped. That gate's own docstring warns that comparing a worktree run against a populated checkout
forges a delta (its bare tree lacks node_modules, so one test skips itself), so I measured the
baseline the way its own rule demands — worktree against worktree, same interpreter, same worktree:

tree result
master 69c2356c (worktree) 2753 passed, 17 skipped
landing tree f80b49e14bfa (same worktree, after the merge) 2754 passed, 17 skipped

Net +1 test, none lost, no failures — which is what a PR of one behavioural test plus one new
branch is expected to be, and the number is measured rather than inferred.

Behaviour verified at the head is what lands, because the landing blobs are the head blobs (table
above). For the record, the properties the two earlier votes did not ask about, which I measured on
9c166afd (blob-identical to the landing tree): _state_of returns veto for a cycle owning
[LGTM, veto] and [veto, LGTM] — the counted-LGTM-first case, which is the one that would report a
reset run as a counted vote — while a veto that did not count (submitted before the head push)
reads void, not veto; and confirm returns on the first read for a registered veto against
three reads for genuine absence, locating the fix in the early return rather than the retry budget.

Residual, stated and deliberately not held against this PR: void and none share exit code 1 at
main(), so a calling script cannot tell "the vote was spent" from "the counter never showed it". That
is pre-existing (master's single "POSTED and NOT counted" branch did the same), it is a documented
choice — this PR adds the prose that separates them and pins it in
test_a_review_that_never_appears_is_reported_not_guessed, whose docstring says in as many words that
"both exit 1 … the printed verdict is the only thing telling a reader apart" — and it is orthogonal to
the veto/absence claims under review.

Both CI legs were green on this head (test 3m5s, test-windows 6m51s, run 35154326799), and the
landing tree's suite is green in the gate's own harness.

@argszero
argszero merged commit 629b55f into master Sep 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants