Skip to content

bees/merger/publisher: reviewer bee as a service, merger reads discounted checks, publisher heading passes check (Closes #5776) - #5777

Open
gHashTag wants to merge 56 commits into
masterfrom
claude/review-bottleneck-issues-64b338
Open

bees/merger/publisher: reviewer bee as a service, merger reads discounted checks, publisher heading passes check (Closes #5776)#5777
gHashTag wants to merge 56 commits into
masterfrom
claude/review-bottleneck-issues-64b338

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Closes #5776

Why

The review column did not drain. These measurements are from 2026-10-03:

  • The merger merged nothing. auto-merge-ready-prs.yml merged 0 pull requests since 2026-09-20; all 300 merges were by hand.
  • Its gate could never pass. It required every check to be green, while advisory checks are red on nearly every PR and spec-guards is red on master itself.
  • No reviewer was running. The reviewer bee was a session someone had to remember to start.
  • The publisher broke every queen PR. Each PR from tools/queen/publish.py failed check, because the publisher wrote (published YYYY-MM-DD) as the last parenthesis of the heading.

What changes

File Change
tools/bees/reviewer.py (new) The reviewer bee runs as a launchd service, 3 reviews at a time. The trusted runner gathers the facts: the checks the ruleset requires, and for each red check its failing step, log tail, and result on master. A claude -p judges the PR with --restricted --safe-mode --strict-mcp-config --tools Read,Grep,Glob and no token in its environment. The runner then validates the verdict, re-reads the head, and approves and labels as t27-bees[bot]. A request for changes is posted as a COMMENT, so it never blocks a manual merge.
.github/workflows/auto-merge-ready-prs.yml A red check passes only when all three hold: the ruleset does not require it, it has concluded, and the bot's approval of that head contains discounted-check: <name> -- <why>. Required checks must have posted and passed. If the ruleset cannot be read, the gate fails closed. The workflow now also runs on pull_request_target: labeled (bee-reviewed); it never checks out PR code.
tools/bees/merger_gate_selftest.py 14 new scenarios. A new drift check confirms the reviewer ignores exactly the checks the merger ignores.
tools/queen/publish.py The heading is now # NOW -- Published: <title> (YYYY-MM-DD). The self-test imports the real check_now_entry_shape.check_entry and runs it on the generated entry.
tools/bees/README.md Documents the service, the discounted-check rule, the event trigger, and the verify counts.

Verified

Not done here

  • The launchd job is not loaded yet.
  • The open queen PRs still carry the old heading until they are republished or amended.
  • The merger self-test is not wired into CI.

Foreign code: owner approval

This branch edits Python (tools/bees/reviewer.py), which the only-t27 rule
(specs/policy/own_language.t27) admits only under the owner-approved-foreign
label. The owner's standing approval of 2026-10-06 (translated): put the
owner-approved-foreign label on yourself and do the work, do not ask about
labels. The owner asked the same day to fix the reviewer bee's second-model rule
(489bae8, plan B34). The debt -- the reviewer in t27 -- is #6198/#5980.

🤖 Generated with Claude Code

@gHashTag @claude
bees/merger/publisher: reviewer bee as a service, merger reads discou…
b242bf4 
…nted checks, publisher heading passes check (Closes #5776)

The merger merged 0 pull requests since 2026-09-20: its all-green gate
cannot pass while advisory checks (and spec-guards on master) are red.

- tools/bees/reviewer.py: the reviewer bee as a launchd service. The runner
  gathers facts (ruleset-required checks, failing step, log tail, the same
  check on master); a sandboxed claude -p (Read/Grep/Glob, no token) judges;
  the runner validates the verdict, re-reads the head, approves and labels
  as t27-bees[bot].
- auto-merge-ready-prs.yml: a red check passes only if not required, concluded,
  and discounted in the bot's approval of that head. Required checks must have
  posted and passed; an unreadable ruleset fails closed. Also runs on the
  bee-reviewed label.
- tools/queen/publish.py: heading ends in (YYYY-MM-DD) as check_now_entry_shape
  requires; self-test runs that checker on the generated entry.

Self-tests: reviewer 65/65, merger 26/26 (master's workflow fails 4), publish
26 shapes (old heading fails).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:40:00 UTC

Summary

Status Count
Total Open PRs 43
PRs with Failing Checks 40
PRs with All Checks Green 3
READY 1
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 40 + 0 + 0 = 41, and there are 43 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions github-actions Bot mentioned this pull request Oct 3, 2026
Open
@gHashTag @claude
bees/reviewer: re-read the PR list once when mergeability is UNKNOWN …
1abadb4 
…(Refs #5776)

The first listing after the base moves reads UNKNOWN for every pull
request; measured on 2026-10-03 it reported 0 to review where a second
listing found 10, so a whole launchd interval passed with nothing reviewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag @claude
bees/reviewer: write the launchd job with plistlib (Refs #5776)
532e95e 
The string template put '2>&1' into XML unescaped; plutil rejected the
plist ('unknown ampersand-escape sequence'), so launchd could never load
it. The self-test now parses the job it writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:45:01 UTC

Summary

Status Count
Total Open PRs 44
PRs with Failing Checks 42
PRs with All Checks Green 2
READY 1
FAILING 42
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 42 + 0 + 0 = 43, and there are 44 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:45:31 UTC

Summary

Status Count
Total Open PRs 44
PRs with Failing Checks 42
PRs with All Checks Green 2
READY 1
FAILING 42
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 42 + 0 + 0 = 43, and there are 44 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag @claude
bees/reviewer: an unreadable pull request skips itself, not the whole…
7b25962 
… run (Refs #5776)

The first launchd run exited 1 on a TLS handshake timeout reading one
linked issue; nothing was reviewed that interval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Oct 3, 2026
Merged
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:49:52 UTC

Summary

Status Count
Total Open PRs 45
PRs with Failing Checks 41
PRs with All Checks Green 4
READY 1
FAILING 41
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 41 + 0 + 0 = 42, and there are 45 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 3, 2026
Merged
Merged
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions github-actions Bot mentioned this pull request Oct 3, 2026
Merged
gHashTag and others added 2 commits October 3, 2026 23:56
@gHashTag @claude
bees/reviewer: an expired agent login stops the run and charges no he…
b2fe74b 
…ad (Refs #5776)

Under launchd the first real run hit 'Failed to authenticate: OAuth
session expired', recorded agent-failed on four heads (two strikes and a
head is never reviewed again), and kept going. Now AgentUnavailable stops
the remaining reviews, records nothing, exits 1, and logs the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag @claude
bees/reviewer: run the agent on a Keychain-held setup-token (Refs #5776)
8fb8496 
The CLI's own OAuth session cannot refresh under launchd. The operator
stores a 'claude setup-token' token as Keychain item t27-bees-claude-token;
the runner hands it to the agent's environment only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:57:15 UTC

Summary

Status Count
Total Open PRs 47
PRs with Failing Checks 44
PRs with All Checks Green 3
READY 1
FAILING 44
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 44 + 0 + 0 = 45, and there are 47 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 16:57:26 UTC

Summary

Status Count
Total Open PRs 47
PRs with Failing Checks 44
PRs with All Checks Green 3
READY 1
FAILING 44
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 44 + 0 + 0 = 45, and there are 47 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions github-actions Bot mentioned this pull request Oct 3, 2026
Merged
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

This was referenced Oct 3, 2026
Merged
Merged
@gHashTag @claude
census: quiet "named a path but not quiet" 162 -> 164 (Refs #5776)
b509a90 
The merger's gate in auto-merge-ready-prs.yml changed shape. One step
left the census (`if [ "$FAILING" = "0" ]`, the all-green rule) and
three entered it: `if [ -n "$PENDING" ]`, `if [ -z "$BLOCKING" ]` (the
discounted-check gate) and the job's `if:` for the bee-reviewed label
trigger. None of them is in a quiet shape: each names its subject and
fails when it is missing. Re-blessed with `tri census pin --bless`;
`tri census pin --gate` passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 17:36:39 UTC

Summary

Status Count
Total Open PRs 48
PRs with Failing Checks 46
PRs with All Checks Green 2
READY 1
FAILING 46
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 46 + 0 + 0 = 47, and there are 48 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

gHashTag added a commit that referenced this pull request Oct 3, 2026
@gHashTag @claude
now: the heading the check gate reads (Refs #5749)
71f2b68 
tools/queen/publish.py wrote `(published DATE)`, which
tools/check_now_entry_shape.py HEADING does not accept; fixed in
the publisher by #5777. Only the first line changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit that referenced this pull request Oct 3, 2026
@gHashTag @claude
now: the heading the check gate reads (Refs #5774)
1f763fb 
tools/queen/publish.py wrote `(published DATE)`, which
tools/check_now_entry_shape.py HEADING does not accept; fixed in
the publisher by #5777. Only the first line changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 6, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Open
Merged
Open
Open
Merged
Merged
Open
Merged
Open
Open
Open
Merged
Open
Merged
Open
Open
Open
Open
This was referenced Oct 6, 2026
Merged
Open
gHashTag added a commit that referenced this pull request Oct 6, 2026
@gHashTag @claude
fix(merger): judge red checks the bee answered for, run on the bee's …
97103e5 
…label, read every bee-reviewed PR (#6826)

* fix(merger): judge red checks the bee answered for, run on the bee's label, read every bee-reviewed PR (Refs #6657, Refs #5776)

The merger has merged zero pull requests since 2026-09-20 for three reasons, each measured:

1. It counts every red check as failing. spec-guards has been red on master since
   2026-09-16 (ring-096-rust drift, fix #5921), so every bee PR inherits it. The
   reviewer bee already writes `discounted-check: <name> -- <why>` lines in its
   approving review; the merger never read them. Now it does (the #5777 slice):
   a red check blocks unless the bee discounted it by exact name, and a check
   the base branch ruleset requires can never be discounted, nor merged around
   by never posting.
2. GitHub ran the */20 cron about every six hours (02:19Z, 09:13Z, 18:28Z,
   00:16Z, 06:25Z). The merger now also runs on pull_request_target labeled
   with bee-reviewed; the only value read from the payload is the PR number,
   validated numeric, and every gate is re-read from the API.
3. `gh pr list --limit 50` read the newest 50 of 108 open PRs and missed 6 of
   the 7 approved ones. It now lists only bee-reviewed PRs, limit 500.

Self-test: 29/29 (tools/bees/merger_gate_selftest.py). Negative control
against master's gate: 6 failures, including "required check never posted".
Live read-only dry run: ready_prs=6729 6728 6724 6723 6722 6718.

Owner standing approval 2026-10-06 for foreign edits (label owner-approved-foreign);
the foreign-exceptions entry is removed again right after merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* census: quiet 154 -> 159 "named a path but not quiet" (Refs #6657)

The merger's new steps (ruleset read, review-body decode, EVENT_PR guard)
add five lines that name a path in a shape the quiet census does not flag.
Nothing about any gate changed; the population grew. Value as measured by
`tri census pin --gate` in cli-tri on 68d2471 ("now: ... 159").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag @claude
fix(bees): the second model seconds a review that fell back, an overl…
489bae8 
…oad spends no attempt (Refs #5776)

B34. The two-model rule dead-ended: when z.ai answered 1305 and the CLI fell
back mid-review, the first review's modelUsage held both free flash models and
second_model() found none left, so every such APPROVE ended incomplete -- 29
rows in all, 18 of the 23 incompletes on 2026-10-06.

The CLI (2.1.283) does not alternate: on 1305 it sets mainLoopModel to the
fallback and stays there, so the verdict was written by glm-4.5-flash. The
rule is kept -- two models, each reaching APPROVE on its own from the same
brief -- and the second model now only has to differ from the model that WROTE
the first verdict (verdict_model). It still runs without a fallback.

An agent-failed on 1305 (the second opinion has no fallback) is z.ai's load,
not the head's, and spends no attempt, as B18 made a fallback free: #6551 and
#6730 are reviewable again. The opinion now names the model that wrote it, so
a review that ran wholly on 4.5 no longer reads "glm-4.7-flash approved,
glm-4.7-flash requested changes" (#6759).

Six new self-test checks failed by name on the old code; self-test 0
failures, bees 0, merger gate 0 of 26, loop-tools 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag gHashTag added the owner-approved-foreign Owner-approved exception to the only-t27 rule: hand-written foreign code allowed in this PR label Oct 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

owner-approved-foreign Owner-approved exception to the only-t27 rule: hand-written foreign code allowed in this PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Review jam: reviewer bee as a service, merger that reads the bee's discounted checks, publisher heading that passes check

1 participant

@gHashTag