Repository navigation
feat(labels): mark a pull request pass when the verdict was a PASS (#739) - #760
Conversation
…739) A merged pull request kept no trace of the verdict that let it in: `agent-pr` stays, and the conclusion lives only in a comment thread, so the review record is unreadable from the pull request list. `pass` is now a label, applied by a workflow that reads the comment with the same parser the gates use. Only a conclusion from someone GitHub ties to this repository counts, and only when its `Written-by:` names a writer the policy lists; a `Correction from …` comment never labels, a replacement PASS labels like any other, and nothing ever removes the label — a later INC on a newer tip is another record, and `hold` stays the one manual stop. `--backfill` labels the merged pull requests whose head carried a PASS, so the existing history is visible without hand work; it is idempotent, and a PASS about another tip does not count.
|
Verdict: PASS at f61f459 #739: a pull request that carried a Ran:
Restored after each cut → 6/6 and 9/9. The workflow itself is well behaved on the points that matter here: it runs base-branch code ( Not blocking, for whoever picks up a follow-up: |
Written-by: Dev2
Fixes #739
Tip SHA: f61f459
Reproduce
A merged pull request keeps no trace of the verdict that let it in.
agent-prstays on the list, and the conclusion lives only in a comment thread: to answer "was this reviewed?" you open the pull request and scroll. Onmaintoday there is nopasslabel at all (.github/labels.jsonhas 12 entries), sogh pr list --label passanswers nothing, and after a branch is deleted the review record is unreadable from the list.What changed
.github/labels.json: apasslabel (#006B75, "Reviewer PASS verdict at the merged head; set by pass-label, never removed"), sosync-labels.mjscreates it. The count assertion insync-labels.test.mjsmoves from 12 to 13 andpassjoins the names the test requires..github/scripts/pass-label.mjs(new):verdictOf(body, { writers, association })reads a conclusion withparseReportfromreview-gate.mjs— the parser the gates already share — and returns it only when the verdict isPASS, the comment'sauthor_associationisOWNER/MEMBER/COLLABORATOR, and theWritten-by:name is in the policy'swriters. A body whose first line isCorrection from …never counts, whateverVerdict:it quotes.planCommentdecides one event;planBackfilldecides the merged ones from raw API pages;mainruns either mode againstgh..github/workflows/pass-label.yml(new): onissue_comment(created) it labels the pull request the conclusion was posted on, andworkflow_dispatchruns--backfill. The comment body is only parsed and never run; the code that runs is checked out frommain, so an untrusted comment cannot bring code with it. Permissions arecontents: read,issues: write,pull-requests: write.docs/testing-layers.json+ the generateddocs/testing-layers.md:pass-label.ymlgets its workflow row (workflow-only, gates nothing), so the layer check docs(testing): pin which layer runs every suite, and check it (#749) #755 added stays honest about the new workflow.holdstays the one manual stop. A test asserts the script contains no removal call at all.What I ran
At this tip
f61f4595b5f3c06183623ac30d3e029b8c136ba7(basee15d14c3, 7 files, +303/−2):node --test .github/scripts/*.test.mjs→ 226 tests, 226 pass, 0 fail (baseline before this diff: 219).node --test .github/scripts/pass-label.test.mjs→ 6/6: a PASS labels; INC,Correction from …, a non-member association and an unknownWritten-by:name never label; aReplaces:PASS labels like any PASS; a second run adds nothing; backfill picks the merged-already-on-a-PASS one and leaves the INC-only, already-labelled, still-open and PASS-at-another-tip rows alone; a 12-character tip names the head.node .github/scripts/testing-layers.mjs --check→310 test file(s) in 12 workflow(s), all classified (the new workflow and the new test file are both listed).node scripts/ds-bot-patches.mjs check→ three ok (untouched by this diff).docs/testing-layers.mdregenerated with--write.Red without the source change
Correctionguard inverdictOf→ the correction case fails, becauseparseReportfinds theVerdict: PASSthe correction quotes.writer/writerscheck → the unknown-Written-by:case fails (and a passer-by withassociation: NONEstill fails on its own rule).already labelledcheck inplanComment→ the "second run adds nothing" case fails.namesHeadcheck inplanBackfill→ the PASS-at-another-tip row is labelled too, so the backfill case sees[11, 15]instead of[11].DELETEcall to the script → the "closed record" case fails.Not run
pass-label.ymldoes not exist onmainyet. The first trigger is the next PASS comment after merge;workflow_dispatchruns the backfill, which I did not run from here — labelling 40+ merge records is the workflow's job, not a by-hand write.actionlintis not installed on this box;packageruns the real one on this PR.npm run verifyas a whole; nothing undercore/,plugins/orpatches/changes.Risk
T3 (
.github/**anddocs/testing-layers.json), so greentests-okat the tip plus a PASS at this tip, then the queue.The write is a single
POST /issues/<n>/labelswith one label, from a workflow that hasissues: writeand nothing else. Three things could go wrong and each is bounded: a mistake labels a pull requestpass; a pass-through of untrusted text would at worst add a label (it cannot remove anything, and it never gates a merge —merge-gatereads the conclusion itself at the entry); and the backfill could label a pull request whose merged head carried a PASS for the wrong reason — the tip check reduces that to a PASS that names the merged head, which is the record the label is supposed to keep. If the label turns out to be unwanted on an old pull request, removing it is a human action, likehold.