From c77ded1a9a6e457b9ae0e2417da2b45b5738c9b1 Mon Sep 17 00:00:00 2001 From: Aidan Daly Date: Thu, 7 May 2026 12:13:56 -0400 Subject: [PATCH 1/4] feat(harness): add verdict prefix requirement to reviewer system prompt The PR reviewer agent now must begin every comment with APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes are immediately visible and parseable. --- .github/harness/prompts/system.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/harness/prompts/system.md b/.github/harness/prompts/system.md index 52a3d2260..cf21e6cca 100644 --- a/.github/harness/prompts/system.md +++ b/.github/harness/prompts/system.md @@ -18,6 +18,16 @@ when users run `agentcore create`. `agentcore-cli` is the main product. It vends CDK projects using constructs from `agentcore-l3-cdk-constructs`. +## Comment Format + +When posting your final review comment on a PR, you MUST begin the comment with one of these exact verdicts: + +- **APPROVED** — no issues found, safe to merge as-is. +- **APPROVED WITH MINOR COMMENTS** — no blocking issues, but you have optional suggestions. The PR can merge without addressing them. +- **REQUESTING CHANGES** — serious issues found that must be fixed before merging. + +The verdict must be the very first word(s) of the comment, on its own line, followed by your explanation. + ## Testing with a bundled distribution Run `npm run bundle` in `agentcore-cli/` to create a tar distribution that includes the packaged From a866627d726da09bba657fc15759063ef1b8ca9b Mon Sep 17 00:00:00 2001 From: Aidan Daly Date: Thu, 7 May 2026 12:18:58 -0400 Subject: [PATCH 2/4] feat(harness): use gh pr review for formal review verdicts Instead of plain comments with text prefixes, the reviewer agent now submits formal GitHub PR reviews via `gh pr review` with --approve, --request-changes, or --comment. This integrates with branch protection rules and makes the review status machine-readable. --- .github/harness/prompts/system.md | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/.github/harness/prompts/system.md b/.github/harness/prompts/system.md index cf21e6cca..9c8276179 100644 --- a/.github/harness/prompts/system.md +++ b/.github/harness/prompts/system.md @@ -18,15 +18,29 @@ when users run `agentcore create`. `agentcore-cli` is the main product. It vends CDK projects using constructs from `agentcore-l3-cdk-constructs`. -## Comment Format +## Submitting Your Review -When posting your final review comment on a PR, you MUST begin the comment with one of these exact verdicts: +When you have finished reviewing, submit a formal GitHub PR review using the `gh` CLI — do NOT post a plain comment. -- **APPROVED** — no issues found, safe to merge as-is. -- **APPROVED WITH MINOR COMMENTS** — no blocking issues, but you have optional suggestions. The PR can merge without addressing them. -- **REQUESTING CHANGES** — serious issues found that must be fixed before merging. +Use one of these commands depending on your verdict: -The verdict must be the very first word(s) of the comment, on its own line, followed by your explanation. +```bash +# No issues — approve the PR +gh pr review --approve --body "" + +# Serious issues that must be fixed before merging +gh pr review --request-changes --body "" + +# No blocking issues, but you have optional suggestions +gh pr review --comment --body "" +``` + +Rules: +- Always use `gh pr review`, never `gh pr comment`. +- Use `--approve` when the PR is safe to merge as-is. +- Use `--request-changes` when there are issues that must be fixed before merging. +- Use `--comment` when you have minor, non-blocking suggestions the author can optionally address. +- The body should summarize your findings. If you flagged individual lines earlier with inline comments, reference them in the summary. ## Testing with a bundled distribution From 5da8cbe717719a83b18be0d1c85b3d93cc4327db Mon Sep 17 00:00:00 2001 From: Aidan Daly Date: Thu, 7 May 2026 12:23:16 -0400 Subject: [PATCH 3/4] refactor(harness): move review verdict to review.md as one-liner Keep system.md as pure workspace context. The review task prompt now ends with a single directive to submit a formal PR review rather than a plain comment. --- .github/harness/prompts/review.md | 3 +-- .github/harness/prompts/system.md | 24 ------------------------ 2 files changed, 1 insertion(+), 26 deletions(-) diff --git a/.github/harness/prompts/review.md b/.github/harness/prompts/review.md index d34c67b95..9ce034749 100644 --- a/.github/harness/prompts/review.md +++ b/.github/harness/prompts/review.md @@ -14,5 +14,4 @@ Review the PR. If there are any serious issues that require code changes before each issue explaining the problem. If there are multiple ways to fix an issue, list the options so the author can choose. Skip style nits and minor suggestions — only flag things that actually need to change. -If all serious issues have already been raised in existing comments, or if you found no new issues, post a single -comment on the PR saying it looks good to merge (or that all issues have already been flagged). +When finished, submit a formal PR review (approve, request changes, or comment) — do not post a plain comment. diff --git a/.github/harness/prompts/system.md b/.github/harness/prompts/system.md index 9c8276179..52a3d2260 100644 --- a/.github/harness/prompts/system.md +++ b/.github/harness/prompts/system.md @@ -18,30 +18,6 @@ when users run `agentcore create`. `agentcore-cli` is the main product. It vends CDK projects using constructs from `agentcore-l3-cdk-constructs`. -## Submitting Your Review - -When you have finished reviewing, submit a formal GitHub PR review using the `gh` CLI — do NOT post a plain comment. - -Use one of these commands depending on your verdict: - -```bash -# No issues — approve the PR -gh pr review --approve --body "" - -# Serious issues that must be fixed before merging -gh pr review --request-changes --body "" - -# No blocking issues, but you have optional suggestions -gh pr review --comment --body "" -``` - -Rules: -- Always use `gh pr review`, never `gh pr comment`. -- Use `--approve` when the PR is safe to merge as-is. -- Use `--request-changes` when there are issues that must be fixed before merging. -- Use `--comment` when you have minor, non-blocking suggestions the author can optionally address. -- The body should summarize your findings. If you flagged individual lines earlier with inline comments, reference them in the summary. - ## Testing with a bundled distribution Run `npm run bundle` in `agentcore-cli/` to create a tar distribution that includes the packaged From 986a5cde0c367fe944255fa769bcb0dafd780a42 Mon Sep 17 00:00:00 2001 From: Aidan Daly <99039782+aidandaly24@users.noreply.github.com> Date: Thu, 7 May 2026 12:26:08 -0400 Subject: [PATCH 4/4] Update PR review submission instructions Clarified instructions for submitting PR reviews. --- .github/harness/prompts/review.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/harness/prompts/review.md b/.github/harness/prompts/review.md index 9ce034749..c0697ff3b 100644 --- a/.github/harness/prompts/review.md +++ b/.github/harness/prompts/review.md @@ -14,4 +14,4 @@ Review the PR. If there are any serious issues that require code changes before each issue explaining the problem. If there are multiple ways to fix an issue, list the options so the author can choose. Skip style nits and minor suggestions — only flag things that actually need to change. -When finished, submit a formal PR review (approve, request changes, or comment) — do not post a plain comment. +When finished, submit a formal PR review (approve or request changes) with individual and inline comments in it. Be specific with line numbers.