Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions .github/workflows/review-swarm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: Review swarm

on:
pull_request:
types: [opened, synchronize, reopened]

permissions:
contents: read
pull-requests: write

concurrency:
group: review-swarm-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
review:
# Per RFC-0001 §2 rule 7: "Every PR is met by a review swarm." No author
# filter — human PRs get reviewed too. If we ever need a rollout-scoped
# narrower filter, express it as a documented rule, not a hardcoded
# username list.
runs-on: ubuntu-latest
# Job cap must exceed the poll deadline (65min) + install overhead.
# 75min gives 10min headroom for checkout, npm install, sync, post.
timeout-minutes: 75
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
RELAY_WORKSPACE_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }}
steps:
- name: Check out PR merge commit
uses: actions/checkout@v4
with:
ref: refs/pull/${{ github.event.pull_request.number }}/merge
fetch-depth: 0

- name: Install agent-relay
run: npm install --global agent-relay@11.8.7

- name: Set review target
run: echo "$PR_NUMBER" > .review-target

- name: Launch review swarm
id: launch
shell: bash
run: |
set -euo pipefail
response=$(agent-relay cloud run workflows/review-swarm.yaml --json)
printf '%s\n' "$response"
run_id=$(jq -er '.runId // .id // .run.id' <<<"$response")
echo "run_id=$run_id" >> "$GITHUB_OUTPUT"

- name: Wait for review swarm
id: wait
shell: bash
env:
RUN_ID: ${{ steps.launch.outputs.run_id }}
# Poll deadline (3900s = 65min) must be >= workflows/review-swarm.yaml's
# own timeoutMs (3600000 = 60min) plus buffer, or CI abandons swarms
# still executing in cloud. The job's timeout-minutes must exceed 65.
# Record the terminal status in an output so the next step can post
# transcripts EVEN when the swarm rejects — otherwise a rejecting
# swarm's evidence never reaches the PR.
run: |
set -euo pipefail
deadline=$((SECONDS + 3900))
while (( SECONDS < deadline )); do
response=$(agent-relay cloud status "$RUN_ID" --json)
printf '%s\n' "$response"
status=$(jq -er '.status // .run.status' <<<"$response")
case "$status" in
completed|failed|errored|cancelled|canceled)
echo "swarm_status=$status" >> "$GITHUB_OUTPUT"
exit 0 ;;
esac
sleep 30
done
echo "swarm_status=timeout" >> "$GITHUB_OUTPUT"

- name: Sync and post review comments
# ALWAYS run, even if wait recorded a non-completed status. A rejecting
# swarm's transcripts and marker are the evidence we need on the PR;
# skipping this step on failure is exactly the evidence-loss class
# ops/DRIVE-LOG.md warns about.
if: always() && steps.launch.outputs.run_id != ''
env:
RUN_ID: ${{ steps.launch.outputs.run_id }}
run: bash .github/workflows/scripts/swarm-post.sh "$RUN_ID" "$PR_NUMBER"

- name: Fail the job when swarm rejected
# Post-transcript gate: the marker comment now records the verdict on
# the PR, so we can safely fail the CI check to block merge.
if: steps.wait.outputs.swarm_status != 'completed'
run: |
echo "Swarm terminated with status: ${{ steps.wait.outputs.swarm_status }}" >&2
exit 1
93 changes: 93 additions & 0 deletions .github/workflows/scripts/swarm-post.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#!/usr/bin/env bash
# Sync a completed cloud review-swarm run and post its transcripts + marker
# back to the PR.
#
# Two load-bearing extraction rules (both hard-won repo history):
# 1. SORT transcripts by FILENAME (starts with YYYYMMDD-HHMM), not by mtime.
# Fresh checkouts give transcripts uniform mtimes and mtime-sort picked
# stale verdicts — commit b2535aa fixed this class of bug elsewhere.
# 2. The verdict is the LAST non-empty line's token, not a whole-file grep.
# A whole-file grep of REVIEW_FAILED misclassifies a passing review that
# quotes the token in prose — commit f59d9cd fixed this elsewhere too.
# Same for the aggregate: LAST SWARM_ token in the log, not a substring match.
#
# Sticky marker: hidden HTML comment identifies the marker; edit-in-place
# across re-runs so N pushes don't accumulate N marker comments.
set -euo pipefail

run_id=${1:-}
pr_number=${2:-}

[[ "$run_id" =~ ^[[:alnum:]-]+$ ]] || {
echo "usage: $0 <runId> <PR-number>" >&2
exit 2
}
[[ "$pr_number" =~ ^[0-9]+$ ]] || {
echo "usage: $0 <runId> <PR-number>" >&2
exit 2
}

agent-relay cloud sync "$run_id"
run_log=$(agent-relay cloud logs "$run_id")
printf '%s\n' "$run_log"

# Aggregate: last SWARM_ token in the log (not a substring anywhere).
overall=FAILED
last_swarm=$(printf '%s\n' "$run_log" | grep -Eo 'SWARM_(PASSED|FAILED)' | tail -1 || true)
[[ "$last_swarm" == SWARM_PASSED ]] && overall=PASSED

declare -A verdicts
for lens in maintainability history structure; do
# Sort lexicographically by filename (YYYYMMDD-HHMM prefix), take newest.
shopt -s nullglob
matches=(ops/reviews/*-pr"$pr_number"-"$lens".md)
shopt -u nullglob
if ((${#matches[@]} == 0)); then
echo "missing $lens transcript for PR #$pr_number" >&2
verdicts[$lens]=MISSING
continue
fi
transcript=$(printf '%s\n' "${matches[@]}" | sort | tail -1)

# Verdict = last non-empty line's token.
last_line=$(awk 'NF { last=$0 } END { print last }' "$transcript")
case "$last_line" in
*REVIEW_PASSED*) verdicts[$lens]=PASSED ;;
*REVIEW_FAILED*) verdicts[$lens]=FAILED ;;
*) verdicts[$lens]=UNCLEAR ;;
esac

gh pr comment "$pr_number" --body-file "$transcript"
done

# Compute the aggregate from lens verdicts DIRECTLY, not from cloud logs. The
# logs-derived `overall` above is a first pass but can disagree with the
# actual transcripts (log parsing missed a lens, aggregate step raced, etc).
# The transcripts are the load-bearing evidence — a single FAILED lens means
# aggregate FAILED, per the review-swarm.yaml aggregate step's own rule
# ("any single honest refusal blocks the merge").
#
# Compute aggregate = ALL lenses PASSED; else FAILED. This is fail-closed:
# MISSING, UNCLEAR, FAILED all degrade to FAILED. The prior bug only
# degraded MISSING/UNCLEAR, so a log-derived PASSED could survive even when
# a lens transcript said FAILED.
overall=PASSED
for lens in maintainability history structure; do
if [[ "${verdicts[$lens]:-MISSING}" != PASSED ]]; then
overall=FAILED
fi
done

marker_id='<!-- review-swarm-marker -->'
body="$marker_id"$'\n'"🎯 review-swarm: $overall (M:${verdicts[maintainability]:-MISSING} H:${verdicts[history]:-MISSING} S:${verdicts[structure]:-MISSING})"

# Sticky comment: find existing by identity marker, edit in place.
existing_id=$(gh api "repos/${GITHUB_REPOSITORY:?}/issues/$pr_number/comments" --jq \
".[] | select(.body | contains(\"$marker_id\")) | .id" | head -1)

if [[ -n "$existing_id" ]]; then
gh api -X PATCH "repos/${GITHUB_REPOSITORY}/issues/comments/$existing_id" \
-f body="$body" > /dev/null
else
gh pr comment "$pr_number" --body "$body"
fi
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ an entire application.
The constitution is [`docs/RFC-0001-everything-is-a-relayflow.md`](docs/RFC-0001-everything-is-a-relayflow.md).
Nothing in this repo may contradict it; changing it is a human decision.

The required `RELAY_WORKSPACE_KEY` repository secret authenticates review-swarm GitHub Actions runs to the canonical Agent Relay Cloud workspace.

## Layout

```
Expand Down
Loading