Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/workflows/review-swarm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
name: Review swarm

"on":
pull_request:
types: [opened, synchronize, reopened]

permissions:
contents: read
pull-requests: write

concurrency:
group: review-swarm-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
review:
if: >-
${{ contains(fromJSON('["kjgbot", "miyaontherelay"]'),
github.event.pull_request.user.login) }}
runs-on: ubuntu-latest
timeout-minutes: 50
env:
GH_TOKEN: ${{ github.token }}
RELAY_WORKSPACE_KEY: ${{ secrets.RELAY_WORKSPACE_KEY }}
PR_NUMBER: ${{ github.event.pull_request.number }}
steps:
- name: Check out pull request merge commit
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Install agent-relay
run: npm install --global agent-relay

- name: Select pull request
run: echo "$PR_NUMBER" > .review-target

- name: Launch review swarm
id: launch
shell: bash
run: |
set -euo pipefail
response=$(agent-relay cloud run workflows/review-swarm.yaml \
--sync-code --json)
run_id=$(jq -er '.runId // .id' <<<"$response")
echo "run_id=$run_id" >> "$GITHUB_OUTPUT"
echo "Launched review swarm run $run_id"

- name: Wait for review swarm
shell: bash
run: |
set -euo pipefail
run_id='${{ steps.launch.outputs.run_id }}'
deadline=$((SECONDS + 2700))
while (( SECONDS < deadline )); do
response=$(agent-relay cloud status "$run_id" --json)
status=$(jq -er '.status' <<<"$response")
echo "Review swarm $run_id: $status"
case "$status" in
completed) exit 0 ;;
failed|cancelled) exit 1 ;;
esac
sleep 30
done
echo "Review swarm $run_id did not complete within 45 minutes" >&2
exit 1

- name: Sync and post reviews
env:
RUN_ID: ${{ steps.launch.outputs.run_id }}
run: bash .github/workflows/scripts/swarm-post.sh "$RUN_ID" "$PR_NUMBER"
47 changes: 47 additions & 0 deletions .github/workflows/scripts/swarm-post.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
set -euo pipefail

if [[ $# -ne 2 || ! $2 =~ ^[0-9]+$ ]]; then
echo "usage: $0 <run-id> <pr-number>" >&2
exit 2
fi

run_id=$1
pr_number=$2
agent-relay cloud sync "$run_id"
run_status=$(agent-relay cloud status "$run_id" --json)

declare -A verdicts
for lens in maintainability history structure; do
transcript=$(find ops/reviews -maxdepth 1 -type f \
-name "*-pr${pr_number}-${lens}.md" -printf '%T@ %p\n' |
sort -nr | head -1 | cut -d' ' -f2-)
if [[ -z $transcript ]]; then
echo "No $lens transcript found for PR #$pr_number" >&2
exit 1
fi

if grep -q 'REVIEW_FAILED' "$transcript"; then
verdicts[$lens]=FAILED
elif grep -q 'REVIEW_PASSED' "$transcript"; then
verdicts[$lens]=PASSED
else
echo "$transcript has no review verdict" >&2
exit 1
fi

gh pr comment "$pr_number" --body-file "$transcript"
done

if grep -q 'SWARM_PASSED' <<<"$run_status"; then
aggregate=PASSED
elif grep -q 'SWARM_FAILED' <<<"$run_status"; then
aggregate=FAILED
else
echo "Cloud run $run_id has no aggregate swarm verdict" >&2
exit 1
fi

marker="🎯 review-swarm: $aggregate (M:${verdicts[maintainability]} H:${verdicts[history]} S:${verdicts[structure]})"
gh pr comment "$pr_number" --body "$marker"
echo "$marker"
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,5 @@ Nine gates, in `docs/RFC-0001` §3. Gate 1 first: a relayflow can run — the he
ladder survives `kill -9` at every boundary.

Private while we build. YC 2026-09-15 runs on this base.

Repository Actions must define the `RELAY_WORKSPACE_KEY` secret so cloud review-swarm runs use the canonical workspace.
8 changes: 8 additions & 0 deletions ops/NEEDS_HUMAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Gate 3 needs human setup

The current environment is not authenticated to GitHub (`gh auth status` reports
no logged-in hosts), so it cannot verify or create the required
`RELAY_WORKSPACE_KEY` Actions secret on `AgentWorkforce/flows`, nor can it run
the definition-of-done posting command against a real pull request. A repository
administrator must add the secret if absent and run the live posting check from
an environment authenticated to that repository.
155 changes: 91 additions & 64 deletions ops/NEXT.md
Original file line number Diff line number Diff line change
@@ -1,87 +1,114 @@
# NEXT — work package for this tick
# NEXT — gate 3 work package

**Scope:** Build a minimal agent worker in the SDK. CODE task, SDK-side.
**Scope:** Wire the review-swarm to fire on PR open via GitHub Actions + `agent-relay cloud run`. CODE task, `.github/workflows/`-side.

This run is pinned to **gate 3** and must not work on any other gate.

## Objective

Promote the throwaway worker the tests already build into a real SDK component
that can execute agent steps by running their declared CLI as a subprocess.
Add `.github/workflows/review-swarm.yml` that automatically invokes the existing `workflows/review-swarm.yaml` workflow on PR open/synchronize/reopened events for drive-loop authored PRs. The swarm produces three independent review transcripts (maintainability/history/structure) plus an aggregate verdict, which are synced back and posted as PR comments.

## Context

Nothing in this repo can execute an agent step. Searching for `workerAttach` /
`step.complete` finds only TESTS (`sdk/tests/live-kernel.test.ts`,
`journal-client.test.ts`, `journal-client-loopback.ts`) and the protocol
definitions. `sdk/src/cli/run.ts` only OBSERVES worker leases and waits for one
that never arrives.
`workflows/review-swarm.yaml` already exists in this repo — three model-diverse reviewers (claude/codex/opencode) that read a PR diff and produce three independent transcripts + one aggregate verdict. Today it only fires when a human writes `.review-target` and runs `agent-relay cloud run` by hand. It has run zero times against a drive PR on cloud.

The kernel's dispatch, lease and claim machinery is real and tested. The worker
side of the protocol is simply unimplemented, and that is what blocks gate 2
("a workload RUNS as a relayflow" — today a run can only be shown CREATED) and
gate 3 ("every claim/lease/retry served by the kernel").
Meanwhile the reviewers this repo actually depends on (CodeRabbit, Devin) are external SaaS bots: CodeRabbit rate-limits into silence and Devin's trial expired. RFC-0001 §2 rule 7 says the review team must be OUR own — the swarm is that team, and it is not firing.

`sdk/tests/live-kernel.test.ts` around the `live-manual-agent` case (line 288)
shows the whole shape: connect, `hello`, `workerAttach` with pins, receive
`step.dispatch`, act, complete. The protocol is already proven there.
The fix is one GitHub Actions workflow: on PR open/synchronize, write `.review-target`, invoke `agent-relay cloud run workflows/review-swarm.yaml`, poll for completion, `agent-relay cloud sync` the transcripts back, and post each as a PR comment with an aggregate marker.

## Files in scope

- `sdk/src/worker.ts` — new file, the worker implementation
- `sdk/src/index.ts` — export the worker
- `sdk/tests/live-kernel.test.ts` OR a new test file — add a test that runs a
real flow with an agent step end to end against a live `relayflowd`, with
this worker attached, and asserts the step reaches `done`.
- `.github/workflows/review-swarm.yml` (CREATE) — the only file this tick should create
- `README.md` or a `docs/*.md` file (EDIT) — document the RELAY_WORKSPACE_KEY secret requirement (one sentence)

## Definition of done

ALL of the following must hold:

1. The worker in `sdk/src/worker.ts`, exported from `sdk/src/index.ts`

2. A test that runs a real flow with an agent step end to end against a live
`relayflowd`, with this worker attached, and asserts the step reaches
`done`. `sdk/tests/live-kernel.test.ts` already starts a daemon — follow
that pattern.

3. **The worker must attach BEFORE the run starts.** A run that finds no worker
parks, and attaching afterwards does not re-drive it — `run.resume` is what
picks a parked run back up. That contract is pinned in the live-kernel
suite; do not fight it.

4. The worker must:
- attach for `agent` steps with the pins it holds
- on `step.dispatch`, run the step's declared `cli` as a subprocess
- report the result back through the existing protocol (`step.complete`, and
the failure path when the CLI exits nonzero)
- nothing speculative: no retries of its own, no scheduling, no LLM calls.
The kernel owns retry and lease policy — do not reimplement it.

5. `cd sdk && npm test` must be green. Run it and paste the literal command and
output tail showing test counts.

6. `cd kernel && sh ../ops/cargo.sh test` must be green. Run it and paste the
literal command and output tail showing test counts.

7. EVERY new test confirmed to FAIL against current code, with the literal
failing output quoted in the summary.

8. As your LAST action, run `git status --porcelain` and paste it.
ALL of the following must pass:

1. **`.github/workflows/review-swarm.yml` exists and passes lint**
```bash
test -f .github/workflows/review-swarm.yml && echo "exists"
```
Output: `exists`

```bash
which actionlint >/dev/null 2>&1 && actionlint .github/workflows/review-swarm.yml || yamllint .github/workflows/review-swarm.yml
```
Output: Must pass with no errors (actionlint preferred, yamllint fallback)

2. **The workflow's `jobs.review.if` correctly gates on drive-loop author only**
Test the expression by hand:
```bash
node -e "console.log(['kjgbot', 'miyaontherelay'].includes('kjgbot'))"
```
Output: `true`

```bash
node -e "console.log(['kjgbot', 'miyaontherelay'].includes('khaliqgant'))"
```
Output: `false`

3. **README.md or docs/ describes the required repo secret**
```bash
grep -l RELAY_WORKSPACE_KEY README.md docs/*.md 2>/dev/null | head -1
```
Output: Must show at least one file path containing the documentation

4. **A dry-run test proves the shell logic works**
A companion shell script `.github/workflows/scripts/swarm-post.sh` (or inline) that TAKES a runId as an argument and posts the comments. Show it working against an EXISTING completed cloud run:
```bash
bash .github/workflows/scripts/swarm-post.sh <some-real-runId> <some-PR>
```
Output: Must show posted comment URL or confirmation

5. **SDK tests remain green** (unaffected by this change)
```bash
cd sdk && npm test 2>&1 | grep "Test Files"
```
Output: `Test Files 14 passed (14)` or similar all-green

6. **EVERY new test confirmed to FAIL against current code**
With the literal failing output quoted in the summary.

7. **Final file status**
```bash
git status --porcelain
```
Output: Must show the created/modified files

## Workflow requirements

Scope it small and honest:

- **Trigger:** `pull_request` events `opened`, `synchronize`, `reopened`
- **Author filter:** Only run if `github.event.pull_request.user.login` is a drive-loop author (`kjgbot`, `miyaontherelay`) — do not review human PRs
- **Concurrency:** Group per PR so a second push cancels the first review
- **Job steps:**
1. Checkout the PR's head at the merge commit
2. Install `agent-relay` (curl the release, or use `mise install` if that's how this repo does it — check `.mise.toml` and `.env.example`)
3. `echo "$PR_NUMBER" > .review-target`
4. `agent-relay cloud run workflows/review-swarm.yaml` with `RELAY_WORKSPACE_KEY` from a repo secret; capture the runId
5. Poll `agent-relay cloud status <runId> --json` every 30s until `status == completed` or 45 min elapse
6. `agent-relay cloud sync <runId>` to fetch the run's artifacts
7. Read `ops/reviews/*-pr<N>-*.md` produced by the swarm; post each as a PR comment via `gh pr comment`
8. Post one aggregate marker comment: `🎯 review-swarm: PASSED|FAILED (M:<v> H:<v> S:<v>)` — the aggregate step of review-swarm.yaml prints `SWARM_PASSED` or `SWARM_FAILED` on stdout; grep for that.

## Prerequisites this brief cannot satisfy

- **`RELAY_WORKSPACE_KEY` must exist as a GitHub Actions secret** on `AgentWorkforce/flows`. `gh secret list --repo AgentWorkforce/flows` will show if it's there. If it is missing, write ops/NEEDS_HUMAN.md saying so and STILL end with ASSESS_DONE — this is a human step.
- **`agent-relay cloud run` invoked from GHA must reach the same cloud workspace as the laptop.** If the run fails with an auth error, DO NOT invent a workaround; file it as a NEEDS_HUMAN and stop.

## Explicitly OUT of scope

- LLM steps — not in the gate 3 scope
- Retry logic in the worker — the kernel owns retry policy
- Scheduling or lease management — the kernel owns lease policy
- Optimizations, abstractions, or speculative features
- Changes to the kernel
- Changes to existing tests (except adding new test cases)
- Work on any gate other than gate 3
- **Creating or verifying the RELAY_WORKSPACE_KEY secret** — document the requirement; human creates it if missing
- **Auth troubleshooting beyond documentation** — if auth fails, document it in NEEDS_HUMAN.md
- **Modifying workflows/review-swarm.yaml** — it already exists and works
- **Reviewing human PRs** — filter ensures only drive-loop PRs trigger
- **Any changes to kernel/, sdk/, testdata/, or other code** — only .github/workflows/ and docs
- **Work on any gate other than gate 3**

## If blocked

If gate 3 is genuinely unreachable from the current state, write
ops/NEEDS_HUMAN.md saying exactly why and still end with ASSESS_DONE. Do not
silently substitute different work: a run that reports progress on the wrong
gate is worse than one that reports it is blocked.
If gate 3 is genuinely unreachable from the current state, write ops/NEEDS_HUMAN.md saying exactly why and still end with ASSESS_DONE. Do not silently substitute different work: a run that reports progress on the wrong gate is worse than one that reports it is blocked.

A working `.github/workflows/review-swarm.yml` that stalls at the auth step, plus a NEEDS_HUMAN naming the missing secret, is a complete deliverable — the runbook is the artifact.
Loading