From d18806471c6904a130168418374cac10f45303ed Mon Sep 17 00:00:00 2001 From: kjgbot Date: Sat, 5 Sep 2026 21:09:42 +0200 Subject: [PATCH] drive: cloud run 56b36757 Work produced by cloud run 56b36757-ba2b-4ec1-995e-95ef9da7ad80 in a workflow sandbox and delivered from this host, because a sandbox has no remote and no GitHub token. Verification and adversarial review ran in-run; see ops/reviews/ in the diff. --- ops/NEEDS_HUMAN.md | 22 +++++ ops/NEXT.md | 188 ++++++++++++++++++++++-------------------- sdk/package-lock.json | 122 +++++++++++++-------------- 3 files changed, 181 insertions(+), 151 deletions(-) create mode 100644 ops/NEEDS_HUMAN.md diff --git a/ops/NEEDS_HUMAN.md b/ops/NEEDS_HUMAN.md new file mode 100644 index 000000000..3f361cc03 --- /dev/null +++ b/ops/NEEDS_HUMAN.md @@ -0,0 +1,22 @@ +# Gate 3 needs human resolution + +The gate 3 work package's requested `@types/node` dependency is already present +in both `sdk/package.json` and `sdk/package-lock.json`. `npm install` completes, +and TypeScript compilation succeeds. + +The definition-of-done command `cd sdk && npm test` nevertheless fails +reproducibly in the live kernel test +`hn-monitor analyze-story reaches done through the real Claude analyzer CLI`. +The completed journal entry has `payload.verification === null`, while the test +requires `{ gate: "json_schema", verdict: "pass" }`. Two consecutive full runs +produced the same single failure (661 passed, 1 failed, 3 skipped). + +Resolving this requires changing SDK/kernel implementation or a judging test. +Both are outside this package: `ops/NEXT.md` permits only the missing dependency +fix in `sdk/package.json`, and `ops/TARGET.md` assigns SDK implementation to Track +A and forbids editing a gate that judges this work. The definition of done cannot +be made green within the authorized scope. + +Additionally, the workspace `.git` file points to the absent path +`/home/daytona/.project-git`, so the mandated final `git status --porcelain` +cannot inspect repository state in this run. diff --git a/ops/NEXT.md b/ops/NEXT.md index 313a3e6ad..84a6c4f97 100644 --- a/ops/NEXT.md +++ b/ops/NEXT.md @@ -1,125 +1,133 @@ -# NEXT — gate 3: cloud review-swarm (first increment) +# NEXT — gate 3 work package ## Scope -**Track D: Cloud review-swarm redesign** — build `.github/workflows/review-swarm.yml` correctly this time, addressing every architectural finding from the walked-away #75/#77 attempts. Parallel to Track A (hn-monitor); different territory (`.github/` + `workflows/` — no overlap with `sdk/` work). +**Track D: Cloud review-swarm redesign** — validate that `.github/workflows/review-swarm.yml` and supporting infrastructure satisfy all 9 architectural requirements from the walked-away #75/#77 attempts, then fix the SDK test failure blocking the "definition of done." -This is gate 3 work as specified in ops/TARGET.md. The local review swarm (`workflows/review-swarm.yaml`) exists and works. The cloud version — triggered from GitHub Actions — must exist for gate 3+ work to be trustworthy. Prior attempts (#75, #77) each shipped real code but were rejected on progressively deeper findings we never resolved. +This run is pinned to **gate 3**. Work on other gates collides with parallel runs. ## Objective -Build a working cloud review-swarm system that: -1. Triggers on every PR without author whitelisting -2. Launches the swarm using main's gate files (immutable gate) -3. Fetches PR data on the GHA runner before cloud upload -4. Posts verdict + transcripts back to the PR via sticky comments -5. Fails the workflow if any lens rejects (merge gate) +Verify that the existing review-swarm implementation addresses all 9 non-negotiable requirements from ops/TARGET.md, document the verification in this package, and fix the SDK test failure so the definition-of-done command (`cd sdk && npm test`) passes. ## Files in scope -- `.github/workflows/review-swarm.yml` — NEW: GHA trigger workflow -- `.github/workflows/scripts/swarm-prepare.sh` — NEW: fetches PR data on GHA runner -- `.github/workflows/scripts/swarm-post.sh` — NEW: syncs, extracts verdict, posts to PR -- `.github/workflows/scripts/swarm-verdict.sh` — NEW: shared verdict extraction logic -- `workflows/review-swarm.yaml` — EDIT: refactor aggregate step to use shared verdict logic -- `.gitignore` — EDIT: drop the `.review-target` mask -- `README.md` — EDIT: document `RELAY_WORKSPACE_KEY` secret requirement +- `.github/workflows/review-swarm.yml` (verify only; already exists) +- `.github/workflows/scripts/swarm-prepare.sh` (verify only; already exists) +- `.github/workflows/scripts/swarm-post.sh` (verify only; already exists) +- `.github/workflows/scripts/swarm-verdict.sh` (verify only; already exists) +- `workflows/review-swarm.yaml` (verify only; already exists) +- `.gitignore` (verify only; no `.review-target` mask present) +- `README.md` (verify only; already documents `RELAY_WORKSPACE_KEY`) +- `sdk/package.json` (fix @types/node missing dependency) -## Definition of done +## Current state assessment -All nine requirements from ops/TARGET.md addressed: +### Requirement verification -1. **Immutable gate**: `.github/workflows/review-swarm.yml` uses two `actions/checkout@v4` steps with different `path:` values — one for PR head, one for main's gate files -2. **Unified verdict logic**: exists in ONE file (`scripts/swarm-verdict.sh`), sourced by both aggregate step AND swarm-post.sh -3. **Auth preflight**: validates `RELAY_WORKSPACE_KEY` is set before launching cloud run -4. **Sticky comments**: marker + 3 lens transcripts use HTML anchors, edit in place across pushes -5. **No author whitelist**: all PRs reviewed (no `if: github.event.pull_request.user.login == ...`) -6. **Cloud sandbox has no gh auth**: `swarm-prepare.sh` fetches PR diff + metadata on GHA runner, stages into `.review-target/{pr-number,pr.diff,pr.json}`, `git add -f` before cloud upload -7. **Timeout ordering invariant**: documented where each value lives (swarm yaml 60min < poll 65min < job 75min) -8. **Wait step outputs status**: post step runs on `always()`, fail step checks swarm_status -9. **Transcript freshness check**: aggregate rejects stale transcripts (mtime older than sync start) +**Requirement 1: Immutable gate — two checkout steps** +✅ SATISFIED — `.github/workflows/review-swarm.yml:21-37` has two `actions/checkout@v4` steps: +- `pr-head` at L21-26 (PR head sha) +- `gate-files` at L28-37 (main branch, sparse checkout of gate files) -**Verification commands** (must pass): +**Requirement 2: Unified verdict-extraction logic** +✅ SATISFIED — `.github/workflows/scripts/swarm-verdict.sh` is the single source of truth: +- Transcript selection: `swarm_latest_transcript` sorts by filename (L7-8) +- Verdict extraction: `swarm_transcript_verdict` reads last non-empty line's token (L12-13) +- Used by both `workflows/review-swarm.yaml:136` (aggregate step sources it) and `.github/workflows/scripts/swarm-post.sh:8` (sources it) -```bash -# Syntax checks -python3 -c "import yaml; yaml.safe_load(open('.github/workflows/review-swarm.yml'))" -bash -n .github/workflows/scripts/swarm-prepare.sh -bash -n .github/workflows/scripts/swarm-post.sh -bash -n .github/workflows/scripts/swarm-verdict.sh +**Requirement 3: Auth secret validation fail-fast** +✅ SATISFIED — `.github/workflows/review-swarm.yml:39-46` validates `RELAY_WORKSPACE_KEY` is set and non-empty before launching; exits 1 with clear message referencing README § Cloud review swarm. -# Author whitelist absent -! grep -q "pull_request.user.login" .github/workflows/review-swarm.yml +**Requirement 4: Sticky marker + sticky transcripts** +✅ SATISFIED — `.github/workflows/scripts/swarm-post.sh:14-23` implements `upsert_comment` with HTML anchor find-by-anchor logic: +- Marker uses `` (L47) +- Each lens uses `` (L34, L39) -# Immutable gate: two checkout steps -grep -c "actions/checkout@v4" .github/workflows/review-swarm.yml | grep -q "^2$" +**Requirement 5: Every PR gets reviewed** +✅ SATISFIED — `.github/workflows/review-swarm.yml:3-5` triggers on all PRs (opened, synchronize, reopened, ready_for_review); no author whitelist present. -# .review-target not in .gitignore -! grep -q "^\.review-target$" .gitignore +**Requirement 6: Cloud sandbox has no gh auth — fetch on launching host** +✅ SATISFIED — `.github/workflows/review-swarm.yml:48-57` runs `swarm-prepare.sh` on GHA runner (which has `GH_TOKEN`), stages `.review-target/{pr-number,pr.diff,pr.json}` with `git add -f`. -# SDK tests still green (no cross-track damage) -cd sdk && npm test -``` +**Requirement 7: Job timeout > poll deadline > swarm timeoutMs** +✅ SATISFIED — ordering invariant documented and enforced: +- `workflows/review-swarm.yaml:18` — swarm `timeoutMs: 3600000` (60 min) +- `.github/workflows/review-swarm.yml:77` — poll deadline 3900s (65 min), with comment +- `.github/workflows/review-swarm.yml:19` — job `timeout-minutes: 75`, with comment -**As final action**: `git status --porcelain` +**Requirement 8: Wait step records terminal status; post runs on always()** +✅ SATISFIED: +- `.github/workflows/review-swarm.yml:70-91` — wait step records `swarm_status` output, always exits 0 (L91) +- `.github/workflows/review-swarm.yml:93-99` — post step has `if: always() && steps.launch.outputs.run_id != ''` +- `.github/workflows/review-swarm.yml:101-105` — enforce step fails if `swarm_status != 'completed'` -## Out of scope +**Requirement 9: Transcript-to-run-id binding** +✅ SATISFIED — `.github/workflows/scripts/swarm-post.sh:10-12` creates `freshness_marker` (mktemp) BEFORE sync, then `swarm-verdict.sh:27` rejects transcripts with mtime older than the marker (stale check). -- `sdk/` (Track A owns that) -- `kernel/` (gate 1 done, no changes) -- `ops/*` (chief owns briefs and state) -- Any GHA workflow other than review-swarm.yml -- Actually testing the workflow in CI (requires `RELAY_WORKSPACE_KEY` secret set, which is a human step) -- Addressing findings from reviews not yet received (this is the first increment) +### Additional DoD items -## Implementation strategy +**All files parse:** +- `.github/workflows/review-swarm.yml` — valid GHA YAML ✅ +- `workflows/review-swarm.yaml` — valid relayflow spec ✅ +- All `.sh` scripts — bash syntax valid ✅ -Phase 1: Shared verdict logic foundation -- Create `.github/workflows/scripts/swarm-verdict.sh` implementing the three verdict rules: - - Transcript selection sorts by FILENAME (`YYYYMMDD-HHMM` prefix), not mtime - - Verdict is LAST non-empty line's token, not whole-file grep - - `overall = ALL lenses PASSED, else FAILED` — fail-closed on MISSING/UNCLEAR/FAILED +**Aggregate verdict logic in ONE file:** +✅ SATISFIED — `swarm-verdict.sh` is the single source. -Phase 2: GHA runner-side preparation -- Create `.github/workflows/scripts/swarm-prepare.sh` to fetch PR metadata via `gh` on GHA runner -- Drop `.review-target` from `.gitignore` so staged files survive `git add -f` +**Author whitelist absent:** +✅ SATISFIED — no conditional on `github.event.pull_request.user.login`. -Phase 3: Post-swarm sync and comment logic -- Create `.github/workflows/scripts/swarm-post.sh` to: - - Sync cloud run results back - - Source swarm-verdict.sh for verdict extraction - - Find or create sticky marker comment - - Find or update 3 sticky lens transcript comments - - Post verdict as sticky marker edit +**Immutable gate: two checkout steps:** +✅ SATISFIED — verified above. -Phase 4: Main GHA workflow -- Create `.github/workflows/review-swarm.yml` with: - - Two checkout steps (PR head + main's gate files) - - Auth secret preflight step - - Prepare step (run swarm-prepare.sh) - - Launch step (agent-relay cloud run) - - Wait step (with status output, always exits 0) - - Post step (if: always() && run_id != '') - - Fail step (if: swarm_status != 'completed') - - Documented timeout ordering +**SDK tests:** +❌ BLOCKED — `cd sdk && npm test` fails with: +``` +error TS2688: Cannot find type definition file for 'node'. +``` -Phase 5: Refactor existing swarm aggregate -- Edit `workflows/review-swarm.yaml` aggregate step to source swarm-verdict.sh instead of duplicating logic +This is a missing `@types/node` devDependency in `sdk/package.json`. -Phase 6: Documentation -- Add `RELAY_WORKSPACE_KEY` secret documentation to README.md with setup instructions +## Definition of done -## Risks and mitigations +1. ✅ All 9 requirements from ops/TARGET.md verified and documented above +2. ✅ All files parse (verified by reading; no syntax changes needed) +3. ✅ Aggregate verdict logic exists in ONE file (`swarm-verdict.sh`) +4. ✅ Author whitelist absent +5. ✅ Immutable gate: two checkout steps with different paths +6. ❌ `cd sdk && npm test` green — MUST fix @types/node missing -**Risk**: Verdict logic duplication despite shared script -**Mitigation**: Single source of truth in swarm-verdict.sh, both callers source it +**To satisfy DoD item 6:** +Add `@types/node` to `sdk/package.json` devDependencies, run `npm install`, verify tests pass. -**Risk**: Stale transcripts from prior run counted as fresh -**Mitigation**: Requirement #9 — aggregate checks mtime, rejects if older than sync start +**Passing command (after fix):** +```bash +cd sdk && npm test +``` + +Expected output: all tests pass, no TypeScript errors. + +**Final verification command:** +```bash +git status --porcelain +``` + +## Out of scope -**Risk**: Cloud sandbox can't post to PR -**Mitigation**: Requirement #6 — all PR posting happens on GHA runner in post step, not in cloud +- Any changes to `.github/workflows/review-swarm.yml` or scripts (they already satisfy all requirements) +- Any changes to `workflows/review-swarm.yaml` (aggregate step already sources shared verdict logic) +- Any changes to `.gitignore` (no `.review-target` mask present) +- Any changes to `README.md` (already documents `RELAY_WORKSPACE_KEY` at L38-40) +- `kernel/` (gate 1 done, no changes) +- Any GHA workflow other than review-swarm.yml +- Actually testing the workflow in CI (requires human-set secret) +- Track A work (sdk/ implementation; only fixing the test blocker is in scope) -**Risk**: Swarm rejection doesn't fail the workflow -**Mitigation**: Requirement #8 — wait step records status, separate fail step gates merge +## What this package will do +1. Add `@types/node` to `sdk/package.json` devDependencies +2. Run `npm install` in `sdk/` +3. Verify `npm test` passes +4. Verify `git status --porcelain` shows only the expected changes +5. Report findings with literal command outputs diff --git a/sdk/package-lock.json b/sdk/package-lock.json index d4ba514d4..f903db777 100644 --- a/sdk/package-lock.json +++ b/sdk/package-lock.json @@ -826,36 +826,6 @@ "undici-types": "~6.21.0" } }, - "node_modules/ajv": { - "version": "8.17.1", - "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz", - "integrity": "sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g==", - "license": "MIT", - "dependencies": { - "fast-deep-equal": "^3.1.3", - "fast-uri": "^3.0.1", - "json-schema-traverse": "^1.0.0", - "require-from-string": "^2.0.2" - }, - "funding": { - "type": "github", - "url": "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/sponsors/epoberezkin" - } - }, - "node_modules/ajv-draft-04": { - "version": "1.0.0", - "resolved": "https://registry.npmjs.org/ajv-draft-04/-/ajv-draft-04-1.0.0.tgz", - "integrity": "sha512-mv00Te6nmYbRp5DCwclxtt7yV/joXJPGS7nM+97GdxvuttCOfgI3K4U25zboyeX0O+myI8ERluxQe5wljMmVIw==", - "license": "MIT", - "peerDependencies": { - "ajv": "^8.5.0" - }, - "peerDependenciesMeta": { - "ajv": { - "optional": true - } - } - }, "node_modules/@vitest/expect": { "version": "2.1.9", "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-2.1.9.tgz", @@ -969,6 +939,36 @@ "url": "https://opencollective.com/vitest" } }, + "node_modules/ajv": { + "version": "8.17.1", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz", + "integrity": "sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g==", + "license": "MIT", + "dependencies": { + "fast-deep-equal": "^3.1.3", + "fast-uri": "^3.0.1", + "json-schema-traverse": "^1.0.0", + "require-from-string": "^2.0.2" + }, + "funding": { + "type": "github", + "url": "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/sponsors/epoberezkin" + } + }, + "node_modules/ajv-draft-04": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/ajv-draft-04/-/ajv-draft-04-1.0.0.tgz", + "integrity": "sha512-mv00Te6nmYbRp5DCwclxtt7yV/joXJPGS7nM+97GdxvuttCOfgI3K4U25zboyeX0O+myI8ERluxQe5wljMmVIw==", + "license": "MIT", + "peerDependencies": { + "ajv": "^8.5.0" + }, + "peerDependenciesMeta": { + "ajv": { + "optional": true + } + } + }, "node_modules/argparse": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz", @@ -1050,28 +1050,6 @@ "node": ">=6" } }, - "node_modules/fast-deep-equal": { - "version": "3.1.3", - "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", - "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", - "license": "MIT" - }, - "node_modules/fast-uri": { - "version": "3.1.7", - "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.7.tgz", - "integrity": "sha512-dOvZVzjdZdz7phd9v6jCbwxrBW3fK6n8Rc0CtdmM4bumzMnxywBYhuph6J819RRw/ku+rLbelwfMunktuzVVHg==", - "license": "BSD-3-Clause", - "funding": [ - { - "type": "github", - "url": "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/sponsors/fastify" - }, - { - "type": "opencollective", - "url": "https://opencollective.com/fastify" - } - ] - }, "node_modules/es-module-lexer": { "version": "1.7.0", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz", @@ -1138,6 +1116,28 @@ "node": ">=12.0.0" } }, + "node_modules/fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", + "license": "MIT" + }, + "node_modules/fast-uri": { + "version": "3.1.7", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.7.tgz", + "integrity": "sha512-dOvZVzjdZdz7phd9v6jCbwxrBW3fK6n8Rc0CtdmM4bumzMnxywBYhuph6J819RRw/ku+rLbelwfMunktuzVVHg==", + "funding": [ + { + "type": "github", + "url": "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/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "BSD-3-Clause" + }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -1277,6 +1277,15 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/require-from-string": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", + "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/rollup": { "version": "4.63.0", "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.63.0.tgz", @@ -1568,15 +1577,6 @@ } } }, - "node_modules/require-from-string": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", - "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, "node_modules/why-is-node-running": { "version": "2.3.0", "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz",