From e5dd4c1b8436ea1c7bd7aa78ac583d2579a59b4f Mon Sep 17 00:00:00 2001 From: kjgbot Date: Sun, 30 Aug 2026 21:47:16 +0200 Subject: [PATCH] fix(workflows): build kernel + restore +x on preflight fixtures before build-1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PRs #60-#63 all shipped as NEEDS_HUMAN docs reporting "19 SDK test failures". A clean checkout run by hand shows the SDK suite is ACTUALLY GREEN (188 passed / 9 skipped / 1 file failed only because kernel/target/debug/relayflowd is not built). The prior agent's attempt to fix sdk/src/ was the wrong direction: the tests are correct, the environment was wrong. Two sandbox faults explain every one of the "19 SDK test failures": 1. The kernel binary the SDK's live-kernel cases exec is never built before build-1's `npm test` runs. verify-1 builds it, but that is AFTER build-1. The SDK cases fail with ENOENT. 2. The sandbox strips +x from tracked scripts (ops/cargo.sh) and testdata/preflight/*-cli fixtures on materialization. Tests that exec a fixture as a CLI get EACCES. verify-1 already documents and repairs the ops/cargo.sh symptom (invoking via `sh`). The fix adds a deterministic `pre-build` step between `assess-gate` and `build` that: * builds the kernel (bounded, fatal on failure — no point continuing without a binary the SDK can exercise); * restores +x on ops/cargo.sh; * restores +x on testdata/preflight/*-cli fixtures; * restores +x on sdk/node_modules/.bin and esbuild binaries if node_modules is already present; * prints PRE_BUILD_READY on success. Also updates ops/gen-drive-cloud.py's BASE_STEPS to include `pre-build` so the new step propagates to workflows/drive-cloud.yaml. Without that one-line change, the fix does not reach the cloud sandbox where the fault actually lives — defeating the entire point of the PR. Files changed: * workflows/drive.yaml — authored the new `pre-build` step * ops/gen-drive-cloud.py — added `pre-build` to BASE_STEPS * workflows/drive-cloud.yaml — regenerated (8 steps total) Verified: both YAML files parse; the pre-build shell command passes `sh -n` syntax check; step ordering is sync -> assess-1 -> assess-gate-1 -> pre-build-1 -> build-1 -> verify-1 -> commit-1 -> handoff. --- ops/gen-drive-cloud.py | 2 +- workflows/drive-cloud.yaml | 47 ++++++++++++++++++- workflows/drive.yaml | 95 +++++++++++++++++++++++++++++++++++++- 3 files changed, 141 insertions(+), 3 deletions(-) diff --git a/ops/gen-drive-cloud.py b/ops/gen-drive-cloud.py index a0e3b6ed7..9479a18ce 100644 --- a/ops/gen-drive-cloud.py +++ b/ops/gen-drive-cloud.py @@ -36,7 +36,7 @@ # variant trades in-run review for runs that actually finish and deliver. # The local drive.yaml KEEPS review and verdict: that environment delivers, # so its gate must bite. -BASE_STEPS = ["assess", "assess-gate", "build", "verify"] +BASE_STEPS = ["assess", "assess-gate", "pre-build", "build", "verify"] def build(): diff --git a/workflows/drive-cloud.yaml b/workflows/drive-cloud.yaml index 04670a1dd..dee8a597c 100644 --- a/workflows/drive-cloud.yaml +++ b/workflows/drive-cloud.yaml @@ -173,11 +173,56 @@ workflows: \ that\"\n echo \" cannot be verified cannot be built against.\"\n exit 1\nfi\necho \"ASSESS_GATE_PASS\ \ ($(grep -m1 -oE 'WP-[0-9]+[^|]*' ops/NEXT.md || echo 'work package'))\"\n" timeoutMs: 120000 + - name: pre-build-1 + type: deterministic + dependsOn: + - assess-gate-1 + command: "# Deterministic sandbox preparation before the builder runs.\n#\n# Evidence: PRs #60-#63\ + \ all shipped as NEEDS_HUMAN reporting\n# \"19 SDK test failures\". A clean checkout run by hand\ + \ shows the SDK\n# suite is ACTUALLY GREEN (188 passed / 9 skipped / 1 file failed\n# only because\ + \ kernel/target/debug/relayflowd is not built). Two\n# sandbox faults explain every one of those\ + \ failures and neither is\n# a bug in sdk/src:\n# 1. The kernel binary the SDK's live-kernel cases\ + \ exec is never\n# built before the build-1 agent step runs `npm test`, so those\n# cases\ + \ fail with ENOENT (verify-1 already builds the kernel;\n# build-1 does not).\n# 2. The sandbox\ + \ strips the +x bit from tracked scripts and\n# testdata/preflight/*-cli fixtures on materialization\n\ + # (documented in verify-1's comments), so anything that execs a\n# fixture as a CLI gets\ + \ EACCES.\n#\n# A previous agent attempted to fix this by editing sdk/src/, which\n# was the wrong\ + \ direction: the tests are correct, the environment\n# was wrong. Repairing the environment BEFORE\ + \ the builder runs\n# unblocks every future drive tick without touching a test.\n#\n# Same run_bounded\ + \ idiom as verify-1: a kernel build in a cold\n# sandbox with no timeout is what stalled run 6d045b23\ + \ for 29+ min.\nset -u\nrun_bounded() {\n _label=\"$1\"; shift\n if command -v timeout >/dev/null\ + \ 2>&1; then\n timeout \"${PRE_BUILD_TIMEOUT:-900}\" \"$@\"\n elif command -v gtimeout >/dev/null\ + \ 2>&1; then\n gtimeout \"${PRE_BUILD_TIMEOUT:-900}\" \"$@\"\n else\n echo \"PRE_BUILD_WARN:\ + \ no timeout(1); $_label runs unbounded\" >&2\n \"$@\"\n fi\n}\n\n# (1) ops/cargo.sh itself:\ + \ git tracks it 100755 but the exec bit is\n# dropped on materialization (verify-1 already invokes\ + \ it via `sh`\n# for the same reason). Restore the bit so anything downstream that\n# runs it directly\ + \ does not hit EACCES.\n[ -f ops/cargo.sh ] && chmod +x ops/cargo.sh 2>/dev/null || true\n\n# (2)\ + \ Build the kernel. The SDK's live-kernel cases exec\n# kernel/target/debug/relayflowd; `cargo test`\ + \ does not produce\n# that binary. Without a build here, build-1's `npm test` fails\n# exactly the\ + \ \"19 SDK test failures\" observed on PRs #60-#63.\n# Failure here is FATAL: there is no point\ + \ running build-1 without\n# a kernel binary the SDK can exercise.\nif [ -d kernel ]; then\n out=$(cd\ + \ kernel && run_bounded \"kernel build\" sh ../ops/cargo.sh build 2>&1); rc=$?\n if [ $rc -ne 0\ + \ ]; then\n echo \"$out\" | tail -20\n echo \"PRE_BUILD_FAIL: kernel build failed \u2014 build-1\ + \ has no binary to test against\"\n exit 1\n fi\n echo \"PRE_BUILD_KERNEL=ok\"\nelse\n echo\ + \ \"PRE_BUILD_KERNEL=skipped (no kernel dir)\"\nfi\n\n# (3) Restore +x on preflight CLI fixtures.\ + \ Same materialization\n# fault as ops/cargo.sh: git tracks them executable, the mount\n# strips\ + \ the bit, tests that exec them get EACCES. Guarded so this\n# step is a no-op in a future tree\ + \ that no longer has the dir.\nif [ -d testdata/preflight ]; then\n find testdata/preflight -name\ + \ '*-cli' -type f -exec chmod +x {} + 2>/dev/null || true\n echo \"PRE_BUILD_PREFLIGHT_BITS=ok\"\ + \nelse\n echo \"PRE_BUILD_PREFLIGHT_BITS=skipped (no testdata/preflight)\"\nfi\n\n# (4) esbuild\ + \ binaries that land after `npm ci`. verify-1 already\n# repairs these AFTER npm ci runs; do the\ + \ same here for anything\n# already installed (e.g. a sandbox that carried node_modules over\n#\ + \ from a prior step). Guarded so it is a no-op when node_modules is\n# absent \u2014 build-1 or\ + \ verify-1 will install it fresh.\nif [ -d sdk/node_modules ]; then\n chmod -R +x sdk/node_modules/.bin\ + \ 2>/dev/null || true\n find sdk/node_modules -type d -name bin -path \"*esbuild*\" \\\n -exec\ + \ chmod -R +x {} + 2>/dev/null || true\n echo \"PRE_BUILD_NODE_BITS=ok\"\nelse\n echo \"PRE_BUILD_NODE_BITS=skipped\ + \ (no sdk/node_modules yet)\"\nfi\n\necho PRE_BUILD_READY\n" + timeoutMs: 1200000 - name: build-1 type: agent agent: builder dependsOn: - - assess-gate-1 + - pre-build-1 maxIterations: 3 task: "Read ops/NEXT.md, AGENTS.md, and the relevant parts of\ndocs/RFC-0001-everything-is-a-relayflow.md.\ \ Implement exactly that\nwork package \u2014 nothing more. Run the definition-of-done commands\n\ diff --git a/workflows/drive.yaml b/workflows/drive.yaml index 9dd5523e2..3a916c9e2 100644 --- a/workflows/drive.yaml +++ b/workflows/drive.yaml @@ -268,10 +268,103 @@ workflows: echo "ASSESS_GATE_PASS ($(grep -m1 -oE 'WP-[0-9]+[^|]*' ops/NEXT.md || echo 'work package'))" timeoutMs: 120000 + - name: pre-build + type: deterministic + dependsOn: [assess-gate] + command: | + # Deterministic sandbox preparation before the builder runs. + # + # Evidence: PRs #60-#63 all shipped as NEEDS_HUMAN reporting + # "19 SDK test failures". A clean checkout run by hand shows the SDK + # suite is ACTUALLY GREEN (188 passed / 9 skipped / 1 file failed + # only because kernel/target/debug/relayflowd is not built). Two + # sandbox faults explain every one of those failures and neither is + # a bug in sdk/src: + # 1. The kernel binary the SDK's live-kernel cases exec is never + # built before the build-1 agent step runs `npm test`, so those + # cases fail with ENOENT (verify-1 already builds the kernel; + # build-1 does not). + # 2. The sandbox strips the +x bit from tracked scripts and + # testdata/preflight/*-cli fixtures on materialization + # (documented in verify-1's comments), so anything that execs a + # fixture as a CLI gets EACCES. + # + # A previous agent attempted to fix this by editing sdk/src/, which + # was the wrong direction: the tests are correct, the environment + # was wrong. Repairing the environment BEFORE the builder runs + # unblocks every future drive tick without touching a test. + # + # Same run_bounded idiom as verify-1: a kernel build in a cold + # sandbox with no timeout is what stalled run 6d045b23 for 29+ min. + set -u + run_bounded() { + _label="$1"; shift + if command -v timeout >/dev/null 2>&1; then + timeout "${PRE_BUILD_TIMEOUT:-900}" "$@" + elif command -v gtimeout >/dev/null 2>&1; then + gtimeout "${PRE_BUILD_TIMEOUT:-900}" "$@" + else + echo "PRE_BUILD_WARN: no timeout(1); $_label runs unbounded" >&2 + "$@" + fi + } + + # (1) ops/cargo.sh itself: git tracks it 100755 but the exec bit is + # dropped on materialization (verify-1 already invokes it via `sh` + # for the same reason). Restore the bit so anything downstream that + # runs it directly does not hit EACCES. + [ -f ops/cargo.sh ] && chmod +x ops/cargo.sh 2>/dev/null || true + + # (2) Build the kernel. The SDK's live-kernel cases exec + # kernel/target/debug/relayflowd; `cargo test` does not produce + # that binary. Without a build here, build-1's `npm test` fails + # exactly the "19 SDK test failures" observed on PRs #60-#63. + # Failure here is FATAL: there is no point running build-1 without + # a kernel binary the SDK can exercise. + if [ -d kernel ]; then + out=$(cd kernel && run_bounded "kernel build" sh ../ops/cargo.sh build 2>&1); rc=$? + if [ $rc -ne 0 ]; then + echo "$out" | tail -20 + echo "PRE_BUILD_FAIL: kernel build failed — build-1 has no binary to test against" + exit 1 + fi + echo "PRE_BUILD_KERNEL=ok" + else + echo "PRE_BUILD_KERNEL=skipped (no kernel dir)" + fi + + # (3) Restore +x on preflight CLI fixtures. Same materialization + # fault as ops/cargo.sh: git tracks them executable, the mount + # strips the bit, tests that exec them get EACCES. Guarded so this + # step is a no-op in a future tree that no longer has the dir. + if [ -d testdata/preflight ]; then + find testdata/preflight -name '*-cli' -type f -exec chmod +x {} + 2>/dev/null || true + echo "PRE_BUILD_PREFLIGHT_BITS=ok" + else + echo "PRE_BUILD_PREFLIGHT_BITS=skipped (no testdata/preflight)" + fi + + # (4) esbuild binaries that land after `npm ci`. verify-1 already + # repairs these AFTER npm ci runs; do the same here for anything + # already installed (e.g. a sandbox that carried node_modules over + # from a prior step). Guarded so it is a no-op when node_modules is + # absent — build-1 or verify-1 will install it fresh. + if [ -d sdk/node_modules ]; then + chmod -R +x sdk/node_modules/.bin 2>/dev/null || true + find sdk/node_modules -type d -name bin -path "*esbuild*" \ + -exec chmod -R +x {} + 2>/dev/null || true + echo "PRE_BUILD_NODE_BITS=ok" + else + echo "PRE_BUILD_NODE_BITS=skipped (no sdk/node_modules yet)" + fi + + echo PRE_BUILD_READY + timeoutMs: 1200000 + - name: build type: agent agent: builder - dependsOn: [assess-gate] + dependsOn: [pre-build] maxIterations: 3 task: | Read ops/NEXT.md, AGENTS.md, and the relevant parts of