From 217318f5b8901e89d85944fc9c064511be18eadd Mon Sep 17 00:00:00 2001 From: kjgbot Date: Fri, 4 Sep 2026 09:11:53 +0200 Subject: [PATCH] ci: let the SDK suite run where no model is reachable #153 enabled the full SDK suite without giving CI a way to run it. One case in live-kernel.test.ts executes the REAL Claude analyzer and fails by default when it cannot -- deliberately, so that a reader never gets a green that proves nothing about gate 2. A GitHub runner has no `claude` binary: LIVE_ANALYZER_UNAVAILABLE: "...analyze-story-claude-cli auth status" exited 1: cannot run "claude": spawnSync claude ENOENT so the step could never pass. That is my defect from #153, not the test's. RELAYFLOWS_ALLOW_ANALYZER_SKIP is the escape hatch the test itself documents, for "environments that knowingly cannot reach a model and are not counting this run as gate evidence". Setting it here is that statement made explicitly: this workflow is not gate-2 acceptance evidence. Gate-2 evidence must come from a machine that can reach a model, and the skipped case prints as much. Every other case in the suite still runs and still gates. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FtQSAcGDta5VH9xiZFT4sR Session-Id: c228933d-4f94-4d83-9a9a-daf3c83b94f1 --- .github/workflows/cloud-runtime-artifact.yml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/.github/workflows/cloud-runtime-artifact.yml b/.github/workflows/cloud-runtime-artifact.yml index 484897c94..0e3dc20e2 100644 --- a/.github/workflows/cloud-runtime-artifact.yml +++ b/.github/workflows/cloud-runtime-artifact.yml @@ -71,6 +71,25 @@ jobs: - name: Test SDK and type-level authoring contracts working-directory: sdk + env: + # `live-kernel.test.ts` runs one case against the REAL Claude analyzer + # and fails by default when it cannot, deliberately: an unavailable + # analyzer is diagnostics, never acceptance, and a reader must not get + # a green that proves nothing about gate 2. + # + # A GitHub runner is exactly the case the flag was written for -- it + # has no `claude` binary and no model access: + # LIVE_ANALYZER_UNAVAILABLE: ... cannot run "claude": spawnSync claude ENOENT + # so without this the full SDK suite can never pass here, which is a + # defect in #153: it enabled the suite without giving CI a way to run + # it. + # + # Setting it is an explicit statement, not a convenience: **this + # workflow is not gate-2 acceptance evidence.** Gate-2 evidence has to + # come from a machine that can actually reach a model, and the skipped + # case says so in its own output. Everything else in the suite still + # runs and still gates. + RELAYFLOWS_ALLOW_ANALYZER_SKIP: '1' run: | # The whole suite, not four named files. Naming files means a test # added to any other file never runs, which is how ~22 of ~26 SDK