From 8f0cbec91e1d34c173d8a35a1f1e33387f1f6ec0 Mon Sep 17 00:00:00 2001 From: Dmitriy Vasilev Date: Sun, 6 Sep 2026 16:57:19 +0700 Subject: [PATCH 1/2] fix(ci): brain-ci reported an unmeasured value as a score of zero brain-ci has failed on every run in its history, including on main. Neither failure was a health regression. 1. `tri stress --health` currently prints "stress-test: TODO - not implemented yet" and emits no "Score:" line. The step's `grep ... || echo "0"` turned that absence into the NUMBER 0, which the threshold check then reported as "Brain health 0 below threshold 80" -- a fabricated measurement of a command that measured nothing. Same class as A36/A37, where an unmeasured field rendered as an FPGA measurement on a public route. It now distinguishes "not measured" from "measured zero" and prints what the command actually said. The job still fails, because an unimplemented health check is a real deficiency -- but it now fails for the true reason instead of accusing the brain of a regression it never had. 2. "Critical State Notification" carried only `if: failure()`. On a push there is no PR, so context.issue.number is undefined and the call becomes POST /repos/.../issues//comments -> 404, failing the step and the job. That is what "Brain Health Report" has been failing on for every push to main: the reporter, not the check. Now guarded on pull_request, matching the two sibling steps that already were. Also fixes a latent parse bug: the score pattern required "[0-9]+\.[0-9]+", so a Score: line reading "Score: 92" would have parsed as empty and reached bc as an empty string. It now accepts an integer, and an unparseable Score: line is reported as unmeasured rather than silently becoming zero. Verified by running the step's logic over five inputs: the real current output (not measured), a decimal score (pass), an integer score (pass -- the old pattern broke here), a genuine 41.2 regression (still fails, as it must), and an unparseable Score: line (not measured). The gate is not weakened. A check that is always red carries as little information as one that is always green, and this one was red for a reason nobody could act on. Co-Authored-By: Claude Opus 5 --- .github/workflows/brain-ci.yml | 44 +++++++++++++++++++++++++++++++--- 1 file changed, 41 insertions(+), 3 deletions(-) diff --git a/.github/workflows/brain-ci.yml b/.github/workflows/brain-ci.yml index 901d768459..6b3ae5cf2f 100644 --- a/.github/workflows/brain-ci.yml +++ b/.github/workflows/brain-ci.yml @@ -49,8 +49,42 @@ jobs: # Run health check and capture output ./zig-out/bin/tri stress --health 2>&1 | tee /tmp/health-output.txt || true - # Extract health score - HEALTH_SCORE=$(grep "Score:" /tmp/health-output.txt | grep -oE "[0-9]+\.[0-9]+" || echo "0") + # An absent measurement is not a measurement of zero. + # + # `grep "Score:" ... || echo "0"` used to turn "the command printed no + # Score: line at all" into the NUMBER 0, which the threshold check + # below then reported as "Brain health 0 below threshold 80". That + # reads as a measured regression. It is not one: `tri stress --health` + # currently prints "stress-test: TODO - not implemented yet", so there + # has never been a score to compare. The gate has failed on every run + # in its history, including on main, for that reason alone -- and a + # check that is always red carries as little information as one that + # is always green. + # + # Same class as A36/A37, where an unmeasured field rendered as an FPGA + # measurement on a public route. Report what happened, not a number + # that was never produced. + if ! grep -q "Score:" /tmp/health-output.txt; then + echo "HEALTH_SCORE=" >> $GITHUB_OUTPUT + echo "HEALTHY=unknown" >> $GITHUB_OUTPUT + echo "::error::brain health was NOT MEASURED -- 'tri stress --health' emitted no 'Score:' line, so there is no number to compare against the threshold of $BRAIN_HEALTH_THRESHOLD. This is not a health regression. What the command actually printed:" + sed -n '1,20p' /tmp/health-output.txt + exit 1 + fi + + # Extract health score. Accepts an integer as well as a decimal: the + # old pattern required "[0-9]+\.[0-9]+", so a Score: line reading + # "Score: 92" would have parsed as empty and reached bc as an empty + # string. A Score: line we cannot parse is also "not measured" -- it + # is not zero either. + HEALTH_SCORE=$(grep "Score:" /tmp/health-output.txt | grep -oE "[0-9]+(\.[0-9]+)?" | head -1 || true) + if [ -z "$HEALTH_SCORE" ]; then + echo "HEALTH_SCORE=" >> $GITHUB_OUTPUT + echo "HEALTHY=unknown" >> $GITHUB_OUTPUT + echo "::error::a 'Score:' line was present but carried no number this step could parse. This is not a health regression. The line was:" + grep "Score:" /tmp/health-output.txt | head -5 + exit 1 + fi echo "HEALTH_SCORE=$HEALTH_SCORE" >> $GITHUB_OUTPUT # Check if healthy @@ -444,8 +478,12 @@ jobs: exit 1 fi + # On a push there is no PR, so context.issue.number is undefined and the + # call becomes POST /repos/.../issues//comments -> 404 -> this step fails + # and takes the job with it. That is what "Brain Health Report" has been + # failing on for every push to main: not the health check, the reporter. - name: Critical State Notification - if: failure() + if: failure() && github.event_name == 'pull_request' uses: actions/github-script@v7 with: script: | From a0c54d9c32c5b9531c50ffde17611e4ff5bdfd0c Mon Sep 17 00:00:00 2001 From: Dmitriy Vasilev Date: Sun, 6 Sep 2026 17:02:38 +0700 Subject: [PATCH 2/2] fix(ci): correct an overstatement in the brain-ci comment The first version of this comment claimed the 404 was why Brain Health Report failed on every push. That is only part of it: Generate Health Report is an aggregator and exits 1 whenever an upstream job failed, which is correct. The 404 fired on top of that and buried the real verdict under a 30-line octokit dump, so the log read as an API problem rather than as a failing health check. Guarding the notification removes the noise, not the signal. Caught by reading the aggregator's own body after the fix ran in CI, rather than inferring it from the job disappearing off a list. Co-Authored-By: Claude Opus 5 --- .github/workflows/brain-ci.yml | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/.github/workflows/brain-ci.yml b/.github/workflows/brain-ci.yml index 6b3ae5cf2f..7d9e0706b2 100644 --- a/.github/workflows/brain-ci.yml +++ b/.github/workflows/brain-ci.yml @@ -479,9 +479,14 @@ jobs: fi # On a push there is no PR, so context.issue.number is undefined and the - # call becomes POST /repos/.../issues//comments -> 404 -> this step fails - # and takes the job with it. That is what "Brain Health Report" has been - # failing on for every push to main: not the health check, the reporter. + # call becomes POST /repos/.../issues//comments -> 404. + # + # This is NOT the whole reason the job goes red: the step above is an + # aggregator and exits 1 whenever an upstream job failed, which it should. + # But this 404 fired on top of that and buried the real verdict under a + # 30-line octokit dump, so the log read as an API problem rather than as + # "the health check did not pass". Guarding it removes the noise, not the + # signal -- the aggregator still fails the job on its own. - name: Critical State Notification if: failure() && github.event_name == 'pull_request' uses: actions/github-script@v7