Skip to content

fix(ci): brain-ci reported an unmeasured value as a score of zero - #957

Merged
gHashTag merged 2 commits into
mainfrom
fix/brain-ci-false-measurement
Sep 6, 2026
Merged

gHashTag merged 2 commits into
mainfrom
fix/brain-ci-false-measurement

Conversation

@gHashTag

@gHashTag gHashTag commented Sep 6, 2026

Copy link
Copy Markdown
Owner

brain-ci has failed on every run in its history, including on main.
Neither of its two failures is a health regression.

1. An unmeasured value reported as a score of zero

tri stress --health currently prints:

⚠️  stress-test: TODO - not implemented yet

and emits no Score: line. The step's grep "Score:" ... || echo "0" turned
that absence into the number 0, which the threshold check then reported as:

##[error]Brain health 0 below threshold 80

That reads as a measured collapse of brain health. It is a fabricated
measurement of a command that measured nothing — the same class as A36/A37,
where an unmeasured field rendered as an FPGA measurement on a public route.

The step now distinguishes not measured from measured zero, and prints what
the command actually said. The job still fails, because an unimplemented
health check is a real deficiency — but it fails for the true reason instead of
accusing the brain of a regression it never had.

2. The reporter, not the check

Critical State Notification carried only if: failure(). On a push there is no
PR, so context.issue.number is undefined and the call becomes:

POST /repos/gHashTag/trinity/issues//comments   ->  404

which fails the step and takes the job with it. That is what 📋 Brain Health
Report
has been failing on for every push to main. Now guarded on
pull_request, matching the two sibling steps that already were. (Audited all
four context.issue.number uses in the file; this was the only unguarded one.)

3. A latent parse bug

The score pattern required [0-9]+\.[0-9]+, so a line reading Score: 92 would
have parsed as empty and reached bc as an empty string. It now accepts an
integer, and an unparseable Score: line is reported as unmeasured rather than
silently becoming zero.

Verified

Ran the step's logic over five inputs rather than reading it through:

input result
the real current output not measured, exit 1
Score: 92.5 HEALTHY pass
Score: 92 (integer) pass — the old pattern broke here
Score: 41.2 fails, as a genuine regression must
Score: n/a not measured, exit 1

The gate is not weakened — a real low score still fails it. A check that is
always red carries as little information as one that is always green, and this
one was red for a reason nobody could act on.

Still open

tri stress --health is unimplemented. That is the actual work behind this
gate, and it is not in this PR.

gHashTag and others added 2 commits September 6, 2026 16:57
brain-ci has failed on every run in its history, including on main. Neither
failure was a health regression.

1. `tri stress --health` currently prints "stress-test: TODO - not implemented
   yet" and emits no "Score:" line. The step's `grep ... || echo "0"` turned
   that absence into the NUMBER 0, which the threshold check then reported as
   "Brain health 0 below threshold 80" -- a fabricated measurement of a
   command that measured nothing. Same class as A36/A37, where an unmeasured
   field rendered as an FPGA measurement on a public route.

   It now distinguishes "not measured" from "measured zero" and prints what the
   command actually said. The job still fails, because an unimplemented health
   check is a real deficiency -- but it now fails for the true reason instead of
   accusing the brain of a regression it never had.

2. "Critical State Notification" carried only `if: failure()`. On a push there
   is no PR, so context.issue.number is undefined and the call becomes
   POST /repos/.../issues//comments -> 404, failing the step and the job. That
   is what "Brain Health Report" has been failing on for every push to main:
   the reporter, not the check. Now guarded on pull_request, matching the two
   sibling steps that already were.

Also fixes a latent parse bug: the score pattern required "[0-9]+\.[0-9]+", so
a Score: line reading "Score: 92" would have parsed as empty and reached bc as
an empty string. It now accepts an integer, and an unparseable Score: line is
reported as unmeasured rather than silently becoming zero.

Verified by running the step's logic over five inputs: the real current output
(not measured), a decimal score (pass), an integer score (pass -- the old
pattern broke here), a genuine 41.2 regression (still fails, as it must), and
an unparseable Score: line (not measured).

The gate is not weakened. A check that is always red carries as little
information as one that is always green, and this one was red for a reason
nobody could act on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first version of this comment claimed the 404 was why Brain Health Report
failed on every push. That is only part of it: Generate Health Report is an
aggregator and exits 1 whenever an upstream job failed, which is correct. The
404 fired on top of that and buried the real verdict under a 30-line octokit
dump, so the log read as an API problem rather than as a failing health check.
Guarding the notification removes the noise, not the signal.

Caught by reading the aggregator's own body after the fix ran in CI, rather
than inferring it from the job disappearing off a list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Sep 6, 2026

Copy link
Copy Markdown
Owner Author

Correction to §2 of the description, pushed as a0c54d9c.

I wrote that the 404 was what Brain Health Report had been failing on. That is
only part of it. Generate Health Report is an aggregator: it exits 1 whenever
an upstream job did not succeed, which is correct behaviour. The job would go red
on that alone.

What the 404 actually did was fire on top of the real verdict and bury it
under a 30-line octokit dump, so the log read as a GitHub API problem rather than
as "the health check did not pass". Guarding it removes the noise, not the
signal.

I caught this by reading the aggregator's body after the fix ran in CI, rather
than concluding the fix worked because a job name dropped off a list.

Verified on the runner, not only locally — the health step now emits:

##[error]brain health was NOT MEASURED -- 'tri stress --health' emitted no
'Score:' line, so there is no number to compare against the threshold of 80.
This is not a health regression.

in place of the old Brain health 0 below threshold 80.

@gHashTag
gHashTag merged commit 47948ad into main Sep 6, 2026
21 of 26 checks passed
@gHashTag
gHashTag deleted the fix/brain-ci-false-measurement branch September 6, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant