Skip to content

The coverage gate cannot be reproduced locally (Go 1.26 counts 404 blocks where CI's 1.27 counts 414) and has no rounding headroom left #704

Open

Description

@tannevaled

Two related facts about the 100% coverage gate, both measured today. Neither is a
test failure; together they mean the gate cannot be reproduced on a developer
machine and has no margin left.

1. Local coverage measurement counts fewer statements than the gate

CI uses actions/setup-go@v7 with go-version: stable, currently Go 1.27.1.
This workstation defaults to Go 1.26.4. They do not agree on how many basic
blocks a function contains. Measured on the same commit (8f2a019), same file:

toolchain basic blocks in internal/vm/thread.go
go1.26.4 404
go1.27.1 414

Go 1.27 splits a block when a preceding statement can panic — e.g.
w = m.enqueue(t) after a call that may raise — where 1.26 folded those statements
into the previous block and counted them covered on the strength of a later line
being reached.

The concrete consequence, from PR #698: a local profile reported mutexLock at
100.0% (21 statements) while the gate read 86.2% (29 statements). A tool
that counts fewer statements than the gate cannot witness the gate's complaint
, so
every local coverage figure is an upper bound rather than a measurement.

Suggested fix: pin the toolchain in the workflow (go-version: '1.27.1' rather
than stable) and document that local runs must use the same one, or the
"diff the below-100 list locally" workflow silently disagrees with CI. Pinning also
removes a second hazard already seen today: stable moving under the repo changed
gofmt's trailing-comment alignment between 1.26 and 1.27, turning a green local
gofmt -l into a red lane.

An upside worth recording: the finer blocks were informative. The block that
looked uncovered turned out to be unreachable — both routes to it divert
elsewhere — so there was no test to write, and it was deleted. Go 1.27 exposed dead
code that 1.26 hid.

2. The gate has no headroom: one function tips it

main's own below-100 lists, from the runner logs:

lane below-100 functions reported total verdict
ubuntu-latest 19 100.0% pass
macos-latest 17 100.0% pass

Both pass only because the total rounds to one decimal. Adding a single
below-100 function to the ubuntu lane took #698 to 99.9% and failed it, while
the same function at the same percentage left macOS green — the platform lists
differ, so ubuntu sits nearer the boundary.

So a PR can be failed by the gate without having broken anything: it merely has to
be the first to add an entry. internal/vm/dir.go:572 cwdStillNamed 77.8%
(from #685) is on main right now, absorbed by rounding, and is one of the entries
holding that margin.

Options, in the order I would consider them: cover the 19 (they include two
Inspect bindings at 0.0%, bigdecimal_bind.go:35 and date_bind.go:44, which
have been invisible behind the rounding for weeks); or make the gate compare the
below-100 list against main's rather than a rounded total, which is what every
contributor is already told to do by hand and is the thing the gate is actually
trying to express.

Activity

  1. tannevaled commented on Oct 12, 2026

    @tannevaled
    ContributorAuthor

    Both halves are resolved — measured on main today

    1. The toolchain divergence is closed by construction

    The suggested fix was "pin the toolchain in the workflow (go-version: '1.27.1'
    rather than stable) and document that local runs must use the same one"
    . The
    pin landed (#808 moved it to 1.27.2, in all four places), and the second half
    turns out to need no documentation — it is automatic:

    $ grep -c "go-version: 1.27.2" .github/workflows/ci.yml
    4
    $ grep '^go ' go.mod
    go 1.27.2
    $ go env GOTOOLCHAIN
    auto
    $ go version
    go version go1.27.2 darwin/arm64

    GOTOOLCHAIN=auto makes a local go follow the go directive, so any
    developer whose Go is new enough runs exactly 1.27.2 — the version CI pins —
    without being told to. And actions/setup-go sets GOTOOLCHAIN=local when
    given an explicit version, so CI cannot silently fetch a different one either.
    Both sides land on the same toolchain, which is what the issue asked for.

    That mechanism has a sharp edge worth recording while it is in view: on this
    machine the binary at …/v1.27.1/bin/go reports go1.27.2, because auto
    downloaded and ran what the directive asked for. The path names a version; it
    does not run one. go version is the only way to find out, and it is why the
    GOTOOLCHAIN=local pin in CI is load-bearing rather than decorative — it is
    exactly what made #792's nine red lanes say
    go: go.mod requires go >= 1.27.2 (running go 1.27.1; GOTOOLCHAIN=local)
    instead of quietly measuring something else.

    2. The rounded total is gone — the gate compares the list

    This issue's preferred option was "make the gate compare the below-100 list
    against main's rather than a rounded total, which is what every contributor is
    already told to do by hand"
    . That is what cmd/covratchet +
    scripts/coverage/BELOW100.<lane> do:

    functions below 100%: 40 measured, 40 recorded
    coverage ratchet OK -- no function newly below 100%
    

    and when something regresses it names it rather than tipping a percentage:

    -- NEWLY BELOW 100% (1) --
       internal/vm/builtins.go explicitlyFrozen  90.0% (was fully covered)
    

    That one is from this week, and it is the mechanism working: a 90% function in a
    67-entry list could not have moved a rounded total at all.

    3. The three functions this issue named are now covered

    bigdecimal_bind.go Inspect, date_bind.go Inspect and date_bind.go ToS were
    all at 0.0%, recorded in both lanes. #815 covers them, and they did not have
    the same answer:

    • Two were untested. One line of Ruby reaches each, and the output already
      matched MRI 4.0.7.
    • One was neither. (*Date).ToS was unreachable from Ruby, because to_s
      was registered separately as strM((*date.Date).String) — and it cannot be
      deleted, since object.Value requires ToS. So the method existed to satisfy
      an interface while a second path did the work, and the two renderings of a
      Date could drift with nothing able to notice. to_s now goes through it.

    Closing this on the strength of parts 1 and 2; the 0.0% entries are the last
    concrete item and are in flight.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions