Skip to content

Raise ruby/spec floor 22205 -> 22475 after wave 32 - #673

Merged
tannevaled merged 1 commit into
mainfrom
wave32/floor
Sep 26, 2026
Merged

tannevaled merged 1 commit into
mainfrom
wave32/floor

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Three units, +283 measured, and the first wave whose numbers are trustworthy throughout: #660 repaired the harness at the end of wave 31, so this is the first measurement in the campaign with no known vacuous assertions beneath it.

PR area gain
#666 core/kernel, numerics, symbols +162, core/rational and core/builtin_constants at the ceiling
#668 IO/Dir encodings resolved once at open, #syswrite reaching the descriptor, IO::Buffer's base/size distinction +56, and 13 of 16 files touched now sit exactly at the ceiling
#665 singleton classes of nil/true/false, post parameters in Method#arity and #parameters, !~/!/!= dispatched

45 examples were invisible, not failing

core/integer/pow_spec.rb and exponent_spec.rb each scored 0 — never finishing. The corpus contains 100000000 ** 1000000000, which rbgo computed unbounded until the harness killed the process. bignum.c's BIGLEN_LIMIT refuses it. Same shape as the shim story: a file reporting 0 needs its ceiling measured before that 0 is read as failures.

One cause, three defects

#663 reported that !~ was compiled to !(a =~ b) and never dispatched. Asking whether its neighbours shared the shape found ! and != doing the same — all three definitions were dead code on main. Any object redefining them (a null object, a matcher, a DSL) lost the definition silently.

Measurement

scripts/conformance/rubyspec/run.sh, pinned corpus 87b1631, 16-way, TIMEOUT=25, three consecutive runs:

passing examples: 22489   (files loaded: 2194, failed to load: 12)
passing examples: 22489
passing examples: 22488
frozen floor:     22205

Floor set thirteen below the lowest.

Three briefing targets dissolved under measurement — the third wave running

  • core/io/write_spec.rb's "9 failures": a real gap of 1. MRI fails the other eight through this shim.
  • core/array/fill_spec.rb's "9": a gap of zero. Ceiling 38, rbgo 38.
  • core/numeric/step_spec.rb's 12.700000000000001: what MRI prints.

Running MRI through our own shim for a per-file ceiling is what catches these, and it is now done before planning rather than after.

The largest lever left is in none of these lanes

core/file/printf_spec.rb scores 0 against a ceiling of 160: rbgo cannot parse a do…end block inside an arrow-lambda that is a paren-less command argument.

foo :a, -> { [1].each do |x| x end }   # rbgo: parse error. MRI: fine.
foo(:a, -> { … })                      # parses
foo :a, lambda { … }                   # parses

One parser fix, 160 examples — issue #669.

🤖 Generated with Claude Code

Three units, +283 measured, and the first wave whose numbers are trustworthy
throughout: #660 repaired the harness at the end of wave 31, so this is the
first measurement in the campaign with no known vacuous assertions under it.

  #666  core/kernel, numerics, symbols (+162). 45 of those were INVISIBLE
        rather than failing: core/integer/{pow,exponent}_spec.rb each scored 0
        because the corpus contains `100000000 ** 1000000000` and rbgo computed
        it unbounded until the harness killed the process. bignum.c's
        BIGLEN_LIMIT refuses it. core/rational and core/builtin_constants are
        now AT the ceiling.
  #668  IO/Dir encodings resolved once at open, #syswrite reaching the
        descriptor, IO::Buffer's base/size distinction (+56). THIRTEEN of the
        sixteen files it touched now sit exactly at the ceiling the harness can
        express.
  #665  singleton classes of nil/true/false, post parameters in Method#arity
        AND #parameters together, and `!~`/`!`/`!=` DISPATCHED rather than
        synthesised — one cause, three defects, found by asking whether #663's
        neighbours shared its shape.

Measured with scripts/conformance/rubyspec/run.sh against the pinned corpus at
87b1631, 16-way, TIMEOUT=25, three consecutive runs: 22489, 22489, 22488. The
floor is set thirteen below the lowest.

Three of my briefing targets dissolved under measurement again — the third wave
running. core/io/write_spec's "9 failures" is a real gap of 1 (MRI fails the
other eight through this shim); core/array/fill_spec's "9" is a gap of ZERO
(ceiling 38, rbgo 38); core/numeric/step_spec's 12.700000000000001 is what MRI
prints. Running MRI through our own shim for a per-file ceiling is what catches
these, and it is now done before planning rather than after.

The largest lever left is not in any of these lanes: core/file/printf_spec.rb
scores 0 against a ceiling of 160 because rbgo cannot parse a do...end block
inside an arrow-lambda that is a paren-less command argument. One parser fix,
160 examples — issue #669.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 3e8e3cc into main Sep 26, 2026
7 checks passed
@tannevaled
tannevaled deleted the wave32/floor branch September 26, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant