Repository navigation
Raise ruby/spec floor 22170 -> 22205 after wave 31 - #664
Merged
Merged
Conversation
Read this number against the previous one with care: the floor FELL from 22345 to 22170 earlier in this same wave, because #660 fixed the harness. The comparison that means something is 22170 -> 22205, both measured with the corrected shim. #660 The shim's ShouldProxy inherited Object, so every predicate Object answers was dispatched on the PROXY and discarded. Eleven spellings, 2806 assertion sites across 811 files. Two were not merely vacuous: `should === y` compared the wrong pair (a false FAILURE) and `should !~ re` ran the POSITIVE match (the exact inverse). Fixed by inheriting BasicObject, as upstream mspec does. 205 examples newly fail; classified against MRI through the same shim, 205/205 are real rbgo defects the vacuous assertion had hidden and 0/205 are limits of the harness. #662 const_source_location 13 -> 39, const_added 4 -> 13, classpath by permanence, `class Foo::Bar` firing a pending autoload — and the #615 crash, FIXED. That flake had distorted every core/module measurement for three weeks. #661 Refinements: the Refinement class surface, a refinement's superclass being the class it refines, `using` replacing the frame cref. Measured with scripts/conformance/rubyspec/run.sh against the pinned corpus at 87b1631, 16-way, TIMEOUT=25, three consecutive runs: 22221, 22221, 22220. The floor is set fifteen below the lowest. Two agents had their briefing premise overturned by measurement this wave, both mine: a gsub defect I routed had been fixed three waves earlier, and a refinement activation lead was fixed between its measurement and the brief. Running MRI through our own shim to get a per-file ceiling is now the standing method, and it is what caught both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read this number against the previous one with care. The floor fell from 22345 to 22170 earlier in this same wave, because #660 fixed the harness. The comparison that means something is 22170 → 22205, both measured with the corrected shim.
ShouldProxyinheritedObject— 2806 assertion sites across 811 files asserted nothing, and two spellings asserted the oppositeconst_source_location13 → 39,const_added4 → 13, classpath by permanence,class Foo::Barfiring a pending autoload — and the #615 crash, fixedRefinementclass surface, a refinement's superclass being the class it refines,usingreplacing the frame crefWhat #660 changed about every number we publish
ShouldProxyinheritedObject, so every predicateObjectanswers was dispatched on the proxy and discarded. Eleven spellings. Two were not merely vacuous:should === ycompared the wrong pair — a false failure — andshould !~ reran the positive match, the exact inverse.205 examples newly fail. Classified against MRI 4.0.5 run through the same corrected shim: 205/205 are real rbgo defects the vacuous assertion had hidden (
IO#internal_encodingreturning nil,Rational#eql?false for(1/1),RUBY_VERSIONnot frozen,Integer#to_sin UTF-8), and 0/205 are limits of the harness.#615 is closed
The intermittent frame-pop crash had distorted every
core/modulemeasurement for three weeks — an aborted run contributes 0, which reads as a ~60-example regression, and several agents lost time re-running because of it. Verified independently: 0 crashes in 12 runs wheremaincrashed about 1 in 4. What it does not fix is named in the closing comment: the frame stacks are still per-VM rather than per-thread.Measurement
scripts/conformance/rubyspec/run.sh, pinned corpus87b1631, 16-way,TIMEOUT=25, three consecutive runs:Floor set fifteen below the lowest.
Two briefing premises overturned by measurement, both mine
A
gsubdefect I routed across three waves had been fixed in the first of them; a refinement-activation lead was fixed between its measurement and the brief I wrote from it. Both were caught by agents who reproduced the lead before chasing it. Running MRI through our own shim for a per-file ceiling is now the standing method — it is what separates a real defect from something the harness cannot express, and it has overturned a premise in each of the last three waves.🤖 Generated with Claude Code