[tannevaled]

Bumps github.com/go-ruby-parser/parser v0.3.0 → v0.4.0. go.mod / go.sum only; no replace directive (checked with grep -n replace go.mod before pushing).

v0.4.0 splits the parser's single noDo flag into noDo (MRI's COND_P()) and noDoBlock (MRI's CMDARG_P()), read through one atDoBlock() helper. One flag could not express the reset MRI performs on entering a nested statement body, because MRI keeps two independent bit-stacks that are reset in different places. The effect for rbgo: a do…end written inside a paren-less command argument now attaches to the call it follows instead of being refused.

What moved — ruby/spec ratchet

Measured on the pinned corpus (SPEC_SHA 87b1631, /Users/Shared/rbgo-measure/rubyspec-corpus) with the vendored spec_helper.rb shim, one rbgo process per file, TIMEOUT=25, 2-way — this corpus is not reproducible under full parallelism. origin/main (3e8e3cc) and this branch were measured back to back in the same conditions, with a purpose-built per-file runner that emits one row per input and fails if rows ≠ inputs (2206 in, 2206 out, both runs).

	passing examples
origin/main, parser v0.3.0	22 488
this branch, parser v0.4.0	22 711
delta	+223

FLOOR (22475) is not touched — it is raised in the wave's consolidated PR.

CI's own ratchet lane on this branch (full parallelism, TIMEOUT=60) reads 22 724 passing, +249 over the floor, with files loaded: 2196, failed to load: 10 — the same per-file status split this local run produced (2196 ok; 7 whole-file failures + 3 timeouts). The 13-example gap between 22 711 and 22 724 is the load/timeout sensitivity the corpus is known for, and it is why the delta was taken from two back-to-back local runs rather than by comparing a local number against a remote one.

Every file whose number changed:

file	before	after	MRI 4.0.5 through the same shim
core/file/printf_spec.rb	0 — whole-file parse error	161	160
core/time/utc_spec.rb	0 — whole-file parse error	61	63
language/string_spec.rb	23	24	34

No file moved down. Whole-file failures 9 → 7, timeouts 3 → 3 (the same three files), files loading OK 2194 → 2196.

The two zeros were parse errors, not hangs — established by running each file directly rather than inferring it:

• core/file/printf_spec.rb:9 — it_behaves_like :kernel_sprintf, -> format, *args { … File.open(@filename, "w") do |f| … end … }: a do…end in a lambda's brace body that is itself a paren-less command argument.
• core/time/utc_spec.rb:89 — guard -> { with_timezone 'right/UTC' do … end } do … end: the same shape.
• language/string_spec.rb gains exactly the example ruby/spec names "allows a dynamic string to parse a nested do…end block as an argument to a call without parens, interpolated", reported on main as error SyntaxError parse error at line 1: unexpected "do" after statement. That is the string-interpolation entry (tSTRING_DBEG).

A second corpus: MRI 4.0.5's own stdlib

The ratchet can only see shapes ruby/spec happens to contain. scripts/conformance/rspec/parsesweep run over MRI 4.0.5's rubylibdir (728 files) measures the front end on a population the corpus does not cover:

	files accepted
parser v0.3.0	712 / 728 — 97.8%
parser v0.4.0	723 / 728 — 99.3%

11 files newly accepted, 0 regressed (every row joined, 728 in / 728 out), all previously unexpected "do" after statement:

optparse.rb                                   pp.rb
tempfile.rb                                   error_highlight/core_ext.rb
prism/translation/ripper.rb                   prism/translation/ruby_parser.rb
syntax_suggest/code_line.rb                   syntax_suggest/display_code_with_line_numbers.rb
syntax_suggest/display_invalid_blocks.rb      syntax_suggest/left_right_lex_count.rb
bundler/vendor/connection_pool/lib/connection_pool/wrapper.rb

The same sweep over the whole pinned ruby/spec tree (4413 .rb files, not just language/ + core/): 4405 → 4407; the two gained are exactly the two spec files above, and the 6 that remain are 2 fixtures that MRI also rejects on purpose plus 4 unrelated gaps (listed below).

Verified in both directions, against local MRI 4.0.5

Every shape below was run byte for byte through ruby, ruby -c, ruby --parser=parse.y -c and both rbgo binaries. The witness prints which call actually received the block (block_given?), so it measures attachment, not just parseability.

Newly accepted, and attaching to the call MRI attaches it to — all were parse error on main:

foo :a, -> { [1].each do |x| x end }      foo :a, begin; y do end; end
foo :a, (y do end)                        foo :a, [y do end]
foo :a, {k: y do end}                     foo :a, "#{y do end}"
foo :a, -> { y do end }                   foo :a, -> do y do end end
foo :a, if c; y do end; end               foo :a, class Q; y do end; end
while -> { y do end }.call; end           foo :a, (y do end) do end
foo :a, [y do end] do end                 foo :a, {k: y do end} do end
foo :a, -> { y do end } do end

Refusal direction — 17 shapes MRI rejects or accepts, checked for over-acceptance. An over-accepting parser is a silent regression, so this is the half that matters most. No shape moved from refused to accepted against MRI:

while y do end; end                  reject / reject / reject
while begin; y do end; end; end      reject / reject / reject
while -> do y do end end.call; end   reject / reject / reject
while if true then y do end end; end reject / reject / reject
while (y do end) do end; end         reject / reject / reject
until y do end; end                  reject / reject / reject
for i in [1] do y do end             reject / reject / reject
foo :a, b do end do end              reject / reject / reject
foo :a, y do end do end              reject / reject / reject
foo :a, begin; y do end              reject / reject / reject
foo :a, "#{y do end                  reject / reject / reject
foo :a, if c; y do end               reject / reject / reject
                                     (MRI / v0.3.0 / v0.4.0)

Two pre-existing over-acceptances were found, identical on v0.3.0 and v0.4.0, so neither is caused by this PR: foo :a, y 1 and case y do end; end are SyntaxError in MRI under both its parsers and are accepted by rbgo. Reported, not touched.

Unchanged shapes that had to stay unchanged, and did: foo :a, y do end (block is foo's), foo(:a, y do end) (block is y's), foo :a, y(1) do end, foo :a, bar_x = y do end, while [1].each { |x| y do end }; end, while y(); end, for i in [1] do (y do end); end, def m(x = y do end); end.

Bibliography for each decision, read from parse.y at v3_4_0 (cached under /Users/Shared/rbgo-measure/mri-c/): the three-way keyword_do / keyword_do_cond / keyword_do_block decision in parser_yylex (10519-10527); COND_PUSH(1) in expr_value_do (3430); CMDARG_PUSH(1) in command_args with its lookahead cancellation for ( / [ (4252-4265); and COND_PUSH(0); CMDARG_PUSH(0) in the lexer cases for ( (11193), [ (11221) and { (11245) and in the tSTRING_DBEG action (6181).

Gates

• CGO_ENABLED=0 GOWORK=off go test ./... — full suite, green (internal/vm 175.0s; exit 0)
• CGO_ENABLED=0 go build ./... and CGO_ENABLED=0 GOOS=wasip1 GOARCH=wasm go build ./... — green
• go vet ./... — clean; gofmt -l internal/ cmd/ — empty
• coverage: measured with -race -coverpkg=$(go list ./internal/... | paste -sd, -) on this branch and on a detached worktree at origin/main, and compared as a LIST, never as a total. Both sides: internal/vm 99.0%, reported total 99.9%, 17 functions below 100% — and the two below-100 lists are byte-identical (sha1 82afa873…). No function was orphaned: the only two at 0.0% (internal/vm/bigdecimal_bind.go:35 Inspect, internal/vm/date_bind.go:44 Inspect) are at 0.0% on origin/main as well.
• internal/vm/prelude_frozen_gen.go: regenerated with go run ./cmd/freeze-prelude from the repo root, byte-identical (sha1 153c04b5…). Because a byte-identical regeneration is not by itself evidence that anything ran, the file was also deleted and regenerated from scratch under both parser versions: both reproduce the same sha1. So the generator does write, and prelude.rb's lowering is genuinely unaffected by the bump.

Left undone, with reasons

1. FLOOR not raised — it belongs to the wave's consolidated PR. Measured total on this branch: 22711.

2. The COND half of the same fix is still missing, upstream of this repo. MRI's lexer pushes COND_PUSH(0) and CMDARG_PUSH(0) at (, [, { and in the tSTRING_DBEG action. v0.4.0 clears noDoBlock (CMDARG) at those entries but never clears noDo (COND), so inside a while/until condition these six shapes are still refused although MRI 4.0.5 accepts them under both its parsers — identical on v0.3.0, so a pre-existing gap and not a regression here:

   while (y do end); end                    while [y do end].first; end
   while {k: y do end}.values.first; end    while "#{y do end}".empty?; end
   while y([1].each do |q| q end); end      until (y do end); end
   

   Owner: go-ruby-parser/parser. v0.4.0's comment in parseStatements states the MRI rule correctly ("only the bracket and interpolation entries also push COND 0") but no bracket or interpolation entry point acts on it.

3. Four further parser gaps, each reduced to a one-liner, found by the stdlib sweep and identical on v0.3.0 and v0.4.0 (owner: go-ruby-parser/parser):

   shape	MRI	rbgo	seen in
   def M::name(v); end / def self::m; end	accepts	expected CONST, got …	rbconfig.rb:310, mkmf.rb:327
   then on the line after the condition	accepts	unexpected token "then"	ripper/filter.rb:65, ripper/lexer.rb:279
   when with its value on the next line	accepts	unexpected token "\n"	rubygems/request.rb:125
   -> x: { } — lambda kwarg + brace body	accepts	expected {, got "x"	core/proc/parameters_spec.rb:65

   BEGIN { } / END { } (language/fixtures/begin_file.rb, shared/kernel/fixtures/END.rb) are also still refused.

4. Two real rbgo defects this PR makes visible for the first time, in core/file/printf_spec.rb, which contributed 0 before: sprintf does not preserve the encoding of the format string (UTF-8 where US-ASCII is expected), and no too many arguments for format string warning is emitted when $VERBOSE is true. Not touched — this PR is go.mod-only.

Four premises the measurement contradicted

• "MRI through our shim is a per-file ceiling." It is a valuable second measurement but it is not an upper bound. rbgo scores above MRI-through-the-shim on 31 files (+190 examples) and below on 531 (−1406); corpus total for MRI through the shim is 23927 against rbgo's 22711.

• On core/file/printf_spec.rb rbgo's 161 beats MRI's 160, and the two failure sets are disjoint. The cause is a real 3.x → 4.0 drift the pinned corpus cannot see: the spec asserts /no implicit conversion of nil into Integer/ for sprintf("%c", nil), while MRI 4.0.5 raises TypeError: no implicit conversion from nil to integer — so MRI fails its own spec on three examples there and rbgo passes them by emitting the 3.x wording. Nothing was weakened for this; it is a decision about the corpus pin (and about which wording rbgo should target), not about this PR.

• On other files the shim limits MRI, not the reverse. In core/dir/glob_spec.rb MRI reads 16 to rbgo's 92 because every Dir.glob assertion returns [] under the shim's fixture setup. So an above-ceiling reading on a file is a reason to look at the shim, never a reason to celebrate.

• "main reports 100.0% while ~17 functions sit below it." Locally the count is exactly 17 and the reported total is 99.9% — on origin/main as much as on the branch — so the CI gate expression (cov >= 100.0) would fail on unmodified main if run on this host. In CI's ubuntu lane on this branch, by contrast, the below-100 list is empty and the total is 100.0%. So those 17 belong to this machine's environment (the two at 0.0% are the bigdecimal and date Inspect bindings), not to main. The method still holds — compare the LIST, never the total — but the number does not describe CI, which is why the local comparison was made against a detached origin/main on the same host rather than against a remembered figure.

🤖 Generated with Claude Code