Repository navigation
compiler: for-loop targets, block-local variables, and what defined? declares (+382 in language/) - #626
Merged
Conversation
The v0.2.0 parser makes 13 more language/*_spec.rb files of the pinned corpus parseable, and adds For.Target and Block.Locals to the AST. None of the compiler work in this branch is visible on v0.1.9. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`{ |a; b, c| … }` declares b and c as fresh, nil-valued locals of the
block that shadow any enclosing binding of the same name. The v0.2.0
parser carries them on Block.Locals; the compiler ignored the field, so
every mention of one resolved to the outer binding — or to nothing, and
language/block_spec.rb stopped at "undefined local variable" before a
single example ran.
MRI read: prism_compile.c v3_4_0:6325-6334, which walks the block
parameters node's `locals` list and calls pm_insert_local_index for each
right after the requireds/optionals/rest/posts and before the "fill in
any locals we missed" pass — so they are table entries, not parameters,
and start nil. parse.y v3_4_0:5128-5148 (opt_bv_decl/bv_decls/bvar) and
new_bv at 13621 show the parser already rejects a block-local that
shadows a parameter, so no dedup is needed at this end.
language/block_spec.rb: 0 -> 139 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four things, all in the lowering of `for`: 1. For.Target. The v0.2.0 parser puts every loop variable Vars cannot spell into For.Target and leaves Vars empty — a lone non-local (`for @v in …`, `for a[0] in …`), or any list with a `*rest`, a nested group, a non-local, or a trailing comma. The compiler read only Vars, so `for i, in [[1,2]]` compiled a loop over nothing and language/for_spec.rb stopped at "undefined local variable". 2. The each-block's parameter shape. MRI picks it by the kind of loop variable (prism_compile.c v3_4_0:6293-6301): a lone local target bumps lead_num, so the block takes one required argument; anything else sets has_rest, so the block collects every yielded value into an Array. rbgo always took one required argument, so a `for x, y in o` over an `each` that does `yield 1, 2` bound y to nil where MRI binds 2. The single-non-local arm is MRI's `expandarray 1, 0` on that rest Array (prism_compile.c v3_4_0:4862-4880), which is why the loop variable takes the FIRST yielded value and drops the others. For the multi-target arm MRI emits `(args.length == 1 && Array.try_convert(args[0])) || args` then expandarray (compile.c compile_for_masgn v3_4_0:8302-8329, prism_compile.c v3_4_0:4881-4919). The try_convert half is redundant in front of an expandarray that already converts: when args holds one element, expanding args[0] yields its Array conversion if it has one and the one-element list [args[0]] otherwise — which is what expanding args gives. So the same lowering is `args.length == 1 ? args[0] : args`, emitted with the same idLength and idAREF sends MRI uses. 3. `for o&.target in m`. storeMultiTarget ignored Call.Safe, so a nil receiver raised NoMethodError instead of skipping the write. A multiple assignment can never reach that branch (`a&.x, b = …` is a SyntaxError), but a for-var can: prism_compile.c v3_4_0:4862 routes PM_CALL_TARGET_NODE through pm_compile_target_node, which honours PM_CALL_NODE_FLAGS_SAFE_NAVIGATION. 4. `for` opens no scope. The each-block is a block ISeq but not a Ruby scope, so a variable FIRST ASSIGNED in the body belongs to the scope around the loop: after `for a in [6]; for b in [7]; c = a * b; end; end`, c is a local of the enclosing method. It hops out of enclosing `for` bodies but not out of an enclosing block, which is a scope — the for_spec "can be nested with blocks in between" example pins both halves. builder.declOwner names that scope; every site that declares a named local now goes through it (assignment, op-assignment, named regexp captures, `rescue => e`), which also removes four copies of the resolve-or-allocate dance. Also fixed on the way, because for_spec is the file that exposes it: mustResolve dropped the block depth, so `g(...)` argument forwarding inside ANY block — `def f(...); [1].each { g(...) }; end` as much as a `for` body — indexed the block's own frame and crashed with a Go index-out-of-range surfaced as ArgumentError. language/for_spec.rb: 0 -> 29 passing, 0 failing. The 5 remaining errors are Kernel#local_variables and #redo, neither implemented, neither in this branch's files. language/rescue_spec.rb: 49 -> 50. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… desugared compound assignment `defined?` inspects its operand's syntactic kind instead of compiling it, so the assignments inside it emit nothing — but they still DECLARE. MRI builds its local table in the PARSER, which walks the operand like any other code, so `defined?(a += 1)` leaves `a` a nil local of the enclosing scope and a later bare `a` is a local-variable read. rbgo declares as it compiles, so `a` got no slot and the next mention of it — in language/defined_spec.rb, `defined?(a.b += 1)` six lines further on, whose receiver the compiler DOES evaluate — failed with "undefined local variable". Verified against MRI: `defined?(a += 1); p a` prints nil, and `p defined?(a = 1), a.inspect` prints "assignment" then "nil". Second, `a.b += 1` and `a[:b] ||= 1` reach the compiler as the parser's textual desugaring into a setter Call, and were tagged "method". MRI tags them "assignment": PM_CALL_OPERATOR_WRITE_NODE, PM_CALL_OR_WRITE_NODE, PM_CALL_AND_WRITE_NODE and the PM_INDEX_* trio are all in the DEFINED_ASGN group (prism_compile.c v3_4_0:4052-4093; compile.c's NODE_OP_ASGN1/NODE_OP_ASGN2 at v3_4_0:6151-6164 say the same). A plain `a[0] = 1` is NOT in that group — it is an ordinary attrasgn call and stays "method", which ruby/spec pins. opassign.go's shared-node recognisers tell the two apart. language/defined_spec.rb: 0 -> 211 passing. Of the 47 that remain, two groups are worth naming: `defined?(super)` answers "expression" instead of "super"/nil (18 examples), and the tag Strings are not frozen (about 12). Half of the latter is a one-line change in internal/vm/defined.go, outside this branch's files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nches Covers every new branch in the package profile: the For.Target shapes and where each declares its locals, the each-block parameter shape and the two target lowerings, the safe-navigation guard, block-local slots, the defined? declaration walk, and the depth mustResolve had been dropping. Also drops an unreachable guard: compileFor's declare() skipped an empty name, but every For.Target shape that carries one (`for i, * in …`) also carries a parallel Targets list with a typed nil in that position, so the all-locals arm — the only caller that passes Names straight through — never sees it. An unreachable branch cannot be covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`A ||= v` and `A::B ||= v` are not assignment nodes: the parser desugars them to `(defined?(A) && A) || A = v`, so they reached compileDefined as a `||` and were tagged "expression". MRI tags them "assignment" — PM_CONSTANT_OR_WRITE_NODE and PM_CONSTANT_PATH_OR_WRITE_NODE are in the DEFINED_ASGN group (prism_compile.c v3_4_0:4059 and 4063; compile.c's NODE_OP_CDECL at v3_4_0:6162 agrees) — and ruby/spec pins both. The `&&=` and `+=` forms already arrived as ConstAssign/ScopedConstAssign and were right. Recognised by AST pointer identity between the `defined?` probe's argument, the `&&`'s right operand and the scoped assignment's target, the same test opassign.go uses: a HAND-WRITTEN `(defined?(A) && A) || A = 1` has three distinct reads and stays "expression", which MRI confirms. language/defined_spec.rb: 211 -> 213 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The note said the parser rejects a block-local that repeats a parameter name. MRI's does (parse.y v3_4_0:13621, new_bv -> shadowing_lvar_0); go-ruby-parser does not yet, which is the one ruby/spec example in "Block-local variables" still failing. Nothing is forced here on its behalf: localLookup answers with the first slot of a name, so the repeated declaration is dead and the name keeps meaning the parameter. Comment only; no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Sep 23, 2026
Seven units since the last bump: #617 core/process: spawn, exec, the rlimits, priorities, process groups, kill, the wait family (pass 20 -> 165) #618 core/kernel + core/string: require_relative with an absolute path, the require/load argument contract, LoadError#path, byteindex/ byterindex, split, chop/chars/each_char (kernel 233 -> 184, string 179 -> 153 failing) #620 language: compound assignment evaluation order, the flip-flop operator, magic-comment encodings, loop break values (pass 1308 -> 1443) #622 object: Float#to_s laid out by MRI's rule rather than Go's %g (+11) #624 the mspec shim: a nested describe can see the helpers of the one around it (+28) #625 deps: parser v0.2.0 (language +333) #626 compiler: for-loop targets, block-local variables, what defined? declares (language +382) Measured with scripts/conformance/rubyspec/run.sh against the pinned corpus at 87b1631, 16-way, TIMEOUT=25, three consecutive runs: 21749, 21750, 21750. The floor is set nine below the lowest, the same margin rule as the last bump: it absorbs the run-to-run jitter without being wide enough to hide the loss of a whole spec file, which is what the ratchet exists to catch. #615 (an intermittent frame-pop crash) is still open and can cost a file when it fires. TWO OF THESE ARE NOT VM CONFORMANCE, and the number should not be read as if they were. #624 fixed our own mspec shim: core/string/valid_encoding/utf_8_spec scored 0/28 under MRI 4.0.5 too, so those 28 were harness noise counted as VM failure. #625 is a dependency bump; the VM did not change, but thirteen language spec files became parseable for the first time and brought 333 already- passing examples into view. Both are real gains in what the measurement can see, not in what the VM can do. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four compiler-side gaps the v0.2.0 parser (#625) exposed, and a fifth found while
chasing them. Three of the four spec files this unblocks had been contributing
zero to both columns of the conformance measurement — not failing, invisible.
language/over the whole tree (find language -name '*_spec.rb', 80 files),main @ 097d86a vs this branch, same binary lineage and the same harness:
+382 passing, and no file loses an example. The only four files that move at
all are the three unblocked ones and
rescue_spec, which gains one:The rise in fail+error (436 → 512) is entirely the examples those three files can
now reach at all; every one of them was previously unmeasured.
rescue_spec's +1 is not incidental: its "using a safely navigated setter methodon a nil target" example (
rescue => o&.captured_error) reaches the same store armthe
forsafe-navigation fix below repairs, and stopped raising NoMethodError.1. Block-local variables —
{ |a; b| … }They are fresh, nil-valued locals of the block that shadow any enclosing binding.
The parser carries them on
Block.Locals; the compiler ignored the field, soblock_spec.rbstopped at "undefined local variable" before one example ran.MRI read:
prism_compile.cv3_4_0:6325-6334 walks the block-parameters node'slocalslist and callspm_insert_local_indexfor each, after therequireds/optionals/rest/posts and before the "fill in any locals we missed" pass
— table entries, not parameters, so they start nil.
parse.yv3_4_0:5128-5148gives the grammar (
opt_bv_decl/bv_decls/bvar).One example in that group still fails, and deliberately:
{ |foo; foo| }is aSyntaxError in MRI, raised by the PARSER (
parse.yv3_4_0:13621,new_bv→shadowing_lvar_0), and go-ruby-parser does not raise it yet. Nothing is forcedhere on its behalf —
localLookupanswers with the first slot of a name, so therepeated declaration is dead and the name keeps meaning the parameter. Raising
belongs where MRI raises it.
2.
for—For.Target, the block's parameter shape, and the scopeFour things, all in the lowering of
for.For.Target. v0.2.0 puts every loop variableVarscannot spell intoFor.Targetand leavesVarsempty — a lone non-local (for @v in …,for a[0] in …), or any list with a*rest, a nested group, a non-local, or atrailing comma. The compiler read only
Vars, sofor i, in [[1,2]]compiled aloop over nothing.
The each-block's parameter shape. MRI picks it by the kind of loop variable
(
prism_compile.cv3_4_0:6293-6301): a lone local target bumpslead_num, so theblock takes one required argument; anything else sets
has_rest, so the blockcollects every yielded value into an Array. rbgo always took one required
argument, so a
for x, y in oover aneachthat doesyield 1, 2boundytonil where MRI binds 2. The single-non-local arm is MRI's
expandarray 1, 0onthat rest Array (
prism_compile.cv3_4_0:4862-4880), which is why the loopvariable takes the FIRST yielded value and drops the others.
For the multi-target arm MRI emits
(args.length == 1 && Array.try_convert(args[0])) || argsthen
expandarray(compile.ccompile_for_masgnv3_4_0:8302-8329, and theidentical prism arm at
prism_compile.cv3_4_0:4881-4919). Thetry_converthalfis redundant in front of an
expandarraythat already converts: whenargsholdsone element, expanding
args[0]yields its Array conversion if it has one and theone-element list
[args[0]]otherwise — which is what expandingargsgives. Sothe same lowering is
args.length == 1 ? args[0] : args, emitted with the sameidLengthandidAREFsends MRI uses, and a redefinedArray#lengthorArray#[]is honoured here exactly as it is there.for o&.target in m.storeMultiTargetignoredCall.Safe, so a nil receiverraised NoMethodError instead of skipping the write. A multiple assignment can never
reach that branch (
a&.x, b = …is a SyntaxError), but a for-var can:prism_compile.cv3_4_0:4862 routesPM_CALL_TARGET_NODEthroughpm_compile_target_node, which honoursPM_CALL_NODE_FLAGS_SAFE_NAVIGATION.foropens no scope. Its each-block is a block ISeq but not a Ruby scope, so avariable FIRST ASSIGNED in the body belongs to the scope around the loop: after
for a in [6]; for b in [7]; c = a * b; end; end,cis a local of the enclosingmethod. It hops out of enclosing
forbodies but not out of an enclosingblock, which is a scope — the for_spec example "can be nested with blocks in
between" pins both halves.
builder.declOwnernames that scope, and every sitethat declares a named local now goes through it (assignment, op-assignment, named
regexp captures,
rescue => e), which also removes four copies of theresolve-or-allocate dance.
3.
defined?— the locals its operand declares, and two tagsdefined?inspects its operand's syntactic kind instead of compiling it, so theassignments inside emit nothing — but they still DECLARE. MRI builds its local
table in the PARSER, which walks the operand like any other code, so
defined?(a += 1)leavesaa nil local of the enclosing scope and a later bareais a local-variable read. rbgo declares as it compiles, soagot no slot andthe next mention of it —
defined?(a.b += 1)six lines further on, whose receiverthe compiler DOES evaluate — failed to compile. Verified against MRI:
defined?(a += 1); p aprintsnil.Two tags were also wrong:
a.b += 1anda[:b] ||= 1reach the compiler as the parser's textualdesugaring into a setter Call and were tagged "method". MRI tags them
"assignment":
PM_CALL_OPERATOR_WRITE_NODE,PM_CALL_OR_WRITE_NODE,PM_CALL_AND_WRITE_NODEand thePM_INDEX_*trio are all in theDEFINED_ASGNgroup (
prism_compile.cv3_4_0:4052-4093;compile.c'sNODE_OP_ASGN1/NODE_OP_ASGN2at v3_4_0:6151-6164 says the same). A plaina[0] = 1is NOT in that group — it is an ordinary attrasgn call and stays"method", which ruby/spec pins.
A ||= vandA::B ||= vdesugar to(defined?(A) && A) || A = vand soarrived as a
||, tagged "expression". MRI tags them "assignment"(
PM_CONSTANT_OR_WRITE_NODE/PM_CONSTANT_PATH_OR_WRITE_NODE,prism_compile.cv3_4_0:4059 and 4063;compile.c'sNODE_OP_CDECLatv3_4_0:6162).
Both desugarings are recognised by AST pointer identity between the shared
read and the write, the test
opassign.goalready uses — a hand-written(defined?(A) && A) || A = 1has distinct reads and stays "expression", which MRIconfirms.
4.
variables_spec— already fixed, and I can show itThe brief listed
cannot compile *ast.SplatArginlanguage/variables_spec.rbasthe largest of the four. It does not reproduce, on either parser version.
Compiling every
*_spec.rbunderlanguage/turns up nocannot compile *ast.SplatArganywhere. The wave-25 work had already closed it; the 113 exampleswere the precedent the brief cites, not a pending item. Nothing to do, so nothing
was done — an honest three.
5. Found on the way:
...forwarding lost its block depthmustResolvereturned a slot and discarded the depth, sog(...)written insideANY block —
def f(...); [1].each { g(...) }; endas much as aforbody —indexed the block's own frame and died with a Go index-out-of-range surfaced as
ArgumentError. Pre-existing and independent of
for; for_spec is simply the filethat exposes it.
Gates
CGO_ENABLED=0 go build ./...,CGO_ENABLED=0 GOOS=wasip1 GOARCH=wasm go build ./...,gofmt -l internal/empty.CGO_ENABLED=0 GOWORK=off go test ./...— full suite green,TestEmbeddedPreludeMatchesSourceincluded (nothing prelude.rb uses changed shape, soprelude_frozen_gen.gois untouched and still matches).function against main, nothing that was at 100% dropped and no function
decreased at all. Total 98.4% → 98.7%.
forshapes(including the hash, multi-yield, attribute, safe-navigation, index and
leak-past-the-loop cases), block-locals, and the
defined?tags anddeclarations —
diffof raw output, identical.Known-remaining, deliberately not in this PR
In
defined_spec's 45 remaining failures, two groups dominate and neither is aregression:
defined?(super)answers "expression" instead of "super"/nil (18examples — it needs a new opcode and VM support), and the tag Strings are not
frozen (about 12; half of that is one line in
internal/vm/defined.go, outsidethis branch's file set).
for_spec's 5 errors areKernel#local_variablesand#redo, neither implemented.assignments_specwants a rescuable SyntaxErrorwhere the compiler currently reports
cannot compile *ast.BlockPass.🤖 Generated with Claude Code