Skip to content

Add RSpec conformance stress test harness + gap map - #7

Merged
tannevaled merged 1 commit into
mainfrom
conformance-rspec
Jun 26, 2026
Merged

tannevaled merged 1 commit into
mainfrom
conformance-rspec

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Confronts rbgo with RSpec (rspec-core / rspec-expectations / rspec-mocks / rspec-support) as a metaprogramming and dynamic-dispatch conformance probe, with MRI 4.0.5 as the oracle. This is a measured gap map, not an attempt to run RSpec green.

What's here

  • scripts/conformance/rspec/parsesweep/ — Go helper that runs rbgo's real front-end (parser.Parse + compiler.Compile, no execution) over a tree and reports acceptance.
  • scripts/conformance/rspec/dsl/01..10.rb — hand-written snippets reproducing RSpec's actual DSL patterns (describe/it via define_method+instance_exec, let memoization, method_missing matchers, matcher-builder DSL, nested-context subclassing, shared-examples registry, expect().to() chain, stubbing via define_singleton_method).
  • scripts/conformance/rspec/run.sh — build + shallow-clone + parse-sweep + DSL-diff vs MRI. Re-runnable; skips gracefully offline or without MRI.
  • CONFORMANCE-RSPEC.md — the full report.

Headline findings (all from real runs)

  • Parse acceptance: 40.1% of RSpec lib/ (79/197), 22.7% including specs. MRI accepts 100% of lib/, so every reject is a genuine rbgo gap.
  • Load: no RSpec entrypoint loads — blocked first by $LOAD_PATH being nil (and require having no load-path search), then by the class << expr parse gap at rspec/support.rb:21.
  • DSL dynamism (the point): the metaprogramming model is sound — 5/10 snippets are byte-identical to MRI (define_method/instance_exec/let, anonymous-subclass nesting, Comparable, blocks/lambdas, expect().to() chain). The 5 failures reduce to a short, isolatable gap list, not a deep architectural problem.

Ranked gaps (minimal repros in the report; no internal/vm / parser-core edits here)

gap construct impact
G1 leading ::Const reference 64 lib files
G2 paren-less command call + do…end block spec-file hot path
G3 masgn to instance-var LHS (@a, @b = …) 12 files
G4 class << self 20 files
G5 class << expr RSpec bootstrap blocker
G6 bare anonymous splat (a, *) 3 files
G7 \ line-continuation + interpolation in continued string ~30 rejects
G8 super(*args) compile-stage error
G-RT1 respond_to? ignores respond_to_missing? runtime
G-RT2 __send__ alias missing (send/public_send work) runtime, near-free fix
G-RT3 &blk param unbound in define_singleton_method runtime
G-LOAD $LOAD_PATH is nil; require has no load-path search loading

Per coordination, this PR adds only the harness + report under distinct paths (scripts/conformance/rspec/, CONFORMANCE-RSPEC.md); the gaps are recorded for a later coordinated fix pass rather than fixed here.

🤖 Generated with Claude Code

Confronts rbgo with RSpec (rspec-core/expectations/mocks/support) as a
metaprogramming/dynamic-dispatch conformance probe against MRI 4.0.5.

Harness (scripts/conformance/rspec/):
- parsesweep: runs rbgo's front-end (parse + compile, no exec) over a tree
- dsl/01..10: hand-written snippets reproducing RSpec's DSL patterns
- run.sh: build + shallow-clone + parse-sweep + DSL-diff vs MRI; re-runnable,
  skips gracefully offline / without MRI

Findings (CONFORMANCE-RSPEC.md), all from real runs:
- Parse acceptance: 40.1% of RSpec lib/ (79/197), 22.7% incl specs. MRI
  accepts 100% of lib/, so every reject is a genuine rbgo gap.
- Load: no RSpec entrypoint loads -- blocked by $LOAD_PATH being nil + the
  `class << expr` parse gap at rspec/support.rb:21.
- DSL dynamism (headline): model is sound -- 5/10 snippets byte-identical to
  MRI (define_method/instance_exec/let, anon-subclass nesting, Comparable,
  blocks/lambdas, expect().to() chain). The 5 failures reduce to a short,
  isolatable gap list.

Ranked gap map with minimal repros (for a later coordinated fix pass; no
internal/vm or parser-core edits here):
  G1 leading :: const ref (64 files)         G5 class << expr (load blocker)
  G2 cmd-call + do-block (spec hot path)     G6 bare anon splat (a, *)
  G3 masgn to ivar LHS (12 files)            G7 line-cont + interpolation
  G4 class << self (20 files)                G8 super(*args) compile error
  G-RT1 respond_to? ignores respond_to_missing?
  G-RT2 __send__ alias missing (send/public_send work)
  G-RT3 &blk param unbound in define_singleton_method
  G-LOAD $LOAD_PATH nil; require has no load-path search

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 26386a3 into main Jun 26, 2026
9 checks passed
@tannevaled
tannevaled deleted the conformance-rspec branch June 26, 2026 05:26
tannevaled added a commit that referenced this pull request Aug 31, 2026
Engine PR #7 merged; repoint from the PR-branch pseudoversion to the
resulting main pseudoversion so the sprintf/format binding builds against
released engine code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tannevaled added a commit that referenced this pull request Aug 31, 2026
…l in sprintf (#522)

* Match MRI %c coercion, Hash#default named refs, and hex-float # / 0-fill in sprintf

Wire the format binding to the new go-ruby-format host hooks so
Kernel#sprintf / Kernel#format / String#% match MRI 4.0.6 for cases the
engine could not reach alone:

- formatValue.CoerceChar resolves a %c operand with MRI's to_str-then-to_int
  protocol via the method table (never #respond_to?, so a BasicObject that
  defines only #to_str is coerced): a String feeds first-character rendering,
  an Integer a code point, a non-String/#to_int result raises the matching
  "can't convert X into String/Integer" TypeError, and an object answering
  neither raises "no implicit conversion of X into Integer".
- formatNamedArgs registers a NamedArgs default resolver so a %<name>/%{name}
  reference to an absent key consults the hash's own #[] (Hash#default /
  default_proc): a non-nil default is used, a nil one raises MRI's KeyError. A
  present key (even a nil value) stays in the map and never hits the resolver.

The hex-float alternate-form (%#a) and zero-fill (%020a), the "%*1$.*2$3$d"
argument-numbered star width/precision, and the width/precision "too big"
ArgumentError are fixed in the engine (go-ruby-format/format#7) and land here
through the dependency bump.

core/string/modulo_spec: 252 -> 263 examples pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Probe %c/to_ary coercion methods without sending #respond_to? to a BasicObject

String#% unpacked its right-hand side by asking respondsToDynamic whether the
object answered #to_ary, whose fallback *sends* #respond_to? — which a
BasicObject operand does not have, so `"%c" % basic_object_with_to_str` raised
NoMethodError instead of coercing. Add formatRespondsTo, which probes the
method table first and only dispatches #respond_to? (and thus
#respond_to_missing?) to an object that actually provides it, matching MRI's
rb_check_funcall. Use it for the %c to_str/to_int coercion and the to_ary
unpacking. Covers CoerceChar and the named-hash default resolver with a new
format_c_named_test.go.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(deps): re-pin go-ruby-format/format to merged main

Engine PR #7 merged; repoint from the PR-branch pseudoversion to the
resulting main pseudoversion so the sprintf/format binding builds against
released engine code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant