[tannevaled]

What this is

MRI-4.0.x conformance for the special globals, the String encoding paths, the
regexp-backed String methods and IO's paragraph reader. Every change is derived
from the MRI C, cited in the commit that makes it, and checked byte for byte
against ruby 4.0.5 before it was kept.

ruby/spec language + core, measured serially on the same corpus and the
same shim: 22227 → 22271 passing (2206 files). No file regressed. The floor is
22140.

The measurement this wave used

Running MRI itself under scripts/conformance/rubyspec/spec_helper.rb gives
a per-file ceiling, which separates a real defect from an artefact of the shim.
It changed what was worth doing:

• language/predefined_spec.rb reads 124 for rbgo and 153 for MRI, not 168 —
  the five $//$-0/$\/$,/$; "warns if assigned non-nil" examples and the
  two $= ones fail for MRI too (rb_warn_deprecated needs
  Warning[:deprecated], which only -w turns on), and so does the
  MSpecScript one. Making them pass would mean diverging from MRI, so they
  were left alone.
• core/io/write_spec.rb, listed as a target, has no headroom at all: MRI
  under this shim scores it at or below rbgo.
• The handed-over gsub slice-anchoring lead was already fixed —
  "Text\n".gsub(/^/, " ") is " Text\n" on this branch's parent. The ten
  gsub_spec failures were all encoding, which is what this PR fixes.

What changed

String#sub / #gsub negotiate the result encoding per append

str_gsub (string.c) seeds dest with the receiver's encoding and a 7BIT
coderange, appends the literal runs with rb_enc_str_buf_cat(…, str_enc) and
every replacement with rb_str_buf_append. Both land in
rb_enc_cr_str_buf_cat, the only place the result's encoding is decided. rbgo
used a plain strings.Builder and tagged the result UTF-8 unconditionally, so a
no-match gsub retagged a US-ASCII receiver, a block returning BINARY bytes
never promoted the buffer, and an irreconcilable pair never raised.
stringSub also now runs rb_reg_prepare_enc — but only for a Regexp pattern,
because rb_pat_search sends a String pattern to rb_str_byteindex instead.

IO paragraph mode swallows the trailing newline run

rb_io_getline_1 brackets a paragraph read with two swallow(fptr, '\n')
calls. rbgo had only the leading one, so an ordinary gets after a paragraph
read returned "\n" instead of the next line. The trailing one runs even when a
limit cut the read short, and stringio.c has no equivalent.

$/ is seeded to "\n"

Init_IO sets rb_default_rs = rb_fstring_lit("\n") before hooking $/ and
$-0 up, so an unassigned $/ reads a frozen US-ASCII "\n" and $-0 is the
same object. rbgo read nil through both.

#lines / #each_line resolve the separator the way MRI does

rb_str_enumerate_lines reads $/ when no argument is given — including a
$/ set to nil, which yields the whole receiver un-chomped — and re-encodes
the default separator into a non-ASCII-compatible receiver's encoding before the
byte search, which is why a dummy UTF-16 string is never split and a dummy UTF-7
one raises.

#chomp and #bytesplice read characters, not bytes

chompped_length's smart_chomp arm removes one newline character and then
a '\r' character, so "abc\r\n".encode("utf-32be").chomp drops eight bytes.
rb_str_bytesplice negotiates with rb_enc_check over the whole of both
strings before the update, so a 7-bit receiver adopts the source's encoding.

#=~, #match, #match?

rb_str_match sends #=~ to any operand that is neither a String nor a Regexp;
rb_str_match_m coerces with get_pat and then dispatches #match on the
pattern, yielding the MatchData to a block; rb_str_match_m_p honours the
position argument. rbgo raised TypeError for a non-Regexp, ignored the block and
dropped match?'s second argument. Regexp#match? now shares the same
regexpMatchP.

The BOM encodings UTF-16 / UTF-32 transcode

String#encode could not reach them at all, so every spec that builds a
dummy-encoded string died before it could measure anything.
transcode.c's converters write a big-endian BOM and read one back, with no
fallback endianness. With the strings constructible, rb_must_asciicompat
became visible: it runs at the head of rb_str_to_dbl, rb_str_to_inum and
rb_str_to_c, before a byte is read and regardless of exception: false.

Grapheme clusters and #chars are questions the encoding answers

rb_str_enumerate_grapheme_clusters falls back to rb_str_enumerate_chars for
a non-Unicode encoding and otherwise clusters in the receiver's own bytes;
WANTARRAY means the block form yields and returns the receiver. Underneath,
strCharPieces was rb_enc_mbclen for ASCII-8BIT and UTF-8 and a UTF-8 guess
for everything else — now it measures each family, including the Shift_JIS and
EUC-JP lead-byte tables.

Per file, before → after (ceiling = MRI under this shim)

file	before	after	ceiling
core/string/gsub_spec.rb	67	76	77
core/string/chomp_spec.rb	61	65	65
core/string/match_spec.rb	19	23	24
core/string/grapheme_clusters_spec.rb	9	13	13
core/string/lines_spec.rb	20	23	23
core/string/each_line_spec.rb	20	23	23
core/string/bytesplice_spec.rb	37	40	40
core/io/gets_spec.rb	39	42	44
core/kernel/Complex_spec.rb	60	62	62
core/string/each_grapheme_cluster_spec.rb	13	15	15
core/string/chars_spec.rb	10	11	11
core/string/each_char_spec.rb	10	11	11
core/symbol/match_spec.rb	7	9	9
core/string/sub_spec.rb	64	65	66
core/string/to_c_spec.rb	37	38	38
core/string/to_f_spec.rb	20	21	21
core/io/readlines_spec.rb	53	54	58

Tree totals: core/string 3566 → 3603, core/io 1354 → 1358, core/kernel 1995 →
1997, language unchanged (everything reachable there lives in files this wave
does not own).

core/random/bytes_spec.rb samples 8 or 9 on both this branch and main over
four runs each — a pre-existing flake, not a regression.

Gates

• CGO_ENABLED=0 go build ./..., CGO_ENABLED=0 GOWORK=off go test ./... — full
  suite green.
• Coverage: the below-100 list is identical to main's, function for function
  (99.9% both). Comparing the list rather than the rounded total found two real
  defects — see the commit "Whitebox cover this wave".
• gofmt -l internal/vm/ empty; GOOS=wasip1 GOARCH=wasm go build ./... green.

Found but not fixed — these live in files this wave does not own

• internal/vm/prelude.rb — Tempfile#initialize passes mode: straight to
  File.open; MRI's tempfile.rb defaults it to 0 and ORs
  File::RDWR|CREAT|EXCL into it, so Tempfile.new(name, mode: File::RDONLY)
  ENOENTs here. 8 examples in core/io/copy_stream_spec.rb. Note the fix
  needs the default to change from "w+" to 0, which existing String-mode
  callers would feel.
• Native methods cannot tell a keyword Hash from a positional one. IO.gets,
  #readlines and IO.foreach must raise TypeError for a positional options
  Hash (prepare_getline_args: StringValue(rs) / NUM2LONG(lim)); rbgo strips
  any trailing Hash as options. Needs a flag at the native-call boundary
  (vm.go/proc.go/compiler). 7 examples across readlines/foreach/gets.
• internal/vm/reflection.go — Method#source_location returns line 1 for
  every method. Kernel#warn(uplevel:) itself is byte-identical to MRI; the four
  core/kernel/warn_spec.rb failures are the fixture computing its expected line
  numbers from source_location. 4 examples, and probably more elsewhere.
• internal/vm/format_bind.go (and the go-ruby-format engine) — sprintf
  does no encoding negotiation, so String#% cannot preserve or promote the
  format string's encoding. 4–5 examples in core/string/modulo_spec.rb.
• internal/object/object.go — String#inspect escapes every byte of a
  non-UTF-8 string instead of decoding it, so a UTF-16BE string inspects as
  "\x00a\x00b" rather than "ab". 3 examples in
  core/string/inspect_spec.rb. Symbols also carry no encoding, which is the
  whole of core/string/to_sym_spec.rb's remaining 3.
• internal/vm/require.go — $LOAD_PATH is empty and has no
  #resolve_feature_path. 4 examples in language/predefined_spec.rb.
• internal/vm/vm.go / exception.go — $! is not cleared when an
  exception is rescued and not restored to the enclosing value afterwards, and
  $~ is global rather than method-scoped. 11 examples in
  language/predefined_spec.rb, 1 in core/string/gsub_spec.rb.
• internal/compiler — $& = "" should be a SyntaxError "Can't set variable
  $&" at parse time, and nil = 1 / self = 1 should carry MRI's messages.
  8 examples in language/predefined_spec.rb.
• internal/vm/object_model.go — alias $new $& copies the value instead of
  sharing the variable, so the alias is not read-only. Part of the 8 above.
• github.com/go-ruby-regexp/regexp (upstream module) — conditional
  backreferences (?(1)a|b), \k<-1>, \k<01>, forward references, (?~), and
  inline (?imx-imx) are unimplemented. 9 examples in
  language/regexp/back-references_spec.rb, 5 in modifiers_spec.rb.
• internal/vm/rational.go — String#to_r and Kernel#Rational still parse
  a non-ASCII-compatible String's bytes as digits; the same
  rb_must_asciicompat one-liner this PR adds elsewhere fixes them.
• internal/vm/io_descriptors.go (syswrite, ungetc) and
  io_class.go (copy_stream) hold the rest of core/io's headroom.

🤖 Generated with Claude Code