[tannevaled]

Wave 32, core/io / core/file / core/dir encodings and the descriptor write path.

Every item was measured per file against the MRI 4.0.5 ceiling — the same corpus run through the same spec_helper.rb shim — before being planned, and the ceiling overturned two of the briefing's targets (see Premises that did not survive measuring). The C source read for each item is named in its commit message.

Result

Whole language/ + core/ suite, 2-way, same corpus and shim: 22222 → 22278 (+56).

file	before	after	MRI ceiling
core/io/internal_encoding_spec.rb	67	79	79	at ceiling
core/io/external_encoding_spec.rb	48	59	59	at ceiling
core/io/syswrite_spec.rb	12	18	18	at ceiling
core/io/write_nonblock_spec.rb	14	18	18	at ceiling
core/dir/children_spec.rb	9	13	13	at ceiling
core/dir/entries_spec.rb	5	7	7	at ceiling
core/dir/foreach_spec.rb	7	8	8	at ceiling
core/dir/to_path_spec.rb	2	3	3	at ceiling
core/io/buffer/empty_spec.rb	10	13	13	at ceiling
core/io/buffer/string_spec.rb	4	7	7	at ceiling
core/io/buffer/free_spec.rb	8	10	10	at ceiling
core/io/buffer/resize_spec.rb	13	15	15	at ceiling
core/io/buffer/null_spec.rb	12	13	13	at ceiling
core/file/expand_path_spec.rb	25	27	29
core/io/puts_spec.rb	14	16	17
core/io/copy_stream_spec.rb	53	54	62	rest is Tempfile, see below

Thirteen of the sixteen files are now at the ceiling the harness can express.

The five items

1. IO encodings resolve ONCE, at open. IO#internal_encoding returned nil where MRI returns the encoding, and IO#external_encoding the wrong object, both from one missing step: rbgo resolved a stream's encodings lazily, at every read, where MRI resolves them once and freezes the answer. rb_io_ext_int_to_encs is the single place the C turns a (named external, named internal) pair into what it stores on a stream, and four call sites reach it — rb_io_extract_modeenc at open with both arguments NULL, which is what snapshots the defaults; parse_mode_enc for a ":ext:int" suffix; rb_io_extract_encoding_option; and io_encoding_set. It is now reproduced once, with the tri-state the C needs (NULL = not named, Qnil = explicitly refused, or an encoding) that "" alone cannot express. It also fixes set_encoding(ext) losing default_internal, the internal_encoding: nil / external_encoding: nil asymmetry (the C tests one against Qundef and the other against Qnil, and it is load-bearing), and IO.readlines / IO.foreach, which reach rb_io_open through open_key_args and take the same snapshot.

2. #syswrite and #write_nonblock write to the DESCRIPTOR. Both went through the buffered write path, which is the one thing their names say they do not do. Four consequences: a non-String argument was Go-ToS'd rather than rb_obj_as_string'd; io_write's zero-length shortcut was applied, so a read-only stream accepted syswrite(""); the external encoding transcoded the bytes, where rb_io_write_memory writes the string's own; and nothing reached disk. Also IO.copy_stream's src_offset now tells apart the two refusals io.c makes — ArgumentError only when the source has no fptr (a StringIO, a duck-typed object), Errno::ESPIPE for a pipe, which is a real IO and gets as far as pread.

3. Dir names carry an encoding. Every name dir.c hands back goes through rb_external_str_new_with_enc; rbgo built a plain UTF-8 String, which was right by accident on a UTF-8 filesystem. The encoding: keyword is now recorded as dp->enc and applied through the block form and the Enumerator form, Encoding.default_internal is honoured, and Dir#to_path keeps the encoding of the argument (dp->path is taken before rb_str_encode_ospath). The conversion is rb_str_conv_enc, not String#encode, and that distinction is what makes it work: the C ends with /* some error, return original */. Writing it with String#encode turned four passing examples into Encoding::UndefinedConversionError — one unrepresentable name made the whole listing raise.

4. IO::Buffer: base and size are two questions. #null? is base == NULL, #empty? is size == 0; rbgo had each exclude the other's case, so a buffer that had given up its memory answered true to one and false to the other. A slice takes parent->base + offset once, which settles two cases in opposite directions: a slice of a 0-sized buffer is null, and a slice whose parent is freed afterwards is not null — it dangles, which is what #valid? reports. rb_io_buffer_resize tests base == NULL before the external check and re-initializes with io_flags_for_size, so a resized 0-sized buffer comes back mapped and a shrunk mapped one internal. rb_io_buffer_type_string builds the String first and the buffer only aliases it, which fixes both size errors and makes #free inside the block stop losing what was written.

5. File.expand_path keeps its argument's encoding; :newline reaches the write path. rb_enc_check(fname, dirname) in append_fspath both refuses an incompatible pair and names the result's encoding. And the :newline option was parsed, validated against binmode, and then dropped; it is an ECONV_*_NEWLINE_DECORATOR the write converter applies.

Premises that did not survive measuring

• core/io/write_spec.rb was listed as a 9-point target. The real gap is 1. MRI scores 40 through this shim, rbgo 39; 8 of the 9 failures are assertions the harness cannot express (MRI records 3 fails and 5 errors on the same file). Nothing to chase.
• core/io/copy_stream_spec.rb's 9 are 8 + 1. Filed as Tempfile#initialize does not OR mode with RDWR|CREAT|EXCL, so mode: File::RDONLY raises ENOENT (8 ruby/spec examples) #670. Eight are one cause and it is not in copy_stream: Tempfile.new(..., mode: File::RDONLY) in the spec's setup. MRI's Tempfile#initialize ORs the mode with File::RDWR|CREAT|EXCL; the prelude passes it straight to File.open, which opens read-only on a path that does not exist yet → ENOENT in before :each, and eight examples never run. The ninth (ESPIPE) is fixed here.
• The largest single item in core/file is not an IO defect at all. core/file/printf_spec.rb scores 0 against a ceiling of 160 — rbgo cannot parse the file. Minimal case: a do…end block inside an arrow-lambda body that is an argument of a parenthesis-less command call. foo :a, lambda { [1].each do |x| p x end } and foo(:a, -> { … }) both parse; only foo :a, -> { [1].each do |x| p x end } does not. That is 160 examples for one parser fix, in internal/compiler. Filed as Parser: a do…end block inside an arrow-lambda body fails after a paren-less command call (160 ruby/spec examples) #669.

Gates

go build ./..., go vet ./..., the full go test ./..., gofmt -l internal/vm/ empty, and the wasip1/wasm build. Coverage: the below-100 list is identical to main's — sixteen functions, the same sixteen, with the same percentages. Two new below-100 entries appeared during the work and were closed rather than rounded away: ioReadEnc's default_internal fallback turned out to be both dead and wrong once the open-time snapshot existed (io_read_encoding has no such fallback), and bufReinit's zero-size branch got the two cases MRI was witnessed on.

One of the new tests went green here and red on both Linux lanes on the first push, and the cause was not in its subject. wave28_file_open_mode_test.go runs Dir.chdir(scratch) with no block, and that scratch directory is removed when the test ends, so the test binary spends the rest of the package in a directory that no longer exists. Any later test that reads the working directory inherits it — and os.Getwd() after the directory is deleted still answers on darwin (stale path, nil error) and fails on Linux. With no working directory, File.expand_path has no directory to prepend, so the rb_enc_check(fname, dirname) the CompatibilityError case depends on is handed an empty string, which is compatible with everything. The two tests now chdir into a directory they created; the un-restored chdir is left alone (another wave's file, breaks nothing on main) but is named where the next test to trip over it will look.

One further platform trap came out of the same test: File.absolute_path? went through path.IsAbs, which only knows the forward-slash root, so a Windows drive-letter root — "C:/x", exactly what File.realpath hands back there — read as relative. isAbsPath exists in file.go for that reason and File.expand_path already uses it; the two have to agree or File.absolute_path?(File.expand_path(p)) is false on Windows. Fixed, with no POSIX change and no new branch (filepath.IsAbs is where the platform difference lives).

One unrelated flake shows up in the table above: core/random/bytes_spec.rb is 50/50 on main (6/6 over twelve runs of the unmodified binary). Random#bytes returns a UTF-8-tagged String, so #size counts characters and a random run sometimes forms fewer than 15 — random.go, not this branch.

Left alone on purpose

• File.mkfifo stays unregistered — it would gain 7 and cost ~55. The NOTE in file.go is corrected: it blamed a blocking File.open, and that is no longer where the deadlock is. File.open on a FIFO returns at once and rbgo's threads really are concurrent; the hang is at CLOSE, in ioFlush's os.WriteFile, because opening a FIFO to write blocks until a reader opens it and the read side never opens the path either. Each end avoids the handshake, so neither can complete it.
• instance_variable_set is a silent no-op on natively-backed objects. Filed as instance_variable_set is a silent no-op on File/StringIO/Dir/IO::Buffer/Array/Hash/String/Proc (ivarTable returns nil) #672. ivarTable returns nil for File, StringIO, Dir, IO::Buffer, Array, Hash, String and Proc, so the write is discarded and the read answers nil — no error either way. That is what costs core/file/open_spec.rb its two "propagates … produced by close" examples: the fixture stores the exception in @close_exception through a singleton attr_accessor, and it never arrives. File.open's own ensure machinery is correct and is witnessed re-raising a #close exception when the override raises unconditionally. object_model.go and internal/object, outside this file set.
• IO::Buffer.for(str) { } does not lock str. rbgo has no rb_str_locktmp, which needs a flag on object.String and a check in every mutator — internal/object and string_*.go, outside this file set. Filed as No rb_str_locktmp: IO::Buffer.for(str) { } does not lock str against modification #671; worth 1 example here and more in pack/unpack.
• A full non-blocking pipe. ioShortWrite models write(2)'s short write but leaves an already-full pipe alone: MRI's answer could not be witnessed, because a second syswrite to a filled non-blocking pipe never returned under ruby 4.0.5 here. Guessing Errno::EAGAIN would be inventing a semantics.

🤖 Generated with Claude Code