Skip to content

Warnings reach stderr, not stdout (#667); Tempfile ORs its mode: with RDWR|CREAT|EXCL (#670) - #675

Merged
tannevaled merged 5 commits into
mainfrom
wave33/warn
Sep 26, 2026
Merged

tannevaled merged 5 commits into
mainfrom
wave33/warn

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Two measured defects, plus the third the measurement turned up while verifying
the second. Closes #667 and #670.

Every expectation below was taken from MRI 4.0.5 running the same program, and
each stream was read separately (2>/dev/null and 2>&1 >/dev/null) rather
than merged.

#667 — warnings reached stdout, the program's data stream

vm.New set errOut = out and cmd/rbgo/main.go passed os.Stdout for both,
so every diagnostic landed on descriptor 1:

$ rbgo warntest.rb 2>/dev/null
warntest.rb:1: warning: given block not used     # should be silent

MRI separates the two at the descriptor. io.c v3_4_0 Init_IO builds
rb_stdout from rb_io_prep_stdout() (the C stdout FILE*) and rb_stderr
from rb_io_prep_stderr() (the C stderr FILE*, FMODE_WRITABLE|FMODE_SYNC),
and every warning reaches the second one: error.c rb_warn →
rb_write_warning_str → Warning.warn → rb_write_error_str, which io.c
writes to rb_ractor_stderr(), falling back to fwrite(..., stderr) while
$stderr is still the original stream.

NewWithStderr(out, errOut) splits them; New(out) keeps merging them, which is
what an embedder capturing a program's whole output in one buffer wants, and
leaves every existing caller (run.go, cmd/wasm, the tests) unchanged. The CLI
passes os.Stdout and os.Stderr.

errOut is only the fallback sink: every diagnostic already resolves through
vm.curStderr(), which prefers the Ruby-level $stderr global exactly as
rb_write_error_str prefers rb_ractor_stderr(). Reassigning $stderr
therefore still captures warnings, so mspec's complain matcher keeps working —
which is also why the conformance suite never observed this defect.

Where each kind of diagnostic now goes (verified byte for byte against MRI on
each stream in isolation):

diagnostic stream honours a reassigned $stderr
rb_warn (given block not used, …) stderr yes
rb_warning (-w only) stderr yes
Kernel#warn (via Warning.warn) stderr yes
$stderr.puts / STDERR.puts stderr yes / n/a (STDERR is the original, as in MRI)
deprecated constant / deprecated gvar stderr yes
circular-require and autoload warnings stderr yes
Kernel#abort's message stderr no — writes the raw sink, see "Left"
uncaught-exception report stderr n/a (front-end, was already correct)
puts / print / p stdout —

Only the stream is asserted there. The circular-require warning's text still
diverges from MRI (MRI prefixes it with the requiring frame and, under -w, adds
a backtrace); that is a pre-existing text defect, not a stream one, and is left
alone here.

#670 — Tempfile ignored its mode:

Tempfile#initialize passed mode: straight to File.open, so

Tempfile.new("x", encoding: Encoding::BINARY, mode: File::RDONLY)

opened flag 0 — read-only — on a path that does not exist yet and raised
Errno::ENOENT. That call is the before :each of the "to a Tempfile" block of
core/io/copy_stream_spec.rb, so eight examples never ran.

mode: is an integer flag set, not an fopen-style String. lib/tempfile.rb
v3_4_0 Tempfile#initialize defaults it to 0 and does
@mode = mode|File::RDWR|File::CREAT|File::EXCL before the open, so RDONLY
(which is 0) still creates a readable-and-writable file;
create_with_filename does the same. Following the default from "w+" to 0
also makes a String mode raise NoMethodError on #|, which is what
library/tempfile/create_spec.rb asserts.

The other keywords now reach File.open instead of being swallowed, as in MRI:
Tempfile hands them to Dir::Tmpname.create, which consumes max_try: and
yields the rest back for the open with perm: forced to 0600. That is how the
spec's encoding: Encoding::BINARY reaches the file. anonymous: is
Tempfile.create's own keyword and is consumed there.

prelude_frozen_gen.go regenerated with go run ./cmd/freeze-prelude from the
repo root (91 insertions / 49 deletions — not byte-identical).

IO.copy_stream honours #to_io before #to_path

With the mode fix the "to a Tempfile" block ran, and 3 of its 8 examples
still failed: #pos stayed 0 after a copy, a preceding #write was lost, and
the handle's own buffer overwrote the copied bytes on the next flush.
copyStreamRead/copyStreamWrite took the String/#to_path branch for a
Tempfile and wrote the file behind the handle's back.

io.c v3_4_0 copy_stream_body checks #to_io first, on both arguments:

VALUE tmp_io = rb_io_check_io(dst_io);
if (!NIL_P(tmp_io)) dst_io = GetWriteIO(tmp_io);
else if (!RB_TYPE_P(dst_io, T_FILE)) { ... rb_class_new_instance(..., rb_cFile) }

An object that is not an IO but answers #to_io is that IO, and the copy
goes through the handle — at its current position, leaving the position at the
last write. A Tempfile answers both #to_io and #to_path (both delegate to the
File it wraps), so the order decides. The new check stays inside the
String/#to_path branch because that is where MRI puts it: an object answering
#to_io but not #to_path gets dst_fptr == NULL and takes
copy_stream_fallback's duck-typed #read/#write instead.

Measured

Through scripts/conformance/rubyspec/spec_helper.rb, serially (2-way), against
MRI 4.0.5 run through the same shim for the per-file ceiling. 422 files of
core/io core/array core/hash core/kernel plus the 10 of library/tempfile,
before and after.

file before after ceiling
core/io/copy_stream_spec.rb 54 62 62
library/tempfile/create_spec.rb 9 13 18
library/tempfile/initialize_spec.rb 0 2 2
library/tempfile/open_spec.rb 5 6 9
every other file of the 432 — unchanged —

Totals: core/{io,array,hash,kernel} 6537 → 6545 (ceiling 6725);
library/tempfile 17 → 24 (ceiling 36).

A stream change is global, so four directories are not a sufficient
regression check. Widened to the whole of core + language — 2206 files,
22485 → 22493 passing — and exactly one file moved: copy_stream_spec.rb
+8. Nothing regressed. (library/, optional/ and security/ were left out of
that widening: they are dominated by network specs that hit the per-file timeout
in both runs, so their FILEFAIL set is not reproducible — and a sibling session
was running its own test suite on this machine at the time.)

The briefed payoff for #667 did not exist, and measuring is what showed it.
The stream fix moves zero conformance examples: the given block not used
family across core/array and core/hash was said to be blocked by it, but
those examples already passed, because rbWarn1 has gone through
vm.curStderr() — and therefore through the reassigned $stderr that
complain installs — since #666. #667 is a real defect about piping and about
rbgo gen.rb > out.json, and it is invisible to this corpus by construction.

The 8 examples #670 promised did arrive, but only 5 of them came from the mode
fix; the other 3 needed the #to_io ordering above.

Left, with the reason

  • Kernel#abort writes vm.errOut directly (internal/vm/kernel_introspect.go),
    not vm.curStderr(). It now reaches the right descriptor, but a program that
    reassigns $stderr will not capture it, where MRI's rb_f_abort calls
    rb_io_puts(1, args, rb_ractor_stderr()). One line, in a file outside this
    lane's set.
  • closed_main.go / closed_main_wasm.go still call vm.New(os.Stdout), so
    a rbgo build --closed binary keeps the merged streams. Same one-line change
    as main.go; both files are outside this lane's set.
  • SystemExit's status never reaches the process. rbgo -e 'exit 3' exits 0
    where MRI exits 3, and abort exits 0 where MRI exits 1. Found while checking
    abort's stream; it is a separate defect from Warnings go to stdout: rbgo prog.rb 2>/dev/null still prints them #667 and needs its own issue.
  • Tempfile.create returns a Tempfile, not a File. MRI's
    create_with_filename hands back the real File (library/tempfile/create_spec.rb
    says so explicitly), which is most of that file's remaining 5-example gap.
    Tempfile.new with a block also yields, where MRI warns
    "Tempfile.new doesn't call the given block." and does not. Both are
    Tempfile surface rather than the mode: defect Tempfile#initialize does not OR mode with RDWR|CREAT|EXCL, so mode: File::RDONLY raises ENOENT (8 ruby/spec examples) #670 names.
  • File::CREAT is 64 and File::EXCL is 128 in rbgo (the Linux values) where
    macOS has 512 and 2048. Self-consistent, so nothing here depends on it, but a
    program that hard-codes a platform's O_* numbers would disagree. Out of scope
    and a constants question, not a Tempfile one.
  • internal/vm/io_class.go was outside this lane's granted file set, which
    named internal/vm/io.go — copy_stream lives next door. Flagging it rather
    than hiding it; no sibling branch or open PR touches that file.

Gates

  • CGO_ENABLED=0 go build ./... — clean.
  • CGO_ENABLED=0 GOWORK=off go test ./... — full suite green, no test needed
    updating: the existing tests all construct the VM with vm.New, whose merged
    behaviour is unchanged by design.
  • Coverage: the below-100% list and its percentages are identical to
    origin/main's — 17 functions, measured by running the same
    -coverpkg=./internal/... profile on a detached worktree at origin/main
    rather than trusting the rounded total (both report 99.9% by this measurement).
    The gate caught a real hole on the way: copyStreamRead sat at 96.9% because
    every copy_stream test exercised the destination side only, so the
    rb_io_check_io step on the SOURCE was never reached. Covering it is the last
    commit, and it pins observable behaviour, not just a line.
  • gofmt -l internal/ cmd/ — empty.
  • CGO_ENABLED=0 GOOS=wasip1 GOARCH=wasm go build ./... — clean.
  • runtime.GOOS guard on the only OS-specific assertion (the 0600 permission
    bits).

Each new test was checked by A/B against the old code, not assumed to be a
witness: forcing errOut back to out fails all 6 diagnostic cases and the
CLI descriptor test; removing the #to_io check fails 3 of the 5 copy_stream
cases; reverting the prelude fails 3 Tempfile cases (2 with Errno::ENOENT) and
the permission test. TestReassignedStderrCapturesWarnings passes both ways —
it is a guard on the $stderr routing, not a witness of #667, and its comment
says so.

🤖 Generated with Claude Code

tannevaled and others added 5 commits September 26, 2026 13:12
vm.New set errOut = out, and cmd/rbgo passed os.Stdout for both, so every
rb_warn, Kernel#warn, $stderr.puts and STDERR.puts landed on stdout:

    $ rbgo warntest.rb 2>/dev/null
    warntest.rb:1: warning: given block not used     # should be silent

MRI separates the two at the descriptor. io.c v3_4_0 Init_IO builds rb_stdout
from rb_io_prep_stdout() (the C stdout FILE*) and rb_stderr from
rb_io_prep_stderr() (the C stderr FILE*, FMODE_WRITABLE|FMODE_SYNC), and every
warning reaches the second one: error.c rb_warn -> rb_write_warning_str ->
Warning.warn -> rb_write_error_str, which io.c writes to rb_ractor_stderr(),
falling back to fwrite(..., stderr) while $stderr is still the original.

NewWithStderr(out, errOut) splits them; New(out) keeps merging them, which is
what an embedder capturing a program's whole output in one buffer wants, and
leaves every existing caller (run.go, cmd/wasm, the tests) unchanged. The CLI
passes os.Stdout and os.Stderr.

errOut is only the fallback sink: every diagnostic already resolves through
vm.curStderr(), which prefers the Ruby-level $stderr global exactly as
rb_write_error_str prefers rb_ractor_stderr(). Reassigning $stderr therefore
still captures warnings, so mspec's `complain` matcher keeps working -- which is
also why the conformance suite never observed this defect.

Verified against MRI 4.0.5 on both streams separately (2>/dev/null and
2>&1 >/dev/null), byte for byte, for rb_warn, Kernel#warn, $stderr.puts,
STDERR.puts, Kernel#abort's message, the deprecated-constant path, and the
uncaught-exception report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tempfile#initialize passed mode: straight to File.open, so

    Tempfile.new("x", encoding: Encoding::BINARY, mode: File::RDONLY)

opened flag 0 -- read-only -- on a path that does not exist yet and raised
Errno::ENOENT. That call is the `before :each` of the "to a Tempfile" block of
core/io/copy_stream_spec.rb, so eight examples never ran.

mode: is an INTEGER flag set, not an fopen-style String. lib/tempfile.rb v3_4_0
Tempfile#initialize defaults it to 0 and does

    @mode = mode|File::RDWR|File::CREAT|File::EXCL

before the open, so RDONLY (which is 0) still creates a readable-and-writable
file. Tempfile.create does the same in create_with_filename. Following the
default from "w+" to 0 also makes a String mode raise NoMethodError on #|, which
is what library/tempfile/create_spec.rb asserts:

    -> { Tempfile.create(mode: "wb") }.should.raise(NoMethodError, /undefined method ['`]|' for .+String/)

The other keywords now reach File.open instead of being swallowed, as in MRI:
Tempfile hands them to Dir::Tmpname.create, which consumes max_try: and yields
the rest back for the open with perm: forced to 0600. That is how the spec's
encoding: Encoding::BINARY reaches the file. anonymous: is Tempfile.create's own
keyword and is consumed there rather than forwarded.

Measured through scripts/conformance/rubyspec/spec_helper.rb against MRI 4.0.5:

    core/io/copy_stream_spec.rb    54 -> 59   (ceiling 62)
    library/tempfile/create_spec.rb      9 -> 13   (ceiling 18)
    library/tempfile/initialize_spec.rb  0 ->  2   (ceiling 2)
    library/tempfile/open_spec.rb        5 ->  6   (ceiling 9)

prelude_frozen_gen.go regenerated with `go run ./cmd/freeze-prelude`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With #670's mode fix the "to a Tempfile" block of core/io/copy_stream_spec.rb
ran, and 3 of its 8 examples still failed: #pos stayed 0 after a copy, a
preceding #write was ignored, and the handle's own buffer overwrote the copied
bytes on the next flush.

copyStreamRead/copyStreamWrite took the String/#to_path branch for a Tempfile,
writing the file behind the handle's back. io.c v3_4_0 copy_stream_body checks
#to_io FIRST, on both arguments:

    VALUE tmp_io = rb_io_check_io(dst_io);
    if (!NIL_P(tmp_io)) dst_io = GetWriteIO(tmp_io);
    else if (!RB_TYPE_P(dst_io, T_FILE)) { ... rb_class_new_instance(..., rb_cFile) }

An object that is not an IO but answers #to_io IS that IO, and copy_stream works
through the handle -- at its current position, leaving the position at the last
write. A Tempfile answers both #to_io and #to_path (both delegate to the File it
wraps), so the order decides which one wins.

The new check stays inside the String/#to_path branch because that is where MRI
puts it: an object answering #to_io but NOT #to_path gets dst_fptr == NULL and
goes to copy_stream_fallback's duck-typed #read/#write instead.

    core/io/copy_stream_spec.rb    59 -> 62   (ceiling 62, MRI 4.0.5)

Nothing else in core/io, core/array, core/hash or core/kernel moved (422 files
measured before and after).

Note for the reviewer: io_class.go was outside this lane's granted file set,
which named internal/vm/io.go -- copy_stream lives next door. No sibling branch
or open PR touches it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every expectation was taken from MRI 4.0.5 running the same program, and each
stream was read separately (`2>/dev/null` and `2>&1 >/dev/null`) rather than
merged -- under the old wiring the warning text was present either way, only on
descriptor 1, which is exactly how this defect stayed invisible.

internal/vm/diagnostic_stream_test.go
  Six diagnostic kinds on a split pair of writers: rb_warn (given block not
  used), Kernel#warn, $stderr/STDERR writes, rb_warning under $VERBOSE=true, the
  deprecated-constant path, and Kernel#abort's message. Plus: New still merges
  them (the embedder contract), $stderr reassignment still captures everything
  (mspec's `complain`), and the two writers share nothing.

cmd/rbgo/stream_test.go
  The descriptor-level half: real os.Pipe pairs swapped in for os.Stdout and
  os.Stderr around run(), so the assertion is about actual file descriptors and
  not about a pair of buffers being distinct. Ungated, so CI runs it.

internal/vm/tempfile_mode_test.go
  Tempfile mode: as an integer flag set (RDONLY still creates a writable file,
  APPEND survives the OR, a String raises NoMethodError on #|), the forwarded
  options (encoding, perm 0600 under a runtime.GOOS guard, max_try consumed,
  anonymous consumed by .create), and IO.copy_stream through #to_io -- including
  the two negative cases that must NOT change: an object with #to_io but no
  #to_path still takes the duck-typed #write fallback, and a String destination
  is still opened and truncated.

Each witness was checked by A/B against the old code, not assumed:
  - errOut forced back to out: all 6 diagnostic cases + the share-nothing test
    fail; the CLI descriptor test fails on both streams.
  - to_io check removed: 3 of the 5 copy_stream cases fail (pos 0, empty file,
    lost prelude); the 2 negative cases pass both ways, as intended.
  - prelude reverted: 3 of the Tempfile cases fail (2 with Errno::ENOENT) and
    the permission test fails.
  TestReassignedStderrCapturesWarnings passes both ways -- it is a guard on the
  $stderr routing, not a witness of #667, and its comment now says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The coverage gate found it: copyStreamRead sat at 96.9% because every test
exercised the DESTINATION side only. The one uncovered block was the
rb_io_check_io step on the source.

It is not just a coverage hole, it is observable behaviour. Read through the
handle, the copy starts at the source's current position and leaves it advanced,
where the old path branch re-read from byte 0:

    src.write("HEADbody"); src.flush; src.rewind; src.read(4)
    IO.copy_stream(src, dst)   # => 4, "body", src.pos == 8

Byte-identical to MRI 4.0.5.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit e3478ec into main Sep 26, 2026
7 checks passed
@tannevaled
tannevaled deleted the wave33/warn branch September 26, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warnings go to stdout: rbgo prog.rb 2>/dev/null still prints them

1 participant