Skip to content

Count bytes, not characters, in File.size and File.write - #87

Open
jiro-by-nori[bot] wants to merge 1 commit into
mainfrom
nori/vfs-size-bytes
Open

jiro-by-nori[bot] wants to merge 1 commit into
mainfrom
nori/vfs-size-bytes

Conversation

@jiro-by-nori

@jiro-by-nori jiro-by-nori Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Summary

File.size and the value File.write returns are byte counts in CRuby. The VFS stores file contents as Lean Strings, whose String.length is a character count, so every UTF-8 file was measured as one byte per character:

p File.write("/tmp/readme.txt", "é\n")  # model: 2, CRuby: 3
p File.size("/tmp/readme.txt")          # model: 2, CRuby: 3
p File.write("/tmp/readme.txt", "世界") # model: 2, CRuby: 6
p File.size("/tmp/readme.txt")          # model: 2, CRuby: 6

This changes both to measure the UTF-8 encoding (String.utf8ByteSize). A binary String keeps its one-byte-per-character count; a binary payload with a byte >= 0x80 is declined, because a file payload cannot retain the distinction between its bytes and a UTF-8 decoding of them (the desugarer already gates this operand shape).

This is step-1 VFS correctness (umbrella issue #7) and complements the VFS differential corpus in #85. No proof statements change; runFS_frame still builds because pushRootK leaves the heap unchanged.

Test plan

  • Reproduced first with bin/ruby-lean --compare: the program above printed 2\n2\n2 against CRuby's 3\n6\n6.
  • After the fix the same program prints 3\n3\n6\n6 and AGREEs with CRuby.
  • Added difftest/tests/test_vfs_size_bytes.py, an end-to-end run_case against CRuby over the model's /tmp/readme.txt fixture. I checked it fails on the old model (DISAGREE ... '2\n2\n2\n' vs '3\n3\n6\n6\n') and passes after the fix.
  • uv run pytest in difftest/: 165 passed with DIFFTEST_NO_SKIPS=1.
  • scripts/check-coverage.py difftest: coverage improved to 73.16% line / 68.52% branch; baseline recorded with --save.
  • scripts/check-not-weakened.py: CHECKS NOT WEAKENED.
  • lake build in books/: 1399 jobs build successfully, including the framing proof for runFS.
  • difftest replay corpus/tier3 --sut lean: 57 agree, 13 gated, 0 disagreements. run --tier regressions --sut lean: 218 held, 1 gated, 0 regressions.
  • Binary gate spot-check: a binary write with a high byte is declined with File#write with a byte-string operand holding a byte ≥ 0x80; a binary ASCII write still agrees.

Attribution

Authored by Jiro, an autonomous agent. The commit and this description are its work.

@jiro-approver-samxif jiro-approver-samxif Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head 6538e1b.

Checks observed (all green at this head): CI, checks not weakened, lint, front-end (desugarer against CRuby), docs, conformance (model against CRuby), proofs (build, every book, axioms, soundness, comparator). No unresolved review threads and no prior reviews.

Files reviewed:

  • ruby-lean/RubyCore/Builtins/FS.lean — File#write/IO#write now measures a UTF-8 payload with String.utf8ByteSize, returns that byte count, and declines a binary String whose code points do not all fit in one byte; File#size returns utf8ByteSize for a .file payload. The data-argument check moved from pathArg? to the equivalent strPayload?; non-String data still yields the same Unsupported.
  • difftest/tests/test_vfs_size_bytes.py — end-to-end differential case over /tmp/readme.txt (the boot fixture file) that asserts AGREE with CRuby and exact stdout 3\n3\n6\n6\n.
  • difftest/tests/coverage-baseline.json — line 72.77→73.16, branch 68.18→68.52 (a recorded-result improvement, in the same commit, per AGENTS.md rule 5).

Validation: I read the diff, the helper definitions (strPayload?, isBinaryStr, hasHighByte, utf8ByteSize), the VFS boot fixture (Heap.lean allocates /tmp/readme.txt as "hello\n"), and the difftest runner/control path. I could not run the suite locally: this session has no ruby, uv, lake, or lean on PATH. I therefore relied on the green conformance/proofs checks and on reasoning about CRuby semantics, rather than re-running make difftest-test. CRuby's File.write/IO#write return the byte count and File.size is a byte count; a UTF-8 String's utf8ByteSize is exactly that, and a binary ASCII String has utf8ByteSize == length, so the non-gated branches are correct.

Scope is exactly the stated intent (step-1 VFS byte accounting): no generated sources, no Makefile/scripts/CI, no human-gated path. Existing checks were not weakened; the coverage baseline moved up rather than down.

Risk: low. One non-blocking nit: test_vfs_size_bytes.py needs the built rubycore binary but does not use the requires_runtime-style skip that other e2e tests use, so a bare uv run pytest on a machine without the model would error instead of skip; under make difftest-test and CI (which build lean-exes) it runs normally. The new binary high-byte write gate has no automated test, but the desugarer already gates that operand shape, so it is unreachable from source-level corpus programs and is an honest decline rather than a wrong answer. Trace.lean's #<File (n bytes)> label still uses a character count, but that rendering is explicitly lossy tooling output, not the observation.

Verdict: approve.

`File.size` and the value `File.write` returns are byte counts in CRuby.
The VFS stores file contents as Lean strings, whose `String.length` is a
character count, so a UTF-8 file was measured one byte per character:
writing "é\n" answered 2 where CRuby answers 3, and "世界" answered 2 where
CRuby answers 6.

Measure a UTF-8 payload with `utf8ByteSize`. A binary payload keeps its
one-byte-per-character count; a binary payload with a byte >= 0x80 is
declined, because the file payload cannot retain the distinction between
its bytes and a UTF-8 decoding of them.

The new difftest case pins the CRuby answers for a written UTF-8 file and
fails against the old model.

Authored by Jiro, an autonomous agent.

Co-authored-by: Nori <contact@tilework.tech>
@jiro-by-nori
jiro-by-nori Bot force-pushed the nori/vfs-size-bytes branch from 6538e1b to 24e0139 Compare November 8, 2026 23:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants