Skip to content

Advance Marshal conformance: Data dump/load and Regexp encoding - #524

Merged
tannevaled merged 2 commits into
mainfrom
feat/agent-marshal2
Aug 31, 2026
Merged

tannevaled merged 2 commits into
mainfrom
feat/agent-marshal2

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Marshal MRI-4.0.6 conformance: Data + Regexp encoding

Advances core/marshal/dump_spec.rb and core/marshal/load_spec.rb against MRI 4.0.6. Two self-contained clusters, entirely within internal/vm/marshal_codec.go plus a new test file.

Data (Data.define, Ruby 3.2+ immutable value objects)

  • Dump via the S struct container (member name/value pairs), the same encoding MRI uses, using the real class name (never a #name override).
  • Load via memberDefNames, reconstructing a frozen instance (Data is immutable regardless of the freeze: kwarg).
  • Fixes all Data examples: dump (a Data, extended, overridden name, object links) and load (a Data, extended, frozen).

Regexp encoding + option byte

  • Record the compiled Regexp's encoding the way MRI does: bare (no encoding ivar) for a binary source, :E => true/false for UTF-8/US-ASCII, a named :encoding ivar otherwise.
  • Emit the option byte's FIXEDENCODING (0x10) and NOENCODING (0x20) bits; treat an ASCII-incompatible source (UTF-16/UTF-32) as fixed even when its bytes are ASCII-only.
  • Restore both on load, so a Regexp round-trips byte-for-byte, while keeping a loaded /café/ equal to the literal.
  • Fixes the four encoding examples in dump_spec (binary, ascii-compatible, UTF-8, another-encoding).

Conformance deltas

  • dump_spec: 162 → 170 pass (+8)
  • load_spec: 210 → 213 pass (+3)
  • Total: +11 examples, no regressions.

Out of scope (object-model / lexer / non-marshal, per task boundaries)

  • String/Array/Hash/Struct instance variables and extend/subclass identity — core value types don't carry ivars or a class override.
  • Regexp subclass identity and ivars — UserRegexp.new yields a plain Regexp.
  • Symbol per-symbol encoding (Symbol#encoding).
  • Time zone/offset fidelity (in time.go), Exception message re-encoding (in the constructor).
  • Array#== uses identity/eql? for custom-== elements (core comparison, not marshal) — this alone accounts for the three "…and with proc" / "loads a user-marshaled object" load failures, whose loaded objects are structurally correct.
  • Wrapped-C-pointer (_dump_data/_load_data) and Random — need builtins.

Verification

  • go test ./internal/vm/ and go test ./... green.
  • wasm build (GOOS=js GOARCH=wasm) OK.
  • gofmt -l empty on changed files.
  • MERGED coverage 100.0% (-coverpkg over ./internal/...).

🤖 Generated with Claude Code

tannevaled and others added 2 commits August 31, 2026 13:28
Data.define instances (Ruby 3.2+ immutable value objects) store their
members in RObject.structVals the same way a Struct does, and MRI marshals
them with the same 'S' container. Emit that on dump and accept it on load
via memberDefNames, reconstructing a frozen Data instance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MRI records a Regexp's encoding the same way it records a String's: bare
(no encoding ivar) for a binary source, an :E => true/false marker for
UTF-8/US-ASCII, and a named :encoding ivar otherwise. The option byte carries
the FIXEDENCODING (0x10) and NOENCODING (0x20) bits, and an ASCII-incompatible
source (UTF-16/UTF-32) is fixed even when its bytes are ASCII-only.

Compute the encoding from the compiled Regexp (via encodingName plus the fixed
source encoding), pick the option byte to match, and restore both on load so a
Regexp round-trips byte-for-byte. A non-ASCII source stays content-fixed (its
fixedEnc field left false) so a loaded /café/ still equals the literal.

Fixes the four encoding examples in core/marshal/dump_spec (binary,
ascii-compatible, UTF-8, another-encoding); the remaining Regexp residuals
need the object model to carry a Regexp subclass identity and ivars.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 8a510ae into main Aug 31, 2026
7 checks passed
@tannevaled
tannevaled deleted the feat/agent-marshal2 branch August 31, 2026 12:33
tannevaled added a commit that referenced this pull request Aug 31, 2026
* Raise ruby/spec floor to 17470 after wave-3 conformance work

Locks in the gains from #521 (regexp \xHH), #523 (String#encode
fallback/xml/IBM437/default_internal), #524 (Marshal Data + Regexp
encoding), #525 (Module#define_method arity/visibility/hooks) and #522
(sprintf %c/named-default/hex-float). Measured 17470 passing on the
pinned corpus, up from 17310.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: re-trigger after transient GOPROXY stream errors

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant