Repository navigation
Property tests and fuzz targets against the reference - #37
Merged
Merged
Conversation
added 7 commits
October 25, 2026 14:40
The mirrors move to tests/common/mod.rs, each with its Type, samples and hand-written inputs, so property tests and the fuzz targets can hold the seed to the same derives.
serde_json's default float parser is best-effort and can land one ulp off: 4.520618856987477e-279 written by json@1 read back as 4.5206188569874766e-279. Enable float_roundtrip, add a unit test, and record the input as format-json/found-f64-exact-digits, the first found-* vector. Found by the property tests.
proptest draws a Type (depth 4, up to 6 fields, 8 elements, fixed lengths up to 40) and a value of it, per format, and checks the round trip, that packed@1 is canonical, that whatever decodes passes check, and the seed against the derive mirrors on mutated and random bytes. 64 cases by default; PROPTEST_CASES raises it. packed_round_trips fails: packed@1 can't read back an array whose elements encode to no bytes (an empty struct, unit, T[0]), since a count may not exceed the bytes left.
Core refuses T[] when T carries no data (unit, T[0], a struct of only such fields), so the generator gives such an element a bool beside it. packed@1 stays strict: format-packed/reject-count-past-input pins that a count past the bytes left is refused even for elements that take none. The seed that found it is dropped; the generator can't reach it now.
runtime/fuzz, outside the workspace, nightly only: - decode_no_panic: a format, a type and bytes; decoding never panics, stays within -rss_limit_mb/-malloc_limit_mb, and what it accepts passes check. - packed_canonical: bytes that decode as a type in packed@1 re-encode to themselves. - framing_decode: bytes through every frame decoder; binary frames re-encode exactly, except the two that read a prefix (a fault envelope, the handshake), JSON-RPC responses re-encode and decode the same. - seed_vs_derive: the conformance mirrors, seed against derive. Types are read one byte per choice, so examples/seed_corpus.rs (and seed-corpus.sh) can write every vector as a seed input. Arrays of data-less elements are never generated, as in the property tests.
.github/workflows/fuzz.yml runs each target for 60s on a pull request, seeded from the vectors, and nightly for 10 minutes on a cached corpus plus proptest at 100000 cases; a nightly failure uploads the crash and opens an issue unless one with the same title is open. The fuzz README covers running, the targets, and how a finding becomes a found-* vector; the conformance README points to it. The data-less array exclusion now cites ComlineProject/core#141.
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 1 step R4 of runtime#29: property tests and fuzz targets against the reference. Formerly stacked on #32, which is now merged.
What's in it
Property tests (
runtime/conformance/tests/properties.rs, proptest, stable, part ofcargo test):Types and values of them;packed@1is canonical;check;The mirrors move to
tests/common/mod.rs. A run is 64 cases by default;PROPTEST_CASESraises it.cargo-fuzz targets in
runtime/fuzz, excluded from the workspace, nightly only:decode_no_panic,packed_canonical,framing_decodeandseed_vs_derive.seed-corpus.sh..github/workflows/fuzz.yml:Docs:
runtime/fuzz/README.mdcovers running, the targets, and how a finding becomes afound-*vector.runtime/conformance/README.mdlinks to it.Findings
f64precision (fixed). serde_json's default float parser read4.520618856987477e-279back one ulp off.float_roundtripfeature incomline-runtime.json@1gets slower (serde_json's docs say about 2x).format-json/found-f64-exact-digitscover it.packed@1couldn't read backT[]whenTtakes no bytes;[Empty {}]encodes as01.validation::array-of-dataless-type).format-packed/reject-count-past-inputpins that.Not canonical, as in comline-typescript:
Envelope::decodeignores bytes after aFaulttag, andHandshakeFrame::decodereads only the first 31 bytes.framing_decodechecks that the re-encoding of those two is a prefix of the input. Say if either should be strict; that would be a vector change for both runtimes.