Repository navigation
Instruction-count benchmark gates - #35
Merged
Merged
Conversation
added 4 commits
October 25, 2026 14:45
Five format shapes in the vectors' value encoding, the framing payload sizes, the handshake frame and the connection step, vendored by every language so each measures the same work. The Rust mirrors of the shapes, which the benches will use, are checked to encode exactly as their definitions do in every format.
Encode and decode per format per shape through serde-derive mirrors, framing encode and decode per framing per payload size, the handshake frame's encode, decode and check, and one connection step. Inputs are built by setup functions and handed back, so neither building nor dropping them is measured. iai-callgrind is pinned to 0.16.1; iai-callgrind-runner must match.
Runs the base's benches in a worktree, saves them as a baseline, then runs the head's against it with an ir limit of 1%. A base with no benches is skipped and the head only recorded. The comparison goes to the job summary and the head's counts to a rust.bench.json artifact. The bench profile builds one codegen unit: with the default partitioning, slowing the handshake encode also moved the JSON-RPC framing benches by up to 2.6%, with no change to their code.
added 2 commits
October 25, 2026 14:55
In one shared target directory Cargo names a workspace's path packages the same from either checkout, and the head's sources, checked out before the base was built, look older than the base's build, so the head reused it and was never measured. iai-callgrind's output moves to its own directory, which both runs share for the baseline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 1 of runtime#29: benchmark gates. Was to be stacked on #32, which has merged, so this targets
main.runtime/conformance/benches/: five format shapes (small scalar, string-heavy, nested, 1000-element array, enum-heavy) in the vectors' value and type encoding, framing payloads of 16 B to 64 KB, the handshake frame and one connection step, with a README of the format and the bench ids. A conformance test checks each shape round-trips throughWireanddecode_typedin all three formats.runtime/core/benches/runtime.rs(58 benchmarks,iai-callgrindpinned to 0.16.1): encode and decode through serde-derive mirrors of each shape, framing encode and decode fordatagram@1,datagram@2andjsonrpc@1, handshake encode, decode and check, andState::encode_requestthenreceive. Inputs come from setup functions. A test asserts each mirror encodes to exactly its definition's bytes in every format..github/workflows/bench.yml: benches the base in agit worktreewith--save-baseline=base, then the head with--baseline=base --callgrind-limits=ir=1%. A base with no benches (this PR) records only. The comparison goes to the job summary, andrust.bench.jsonis uploaded as an artifact (not committed).jsonrpc@1encode by up to +2.6%, with no change to its code.[profile.bench] codegen-units = 1removes that.ir=1%.HandshakeFrame::encodefailed the job withhandshake/encode336 → 357 (+6.25%); the other 57 benches were unchanged.--regression-fail-fast: it stops at the first regression and prints no summary. Without it, every bench is compared and the run still exits non-zero.