Repository navigation
Conversation
The search benchmark dataset fixtures pass `use_legacy_format=False` and `index_cache_size=...` to `write_dataset` / `lance.dataset`, both of which are deprecated. Because the test config sets `filterwarnings = error::DeprecationWarning`, fixture setup raised `DeprecationWarning` and every benchmark in `test_search.py` errored out before running. Switch to the current parameters (`data_storage_version="stable"` and `index_cache_size_bytes`), restoring the benchmark suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Xuanwo
added a commit
that referenced
this pull request
Oct 28, 2026
…rmat#8878) ## What this changes A `linalg-asan` job in `nightly_run.yml` runs `cargo test -p lance-linalg --tests` under `-Zsanitizer=address`. The public entry points already validate the lengths a caller passes: `sum_4bit_dist_table` asserts `dists.len() >= n`, and `hamming_batch_u64` asserts the output length with a comment explaining why it is not a `debug_assert`. What nothing checks is the pointer arithmetic the kernels then do inside those bounds. `_mm256_storeu_si256(dists.as_mut_ptr().add(16), ...)` in the AVX2 dist-table kernels and `*results_ptr.add(1)` through `.add(3)` in `hamming_batch_avx2` are raw stores at offsets the kernel computes itself, so an off-by-one in a chunk count writes past the buffer while every entry assert still passes. It lands in `nightly_run.yml` rather than `rust.yml` because `-Zsanitizer` needs a nightly toolchain, and the only nightly this repo uses is in `daily-code-coverage.yml`, which is `schedule` plus `workflow_dispatch`. The pin is the same one that file uses, so there is one nightly date in the repo rather than two. Three details in the job are load-bearing and each is there for a reason a reader would otherwise have to rediscover. `RUSTUP_TOOLCHAIN` rather than `rustup default`. A rustup override file outranks the default toolchain, and `rust-toolchain.toml` pins a stable release, so `rustup default nightly-...` followed by `cargo test` runs the sanitizer flag into a stable compiler and stops at `error: the option 'Z' is only accepted on the nightly compiler`. `daily-code-coverage.yml` solves the same problem the other way, with `cargo +nightly-...`. `RUSTFLAGS` repeats the baseline from `.cargo/config.toml`. Setting the variable replaces that file's per-target `rustflags` wholesale rather than appending, so leaving it at just the sanitizer flag would instrument a build no release ships, and would compile out the arms gated on `target_feature = "avx2"`. One of those is where the batch call that writes into a heap buffer lives (`cosine_batch_avx512` into `vec![0.0; batch.len() / dimension]`), which is the case a sanitizer detects best. `--target x86_64-unknown-linux-gnu` keeps the sanitizer off build scripts and proc macros, which run on the host. There is no protoc step, unlike every other Rust job here. `lance-linalg`'s dependency closure contains no `prost-build` or `tonic-build`, dev-dependencies included, and the job runs green without it. ## Test plan On aarch64-apple-darwin, with one deliberate 2-byte store planted past the 32-slot output of `sum_dist_table_32bytes_batch_neon`, ASan reports `heap-buffer-overflow, WRITE of size 2` at `dist_table.rs` with the chain up through `sum_4bit_dist_table_uninit` to the test. The same binary without the sanitizer dies with a bare `SIGTRAP` and no location, so what this job adds on that mutation is the location rather than the detection. It has run for real on Linux, four times, on a fork branch with the `github.repository` guard dropped since that guard is what stops it running anywhere but here. Two clean runs: [33305364459](https://github.com/LuciferYang/lance/actions/runs/33305364459) and, after dropping the protoc step, [33312991796](https://github.com/LuciferYang/lance/actions/runs/33312991796). Both green, about a minute and a half to build and three to four minutes to run. Then two runs with a deliberate out-of-bounds store planted in a kernel, to check the claim that this job names a file and a line. That is where the `ASAN_SYMBOLIZER_PATH` step comes from. Without it the report was detected but useless: ``` WRITE of size 4 at 0x7cb5cf675100 thread T428 #0 0x5620f81ba20f (/home/runner/work/lance/lance/target/.../lance_linalg-...+0x...) #1 0x5620f81b1619 (/home/runner/work/lance/lance/target/.../lance_linalg-...+0x...) ``` ubuntu-24.04 ships `llvm-symbolizer` only under a versioned name, and ASan looks for the unversioned one on `PATH`. With the step, the same overflow reports: ``` WRITE of size 4 at 0x7ce193dcf280 thread T428 #0 ... sum_hacc_dist_table_32bytes_batch_avx2 rust/lance-linalg/src/simd/dist_table.rs:479:5 #1 ... sum_4bit_hacc_dist_table_avx2 rust/lance-linalg/src/simd/dist_table.rs:361:21 #2 ... sum_4bit_hacc_dist_table_uninit rust/lance-linalg/src/simd/dist_table.rs:244:13 #3 ... sum_4bit_hacc_dist_table rust/lance-linalg/src/simd/dist_table.rs:214:14 #4 ... tests::test_sum_4bit_hacc_dist_table_matches_u16_reference_multi_batch ``` So the failing test names itself in the stack, which is why the run does not need `--test-threads=1` to attribute a report. That exercise also turned up a coverage limit worth stating. I first planted the store in `sum_dist_table_32bytes_batch_avx2` and the job stayed green: on a runner with AVX-512BW, `sum_4bit_dist_table_uninit` selects the AVX-512 C kernel, so the Rust AVX2 one never executes. The hacc entry point has no AVX-512 kernel, which is why the planted store there did fire. So on an AVX-512 runner this job covers the hacc AVX2 kernel, the scalar paths and everything else in the crate, but not the AVX2 dist_table kernel that a smaller host would take. One gap remains: the C is not instrumented. `dist_table.c` compiles without `-fsanitize=address`, so the AVX-512 dist_table kernel is not covered. I tried `CFLAGS=-fsanitize=address` and it broke a host link in `lance-arrow`'s build script, so it is left out rather than half-applied. ## Scope note `--tests` selects the lib's unit tests and every integration target, so `tests/panic_location.rs` runs under the sanitizer too, and a target added later is included without editing this job. --------- Co-authored-by: Xuanwo <github@xuanwo.io>
Owner
Author
|
Merged upstream in lance-format#7285. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The dataset fixtures in
python/benchmarks/test_search.pypass deprecated parameters to the dataset APIs:lance.write_dataset(..., use_legacy_format=False)lance.dataset(..., index_cache_size=64 * 1024)The test config sets
filterwarnings = ['error::DeprecationWarning', ...], so these emitDeprecationWarningas errors during fixture setup. As a result every benchmark intest_search.pyerrors out before running:Fix
Switch to the current parameters:
use_legacy_format=False→data_storage_version="stable"(the exact mapping the deprecation shim applies).index_cache_size=64 * 1024→index_cache_size_bytes=512 * 1024 * 1024(512 MiB comfortably caches these 100k-row IVF_PQ indices).Verification
ruff check/ruff format --checkclean.