[test](lance) Add query coverage for the remaining ANN index types and vector element types by FANNG1 · Pull Request #66512 · apache/doris

FANNG1 · 2026-08-06T00:55:52Z

What problem does this PR solve?

Issue Number: Part of #66495

Problem Summary:

vector_search() delegates index selection and search parameters to lance-c, but Doris
previously had end-to-end coverage only for IVF_PQ. The remaining Lance ANN algorithms
were supported by the same code path but had not been verified with physical indexes.

This PR adds Doris query coverage for all six ANN algorithms using Float32 + L2. It
also tests the query-vector encoding of supported element types with flat search. There
is no FE or BE production-code change.

What is changed?

ANN index algorithms

The committed fixture now contains one indexed table for each algorithm:

IVF_FLAT
IVF_PQ
IVF_SQ
IVF_HNSW_FLAT
IVF_HNSW_SQ
IVF_HNSW_PQ

Each table contains the same deterministic 1024-row, two-fragment, 16-dimensional
Float32 dataset with an L2 index. The fixture self-check verifies:

the expected index type and name;
index coverage of every fragment;
indexed and flat physical-plan shapes;
a low-nprobes boundary query that detects silent flat fallback.

IVF_FLAT additionally asserts that probing every partition matches flat search. The
lossy and graph indexes use refine_factor for their normal queries. IVF_HNSW_SQ also
has an ef=5 versus ef=50 discriminator to prove that ef reaches graph search.

Vector element encoding

test_lance_vector_search_element_types uses flat search to verify Doris query-vector
encoding for:

Float16, Float32 and Float64 with L2;
UInt8 with Hamming.

It also verifies null-vector handling and an unsupported metric error. Int8 is excluded
because searching the existing nullable Int8 fixture aborts the BE with the Lance version
used by lance-c v0.1.6; the suite records this limitation.

Scope

This PR does not cover the complete algorithm x element-type x metric matrix:

indexed fixtures are Float32 + L2 only;
Cosine and Dot indexes are not included;
Float16, Float64 and UInt8 index coverage remains follow-up work;
Int8 is not covered for the reason above.

The fixture writer remains pinned to pylance 4.0.1 for compatibility with the other
fixture readers. The committed indexes pass the same self-check under pylance 4.0.1 and
7.0.0, the Lance generation used by lance-c v0.1.6.

Test

Fixture self-check passed with pylance 4.0.1 and 7.0.0.
external_table_p0/lance: 8 suites passed, 0 failed, including the Spark and REST
catalogs that share the same Directory Namespace manifest.

Release note

None

hello-stephen · 2026-08-06T00:55:58Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

yiguolei · 2026-08-11T01:48:23Z

@FANNG1 Have you finished this PR?

FANNG1 · 2026-08-11T03:03:20Z

@FANNG1 Have you finished this PR?

Not yet, I'm verifying the PR in my local mac machine.

zhangstar333 · 2026-08-14T07:23:30Z

@FANNG1 now the teamCity thirdparty lib use lance_c.a from v0.1.6 .
u could continue the pr

FANNG1 · 2026-08-14T12:06:43Z

@FANNG1 now the teamCity thirdparty lib use lance_c.a from v0.1.6 . u could continue the pr

Thanks for notice, is there any plan to upgrade lance c in branch-4.1?

zhangstar333 · 2026-08-14T14:16:22Z

@FANNG1 now the teamCity thirdparty lib use lance_c.a from v0.1.6 . u could continue the pr

Thanks for notice, is there any plan to upgrade lance c in branch-4.1?

#66786

FANNG1 · 2026-08-17T09:07:05Z

Exploratory Doris + Lance compatibility notes

While preparing this PR, I also probed Doris + Lance combinations beyond the committed Float32 + L2 fixtures. This is an integration capability matrix rather than a standalone Lance test: it covers fixture/index creation and Doris queries through FE → BE → lance-c → Lance. The failures found during these probes were traced to Lance-side limitations or bugs; none currently points to a problem in the Doris vector_search() implementation. This matrix is retained for follow-up work on #66495 and is not regression coverage added by this PR.

Environment: indexes generated with pylance 7.0.0 using 1024 × 16 vectors, 4 IVF partitions, and V3 index files, then queried through Doris with lance-c v0.1.6. yes means the physical index was created and verified, Doris could read it through lance-c, and the Doris query returned rows through the indexed path.

Algorithm	Float16/L2	Float16/Cosine	Float16/Dot	Float32/L2	Float32/Cosine	Float32/Dot	Float64/L2	Float64/Cosine	Float64/Dot	Int8/L2	UInt8/Hamming
IVF_FLAT	hang	yes	yes	yes	yes	yes	yes	yes	yes	no (writer)	yes
IVF_PQ	hang	yes	hang	yes	yes	yes	yes	yes	yes	no (writer)	no
IVF_SQ	hang	yes	yes	yes	yes	yes	yes	yes	yes	no (writer)	no
IVF_HNSW_FLAT	hang	yes	yes	yes	yes	yes	yes	yes	yes	no (writer)	yes
IVF_HNSW_SQ	hang	yes	yes	yes	yes	yes	yes	yes	yes	no (writer)	no
IVF_HNSW_PQ	hang	yes	yes	yes	yes	yes	yes	yes	yes	no (writer)	no

Issues found in the Doris + Lance probes

Float16 index-build hangs: Float16 + L2 can overflow during Lance KMeans centroid accumulation for large, unnormalized values and a low partition count. IVF_PQ + Float16 + Dot can hit a similar overflow in its sub-quantizer training. These are data-dependent Lance index-builder issues, not Doris query-path issues. I am working on a fix and plan to submit it to the Lance community.
Nullable Int8 abort: Searching the existing nullable Int8 fixture aborts the BE because the Lance version used by lance-c v0.1.6 mishandles nulls while converting Int8 vectors. The fix exists in a newer Lance version (fix(arrow): preserve inner nulls in convert_to_floating_point lance-format/lance#7498), so Doris should add this coverage after upgrading to a lance-c release that includes the fix.
Int8 indexing: Lance currently has no complete Int8 index-build path. This is a Lance writer/core capability limitation, although flat Int8 distance kernels exist.
UInt8 indexing: UInt8 is Hamming-only; the probed Lance version accepts it for IVF_FLAT and IVF_HNSW_FLAT but rejects it for PQ/SQ variants.
Invalid Hamming builds: Building Float32/Float64 indexes with Hamming can reach a Lance KMeans panic instead of returning a validation error. Doris cannot reach this build-time path because Doris only reads and queries Lance indexes.
Metric mismatch: Lance can silently fall back to flat search when the query metric does not match the physical index metric. Therefore compatibility evidence for Cosine or Dot requires a matching physical index and an ANN-execution discriminator; changing only the query metric is not sufficient.

Question for reviewers

This PR currently gives Doris end-to-end coverage for all six ANN algorithms with Float32 + L2. It separately covers Doris query-vector encoding for Float16, Float32, Float64, and UInt8 through flat search.

Do you think this PR should also add indexed fixtures for non-Float32 element types and Cosine/Dot metrics? These would be Doris + Lance integration tests, but they exercise the same Doris implementation path already covered by Float32 + L2; the additional matrix cells mainly expand dependency compatibility coverage while multiplying the binary fixtures and regression cases significantly. My preference is to keep this PR focused on one representative indexed combination per algorithm and not add the full algorithm × element type × metric matrix, but I would like reviewers' opinions.

zhangstar333 · 2026-08-18T06:39:28Z

Exploratory Doris + Lance compatibility notes

While preparing this PR, I also probed Doris + Lance combinations beyond the committed Float32 + L2 fixtures. This is an integration capability matrix rather than a standalone Lance test: it covers fixture/index creation and Doris queries through FE → BE → lance-c → Lance. The failures found during these probes were traced to Lance-side limitations or bugs; none currently points to a problem in the Doris vector_search() implementation. This matrix is retained for follow-up work on #66495 and is not regression coverage added by this PR.

Environment: indexes generated with pylance 7.0.0 using 1024 × 16 vectors, 4 IVF partitions, and V3 index files, then queried through Doris with lance-c v0.1.6. yes means the physical index was created and verified, Doris could read it through lance-c, and the Doris query returned rows through the indexed path.

Algorithm Float16/L2 Float16/Cosine Float16/Dot Float32/L2 Float32/Cosine Float32/Dot Float64/L2 Float64/Cosine Float64/Dot Int8/L2 UInt8/Hamming
IVF_FLAT hang yes yes yes yes yes yes yes yes no (writer) yes
IVF_PQ hang yes hang yes yes yes yes yes yes no (writer) no
IVF_SQ hang yes yes yes yes yes yes yes yes no (writer) no
IVF_HNSW_FLAT hang yes yes yes yes yes yes yes yes no (writer) yes
IVF_HNSW_SQ hang yes yes yes yes yes yes yes yes no (writer) no
IVF_HNSW_PQ hang yes yes yes yes yes yes yes yes no (writer) no

Issues found in the Doris + Lance probes

Float16 index-build hangs: Float16 + L2 can overflow during Lance KMeans centroid accumulation for large, unnormalized values and a low partition count. IVF_PQ + Float16 + Dot can hit a similar overflow in its sub-quantizer training. These are data-dependent Lance index-builder issues, not Doris query-path issues. I am working on a fix and plan to submit it to the Lance community.

Nullable Int8 abort: Searching the existing nullable Int8 fixture aborts the BE because the Lance version used by lance-c v0.1.6 mishandles nulls while converting Int8 vectors. The fix exists in a newer Lance version (fix(arrow): preserve inner nulls in convert_to_floating_point lance-format/lance#7498), so Doris should add this coverage after upgrading to a lance-c release that includes the fix.

Int8 indexing: Lance currently has no complete Int8 index-build path. This is a Lance writer/core capability limitation, although flat Int8 distance kernels exist.

UInt8 indexing: UInt8 is Hamming-only; the probed Lance version accepts it for IVF_FLAT and IVF_HNSW_FLAT but rejects it for PQ/SQ variants.

Invalid Hamming builds: Building Float32/Float64 indexes with Hamming can reach a Lance KMeans panic instead of returning a validation error. Doris cannot reach this build-time path because Doris only reads and queries Lance indexes.

Metric mismatch: Lance can silently fall back to flat search when the query metric does not match the physical index metric. Therefore compatibility evidence for Cosine or Dot requires a matching physical index and an ANN-execution discriminator; changing only the query metric is not sufficient.

Question for reviewers

This PR currently gives Doris end-to-end coverage for all six ANN algorithms with Float32 + L2. It separately covers Doris query-vector encoding for Float16, Float32, Float64, and UInt8 through flat search.

Do you think this PR should also add indexed fixtures for non-Float32 element types and Cosine/Dot metrics? These would be Doris + Lance integration tests, but they exercise the same Doris implementation path already covered by Float32 + L2; the additional matrix cells mainly expand dependency compatibility coverage while multiplying the binary fixtures and regression cases significantly. My preference is to keep this PR focused on one representative indexed combination per algorithm and not add the full algorithm × element type × metric matrix, but I would like reviewers' opinions.

Hi @FANNG1
maybe this pr could merge first, other element types could be laster.
and the integration capability matrix, maybe u could also update some infos in the doc.
https://github.com/apache/doris-website/blob/master/docs/lakehouse/catalogs/lance-catalog.mdx

zhangstar333 · 2026-08-18T06:39:39Z

run buildall

github-actions · 2026-08-18T15:46:33Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-08-18T15:46:37Z

PR approved by anyone and no changes requested.

Issue Number: Part of apache#66495 The vector_search() path is algorithm-agnostic - the BE forwards metric, nprobes, refine_factor and ef to lance-c and lets Lance choose the index - but the only algorithm it has ever been exercised against is IVF_PQ (apache#66779). Nothing in the repository could say whether Doris reads an IVF_FLAT, IVF_SQ or graph index at all, so five of the six entries on the issue checklist rested on an argument about the code path rather than on a test. Extend the committed fixture with one table per remaining algorithm and add the suites that query them: - lance_build_preinstalled_catalog.py gains five VECTOR_TABLES entries - vs_ivf_flat_f32, vs_ivf_sq_f32, vs_ivf_hnsw_flat_f32, vs_ivf_hnsw_sq_f32 and vs_ivf_hnsw_pq_f32 - each carrying one index of its own algorithm over the same 1024-row, two-fragment, 16-dim Float32 data the IVF_PQ table already uses, so a table is exactly one cell of the algorithm matrix. The self-check verifies every index the same way it verified IVF_PQ: exactly one index of the expected type and name, covering every fragment, ANNSubIndex and ANNIvfPartition in the indexed plan, KNNVectorDistance and no ANN node in the flat plan, and the row-256 nprobes=1 discriminator that fails if a query silently falls back to a flat scan. - Only IVF_FLAT gets its indexed-equals-flat equality asserted, since it stores the original vectors and a full-partition probe is an exhaustive scan by another name. IVF_SQ, IVF_HNSW_* and IVF_PQ quantize or traverse a graph, so their agreement with the flat search is recorded, never asserted, and their queries use refine_factor. - The graph indexes need a second discriminator, because nprobes alone cannot show that ef reached the index. Measured on this data, a query at row 512 loses a true neighbour at ef=5 that ef=50 finds - but only on IVF_HNSW_SQ; on 1024 collinear vectors the FLAT and PQ graphs still return the exact rows at ef=5. The generator therefore asserts the ef discriminator on the one table that can carry it, records it for the others, and the suite queries that same table. - BOUNDARY_TOP_K makes the generator probe the boundary at the k the suites actually use. It checked k=10 while the suites query k=9, so a retrained index could have passed the generator and failed the suites. test_lance_vector_search_ivf_flat asserts the IVF_FLAT guarantee against Doris at both ends of the dataset; test_lance_vector_search_index_types covers the other four with per-table nprobes=1 discriminators, the ef pair, and the "ef must be greater than or equal to k" error a too-narrow refined graph query must keep producing. The writer pin stays at pylance 4.0.1, now with the evidence that matters after apache#66786 moved branch-4.1 to lance-c v0.1.6: pylance 7.0.0 - the Lance generation the BE reads through v0.1.6 - reproduces the generator self-check line for line on all six tables, and all 35 queries these three suites issue return identical rows and distances under both. The goldens therefore depend on the frozen fixture bytes, not on which Lance version reads them. Claude-Session: https://claude.ai/code/session_01BQAjAdyfr5Rr94WA4WE45S

Issue Number: Part of apache#66495 Generated with `run-regression-test.sh -forceGenOut` against a local FE/BE cluster built from branch-4.1 with lance-c v0.1.6, reading the committed fixture from the iceberg docker MinIO stack. This is the first end-to-end exercise of the FE -> BE -> lance-c path against an IVF_FLAT, IVF_SQ or graph index. Every discriminator holds against Doris, and every result matches what the fixture generator recorded when it probed the same physical index directly through pylance: IVF_FLAT nprobes=1 at row 256 -> 256,255,257,254,258,253,259,252,251 flat -> 256,255,257,254,258,253,259,252,260 IVF_PQ nprobes=1 at row 256 -> 256,255,257,258,259,260,261,262,263 IVF_HNSW_SQ ef=5 at row 512 -> 512,511,513,510,509 ef=50 at row 512 -> 512,511,513,510,514 IVF_FLAT reproduces the flat search exactly at full partition probes - the distance ladder 0, 16, 64, 144, 256 - which is its algorithm guarantee and is asserted, not just frozen. The four lossy algorithms return that same ladder once refine_factor reranks their candidates with exact distances, which the goldens record without claiming it holds in general. test_lance_vector_search.out changes in one block: rebuilding the fixture retrains every IVF clustering, and the IVF_PQ partition edge moved to the other side of row 256, so its single-probe result is now the 256..263 run above instead of the previous symmetric neighbourhood. The suite's own assertion - that a single-partition probe differs from the flat search - still holds; only the frozen rows moved, and they moved to exactly what the generator's self-check recorded for the rebuilt index. The whole external_table_p0/lance directory passes with normal golden comparison: 8 suites, 0 failed, including the Spark-preloaded catalogs and the REST catalog that share the same __manifest. Claude-Session: https://claude.ai/code/session_01BQAjAdyfr5Rr94WA4WE45S

Issue Number: Part of apache#66495 Every vector suite so far is Float32, but the frontend packs the query vector differently for each element type - Float16 through Arrow's half-float conversion, int8 and uint8 as single bytes, Float64 as doubles - and the backend decodes it symmetrically before handing it to lance-c. A mistake anywhere in there returns plausible but wrong rows rather than an error, and nothing covered it: all_types.lance already carries a fixed_size_list column of every element type, and no suite had ever run vector_search() against one. The new suite searches those existing columns flat, so it needs no new fixture and no index. Index behaviour is algorithm-level and belongs to the Float32 suites; encoding is per element type, and a flat search isolates exactly that. Querying with row 1's own vector must come back at distance 0, which for Float16 only holds if both sides round to half precision the same way - the check is of the encoding, not of the literal. It also pins that Lance skips null rows rather than scoring them (11 of the 12 rows are null in every vector column), and that asking for a distance an element type does not support is a query error rather than a wrong answer. int8 is deliberately left out, with the reason recorded in the suite: on a nullable int8 column that query does not fail, it aborts the backend. Lance rebuilds the batch as float32 through convert_to_floating_point, whose int8 arm drops null inner elements while keeping the original null buffer, so FixedSizeListArray::new gets a values buffer shorter than that buffer implies and its unwrap panics - and a Rust panic crossing lance-c's FFI boundary has nowhere to be caught. Fixed upstream by lance 15bbd4a85 (apache#7498), first released in Lance v9.0.0-beta.11, while lance-c pins Lance 7.0.0-beta.7 through v0.1.6. The coverage can be added once lance-c ships a release built on Lance 9 or newer. Claude-Session: https://claude.ai/code/session_01BQAjAdyfr5Rr94WA4WE45S

zhangstar333 · 2026-08-20T09:58:56Z

run buildall

…y matrix (#4082) Documents the Lance vector index compatibility matrix requested in [apache/doris#66495](apache/doris#66495), whose completion criteria include "User documentation contains a supported algorithm compatibility matrix". Follows up on the [review comment on apache/doris#66512](apache/doris#66512 (comment)) asking for the integration capability matrix to be added to this page. Companion code PR: [apache/doris#67039](apache/doris#67039), which adds the regression coverage for the combinations documented here. Both `docs/` and the `zh-CN` translation are updated together. ## What is added **A vector element type x distance metric matrix** under "Supported Vector Index Types", describing which combinations can serve an index-backed search with the embedded Lance version, with footnotes for: - the Float16 L2 index-build limitation (and that Flat Search over Float16 is fine); - UInt8 being hamming-only, and available only for `IVF_FLAT` and `IVF_HNSW_FLAT`; - Int8 having no complete index build path, plus the nullable-Int8 crash and the upstream fix it is waiting on. **A "The Query Metric Must Match the Index Metric" section**, because it is the most likely user-visible surprise: Doris selects a vector index only when the requested `metric` equals the metric the index was built with, and otherwise runs Flat Search. `EXPLAIN` reports this as `lanceSearchIndexSegments=0`. It also notes that only one index per vector column is considered. ## Two corrections to existing text 1. **Version binding.** The page said the BE data reader is bound to Lance `9.1.0-beta.3` at commit `e934cc2c`. That is the `lance-java` version the **FE** uses (`fe/pom.xml`). The BE reader is `lance-c v0.1.6`, which pins the Lance Rust crates at **`7.0.0-beta.7`**, Lance commit **`e0e977a6`**. Both are now stated separately. The `data_storage_version` table below it stays correct: `rust/lance-encoding/src/version.rs` at `e0e977a6` has default `V2_1`, `Stable -> V2_1`, `Next -> V2_3`, and `2.2` stable but not default, which is what the table already describes. 2. **Default `metric`.** The `metric` parameter row and the paragraph after the parameter table both said that omitting `metric` makes Doris use the metric the index was created with. `LanceScanNode.metricMatches` maps an unset metric to `l2`, so on a cosine-indexed table an omitted `metric` plans a Flat Search instead. The documented behavior now matches the code, and says to pass `metric` explicitly for a non-L2 index.

…ement types (#67039) ### What problem does this PR solve? Issue Number: Part of #66495 Problem Summary: Companion documentation PR: [apache/doris-website#4082](apache/doris-website#4082). #66512 gave `vector_search()` end-to-end coverage for all six Lance ANN algorithms, but only with **Float32 + L2**. Reviewer feedback there was to merge it and follow up with the other element types, which is what this PR does. The gap is not only a dependency-compatibility one. #66841 made the FE metric-aware: Doris plans an indexed split only when the requested metric equals the metric the index was built with (`LanceScanNode.metricMatches`), and otherwise plans flat splits rather than letting Lance fall back to brute force silently. Until now the **cosine and dot branches of that comparison had never run against a real index**, because no non-L2 index existed anywhere in the fixture. This PR covers them, in both directions. ### What is changed? There is no FE or BE production-code change. #### The matrix is now covered in full, in two tiers The support matrix has 96 cells (4 element types x 4 metrics x 6 algorithms). All 96 were built against the embedded Lance generation, one per subprocess: **56 build and answer a search, 40 do not**, and the 40 fall into three groups — a float type with `hamming` (IVF training rejects it), `uint8` with a non-hamming metric (it is read as a binary vector), and `uint8` under a quantizing builder. `buildable_combos()` in the generator yields exactly those 56; the measurement and the three failure groups are recorded next to it. All 56 are covered: - **12 tables in depth.** All six algorithms on Float32 + L2, plus cosine and dot, plus Float64, Float16 and UInt8. Committed goldens, a closed-form distance ladder where the data shape has one, and a discriminator per table. Data shape is per metric, because one shape cannot serve them all: the collinear ladder is degenerate under cosine (directions converge and the top distances collapse to zero) and under dot (the answer stops depending on the query), so those tables use a directional shape, and UInt8 uses a thermometer code that makes hamming an exact ladder. - **44 cells in one table**, `doris.vs_index_matrix`, at 64 rows with one vector column per cell. One column per cell rather than several indexes on one column, because only the first index built on a column is reachable — Lance answers the others with a brute-force scan, and Doris arrives at the same place by another route, since `selectIndexSegments` keeps only the segments of the first index it finds for a field id. `vs_index_matrix` asserts three things per cell, and no goldens, so a fixture rebuild leaves it untouched: 1. Doris plans an indexed split, with `lanceSearchUnindexedFragments=0` — the only check that would catch an index reaching one of the two fragments. 2. `nprobes=1` answers differently from `nprobes=4`. A flat scan has no partitions and cannot, so this is what separates a real indexed search from a silent fallback. Four query rows are tried and the first that discriminates is enough. 3. Reranked with `refine_factor`, the indexed search returns exactly the rows an exhaustive scan returns. Recall is not measured, and (3) compares two paths inside the same backend, so it does not pin absolute distances — that is what the 12 depth tables' closed-form ladders do. #### Fixture self-check The self-check is the whole contract for a fixture whose bytes are not reproducible, so it now also pins a digest of every profile's vectors and reads rows back from each table; asserts the metric each index was **actually built with**, read from `stats["indices"][*]["metric_type"]` rather than searched for, because a cosine IVF_PQ reports `l2` on its sub-index; keeps the indexed-vs-flat comparison bit-exact for the integer ladders; and enforces the `pylance` and `lance-namespace` pins before building or verifying. Discriminator margins are reported and warned on when thin. `--repin` downgrades a stale discriminator row to a warning so a rebuild can complete and the row can then be re-measured against the fixture that was promoted. #### Fixture publishing `iceberg.yaml.tpl` now uses `mc mirror --overwrite --remove` instead of `mc cp -r`. The `__manifest` version is a commit count that a rebuild can lower, and Lance names version files `u64::MAX - version` so a listing resolves newest first; a stale higher-versioned manifest left in the bucket therefore wins over the one just published. A live test environment's MinIO had accumulated several. Also removes two `entrypoint.sh` files that a `git add -A` had picked up — they are rendered from the `.tpl` beside them by `run-thirdparties-docker.sh` — and adds the `.gitignore` rule that was missing for them. ### Release note None ### Check List (For Author) - Test  - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [x] Yes.  apache/doris-website#4082 ### Check List (For Reviewer who merge this PR) - [x] Confirm the release note - [x] Confirm test cases - [x] Confirm document - [ ] Add branch pick label

FANNG1 force-pushed the lance-ivf-flat-66495 branch from 6c45e29 to 4199072 Compare August 6, 2026 01:09

FANNG1 changed the title [feature](lance) Add verified vector index query coverage for IVF_FLAT/IVF_SQ/IVF_HNSW_* [feature](lance) Add verified IVF_FLAT vector index query coverage Aug 6, 2026

FANNG1 force-pushed the lance-ivf-flat-66495 branch from 4199072 to efbe36d Compare August 6, 2026 01:45

FANNG1 mentioned this pull request Aug 12, 2026

[Bug](lance) Vector search on a Float16/Float64 column with an IVF index aborts the BE #66703

Open

2 tasks

FANNG1 changed the title ~~[feature](lance) Add verified IVF_FLAT vector index query coverage~~ [feature](lance) Verified IVF_FLAT vector index coverage and lance-c 0.1.6 upgrade Aug 12, 2026

FANNG1 mentioned this pull request Aug 14, 2026

[Feature](lance) Support additional vector index algorithms #66495

Open

10 tasks

FANNG1 force-pushed the lance-ivf-flat-66495 branch from 7380b3f to deef928 Compare August 17, 2026 03:05

FANNG1 changed the title ~~[feature](lance) Verified IVF_FLAT vector index coverage and lance-c 0.1.6 upgrade~~ [test](lance) Add query coverage for the remaining ANN index types Aug 17, 2026

FANNG1 changed the title ~~[test](lance) Add query coverage for the remaining ANN index types~~ [test](lance) Add query coverage for the remaining ANN index types and vector element types Aug 17, 2026

FANNG1 marked this pull request as ready for review August 17, 2026 08:59

FANNG1 requested a review from yiguolei as a code owner August 17, 2026 08:59

zhangstar333 approved these changes Aug 18, 2026

View reviewed changes

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Aug 18, 2026

github-actions Bot added the reviewed label Aug 18, 2026

FANNG1 added 3 commits August 20, 2026 17:58

zhangstar333 force-pushed the lance-ivf-flat-66495 branch from 2cda885 to bbef0ec Compare August 20, 2026 09:58

yiguolei approved these changes Aug 21, 2026

View reviewed changes

yiguolei merged commit f44b3fa into apache:branch-4.1 Aug 21, 2026
30 of 32 checks passed

Doris-Breakwater mentioned this pull request Aug 21, 2026

[Bug](lance) test_lance_vector_search boundary_single_probe returns a one-sided candidate set on branch-4.1 #67026

Closed

1 task

This was referenced Aug 21, 2026

[test](lance) Add indexed coverage for cosine, dot and non-Float32 element types #67039

Merged

[doc](lance) Document the vector element type and metric compatibility matrix apache/doris-website#4082

Merged

Uh oh!

Conversation

FANNG1 commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed?

ANN index algorithms

Vector element encoding

Scope

Test

Release note

Uh oh!

hello-stephen commented Aug 6, 2026

Uh oh!

yiguolei commented Aug 11, 2026

Uh oh!

FANNG1 commented Aug 11, 2026

Uh oh!

zhangstar333 commented Aug 14, 2026

Uh oh!

FANNG1 commented Aug 14, 2026

Uh oh!

zhangstar333 commented Aug 14, 2026

Uh oh!

FANNG1 commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Exploratory Doris + Lance compatibility notes

Issues found in the Doris + Lance probes

Question for reviewers

Uh oh!

zhangstar333 commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Exploratory Doris + Lance compatibility notes

Issues found in the Doris + Lance probes

Question for reviewers

Uh oh!

zhangstar333 commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

zhangstar333 commented Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

FANNG1 commented Aug 6, 2026 •

edited

Loading

FANNG1 commented Aug 17, 2026 •

edited

Loading

zhangstar333 commented Aug 18, 2026 •

edited

Loading