Skip to content

fix(search): order CLI semantic results by rerank score, matching the web - #1976

Merged
alishakawaguchi merged 2 commits into
mainfrom
ent-1595-cli-rerank-ordering
Aug 13, 2026
Merged

alishakawaguchi merged 2 commits into
mainfrom
ent-1595-cli-rerank-ordering

Conversation

@alishakawaguchi

@alishakawaguchi alishakawaguchi commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/1032

Problem

entire search (CLI) and entire.io (web) return different results for the same query + repo + user — different order and different totals. Root cause: the CLI's cross-cell semantic merge orders by the wrong key.

  • The CLI ordered tier 0 by BM25 and tier 1 by the per-namespace retrieval Score, and its result model had no rerankScore field at all (it silently dropped it when parsing the response).
  • The shared server-side reranker (Cohere, in entire-search query-serve) is identical for both clients, but the ordering that consumes the rerank score is a separate re-implementation in each client (Go in the CLI, TS in the BFF), and the CLI's copy never got the fixes the web did.
  • query-serve's own docs on the Score field: "It is NOT the field results are ordered by … sorting by it undoes reranking." The CLI was sorting by exactly that. The web BFF orders both tiers by rerank score (ENT-1425 / ENT-1431); the CLI was sitting at the pre-fix behavior.

Measured on EU prod, same user/repo:

  • "dependabot" on entirehq/entire.io: 118 of 119 tier-0 positions differed between CLI and web; totals 100 vs 121.
  • "antithesis reranking": different use github runners #1, 15 of 18 positions differed; CLI output carried no rerankScore.

This is the same divergence class as the "wildly different results" reports.

Fix

Bring the CLI merge to parity with the web:

  • add RerankScore to the search result Meta so the CLI parses and surfaces query-serve's cross-encoder score;
  • order tier 0 by rerank score desc (BM25 tie-break) when every row was reranked, else the BFF's positional mixed-capability fallback (interleave by each cell's selection rank, BM25 tie-break);
  • order tier 1 by rerank score desc (retrieval Score tie-break);
  • tests assert rerank ordering with BM25/Score set to the opposite order, so a regression to the old keys fails, plus a mixed-capability fallback test.

Repos, tier-2 promotion, dedupe, caps, and counts are unchanged.

Verification

  • go build ./cmd/entire/..., go vet, and the search test suites pass.
  • End-to-end: the patched binary's top results for "dependabot" now match the web's rerankScore order exactly (0.6276, 0.6036, 0.5790, 0.5637, …), and rerankScore is now present on every reranked row.

Follow-up (not in this PR)

  • Tier-2 handling still differs (the web hard-drops tier-2; the CLI promotes/keeps it). Dormant today since query-serve no longer emits tier-2, but worth reconciling.
  • The real durable fix is to stop maintaining two orderers — a single shared ordering contract, or have query-serve return final merged order so neither client re-sorts.

🤖 Generated with Claude Code


Note

Cursor Bugbot is generating a summary for commit 9175836. Configure here.

… web

The CLI's cross-cell semantic merge ordered tier 0 by BM25 and tier 1 by
the per-namespace retrieval Score, and its result model dropped the
rerankScore field entirely. query-serve documents that Score "is not the
field results are ordered by ... sorting by it undoes reranking," so the
CLI was effectively showing un-reranked results while the entire.io web
BFF orders both tiers by the Cohere rerank score (ENT-1425/ENT-1431).

Same query + repo + user returned wildly different order and counts
between `entire search` and the web (e.g. 118/119 tier-0 positions
differed on one query). This brings the CLI merge to parity:

- add RerankScore to the search result Meta so the CLI parses and
  surfaces query-serve's cross-encoder score
- order tier 0 by rerank score desc (BM25 tie-break) when every row was
  reranked, else the BFF's positional mixed-capability fallback
- order tier 1 by rerank score desc (retrieval Score tie-break)
- tests assert rerank ordering with BM25/Score set to the opposite order
  so a regression to the old keys fails, plus the mixed-cell fallback

Verified end-to-end: the patched CLI's top results now match the web's
rerankScore order exactly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012duPNCzBFuJLC9Axwo9oHA
@alishakawaguchi
alishakawaguchi requested a review from a team as a code owner August 12, 2026 22:30
Copilot AI lite review requested due to automatic review settings August 12, 2026 22:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR brings the CLI’s semantic cross-cell merge ordering into parity with the web client by parsing and using query-serve’s cross-encoder rerank score, so identical queries yield consistent ordering and totals across CLI and entire.io.

Changes:

  • Add Meta.RerankScore to the search result model so the CLI parses rerank scores returned by query-serve.
  • Update semantic merge ordering to sort tier 0 and tier 1 by rerank score (with appropriate tie-breakers) and implement the mixed-capability tier-0 fallback to match the BFF behavior.
  • Add/adjust tests to assert rerank-first ordering and the tier-0 mixed-capability fallback behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
cmd/entire/cli/search/search.go Extends search result metadata to include rerankScore from the backend response.
cmd/entire/cli/search_v4.go Reworks cross-cell semantic ranking to use rerank score and adds tier-0 mixed-capability fallback ordering logic.
cmd/entire/cli/search_v4_test.go Updates tier ordering expectations and adds coverage for the tier-0 mixed-capability fallback.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Address review caveats: assert query-serve's rerankScore is parsed off the
wire (a dropped field silently reverts ordering) and that absent stays nil,
and assert the tier-0 (BM25 desc) and tier-1 (Score desc) secondary keys
when rerank scores tie.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012duPNCzBFuJLC9Axwo9oHA
@alishakawaguchi

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 87b8942. Configure here.

@alishakawaguchi alishakawaguchi self-assigned this Aug 12, 2026
@alishakawaguchi
alishakawaguchi merged commit 9862b0d into main Aug 13, 2026
12 of 13 checks passed
@alishakawaguchi
alishakawaguchi deleted the ent-1595-cli-rerank-ordering branch August 13, 2026 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants