Skip to content
This repository was archived by the owner on Oct 27, 2026. It is now read-only.

Demo: compare Algolia with Fumadocs built-in search - #1257

Draft
sdserranog wants to merge 2 commits into
mainfrom
exp/builtin-search-demo
Draft

sdserranog wants to merge 2 commits into
mainfrom
exp/builtin-search-demo

Conversation

@sdserranog

@sdserranog sdserranog commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Not for merge. This adds a test page so we can compare Algolia and Fumadocs' built-in search side by side.

Try it live: https://docs-git-exp-builtin-search-demo-arcade-ai.vercel.app/en/search-compare

Quick examples: gmial, Gmail.SendEmail, on prem, cli.

Question for the team

Once I tuned it, Fumadocs' built-in search did about as well as Algolia in my tests. Is there a reason to keep Algolia beyond the analytics, or should we try the built-in one?

Results from 62 queries (how often the right page comes up first):

Type of search Algolia Built-in
Page names (18) 18 15
Integrations (16) 11 15
Typos (13) 12 12
Questions (15) 3 5
Total (62) 44 47

Beyond the first result

Algolia's lists can look better in the screenshots, so I checked results 2 to 5 too. I took the top 5 from each engine for all 62 queries and had two separate sets of AI judges grade every page (useful, somewhat useful, or not useful) without knowing which engine it came from. The two sets agreed on 89% of pages.

Useful results in spots 2 to 5 (out of 4, average of both judges):

Type of search Algolia Built-in
Page names 1.8 1.9
Integrations 1.4 1.9
Typos 2.1 1.8
Questions 1.0 0.9
All 1.6 1.6

Comparing the whole top 5 query by query, the built-in list was better 23 to 25 times, Algolia's 19 times, and they tied 18 to 20 times.

So overall it's a tie. Algolia's later results are better for typos, and the built-in ones are better for integrations. Algolia still looks more precise because each result shows the section it matched (like "Gmail > Available tools"), and the judges only graded pages, so that isn't counted above. The built-in one's weak results also look more random, like Forkable for gmial.

Reasons to keep Algolia

  • Analytics: top searches, searches with no results, and clicks
  • Results link to the exact section of a page and show it, which makes the list easier to scan. The built-in one only links to whole pages.
  • Better at exact page names and typos, and fewer odd matches on short words (cli also shows ClickUp)

Reasons to try the built-in one

  • Free, with no third-party account or crawler to maintain
  • Integration pages are indexed straight from the toolkit data, so tool names like Gmail.SendEmail work

What's in this PR

  • lib/builtin-search.ts: built-in search setup (one result per page, title boost, English stemming, up to 2 typos, toolkit pages added)
  • app/api/search/route.ts: the search endpoint
  • app/en/search-compare/: the comparison page (hidden from search engines)

The real search dialog still uses Algolia, so nothing changes for readers.

I wrote the queries myself. 22 of them were written after tuning so the settings weren't fitted to the test.


Note

Low Risk
Isolated demo page and new search API; no change to production search behavior, though /api/search could add server load if widely used.

Overview
Adds an experimental, noindex comparison page at /en/search-compare so the same query can be run against Algolia (current docs search) and Fumadocs built-in search (ZBSearch) in parallel.

A new GET /api/search route wires up lib/builtin-search.ts, which builds indexes from MDX page structured data plus generated toolkit pages (tool names/descriptions from toolkit JSON), using simple mode with title boost, English stemming, and typo tolerance of 2.

The SearchCompare client UI debounces input, syncs ?q= in the URL, shows example queries, and displays timing, errors, and ranked results in two columns. The main site search dialog is unchanged and still uses Algolia.

Reviewed by Cursor Bugbot for commit a1ac07c. Bugbot is set up for automated code reviews on this repo. Configure here.

Adds /en/search-compare, which runs the same query against Algolia and
Fumadocs' built-in search (ZBSearch) side by side. The live search dialog
still uses Algolia. Not meant to merge.
@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 21, 2026 7:24pm UTC

Request Review

This branch was successfully deployed

1 active deployment
Preview — a1ac07ca Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant