[bug] Search not finding all files

[weberpatr]

Describe the bug

Hello,

We setup Sourcebot last week and connected it to our self-hosted GitLab instance.
After a couple of days it figured out, the search is not finding all occurences of a specific filename or string.
Indexing should be done for all ~600 repos. Any idea?

To reproduce

Example 1:
repo:<team1> file:pom.xml "xml version" -> 7 search results

Same search with npm tool gitlab-search lists around 50 results in the same group:
gitlab-search -g <team1> -f pom.xml "xml version "

Example 2:
repo:<team2> "test" results in 158 matches in 44 files, but only 4 repos of that group.
Same search with npm tool gitlab-search lists around 153 different files
gitlab-search -g <team2> "test"

Sourcebot deployment information

Sourcebot-Version: 4.6.6

Additional information

Configration:

{
  "all": true,
  "url": "https://gitlab.company.io",
  "type": "gitlab",
  "token": {
    "secret": "SourceBot-GitLab-Token"
  },
  "exclude": {
    "archived": true
  }
}

[brendan-kellam]

heya thx for raising - I suspect you may be hitting #492. Are you seeing any "Failed to sync repository" logs?

[weberpatr]

Yes, I have one failed repository - which is also an empty repository:

[backend] | 2025-09-09T05:24:06.460Z error: [repo-manager] Failed to sync repository gitlab.xxxxxx.io/wccb/xyxyxyxyxy (id: 124), attempt 1/3. Sleeping for 60s... Error: Error: Failed to fetch repository: /data/.sourcebot/repos/124. Reason: fatal: couldn't find remote ref HEAD

[weberpatr]

Just updated to v4.6.7.
The failed sync is now solved, but the mentioned search results did not change. Still missing a lot of results.

[musabshak]

Still missing a lot of results.

We were running into this as well. With a search query that specified a particular repository and a keyword, there would be lots of results. But searching for the same keyword without specifying repository didn't yield results, but there were also no error logs in the UI or in the server.

The solution turns out lies in two things:

• Increasing the ZOEKT_MAX_WALL_TIME_MS timeout from default 10 seconds to something higher
• Giving your Sourcebot instance more memory (and more cpu)

The action item on Sourcebot-dev's end is to surface the Zoekt timeout errors up to the Sourcebot server logs and to the UI so the users at least know what's going on.

This is really important from an accuracy POV, so one could rely on sourcebot w.r.t questions of the form "do any references to this keyword exist in my codebase at all"?

Currently, sourcebot is like the opposite of a bloom filter: if it returns matches, you know for sure the matches exist, but if it doesn't return a match, you don't know for sure that a match does not exist, and this is not good enough.

[brendan-kellam]

@weberpatr how long are these searches typically taking? Assuming they are around 10 seconds, then @musabshak theory around ZOEKT_MAX_WALL_TIME_MS is likely correct.

Currently, sourcebot is like the opposite of a bloom filter: if it returns matches, you know for sure the matches exist, but if it doesn't return a match, you don't know for sure that a match does not exist, and this is not good enough.

100% agreed

The action item on Sourcebot-dev's end is to surface the Zoekt timeout errors up to the Sourcebot server logs and to the UI so the users at least know what's going on.

Also agreed - having some kind of "stop reason" would be helpful. Tracking in #504

[weberpatr]

The search is not taking longer than 1-2 secs. Anyway I increased ZOEKT_MAX_WALL_TIME_MS from 10000 to 30000.
Is there any caching of search results?

Maybe I'm trying to delete the local data directory and force a re-index of all 500 repos?

[brendan-kellam]

Is there any caching of search results?

There is client side caching, but refreshing the page will flush the cache and refetch.

Maybe I'm trying to delete the local data directory and force a re-index of all 500 repos?

Yea it could be worth the shot. If the error persists, would be great to jump on a call if you have a minute this week.

[weberpatr]

at the moment I'm stucked because of #509 :)

[weberpatr]

After a clean re-setup on v4.7.0 it seems to work now.
The re-index succeeded without any error message and the test searches (from the first post) now gives a more plausible result than before. I also increased the ZOEKT_MAX_WALL_TIME_MS to 30000.