[FR] Improve support for asking questions over a large number of repositories

[brendan-kellam]

Problem

Sometimes, users want to ask questions that cover the entire breadth of the corpus, e.g., "what repositories contain X, Y, and Z", or "show me examples of foo and bar". For these types of questions, our repository selector dropdown is a bit of a obtuse UX, especially when you have lots (1k+) repos. Whgvat repositories should you select? Pressing "select all" doesn't help either as the agent isn't well equipped to navigate over such a large number of repos.

Proposed solution

Ideally, users shouldn't even need to make a selection at all - the agent is intelligently able to filter down the set of repositories that it should search while efficiently managing its context window.

[Avngrstark62]

Hi @brendan-kellam

I’ve been looking into the code search tool (packages/web/src/features/chat/tools.ts) and noticed a potential bottleneck: it performs a flat global search across all selected repos without repo-level narrowing, which could become problematic as the corpus grows large.

As an initial idea (based on my current understanding), I’m considering a simple two stage retrieval: first, a lightweight search across all repos to identify the most relevant ones, then a focused search over the top K repos before returning results to the agent. I’d explore the space more thoroughly before finalizing any approach.

Before proceeding, I wanted to check the current status and whether anyone is already working on this. If not, I’d be happy to take a first pass and dive deeper.

[brendan-kellam]

Hey @Avngrstark62 - that's a good intuition and also how I'm thinking about approaching the problem. I'm planning on refactoring how the ask agent works to support this use case, so I think it's best if I take this one. Appreciate you taking the time to share your thoughts