Add server-backed documentation search to the Reference drawer

[BorisTyshkevich]

Product goal

Add server-backed documentation search to the existing persistent Reference drawer.

Search must describe the documentation and metadata exposed by the connected ClickHouse server so results remain version-exact, permission-aware, and usable without a remote web index.

The drawer remains non-modal and the editor remains usable while search results or documentation entries are open.

Core design decisions

• Add a search input and submit button to the Reference drawer header.
• Search is explicitly submitted; the first version does not query on every keystroke.
• A valid search executes one capability-generated UNION ALL query over the usable documentation-bearing system tables.
• Every query branch reports its actual physical source_table.
• Query branches are generated only after table and column capabilities are confirmed for the active connection.
• Heterogeneous rows normalize into the existing DocKind / DocTarget domain.
• Duplicate logical entities from different physical sources merge while retaining source provenance.
• Final relevance ranking is deterministic and performed in JavaScript.
• Selecting a result uses the existing docEntry(target) lookup and entry-rendering path.
• Search, caches, and UI responses are connection-generation-safe and protected from stale repaint.
• Server payloads, snippets, result counts, cache size, and retries remain bounded.

Work breakdown

• Build capability-aware documentation search service #421 — Build capability-aware documentation search service
• Add documentation search UI to the Reference drawer #422 — Add documentation search UI to the Reference drawer
• Improve documentation search ranking with match position and proximity #423 — Improve documentation search ranking with match position and proximity

The child issues are the authoritative implementation specifications. Detailed SQL construction, interfaces, ranking weights, lifecycle mechanics, UI behavior, and test matrices belong there rather than in this umbrella issue.

Dependency order

• Build capability-aware documentation search service #421 establishes the server/application search contract and baseline ranking.
• Add documentation search UI to the Reference drawer #422 depends on Build capability-aware documentation search service #421 and adds the complete drawer interaction.
• Improve documentation search ranking with match position and proximity #423 depends on Build capability-aware documentation search service #421 and refines ranking and contextual snippets. It may be implemented independently of Add documentation search UI to the Reference drawer #422 once the service contract exists.

Completion definition

#420 closes only after all three child issues are complete.

#421 and #422 produce the functional search experience. #423 is also required before closing this umbrella because position, phrase, proximity, lexical-boundary ranking, and match-centered snippets are part of the agreed search-quality design.

Cross-cutting invariants

Capability and compatibility

• Never reference an unavailable table or unconfirmed optional column.
• Degrade safely across ClickHouse versions and partial permissions.
• Capability state is scoped to the active connection and invalidated on reconnect/sign-out.
• A stale table/column failure may trigger one bounded reprobe-and-retry attempt, never an unbounded source-removal loop.

Query and provenance

• One submitted search produces one dynamic UNION ALL request.
• Each branch emits its physical source table explicitly.
• Search results preserve the distinction between physical source and logical documentation kind.
• Search candidate retrieval is bounded; final merge and ranking are JavaScript-owned.

Safety and lifecycle

• User input is bound or escaped through the existing SQL-safe seam.
• Result summaries are rendered as text, never trusted HTML.
• Full entries continue through the existing bounded safe Markdown pipeline.
• Late responses from an older search, closed drawer, selected entry, reconnect, or sign-out cannot repaint current UI.
• Search caches and in-flight deduplication are bounded and connection-generation-aware.

Drawer behavior

• The Reference title and close control remain accessible.
• Search results are keyboard reachable and screen-reader announced.
• Selecting a result reuses existing documentation entry navigation.
• Back restores the prior bounded search snapshot, selection, focus, and scroll when still valid.
• Existing hover, completion, F1, alias, related-entry, disambiguation, Retry, resize, Escape, and focus-restoration behavior remains intact.

Non-goals

• Remote search against clickhouse.com or another web index.
• Semantic, vector, fuzzy edit-distance, or regular-expression search.
• Search-as-you-type server requests in the first version.
• Searching full examples in the initial candidate query.
• Persisted cross-session search history.
• A large kind-filter UI in the initial release.
• Replacing the existing context-aware documentation entry and rendering flows.

[BorisTyshkevich]

Implementation split

Keep this issue as the umbrella design and close it only after all child issues are complete.

• Build capability-aware documentation search service #421 — Build capability-aware documentation search service
• Add documentation search UI to the Reference drawer #422 — Add documentation search UI to the Reference drawer
• Improve documentation search ranking with match position and proximity #423 — Improve documentation search ranking with match position and proximity

Recommended order:

1. Build capability-aware documentation search service #421 establishes the search API, dynamic capability-safe UNION ALL, normalization, merge, baseline ranking, caching, and lifecycle invariants.
2. Add documentation search UI to the Reference drawer #422 builds the complete drawer interaction on that API.
3. Improve documentation search ranking with match position and proximity #423 refines relevance and match-centered snippets without changing the public search interaction.

#422 and #423 both depend on #421. #423 may land before or after #422 once the foundational result contract is stable.

[BorisTyshkevich]

https://chatgpt.com/g/g-p-6a5b45669bf881919dba4255bbac396a/c/6a61f427-fbac-83eb-8958-30bf1585be50

[BorisTyshkevich]

https://chatgpt.com/c/6a671c49-0fe8-83eb-af03-7dd8a61d427f