[LightRAG][Step 1] Define pluggable retrieval contracts and preserve the embedding backend #163

Parent Epic: #162

Goal

Decouple knowledge retrieval from Service.knowledge_retriever() and define a backend-neutral asynchronous contract without changing current production behavior.

This step must preserve the existing EmbeddingEngine + KnowledgeTable result path as the default implementation.

Required Design

Define lightweight, import-safe contracts equivalent to:

• KnowledgeCandidate
  • canonical CacheRoute kid
  • relevance score
  • token or stored length
  • source/backend identifier
  • bounded metadata
• RetrievalResult
  • ordered candidates
  • backend name
  • latency
  • degraded/fallback status
  • structured trace fields
• asynchronous KnowledgeRetriever.retrieve(...)

Implement EmbeddingKnowledgeRetriever by wrapping the existing embedding/FAISS path.

Scope

• Identify the current retrieval call chain from Scheduler request handling through Request.build_request() and Service.knowledge_retriever().
• Extract backend-neutral result contracts.
• Wrap the current embedding implementation without changing ranking, threshold, KID, or length semantics.
• Provide a retriever factory/configuration seam whose default is embedding.
• Keep imports lightweight and free of model-loading side effects.
• Add focused CPU-only tests for empty queries, top-k, score thresholds, ordering, length calculation, failure behavior, and legacy compatibility.

Repository Constraints

• Do not add a functional top-level directory.
• Prefer src/cacheroute/<domain>/ if the package migration is ready; otherwise use the existing Scheduler/knowledge component hierarchy and document the later package destination.
• Shared contracts must not import FastAPI, torch, sentence-transformers, FAISS, LightRAG, KDN service modules, or Proxy modules at import time.
• Do not modify root README content except for a concise future overview when the feature is actually available.

Non-Goals

• Do not add LightRAG as a dependency in this step.
• Do not change Scheduler pool selection.
• Do not change Proxy Instance selection or injection behavior.
• Do not change KDN persistence or protocols.
• Do not make Request.build_request() asynchronous yet unless a separately justified minimal compatibility adapter is required.

Acceptance Criteria

• Default CacheRoute retrieval output remains behaviorally compatible with the pre-change embedding path.
• A backend-neutral asynchronous interface exists and is covered by tests.
• Importing shared retrieval contracts does not load embedding models or GPU libraries.
• CPU-only tests pass without CUDA, LightRAG, Redis, vLLM, or external services.
• Existing public imports and demos remain available.
• The PR documents the exact migration seam to be used by Step 4.

Validation

At minimum, run focused retrieval tests, public import compatibility checks, and the existing Scheduler request-building tests. The PR must include actual command output and note any unrelated historical test failures.