[LightRAG][Epic] Integrate LightRAG as a pluggable knowledge retrieval backend #162

Objective

Integrate LightRAG into CacheRoute as an optional, pluggable knowledge indexing and retrieval backend.

The integration must preserve CacheRoute's existing hierarchy:

LightRAG: identify which knowledge is relevant
KDN: own canonical knowledge IDs, source text, artifacts, replicas, and KV lifecycle
Scheduler: select KDN and Proxy pools
Proxy: select a concrete Instance and execute the injection path
Instance: run inference

LightRAG must not become a Scheduler, replace KDN, own KV artifacts, or enter the Instance hot path.

Motivation

CacheRoute currently represents online knowledge demand mainly as a flat Knowledge_List produced by embedding similarity. This Epic establishes a safe engineering path from the existing embedding/FAISS pipeline to an optional graph-augmented retrieval backend, while retaining deterministic fallback and the existing pool-level and instance-level routing boundaries.

The research-facing extension is to later derive a structured Knowledge Demand Profile and use it for knowledge-compute co-routing. That work is intentionally separated from the initial integration steps.

Target Architecture

Knowledge registration
        |
        v
KDN authoritative Text/Artifact state
        |
        +--> asynchronous derived index --> LightRAG service

Online request
        |
        v
Scheduler KnowledgeRetriever
        |-- embedding backend
        |-- LightRAG backend
        `-- shadow/fallback policy
        |
        v
canonical CacheRoute KIDs
        |
        v
Scheduler selects KDN + Proxy
        |
        v
Proxy selects Instance and text/KV injection path

Invariants

• KDN-generated canonical KIDs remain authoritative.
• LightRAG document/chunk IDs never leak into the CacheRoute data plane.
• LightRAG indexes are derived and rebuildable from KDN state.
• LightRAG failure must not block ordinary inference.
• The existing embedding retrieval path remains available as the default and fallback.
• Scheduler selects KDN and Proxy only; Proxy continues to select Instance.
• Instance and vLLM processes do not import or initialize LightRAG.
• CPU-only unit tests must not require CUDA, a live LightRAG service, a live LLM, Redis, or vLLM.

Repository Constraints

• Do not add a new functional top-level directory without a separate architecture review Issue.
• New Python capability should prefer src/cacheroute/<domain>/ when that package layout is available, or an existing component hierarchy during migration.
• Cross-component contracts must live in a lightweight cacheroute namespace without import side effects.
• Any directory move must audit static/dynamic imports, string module references, service/script entry points, Docker/CI/config paths, monkeypatch targets, packaging, and Markdown links.
• Do not replace or delete existing root README content. The root README should contain only a concise overview; detailed LightRAG design belongs in dedicated documentation.

Delivery Steps and Status

• [LightRAG][Step 1] Define pluggable retrieval contracts and preserve the embedding backend #163 — Step 1: define pluggable retrieval contracts and preserve the embedding backend.
• [LightRAG][Step 2] Add a LightRAG REST client and CPU-only contract tests #164 — Step 2: add a LightRAG REST client and CPU-only contract tests.
• [LightRAG][Step 3] Add asynchronous KDN indexing and canonical KID lifecycle mapping #165 — Step 3: add asynchronous KDN indexing and canonical KID lifecycle mapping.
• [LightRAG][Step 4] Refactor Scheduler retrieval into an asynchronous pre-routing stage #166 — Step 4: refactor Scheduler retrieval into an asynchronous pre-routing stage.
• [LightRAG][Step 5] Add shadow retrieval, unified tracing, and evaluation metrics #167 — Step 5: add shadow retrieval, unified tracing, and evaluation metrics.
• [LightRAG][Step 6] Enable controlled LightRAG retrieval with fallback, caching, and rollback #168 — Step 6: enable controlled LightRAG retrieval with fallback, caching, and rollback.
• [LightRAG][Step 7] Prototype Knowledge Demand Profiles and knowledge-compute co-routing #169 — Step 7: prototype Knowledge Demand Profiles and knowledge-compute co-routing.

Dependency Order

#163
  |
#164
  |\
  | #166
  | /
#165
  |
#167
  |
#168
  |
#169

Operationally:

1. Complete [LightRAG][Step 1] Define pluggable retrieval contracts and preserve the embedding backend #163 first.
2. Complete [LightRAG][Step 2] Add a LightRAG REST client and CPU-only contract tests #164 next.
3. [LightRAG][Step 3] Add asynchronous KDN indexing and canonical KID lifecycle mapping #165 and [LightRAG][Step 4] Refactor Scheduler retrieval into an asynchronous pre-routing stage #166 may proceed in parallel after their dependencies, but their integration contracts must be reviewed together.
4. Complete [LightRAG][Step 5] Add shadow retrieval, unified tracing, and evaluation metrics #167 before making LightRAG active.
5. Complete [LightRAG][Step 6] Enable controlled LightRAG retrieval with fallback, caching, and rollback #168 only after shadow evidence and regression gates are available.
6. Treat [LightRAG][Step 7] Prototype Knowledge Demand Profiles and knowledge-compute co-routing #169 as a research prototype after the engineering path is stable.

Version and Dependency Guidance

• Steps 1-2 can be developed as isolated, compatibility-preserving foundations.
• Steps 3-6 should align with the v0.1.10 state, protocol, trace, and regression contracts tracked by v0.1.10-2: Define v1 Runtime, Artifact, LMCache Observation, Operation Task, and Queue State Models #139, v0.1.10-3: Define Versioned KDN Cache Service Facade and LMCache Gateway Contracts #140, v0.1.10-4: Add Unified v1 LMCache Gateway, Tier, Adapter, and Queue Observability #141, and v0.1.10-5: Add v1/Legacy Dual-Profile and LMCache Gateway Regression Validation #142.
• Step 7 is research-oriented and should start only after online retrieval behavior and observability are stable.

Epic Acceptance Criteria

• All child Issues have independent tests and bounded PR scope.
• CacheRoute runs with LightRAG disabled and retains prior behavior.
• CacheRoute runs with LightRAG enabled without changing Scheduler/Proxy/Instance responsibility boundaries.
• Canonical KID mapping and delete/update consistency are validated.
• Shadow-mode evidence demonstrates retrieval quality and system overhead before production activation.
• Failure injection demonstrates timeout, fallback, circuit-breaker, and rollback behavior.
• The research prototype measures both answer-quality effects and system effects such as TTFT, E2E latency, cross-KDN traffic, injected tokens, and KV reuse.