Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 39 additions & 19 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# AGENTS.md

author: heyao
version-date: 26-03-15
version-date: 26-07-30

## 1. Project Overview

Expand Down Expand Up @@ -46,13 +46,16 @@ Scheduler maintains resource pools for:

### 1.3 CacheRoute Strategy Development Area

Scheduling strategies should be implemented under:
The current compatibility implementations of scheduling strategies are under:

scheduler/strategy/
proxy/strategy/
`scheduler/strategy/`
`proxy/strategy/`

Strategies must not modify core scheduler logic.
They should operate through the existing selection interfaces.
Compatibility-preserving fixes may continue in these root modules. New reusable
strategy code belongs to `cacheroute.routing` once its focused migration phase
is approved; do not create empty target packages in anticipation of that work.

Example strategies include:
- round_robin
Expand Down Expand Up @@ -307,7 +310,7 @@ The user typically prefers **incremental, directly actionable code guidance**.

## 13. If You Add New Code

No new functional top-level directory may be added without a dedicated architecture Issue. New Python capabilities must first be placed under `src/cacheroute/<domain>/` or an existing component hierarchy. Cross-component shared code must use a dependency-light, side-effect-free package under the `cacheroute` namespace.
No new functional top-level directory may be added without a dedicated architecture Issue. Fix existing behavior in its current owning component hierarchy. New cross-component contracts and domain capabilities must follow the approved `cacheroute.*` ownership list in Section 18 and remain dependency-light and side-effect-free. Do not create speculative or empty target packages before their focused implementation or migration work is approved.

When introducing a new helper, field, or branch:
- keep naming consistent with existing repository terms,
Expand All @@ -324,15 +327,20 @@ If adding config/arguments:

## 14. Key Files / Entry Points

The exact filenames below are important. Prefer editing these files in place instead of introducing parallel logic elsewhere.
The exact current paths below are important transitional compatibility locations. Prefer editing existing behavior in place instead of introducing parallel logic; they are not permanent package-placement guidance.

- `scheduler/...`: request building, scheduling decisions, resource selection
- `proxy/...`: request execution path, runtime load updates, queue behavior
- `kdn/...`: knowledge registration, status fields, embedding / KVCache readiness
- `kdn_server/...`: knowledge registration, status fields, embedding / KVCache readiness
- `instance/...`: interact layer between proxy and vLLM engine.
- `client.py`: single-request testing path
- `perf_client.py`: workload generation and benchmark execution
- `demo_kdn.py`,`demo_scheduler.py`,`demo_proxy.py`,`demo_instance.py`,`demo_client.py`: startup arguments only; do not place business logic here
- `client/client.py`: single-request testing path
- `client/perf_client.py`: workload generation and benchmark execution
- `test/demo_kdn.py`, `test/demo_scheduler.py`, `test/demo_proxy.py`, `test/demo_instance.py`, `test/demo_client.py`: transitional test/demo launchers only; do not place business logic here

Existing Scheduler, Proxy, Instance, and KDN service code remains in
`scheduler/`, `proxy/`, `instance/`, and `kdn_server/` until separate, focused
migration PRs are approved. Section 18 governs all new package placement and
all migration decisions.

---

Expand Down Expand Up @@ -377,15 +385,21 @@ When in doubt, choose:

## 18. Target Package Architecture and Placement Rules

This section is normative for new architecture and directory decisions. It takes precedence over older current-layout examples in this file. Existing root packages such as `scheduler/`, `proxy/`, `instance/`, and `kdn_server/` are transitional locations until their reviewed migration phases are complete.
This section is authoritative for all new package placement and migration work. It takes precedence over older current-layout examples in Sections 1–17. Existing root packages such as `scheduler/`, `proxy/`, `instance/`, and `kdn_server/` are transitional compatibility locations until their separate reviewed migration phases are complete.

`doc/architecture/package-architecture-rfc.md` is the maintained architecture
source. A focused architecture Issue and a complete reference audit are required
before any directory or service migration.

The governing architecture is:
Use this decision rule:

- Issue #159: package architecture RFC;
- Issue #157: migration Epic;
- `doc/architecture/package-architecture-rfc.md`: maintained repository architecture document.
- **Existing behavior fix**: edit the current owning module with the smallest compatible patch.
- **New cross-component contract/domain capability**: place it according to the approved `cacheroute.*` ownership boundaries below.
- **Directory or service migration**: require a focused Issue and complete reference audit before moving files.

Do not begin a broad package move unless the relevant phase of #157 has an approved, focused Issue and a complete reference audit.
These rules do not authorize speculative target directories or placeholder
packages. Create a canonical package only as part of its approved, substantive
implementation or focused migration work.

### 18.1 Canonical Python namespace

Expand Down Expand Up @@ -616,18 +630,24 @@ Use the following ownership map when planning focused migration Issues:
- `kdn_server/contracts` -> `cacheroute.contracts.v1`;
- shared KDN domain concepts -> `runtime`, `knowledge`, and `cache`;
- `kdn_server/gateway` -> `integrations.lmcache` and `integrations.redis`;
- Scheduler strategies -> `routing`;
- `data` -> runtime package data co-located with its canonical consumer after a focused consumer and package-data audit; proven reusable knowledge behavior may move to `knowledge`, while research datasets remain outside the final wheel; do not create `cacheroute.data`;
- `doc`, `doc.blog`, `doc.integrations`, and any other documentation namespace -> repository-only `doc/`, removed from the installed package list during packaging cleanup; preserve Markdown links and do not add `__init__.py` or package documentation to resolve discovery mismatches;
- Scheduler and Proxy reusable strategies -> `routing` once the focused migration phase is approved; existing root strategy modules remain available for compatibility-preserving fixes;
- Scheduler and Proxy registries -> `topology`;
- reusable Proxy queue policy -> `routing`, while process queue state remains in `services.proxy`;
- Instance vLLM discovery -> `integrations.vllm`;
- embedding implementation -> `integrations.embeddings`;
- UI assets -> their owning service package;
- demo launchers -> `entrypoints.dev` or `examples`;
- generic `util` helpers -> their owning domain; do not preserve a utility dumping ground.
- generic `util` helpers -> their owning domain after a focused ownership audit; do not preserve a utility dumping ground.

Until their separate service migrations are approved, Scheduler, Proxy,
Instance, and KDN implementations remain in the transitional `scheduler/`,
`proxy/`, `instance/`, and `kdn_server/` root packages.

### 18.8 Migration phases

Follow the #157 phases. Do not combine them into a repository-wide move.
Follow these phases in focused, reviewable changes. Do not combine them into a repository-wide move.

1. Architecture guidance and automated boundaries.
2. Packaging cleanup and unified entrypoints.
Expand Down
5 changes: 4 additions & 1 deletion doc/architecture/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Architecture documents

- [Package architecture RFC](package-architecture-rfc.md) — target Python package boundaries, dependency direction, integration adapters, service composition, packaging policy, and migration phases.
- [Package architecture RFC](package-architecture-rfc.md) — accepted target Python package boundaries, dependency direction, integration adapters, service composition, packaging policy, and migration phases.
- [Phase A package migration audit](../package_migration_phase_a.md) — audited namespace foundation and transitional packaging inventory.

The [root README](../../README.md) remains the high-level project overview; detailed package architecture and migration governance live in this directory.
86 changes: 81 additions & 5 deletions doc/architecture/package-architecture-rfc.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# CacheRoute package architecture RFC

This document records the target Python package architecture adopted by the repository architecture RFC and Issue #157 migration epic.
> Status: Accepted; implementation in progress.

This document records the accepted target Python package architecture and the
focused migration sequence used to reach it.

## Decision

Expand Down Expand Up @@ -40,14 +43,18 @@ CacheRoute/
└── AGENTS.md
```

`crates/` is reserved for independently built native components such as the Resource Agent. Adding or moving a native component still requires a focused architecture review.
`crates/` is reserved for independently built native components. Placing the
Resource Agent there remains a proposal until a focused migration Issue approves
it; this RFC alone does not authorize adding or moving it.

## Package responsibilities

### `contracts`

Versioned, JSON-serializable cross-process contracts only.

`contracts` owns wire schemas and enums, not service request handlers or framework adapters.

It must not import FastAPI, Redis, Torch, vLLM, LMCache, or service implementations.

### `runtime`
Expand Down Expand Up @@ -78,6 +85,9 @@ Reusable policy must not depend on FastAPI applications or process-global servic

Dependency-light trace contracts, collectors, events, exporters, and Legacy projections.

Observability exporters may implement reviewed plugin protocols, but canonical
observability contracts remain in this domain package.

### `integrations`

External-system adapters:
Expand All @@ -89,6 +99,10 @@ External-system adapters:

Domain packages must not import integrations.

Integrations implement domain-owned ports; they do not define canonical domain
models or move external SDK semantics into domain packages. A port belongs with
the domain that owns its semantics.

### `services`

Deployable process implementations:
Expand All @@ -100,9 +114,12 @@ Deployable process implementations:

Services use contracts and domain capabilities. They must not import another service's internal implementation.

Services own process-specific state, request handlers, and framework adapters.

### `entrypoints`

CLI, Uvicorn application factories, development orchestration, and process composition.
CLI, construction and dependency injection, Uvicorn application factories,
development orchestration, settings composition, and process lifecycle.

The long-term command surface is:

Expand All @@ -124,7 +141,8 @@ Stable extension protocols and discovery for:
- runtime/engine connectors;
- observability exporters.

Use Python entry points or explicit module paths. Do not pluginize every internal strategy in the first phase.
Use Python entry points or explicit module paths. Plugins are limited to reviewed,
stable extension protocols; do not pluginize every internal strategy.

### `compat`

Expand Down Expand Up @@ -236,6 +254,43 @@ Examples:

Do not place the whole project under `cacheroute.v1`.

## Current layout versus target ownership

The repository still runs from root Python packages and repository-support
directories that predate this RFC. They are transitional, not alternative
permanent architecture. Existing behavior fixes should remain in the current
owner until a focused migration is approved; this table does not authorize
speculative target packages or file moves.

This mapping was audited against the current tree and the
[Phase A inventory](../package_migration_phase_a.md). Where one root package
mixes responsibilities, the final file-level destination requires a focused
migration audit rather than an invented one-to-one replacement.

| Current path | Current role | Target owner/package | Migration phase | Interim rule |
|---|---|---|---|---|
| `core` | Shared request, configuration, forwarding, tokenizer, model-calculation, and runtime-compatibility code | Split by proven semantics among `cacheroute.contracts`, `cacheroute.runtime`, relevant domains, and temporary `cacheroute.compat`; exact file ownership requires a focused audit | Phase 2, with compatibility cleanup in Phase 6 | Keep compatibility fixes in `core`; do not reproduce it as `cacheroute.core`. |
Comment on lines +270 to +272

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Account for omitted installed packages

Complete this supposedly tree-audited mapping with dispositions for data and doc: pyproject.toml lines 49–51 currently install data, data.CacheRoute_dataset, doc, and related namespaces, and the Phase A inventory also records them, but neither appears in this table. A later migration using this authoritative map therefore has no owner or phase for removing these top-level wheel packages—and may either leave them in the final wheel or drop runtime data without the required package-data audit.

AGENTS.md reference: AGENTS.md:L600-L611

Useful? React with 👍 / 👎.

| `data` | Installed package containing dataset and knowledge-document material, YAML knowledge/configuration inputs, and a Python package marker; a focused audit must determine whether Python helpers are present or later added | Co-locate runtime package data with the canonical consuming package after a consumer and package-data audit; move proven reusable knowledge-domain Python behavior to `cacheroute.knowledge`; keep research datasets and repository artifacts outside the final wheel | Packaging audit in Phase 1; proven semantic Python movement in the applicable Phase 2 migration | Preserve current imports and data paths. Do not move or delete datasets without auditing runtime consumers, Docker/config paths, tests, package data, and documentation. Do not create `cacheroute.data`. |
| `doc` | Repository documentation currently listed with `doc.blog` and `doc.integrations` as transitional installed namespaces; it is not runtime Python architecture | Remain under repository `doc/`; remove `doc`, `doc.blog`, `doc.integrations`, and any other documentation namespace from the installed package list | Phase 1 packaging cleanup | Preserve Markdown links and explicitly exclude repository-only documentation subdirectories from namespace discovery where needed. Do not add `__init__.py`, package documentation in the final wheel, or solve discovery mismatches by adding documentation packages to `pyproject.toml`. |
| `scheduler` | Scheduler process, resource/knowledge registries, and selection strategies | `cacheroute.services.scheduler` for process state; `cacheroute.topology`, `cacheroute.knowledge`, and `cacheroute.routing` for reusable domain behavior | Phase 4 after prerequisite domain work | Preserve the root service and strategy interfaces until a Scheduler-specific migration PR audits all references. |
| `proxy` | Proxy process, queue state, resource reporting, metrics, and strategies | `cacheroute.services.proxy` for process state; `cacheroute.routing`, `cacheroute.topology`, and `cacheroute.observability` for proven reusable semantics | Phase 4 after prerequisite domain work | Apply compatible fixes at root; separate reusable policy only in an approved Proxy migration. |
| `instance` | Instance service, vLLM-facing control, prediction tools, Resource Agent, and dashboards | `cacheroute.services.instance` plus `cacheroute.integrations.vllm`; predictor, dashboard, and native-agent ownership requires focused audits | Phases 3–5 | Keep the root service intact; `crates/` placement for the Resource Agent remains only a proposal. |
| `kdn_server` | KDN service, contracts, domain models, gateways, databases, and CLI | `cacheroute.services.kdn`, `cacheroute.contracts.v1`, `cacheroute.runtime`, `cacheroute.knowledge`, `cacheroute.cache`, and reviewed integration adapters | Phases 2–4 | Preserve current wire behavior and root imports; migrate each responsibility only after its dependencies and references are audited. |
| `client` | Interactive client, performance workload client, task sets, and request tools | Production CLI/construction may belong to `cacheroute.entrypoints`; reusable or example ownership is not yet proven and requires a focused audit | Phase 1 or Phase 5, subject to audit | Keep `client/client.py` and `client/perf_client.py` working; do not pre-emptively package them as a service. |
| `store` | Knowledge-base construction, storage interfaces, and embedding helpers | Likely `cacheroute.knowledge` ports and `cacheroute.integrations.embeddings`, but file-level ownership requires a focused audit | Phases 2–3 | Maintain current imports and behavior until storage semantics and adapter boundaries are proven. |
| `model` | Model configuration and embedding implementation | `cacheroute.runtime` for runtime identity/configuration and `cacheroute.integrations.embeddings` for external implementation; exact split requires audit | Phases 2–3 | Keep runtime configuration data with its current consumer until package-data and references are audited. |
| `util` | Cross-cutting address, flag, timer, and KDN helper scripts | Each helper's owning domain, integration, entrypoint, or repository `scripts/`; no single target package | Relevant focused phase | Do not create a utility dumping ground; leave helpers in place until ownership is demonstrated. |
| `UI` | Client, KDN, and Proxy web assets and Python packages | Co-locate each asset set with the `cacheroute.services.*` package that serves it; client UI ownership requires audit | Phase 5 | Preserve current asset paths and package data until each serving application migrates. |
| `test` | Test suite and transitional demo launchers | Repository `tests/` and, where reviewed, `examples/` or `cacheroute.entrypoints.dev` | Phase 5 and final cleanup | Tests and demos remain outside the canonical wheel architecture; existing launchers may receive compatibility fixes. |
| `scripts` | Repository validation and operational scripts | Remain repository `scripts/`; production lifecycle behavior may move to `cacheroute.entrypoints` only after audit | Phase 1 or relevant focused phase | Do not install scripts as Python packages; preserve operational paths until replacements are validated. |
| `env` | Deployment configuration, Docker support, environment checks, and setup tools | Repository `env/`, not a Python wheel package; any runtime entrypoint extraction requires audit | Phase 1 or Phase 5 as applicable | Keep deployment files at root and out of the final wheel. |
| `log` | Checked-in experiment log material | Repository artifacts outside the wheel; long-term retention/location requires a focused data-governance audit | Phase 6 cleanup or separate audit | Do not import or package logs as application code; do not move research artifacts without review. |

Tests, documentation, repository scripts, deployment configuration under `env`,
logs, and generated files must not become wheel packages. UI assets are runtime
package data only when co-located with and declared by the service that serves
them.

## Migration mapping

- `kdn_server/contracts` -> `cacheroute.contracts.v1`
Expand All @@ -254,7 +309,7 @@ Do not place the whole project under `cacheroute.v1`.

### Phase 0 — architecture approval

Approve the RFC and make Issue #157 reference it as the target architecture.
Approve this RFC as the target architecture before implementation phases begin.

### Phase 1 — packaging and entrypoint boundary

Expand All @@ -280,6 +335,27 @@ Move UI assets to their owning service packages, replace test demos with entrypo

Remove obsolete root runtime packages, source bootstraps, transition package lists, and expired compatibility shims.

## Architecture decision checklist

Use this checklist in every future architecture Issue and implementation PR:

- Is this a wire contract, domain model, adapter, service implementation, plugin protocol, or entrypoint?
- Which package owns the semantics?
- Does the proposed dependency follow the allowed direction?
- Does it import a framework or external SDK from a domain package?
- Is a compatibility shim required?
- What is the shim removal milestone?
- Which dynamic strings and operational paths must be audited?
- Does the clean wheel contain only approved packages?
- Can the feature import from outside the source checkout?

For a directory or service migration, the operational-path audit must cover
normal and dynamic imports; module and method strings; Uvicorn/FastAPI targets;
subprocess and `python -m` commands; Docker, Compose, CI, configuration, and
mounted paths; tests, fixtures, monkeypatch, and mock targets; packaging,
editable installs, and installed-wheel behavior; and every README and Markdown
relative link.

## Required architecture tests

Add automated checks for:
Expand Down
Loading