You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration #138
Cache artifact reuse must verify compatibility across the model, tokenizer, adapter stack, KV layout, precision, and parallel configuration. The current instance registration payload does not expose enough information to make that decision safely.
Objective
Introduce a stable, serializable capability fingerprint for each inference instance and extend instance registration and heartbeat reporting without breaking existing clients.
Implementation Scope
Add a capability/fingerprint model covering at least:
model identity and revision;
tokenizer identity and revision;
adapter identity and configuration, when applicable;
KV-cache layout and schema version;
KV dtype and relevant precision settings;
tensor, pipeline, and data-parallel configuration;
vLLM and LMCache capability/version information required for reuse decisions.
Define deterministic fingerprint generation and normalization rules.
Extend instance registration and heartbeat payloads with optional capability fields.
Preserve existing registration fields and default behavior for legacy instances.
Add a compatibility-checking API or utility that returns a structured result and explicit mismatch reasons.
Keep the implementation in production modules rather than demo_*.py.
Acceptance Criteria
A newly registered instance exposes a deterministic capability fingerprint.
Equivalent capability descriptions generate the same fingerprint regardless of field ordering.
A model, tokenizer, KV-layout, dtype, or parallel-configuration mismatch is detected before cache reuse.
Activity
[-]v0.1.10-1: 增加 Instance 能力指纹与兼容性注册模型[/-][+]v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration[/+]