prefix-cache

vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm

Freeze Claude Code's prompt prefix so DeepSeek's automatic cache always hits — alignment proxy + coalescing + keepalive, installable as a CC plugin. Measured 64% cheaper on real Claude Code traffic.

Radix trie-based prompt prefix cache optimizer matching longest common prompt token prefixes to eliminate redundant prefill computation and reduce TTFT.

Radix trie-based prompt prefix cache optimizer matching longest common prompt token prefixes to eliminate redundant prefill computation and reduce TTFT.

A prefix-cache profit-and-loss layer for DeepSeek coding agents — wrap your client in two lines, see per-request cache HIT/PARTIAL/MISS, the ¥ each prefix-bust wasted, and the reorder that restores the discount.

ComfyUI Qwen-Image 2.1 multi-GPU inference: Ulysses sequence parallelism for 1/2/4/8 GPUs, T2I, image editing, INT8/FP8, and distributed prefix cache.

DeepSeek缓存优化器 v1.1 — Reasonix四支柱 + 语义压缩 (命中率+30%)

Codex + DeepSeek 前缀缓存医生：本地透明代理，稳定请求前缀提升缓存命中率，附网页控制台

Hermes plugin: DeepSeek prefix-cache wire shaping (Reasonix-inspired)

Persistent KV prefix caching middleware for llama.cpp and OpenAI-compatible agents. Detects stable system/developer prompts and tools, persists slot snapshots, and restores them after model restarts.

A lightweight LLM inference and tool-calling agent system with KV cache experiments.

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

Consumer AI that remembers your whole life — affordable on DeepSeek prefix-cache

ctxfeed is a local MCP project-context backend that shards a whole repo into GLM-5.2's 1M-token window with cache-aware ingest ordering

Pure-PyTorch DeepSeek-V3/K3-style LLM stack where every serving optimization ships with a bit-exactness oracle: disk-tier expert streaming, KV prefix pool, MTP break-even telemetry, oracle-gated config evolution. 60 tests, all CPU-verified.

Correctness-fixed Rust/PyO3 flat-array DFA prefix cache — rewrite of BCR-memory v1 with regression tests for four bugs and an SGLang/vLLM head-to-head harness.

Measures real prefix cache costs on GPU across 3 experimental versions. Key findings: 2.41x speedup with prefix=512; multi-turn speedup grows to 2.06x over 10 turns; batch sharing breakeven at n=2 (prefix=512) vs n=12 (prefix=128); LFU cache with Zipf alpha=2.0 achieves 82% hit rate with only 4 cache slots.

LLM API caching middleware that combines exact-match, semantic, and upstream prefix-cache accounting for OpenAI-compatible providers.

What does per-tenant KV-cache isolation cost you? Measure it on your own traffic — the prompts never leave

Dependency- and session-aware scheduler for agent workloads — C++ decision core compiled to both a pybind11 module and a Go gRPC service, with trace replay validation and an honest negative result.