LMCache
A KV-cache management layer that stores, reuses and transfers language-model inference state across requests, serving engines and storage backends.
About LMCache
Repeated prompt processing and limited GPU memory make it difficult to reuse inference state across requests and serving workers.
Who it’s for
- Engineers operating self-hosted LLM inference
- Teams serving long-context conversations and RAG workloads
When to consider it
Consider LMCache when you need to offload and share KV caches across an inference-serving stack with configurable storage and transport backends.
Tradeoffs & limitations
- Apache-2.0 licensed. Installation must match the inference engine and hardware runtime; the documented ROCm wheels are tied to specific PyTorch, Python and ROCm versions, with a source build required for other combinations.
SOTA overview · Documentation-based assessment · Sources & review method
Updates
No updates shared yet.
Discussion
Newest firstAsk a question or share how you use LMCache.
Keep it helpful. Community rules
Loading discussion…
Sources & review method
Documentation-based assessment · Oct 30, 2026 · Prepared with AI assistance; not a hands-on benchmark.
- Official product overview
- Official repository, capabilities and setup
- Apache-2.0 code licence
- Installation and runtime requirements
Checked by SOTA · AI-assisted documentation review. Selection advice is our assessment; verify current requirements for your deployment.
Import history & original evidence
LMCache official website
A KV-cache management layer that stores, reuses and transfers language-model inference state across requests, serving engines and storage backends.
Based on official pages and announcements checked on 2026-10-30. No hands-on test, performance benchmark or popularity ranking is claimed.
LMCache official repository and licence
LMCache code is Apache-2.0 licensed; capabilities and setup were checked in the official repository.
Based on official pages and announcements checked on 2026-10-30. No hands-on test, performance benchmark or popularity ranking is claimed.
LMCache official installation documentation
Apache-2.0 licensed. Installation must match the inference engine and hardware runtime; the documented ROCm wheels are tied to specific PyTorch, Python and ROCm versions, with a source build required for other combinations.
Based on official pages and announcements checked on 2026-10-30. No hands-on test, performance benchmark or popularity ranking is claimed.