LMDeploy
A toolkit for compressing and serving language and vision-language models with TurboMind and PyTorch inference engines.
About LMDeploy
Teams deploying open models need inference serving, quantization and configuration for different model architectures.
Who it’s for
- ML engineers serving language and vision-language models
- Infrastructure teams deploying models across multiple GPUs or machines
When to consider it
Consider LMDeploy when you want quantization, offline inference and an OpenAI-compatible server in one model-deployment toolkit.
Tradeoffs & limitations
- Apache-2.0 licensed; the README recommends Python 3.10–3.13 and states that PyPI wheels from v0.13.0 target CUDA 12.8. Model and data-type support differ between its inference engines.
SOTA overview · Documentation-based assessment · Sources & review method
Updates
No updates shared yet.
Discussion
Newest firstAsk a question or share how you use LMDeploy.
Keep it helpful. Community rules
Loading discussion…
Sources & review method
Documentation-based assessment · Oct 30, 2026 · Prepared with AI assistance; not a hands-on benchmark.
- LMDeploy official repository, serving features and installation requirements
- LMDeploy tutorials and deployment interfaces
- Apache-2.0 license
- Official repository commit history
- Maintainer organization and institutional email domain
Checked by SOTA · AI-assisted documentation review. Selection advice is our assessment; verify current requirements for your deployment.
Import history & original evidence
LMDeploy official repository
A toolkit for compressing and serving language and vision-language models with TurboMind and PyTorch inference engines.
Based on official pages and announcements checked on 2026-10-29. No hands-on test, performance benchmark or popularity ranking is claimed.