LLM Compressor
A model-compression library for quantizing weights, activations and KV caches and exporting compressed checkpoints for inference with vLLM.
About LLM Compressor
Serving large models requires fitting their weights and runtime state into the available hardware.
Who it’s for
- Engineers preparing models for vLLM deployment
- Researchers comparing quantization schemes
When to consider it
Consider LLM Compressor when you want configurable quantization recipes and compressed checkpoints for your model-serving hardware.
Tradeoffs & limitations
- Apache-2.0 licensed and requires Python 3.11 or newer; Linux is recommended for GPU support. Hardware acceleration depends on the chosen compression scheme and GPU architecture.
SOTA overview · Documentation-based assessment · Sources & review method
Updates
No updates shared yet.
Discussion
Newest firstAsk a question or share how you use LLM Compressor.
Keep it helpful. Community rules
Loading discussion…
Sources & review method
Documentation-based assessment · Oct 30, 2026 · Prepared with AI assistance; not a hands-on benchmark.
- Official compression library and vLLM examples
- Apache-2.0 licence
- Python and operating-system requirements
- Compression schemes and GPU requirements
Checked by SOTA · AI-assisted documentation review. Selection advice is our assessment; verify current requirements for your deployment.
Import history & original evidence
LLM Compressor official repository
A model-compression library for quantizing weights, activations and KV caches and exporting compressed checkpoints for inference with vLLM.
Based on official pages and announcements checked on 2026-10-30. No hands-on test, performance benchmark or popularity ranking is claimed.