OptiLLM
An OpenAI-compatible inference proxy that applies techniques such as best-of-N, planning and mixtures of agents to model requests.
About OptiLLM
Developers want to experiment with inference strategies around existing models without retraining them.
Who it’s for
- Developers testing inference strategies
- Researchers composing multiple model calls
When to consider it
Consider OptiLLM when you want to select and combine inference techniques through an OpenAI-compatible proxy.
Tradeoffs & limitations
- Apache-2.0 licensed. You must run the proxy and configure a model backend; the hosted-provider quick start requires your own API key.
SOTA overview · Documentation-based assessment · Sources & review method
Updates
No updates shared yet.
Discussion
Newest firstAsk a question or share how you use OptiLLM.
Keep it helpful. Community rules
Loading discussion…
Sources & review method
Documentation-based assessment · Oct 13, 2026 · Prepared with AI assistance; not a hands-on benchmark.
- Official repository overview
- Official repository metadata snapshot
- Official license file
- MARS author-ID fix — October 11, 2026
- Official organization and owner domain
- Official maker website
- Product mark from the official README logo
Checked by SOTA · AI-assisted documentation review. Selection advice is our assessment; verify current requirements for your deployment.
Import history & original evidence
OptiLLM on GitHub
An OpenAI-compatible inference proxy that applies techniques such as best-of-N, planning and mixtures of agents to model requests.
Based on official pages and announcements checked on 2026-10-13. No hands-on test, performance benchmark or popularity ranking is claimed.
