What is litellm?

LiteLLM solves a common headache for teams building AI features: every AI provider (OpenAI, Anthropic, Google, Amazon, and dozens more) has its own way of accepting requests and returning responses. If you've ever wanted to switch from one model to another, or use several at once, you know it means rewriting code, managing different authentication styles, and handling different error types. LiteLLM gives you a single, consistent way to talk to over 100 different AI models, so you write your code once and can swap providers without touching it again. You can use it two ways. The first is as a Python library that you install directly in your project, you just tell it which model you want (say, "anthropic/claude" or "openai/gpt-4o") and it handles the translation behind the scenes. The second is as a proxy server, which is a standalone service that sits between your app and the AI providers. Your application talks to this proxy using the standard OpenAI format, and the proxy figures out how to route each request to the right provider. The proxy also adds production features like spend tracking, virtual API keys for different team members, load balancing across providers, guardrails, and an admin dashboard. This is built for teams and companies that are using AI in real products and need reliability, cost control, and flexibility. For example, if you're a startup that starts on OpenAI but later wants to try Anthropic or a cheaper model for certain tasks, LiteLLM lets you make that switch without rewriting your application. It's also useful if different teams in your organization need their own API keys and you want to track how much each one is spending. The project lists companies like Netflix, Stripe, and others among its adopters. A few things stand out. The proxy is designed to be fast, the project reports about 8 milliseconds of added latency at high traffic volumes. It also supports newer patterns like connecting to external tool servers (MCP) and agent-to-agent communication, so it's keeping up with where the AI ecosystem is heading. You can self-host it or use a hosted enterprise tier if you don't want to manage infrastructure.