What is litellm?

LiteLLM lets you talk to over 100 different AI models, like those from OpenAI, Anthropic, Google, and Amazon, through a single, consistent interface. Instead of learning and managing a different set of code instructions for every AI provider you might want to use, you write your code once. If you ever decide to switch from OpenAI's GPT-4 to Anthropic's Claude, you don't have to rewrite your application, you just change the model name. The project is used by major companies like Stripe and Netflix. You can use it in two ways: as a Python library directly inside your application's code, or as a central "gateway" server that sits between your apps and the AI providers. The gateway approach is particularly useful for teams. It adds a management layer that handles things like tracking how much money you are spending across different AI models, balancing incoming requests across multiple providers to avoid slowdowns, and issuing virtual API keys so you can control exactly who on your team can use which models and at what cost. A startup founder building an AI feature might use this so their engineering team can easily experiment with new models the moment they launch, without waiting for someone to integrate a new software kit. A product manager might appreciate the built-in dashboard for seeing exactly how much the company's AI features cost each month. It also includes guardrails, which help prevent your application from returning unwanted or inappropriate responses. Beyond standard text and chat models, it also supports working with AI agents and external tools through emerging standards like A2A and MCP. This means you can use it to route requests to specialized AI agents or let models interact with outside software. It is open source, can be self-hosted on your own infrastructure, and is designed to add virtually no noticeable delay to your AI requests, making it suitable for production-level apps.