What is llmix?

LLMix is a middleware layer, meaning a piece of software that sits between your application code and the AI services you call, such as OpenAI, Anthropic, or Gemini. Instead of hard coding which AI model your app talks to, LLMix lets you define model behavior in configuration files called MDA presets, so you can change which model runs, or how it behaves, by editing a config file and reloading, with no need to redeploy your application. The problems it addresses tend to show up once real traffic hits an app: API keys getting rate limited, a provider slowing down, many users sending the same request at the same moment, and different services in a company needing the same reliability behavior. To handle this, LLMix adds a response cache so repeated identical requests can skip the AI call entirely, automatic retry logic for failed calls, a circuit breaker that stops repeatedly hitting a provider that is struggling, rotation across multiple API keys, and a feature that collapses duplicate simultaneous requests into a single call. It is built to work the same way across Python, TypeScript, and Rust, so teams using more than one programming language can rely on identical behavior everywhere. Your existing OpenAI, Anthropic, or Gemini client code stays in place. LLMix wraps around it rather than replacing it, meaning you still write your prompts and keep your existing SDK calls exactly as before. In short, you keep control of the prompt and the SDK you already use, while LLMix takes care of the reliability layer built around every AI call it wraps. The full README is longer than what was provided.