What is fork-tensorzero?

TensorZero is a complete platform for building and running AI applications that use large language models (LLMs) in production. Think of it as a single toolkit that handles everything you need: connecting to different AI services, tracking how well your AI is performing, automatically improving your AI over time, and safely testing new versions before shipping them. Instead of stitching together five different tools from different vendors, you get one unified system. At its core, TensorZero acts as a smart middleman between your application and LLMs from providers like OpenAI, Anthropic, Google, and others. It does this through a "gateway", essentially a fast, lightweight proxy that sits between you and the AI providers. You point your code at TensorZero instead of directly at OpenAI or whoever else, and it routes your requests efficiently. This single connection point means you can switch models, add fallbacks if one provider goes down, or load-balance across multiple providers without changing your application code. The gateway is built in Rust, a programming language known for speed, so it adds almost no delay (less than a millisecond even under heavy load). But the real power is what happens after you get an answer back from an AI model. TensorZero stores detailed records of every inference, what you asked, what the AI said, and how well it performed. You can see this data in a built-in dashboard or pull it programmatically. More importantly, you can use this production data to automatically improve your AI: optimize the prompts you send, fine-tune the models you're using, or switch to cheaper/faster models that work just as well. You can also run experiments, like A/B testing two different prompts to see which one users prefer, before rolling out changes to everyone. Who uses this? Companies ranging from startups building AI products to Fortune 500 companies handling mission-critical AI applications. Someone might use it to build a customer service chatbot where they can track answer quality, automatically improve it based on customer satisfaction scores, and safely test new prompts on a small subset of users before going live. Another team might use it to run data extraction across millions of documents, optimizing both the AI model and the cost per document as they learn what works. The platform is open-source and self-hosted, meaning it lives on your own servers, you're not sending your data to a third-party SaaS. This makes it especially appealing for companies with strict privacy or compliance requirements.