What is localaihotswap?

LocalAIHotSwap is a small control layer that sits in front of a locally running AI model server called llama.cpp. Its job is to let you swap out one AI model for another while keeping your conversation history intact, without needing to restart everything from scratch. When you run AI models locally, each model is a large file that must be loaded into your GPU's memory before it can respond to questions. Switching between models normally means losing whatever context the conversation had built up. This project solves that by acting as a middleman: it stores your conversation messages and any preset instructions separately from the model server itself, then coordinates the unloading and reloading of model files in a way that keeps memory usage low. The default approach, called zero_overlap, unloads the current model completely before loading the next one. This avoids having two large model files in GPU memory at the same time, which matters a lot on machines with limited VRAM. Once the new model is loaded, the saved conversation history is fed back in so the chat can continue. The project is honest that this replay step does take time, and that there is no shortcut around it. From the user's perspective, the proxy accepts chat requests in the same format that OpenAI's API uses, so existing tools that speak that format work without changes. You can also send a switch command to change which model is active, and the proxy makes sure any in-progress responses finish before the old model is unloaded. Configuration is handled through a TOML file where you point to your local model files and set limits like how many messages to keep in history, how large the context window can be, and how long to wait for active requests to drain before switching. The project is aimed at developers running experiments across multiple local models who want to compare outputs without constantly losing their session state.