What is cardputer-ai?

Cardputer AI puts a small AI chatbot directly onto a tiny handheld device called the M5Stack Cardputer, which runs on an ESP32-S3 microcontroller. The entire language model lives inside the firmware that gets flashed to the device, so the chatbot works completely offline with no internet connection, no Wi-Fi, no SD card, and no external server calls. Once the firmware is installed, the device is a standalone chatbot. The AI model used is a 3-million-parameter language model trained on simple English stories and conversations. It generates about 7 words per second on the hardware. Because the microcontroller has very little memory (around 280 kilobytes of usable working memory), the project uses a heavily compressed version of the model weights stored in a format called Q4_0, which reduces each number to 4 bits. The vocabulary was also pruned from about 50,000 possible word pieces down to roughly 12,000 to reduce memory further. All of this fits into a firmware file just under 2 megabytes. The device runs in three modes. Chat mode is the default: you type something, the device replies, and it remembers the last few exchanges. Story mode takes a short description and writes a brief story based on it. Raw mode passes your input directly to the model without any formatting, which is useful for testing. The README is honest about the model's limits: it speaks only simple English, has no real-world knowledge, and can only hold about 80 tokens (roughly three short exchanges) in memory at once. The firmware works on two versions of the Cardputer hardware. It detects which one it is running on at startup and loads the correct keyboard driver automatically, so the same firmware file covers both. For people who want to retrain the chat model from scratch, the repository includes Python tools to download the base model, prepare training data from a public dialogue dataset, run the fine-tuning, and then convert the result into the C++ source files the firmware needs. The fine-tuning takes about 30 minutes on an Apple Silicon Mac. Building and flashing the firmware uses either PlatformIO or the standard ESP-IDF toolchain.