What is bitbop?

BitBop is a research project that explores a new way to train small AI language models so they take up much less memory. Normal AI training keeps every weight in a model as a full precision number, which is accurate but memory heavy. Some newer methods try training with weights limited to just three values, negative one, zero, or positive one, but they still secretly keep a full precision copy of every weight in the background during training, which erases most of the memory savings. BitBop removes that hidden full precision copy entirely. Instead, it tracks a much smaller piece of information per weight, which the author says cuts the memory needed during training by roughly five times compared to common alternatives. This means a mid sized model that would normally require a large, expensive graphics card can instead be trained on a much smaller one, such as a 6 gigabyte consumer GPU. The README is careful to state this is a proof of concept, not a finished or production ready model, and it explains clearly what is and is not being claimed. Quality wise, the ternary models trained this way perform close to a similarly sized regular model on a standard language understanding test, though the author notes the difference is small enough that it counts as a tie rather than a clear win. There is no claim of faster training speed yet, since that would require specialized software that has not been built. The project includes ready made model weights, scripts to reproduce the results, and a written technical report with full details. It is released under the MIT license, and building on public research datasets that should be credited separately.