What is nemotron-twotower-mlx?

This project lets you run a large NVIDIA AI language model called Nemotron TwoTower on Apple computers with M-series chips, using a framework called MLX that Apple built for running machine learning on its own hardware. Normally this kind of large model would need an NVIDIA graphics card and specialized software, but this repo converts and adapts it to run natively on a Mac instead. What makes this particular model unusual is how it generates text. Most AI language models write one word at a time, left to right. This one uses a technique the author calls mask diffusion, where a block of upcoming words starts out completely hidden or masked, and the model gradually fills in and refines those words over several passes until it is confident enough to commit to them, then moves on to the next block. The repo actually offers two related things: a standard, ordinary text generation model that works out of the box with existing Mac AI tools, and the fuller two tower diffusion version that needs this repo's own code to run. The model comes in several size and precision options, from a smaller compressed 4-bit version up to a larger, more precise but much heavier bf16 version, with memory requirements ranging from about 17 gigabytes up to around 118 gigabytes depending on the version and precision chosen. Which one you can run depends on how much memory your Mac has. The README recommends starting with the smallest 4-bit versions if you are unsure. Getting started involves having a Mac with Apple Silicon, Python 3.9 or newer, cloning the repository, and installing the listed Python packages. The author reports that the converted model produces identical output, token for token, compared to running the original model on NVIDIA hardware, and provides speed benchmarks measured on a high-end Mac Studio showing how many words per second each version can generate.