What is llama.cpp-fusion?

llama.cpp-fusion is a fork of the popular llama.cpp project, focused on running very large AI language models, especially Mixture of Experts models like DeepSeek and Qwen3, on hardware that does not have enough graphics card memory to hold the whole model at once. It brings together several techniques that let a computer combine its CPU, its GPU or GPUs, and multiple processors to squeeze more speed out of large models. One feature, called Thread Copy, is built for computers with two or more physical CPUs, splitting the work into groups of processor cores so each group stays close to its own memory bank, which cuts down on slow cross-processor data transfers. Another feature, called MoE Cache, keeps the most frequently used parts of the model on the graphics card, so the computer does not have to keep copying them back and forth from system memory, which the readme reports can raise generation speed significantly. The project also supports splitting a model across several graphics cards at once, running only certain parts of the model on the CPU while attention runs on the GPU, and a technique called Multi-Token Prediction, where the model tries to guess several words ahead at once to speed up generation. There is also a feature that predicts, based on how the model processes the initial prompt, which experts inside the model are likely to be needed soon, so they can be loaded in advance. The readme includes detailed benchmark tables and command line examples run on a specific test machine with four graphics cards and two server-grade processors, and gives command line flags and environment variables for turning each feature on. Because it deals with multi-GPU setups, NUMA-aware processors, and large models measured in tens of gigabytes, this project is aimed at people already comfortable running and tuning large language models locally, not beginners.