What is comfyui-gguf?

ComfyUI-GGUF is an add-on for ComfyUI, a node-based interface for generating AI images. The add-on lets ComfyUI load AI models stored in a file format called GGUF, which originated from a project called llama.cpp and is designed for running large AI models on consumer hardware. The core problem this solves is VRAM. High-quality AI image generation models can be too large to fit on the graphics card memory of most consumer GPUs. GGUF files are quantized versions of those models, meaning the numbers inside them are stored at lower precision to shrink the file size and memory footprint. For some model types, specifically newer transformer-based image models like Flux, quantization works well without a major quality loss. Older models based on a different architecture (called conv2d or UNET) do not benefit as much from quantization, so this add-on is most useful for the newer model families. To use it, you clone the repository into the custom_nodes folder inside your ComfyUI installation and install one Python dependency. After that, a new loader node called "Unet Loader (GGUF)" becomes available in ComfyUI. You place your .gguf model files in the unet models folder and connect that loader into your workflow wherever you would normally load a diffusion model. The author also provides pre-quantized versions of several popular models on HuggingFace, so you do not need to create the GGUF files yourself. Support for loading a quantized version of the T5 text encoder is included as well, via separate loader nodes. These nodes accept both GGUF files and the standard safetensors or bin formats, so you can mix and match depending on what you have. The README notes that LoRA loading is experimental but functional, and that macOS users on Apple Silicon may need a specific version of the PyTorch library due to a buffer size bug in newer nightly builds. The project is described as a work in progress.