Speed up the attention computation inside a language model running on NVIDIA SM100 GPUs.
Swap in sparse top-k attention to reduce compute for long-context inference.
Pull the compiled kernels directly through the Hugging Face kernels library instead of building from source.
Install and build the project from source for development or research on attention kernels.
| minimax-ai/msa | feder-cr/invisible_playwright | pixel-talk/pear | |
|---|---|---|---|
| Stars | 258 | 258 | 257 |
| Language | Python | Python | Python |
| Setup difficulty | — | moderate | hard |
| Complexity | — | 3/5 | 5/5 |
| Audience | researcher | developer | researcher |
Figures from each repo's GitHub metadata at analysis time.
MSA, short for MiniMax Sparse Attention, is a library of specialized GPU code for one specific piece of how large AI language models work internally: the attention step, which is the part of the model that figures out which pieces of input text matter most to each other. This project provides fast, low-level implementations of that step, written to run on a very specific and recent generation of NVIDIA graphics card, referred to in the README as SM100. The library offers two related versions of this attention code. A dense version processes every piece of data in full, following an approach known as FlashAttention. A sparse version instead first identifies the most important chunks of data using a top-k selection method, meaning it keeps only the highest-scoring pieces and skips the rest, which can make the whole process noticeably faster since less work is done per step. The project bundles both approaches as two separate compiled code paths inside one Python package, along with a bridge module that connects the two. Because this is low-level GPU code, using it comes with strict requirements: an NVIDIA SM100 graphics card, the CUDA toolkit installed and available on the system, Python 3.10 or newer, and a Linux system, since other operating systems are not tested. The code is compiled just-in-time the first time it runs, so the very first use can take anywhere from thirty seconds to a few minutes while it compiles, which the README explicitly notes is expected behavior rather than the program hanging. For a simpler starting point, the README shows that MSA kernels can also be pulled in through the separate Hugging Face kernels library with a short Python snippet, rather than requiring a full manual install and build. Installation instructions cover both a standard install and an editable, development-focused install using pip, and note that the project includes a required third-party dependency for its lower-level components that must be pulled in alongside the main code. The project is written in Python with underlying CUDA code, and it is released under the MIT license, meaning the project's own code can be freely used, modified, and redistributed, though some bundled third-party components carry their own separate license terms.
MSA is a GPU code library that speeds up the attention step inside AI language models on NVIDIA's newest chips, offering both a full dense version and a faster sparse version.
Mainly Python. The stack also includes Python, CUDA, NVIDIA CUTLASS.
MIT license for the project's own code: use freely for any purpose, including commercial use, as long as you keep the copyright notice. Bundled third-party code keeps its own separate license.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.