Run DeepSeek V4 Flash or PRO locally on a high-memory Mac, NVIDIA, or AMD system.
Use SSD streaming to run a model larger than available RAM would normally allow.
Integrate the built-in server API or CLI with a coding agent workflow.
Generate and test custom GGUF quantizations for DeepSeek V4 models.
| mitkox/ds4-zgx-gb10 | amcchord/classicmac | omgrant/ft9201-libfprint | |
|---|---|---|---|
| Stars | 21 | 21 | 21 |
| Language | C | C | C |
| Setup difficulty | hard | moderate | hard |
| Complexity | 5/5 | 3/5 | 5/5 |
| Audience | developer | general | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires 96GB+ RAM class hardware (Mac Metal, NVIDIA CUDA, or AMD Strix Halo) and specially prepared GGUF model files.
DwarfStar is a small, purpose built inference engine, meaning software that runs an AI language model and produces its responses, built specifically to run DeepSeek V4 Flash and, on machines with a lot of memory, DeepSeek V4 PRO. Unlike general purpose tools that can load many different models, this project deliberately supports only these specific models, handling their loading, prompt formatting, tool calling, and memory state in a way tailored to how they work. It supports three types of computer hardware: Apple computers with Metal graphics starting around 96 gigabytes of memory, NVIDIA GPUs including systems called DGX Spark, and AMD based systems using a chip design called Strix Halo, found in machines like the Framework Desktop. The project openly credits an existing project called llama.cpp and its GGML library as the foundation that made this possible, even though DwarfStar does not directly link against that library, and keeps its original creators credited under the MIT license. A key design idea here is treating a model's working memory, called the key value cache, as something that can live on a fast solid state drive rather than only in RAM. This means someone with less memory than a model normally requires can still run it, trading some speed for flexibility, instead of simply being unable to run it at all. The README states plainly that this project was built with heavy assistance from an AI system alongside human developers leading the design, testing, and debugging, and says people who are uncomfortable with AI assisted code should not use it. It also warns that the code is beta quality, since the project is only a few days old at the time of writing, and that a companion coding agent feature is even earlier, alpha quality. There is a known bug where running the plain CPU code path on current macOS versions can crash the operating system. This tool only works with specially prepared model files matching DeepSeek V4 Flash or PRO, not arbitrary model files. The full README is longer than what was shown.
A narrow, high-performance inference engine built specifically to run DeepSeek V4 Flash and PRO models on Mac, NVIDIA, or AMD hardware.
Mainly C. The stack also includes C, CUDA, Metal.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice, some components are retained from llama.cpp under the MIT license.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.