Run a small 7B coding model fully offline on a 16 GB machine while keeping high accuracy on verifiable coding and math tasks.
Study how automated test execution and repair loops can substitute for larger model size on checkable problems.
Benchmark a base model's coding performance before and after adding a verification and repair layer.
| vexp-ai/horizon | alex-nlp/denoiserl | bytedance-seed/cola-dlm | |
|---|---|---|---|
| Stars | 35 | 35 | 35 |
| Language | Python | Python | Python |
| Setup difficulty | hard | hard | hard |
| Complexity | 5/5 | 5/5 | 4/5 |
| Audience | researcher | researcher | researcher |
Figures from each repo's GitHub metadata at analysis time.
Requires a local 16GB machine, a 7B base model, and llama.cpp with a matching quantized drafter model.
Horizon is a research project that adds a verification layer on top of small, open AI coding models so they can run entirely offline on a regular 16 GB computer. Instead of building a bigger model, the project wraps an existing 7 billion parameter model with tools that check its work: sandboxed test execution, symbolic math checking, and an automatic repair loop that reruns failed code and tries again with the concrete error fed back in. The core claim is that verification, not model size, is what recovers missing capability. In the project's own measurements, adding this layer to a small local model brought its coding and math scores close to those of a much larger 671 billion parameter model it was distilled from, on tasks where correctness can be checked automatically, like code with tests or solvable math problems. The team is upfront that this only applies to verifiable tasks, not general purpose coding, and they published results on multiple different base models to show the layer helps regardless of which one is used. A notable design piece is a tiny helper model used for speculative decoding, a technique that speeds up text generation by having a small model draft likely next words for the larger model to confirm. The project trained its own small drafter model cheaply so it matches the target model's vocabulary exactly, and found this speeds up code generation meaningfully while offering little benefit during reasoning steps. Horizon comes from the team behind vexp, a local first context engine for AI coding, and is released under the MIT license. The project documents its methodology, decontamination steps, and even failed experiments openly, including quarantined invalid runs rather than deleting them.
Horizon is an offline verification layer that wraps a small open-weight coding model with test execution and repair loops, bringing it near the accuracy of a much larger model on checkable tasks.
Mainly Python. The stack also includes Python, llama.cpp.
Permissive MIT license, use freely for any purpose including commercial use.
Setup difficulty is rated hard, with roughly 1day+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.