What is arc-encoder?

ARC-Encoder is a tool that compresses text so large language models can process more of it at once, like fitting a longer document into the model's memory without needing a bigger model. The core benefit is letting you feed substantially more context (longer documents, more background material) into models like Mistral 7B or Llama 3.1 8B before hitting memory or speed limits. The way it works is by training a small encoder model that sits in front of the main language model. This encoder takes chunks of text and compresses them into a condensed representation, roughly 8 times smaller than the original, which the main model can still read and understand. Think of it as a translator that summarizes text into a format the big model consumes efficiently. The project comes with pretrained encoders you can download and use directly, or you can train your own from scratch on your preferred model. This is primarily aimed at AI researchers and engineers working on retrieval-augmented generation, question answering, or any application where context length is a bottleneck. For example, if you're building a system that feeds Wikipedia passages into a model to answer trivia questions, ARC-Encoder could let you include more source material without upgrading to a larger, more expensive model. The project includes evaluation scripts for exactly these kinds of QA benchmarks, along with long-context retrieval tasks. The project is built around multi-GPU distributed training and closely integrates with specific open-weights models (Mistral, Llama, Olmo). It comes with pretrained checkpoints trained on 2.6 billion tokens, and the architecture shares components between the encoder and decoder, which keeps things modular. The retrieval utilities are licensed separately under a non-commercial license, which is worth noting if you're considering commercial use. This is research code accompanying a published paper rather than a polished product, so expect some manual setup and configuration.