What is marlin-2b-breakdown?

This repository documents the internal architecture of Marlin-2B, an AI model that understands both video and text, and provides scripts for shrinking it down and running it. The model combines a vision component that processes video frames with a text based decoder that alternates between two kinds of attention mechanisms, a faster linear style and a more traditional style, repeated across a stack of layers before producing an output. Because a model like this can be large and slow to run on ordinary hardware, the repository includes scripts for quantization, a process that reduces the precision of the model's internal numbers to make it smaller and faster, at some cost to accuracy. Three different quantization methods are supported, each with its own command and settings for things such as how much GPU memory to use during the process. One of these methods is described as still experimental for this particular model, since it can currently produce corrupted output text. Once a version of the model has been prepared, the repository provides a script to run it directly on a video file, either to describe everything happening in the video over time or to search for a specific event within it, such as a particular action taking place. There is also a chunking pipeline for processing longer videos in segments with some overlap between chunks. For deployment, the project can launch the model behind either of two existing serving frameworks, vLLM or SGLang, exposing it through a standard API that other applications can send requests to in the same format used by well known chat AI services. A dry run option lets you preview the exact command that would be used without actually starting a server, which is useful for verifying configuration before committing GPU resources to it.