What is vllm-omni?

vLLM-Omni is a framework for running AI models that can work with text, images, video, and audio at the same time, sometimes called omni-modality models. It is an extension of vLLM, a widely used open-source tool for running large language models efficiently at scale. Where vLLM focused on text-in, text-out models, vLLM-Omni expands that to handle models that accept any mix of inputs and produce outputs that can include generated images, audio, or video alongside text. The framework is designed for developers and companies that need to host these AI models as a service, letting many users send requests simultaneously. It exposes an OpenAI-compatible API, meaning applications already built to talk to OpenAI's services can switch to using vLLM-Omni without major code changes. Beyond the language model side, vLLM-Omni also supports diffusion models, which are a different type of AI architecture used for image and video generation (rather than generating tokens one at a time, they refine images from random noise). Supporting both types in one framework lets a single deployment serve a broader range of model types. For running on multiple GPUs or across multiple machines, the framework provides several parallelism strategies. It supports popular open-source models from Hugging Face, including Qwen-Omni and similar multimodal models. Streaming outputs are supported so responses can start arriving before the full generation is complete. The project is backed by a published research paper on the architecture, released under the Apache 2.0 license. It is actively maintained and receives regular versioned releases. Documentation, a quickstart guide, and a list of supported models are available at the project's documentation site.