What is lomo?

LoMo is a research project from researchers at Fudan University, Shanghai Jiao Tong University, and partner institutions. It addresses a specific weakness in AI models that work with both text and images, sometimes called Vision-Language Models. The problem these models have is called carrier sensitivity: if you take a written question and instead render it as an image of text, the model often answers less accurately, even though the meaning is identical. LoMo is a method for reducing that gap. The approach works at the data level, during the training process. LoMo takes a stretch of text within a training example, renders it visually as an image, and then slots that image back into the surrounding text context. The model then sees a sequence that mixes text and a visual version of part of that same text. Training on examples like this pushes the model to treat equivalent information the same way regardless of whether it arrives as written words or as a rendered image. Importantly, LoMo does not require any changes to the model architecture itself, any extra human-labeled data, or any extra steps at inference time when the model is actually being used. It works as a data preparation step that can be added to existing training workflows. The researchers tested it on two different model backbones and reported accuracy gains across several benchmarks. At the time of the release, the project has published a technical report, a project website, and pre-trained model checkpoints on Hugging Face. The code for constructing LoMo training data and the training scripts have not yet been released but are listed as planned. Evaluation was run using the Hugging Face and vLLM inference backends, with ModelScope as the evaluation framework. This repository is primarily aimed at machine learning researchers and practitioners working on multimodal AI. For a non-technical reader, it is a method paper paired with model weights, not a ready-to-use tool for general tasks.