What is local-multimodal-rag?

This is a local pipeline for searching through your own documents, images, and code files without sending anything to the cloud. You point it at folders on your machine, it reads and processes those files, then lets you search them by meaning rather than exact keywords. It handles images, scanned PDFs, Office files like Word or Excel, and source code files, all on your own hardware. The processing works in stages. Images go through text extraction (OCR), and if that fails, a locally-running vision AI model describes the image content as a fallback. PDFs get rendered page by page and read similarly. Office files are converted using LibreOffice before processing. All the extracted text gets broken into chunks, turned into numeric representations called embeddings, and stored in a local SQLite database. When you search, the system compares your query to those stored embeddings using math, then returns a ranked list of matching files and snippets. No external database service is needed. There is a web interface you open in a browser (at port 8100 by default) that works like a search page, plus a settings panel for changing configuration. A REST API is also available if you want to connect this to other software. A command-line tool lets you run each pipeline step manually: scan, extract metadata, process and embed, then search. The README notes this was built from production experience and includes defenses against specific real-world failures: GPU crashes during processing, memory leaks from the OCR library, and subprocess deadlocks. The vision fallback model can be any locally-running AI that speaks the standard OpenAI API format, such as LMStudio or Ollama. Setup requires Python 3.10 or later. LibreOffice and the PaddleOCR library are optional installs that unlock Office and image support respectively. All settings live in a single YAML file where you specify which folders to scan and which AI model to use.