What is rupixel?

rupixel is a search tool that finds documents based on what you mean, not just the exact words you type. It works in two ways. First, it can search the text of your documents, the classic approach. Second, it can search by how a page looks, taking a screenshot of each page and searching the pictures. This second method matters when text is locked inside a scan, an image, a table, or a chart that ordinary text search cannot read. The project includes a real-time video search demo that runs entirely in your browser. It samples a live feed from your camera or screen a few times per second, skips frames that barely changed, and turns the rest into searchable representations using a model called CLIP. It can even generate live captions for each keyframe using a vision model called Qwen3-VL, narrating what happens on screen word by word. Your API key stays in your browser tab and is cleared when you close it, or you can run a proxy server so the key never reaches the browser at all. To explain the core idea without jargon: each document gets turned into a long list of numbers that captures what it is about. Documents with similar meanings get similar numbers. When you ask a question, it gets turned into numbers the same way, and the system finds the documents whose numbers are closest. A small model called MiniLM handles text, while CLIP handles both pictures and text in the same number space, which lets you type words and get back images. The creators tested it on 8 real Wikipedia pages using 8 questions phrased in everyday words that avoid reusing the page's vocabulary. Both text and visual search got the right page every time on the Rust side, though the browser version got 7 out of 8. The authors are honest that these test pages are easy, and the real value of visual search shows up on harder material like scanned paper, tables, and charts, which is the next thing they plan to build. The project is early-stage work in progress, but everything described actually runs with no placeholder code. You can try a command-line helper with npx rupixel, which explains the project and runs the benchmark harness. The full README is longer than what was shown.