What is llm-case-benchmark?

This project, called Qiaomu LLM Arena, lets people compare how different AI language models answer the same prompt, side by side. Instead of testing models yourself from scratch, you open a case in the app and see a snapshot of results that an administrator already ran and reviewed, covering multiple providers and their sub models at once. You can then look at the original prompt, switch between older saved versions of the results, or rerun the case yourself if you want fresh output. The tool supports two kinds of tasks: plain text writing tasks shown as Markdown, and front end tasks where the model generates HTML or SVG code that gets rendered and previewed live in the browser. Results stream in one card at a time, so you do not have to wait for every model to finish before you start reading. Each result can be rerun individually, copied, opened full screen, exported, or scored by hand. There are two separate tracks for history. Everyday runs that a regular visitor makes are saved just for that person to look back on later. Administrator approved runs are different: once reviewed and published, they become an immutable snapshot with a version number, and older versions stay available for public comparison even after a newer one is published. To run it yourself you need Node.js version 22 or newer. After cloning the repository, you install dependencies and start the server, then open it in a browser. A fresh install has no admin password by default. Admin login only turns on once you set two secret environment variables yourself, one for the admin password and one used to encrypt any API keys the admin configures. Regular visitors who bring their own API key have it stored only in their own browser, never uploaded to the server. The project is self hosted using Node.js, Express, and a local SQLite database, with example deployment configs for systemd and Nginx included. It is released under the MIT license.