What is industrybench?

IndustryBench is a dataset and evaluation toolkit designed to test how well large language models (LLMs, AI systems that understand and generate text) understand specialized knowledge from industrial and manufacturing sectors. The problem it addresses is that most AI benchmarks test general knowledge or programming ability, leaving it unclear how well these models handle technical industrial topics like procurement standards and product specifications. The dataset contains 2,049 question-and-answer items drawn from Chinese national standards (GB/T) and structured industrial product records. Each item has been translated into English, Russian, and Vietnamese by human reviewers, all tied back to the same original Chinese source material. Items are tagged across 7 capability dimensions, 10 industry categories, and three difficulty levels. To score a model's performance, an evaluator feeds a question to the model without giving it any reference material (closed-book style), then uses a separate calibrated AI judge to score the response on a scale of 0 to 3. There is also a safety check that can penalize answers that contradict the source document. You would use this project if you are a researcher wanting to measure how well an AI model handles industrial domain questions, especially across multiple languages. The evaluation script requires Python 3.10 or later and an OpenAI-compatible API endpoint, meaning it works with many hosted AI services. The dataset itself is freely available on Hugging Face without needing to clone the repository.