whatisgithub

What is unibench?

facebookresearch/unibench — explained in plain English

Analysis updated 2026-07-17 · repo last pushed 2026-06-18

★ 228Jupyter NotebookAudience · researcherComplexity · 3/5ActiveSetup · moderate

In one sentence

A benchmarking toolkit that tests vision-language AI models against 40+ tasks and 60+ pre-built models, letting you compare results or evaluate your own models and datasets.

Mindmap

mindmap
  root((repo))
    What it does
      Benchmarks vision language models
      Compares 60 plus models
      Runs 40 plus tasks
    Tech stack
      Jupyter Notebook
      Python
      Embeddings
    Use cases
      Compare model performance
      Test custom models
      Add new benchmarks
      View precomputed results
    Audience
      AI researchers
      Product teams
    Setup
      Install via pip
      CLI or Python scripts

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Compare how 60+ existing vision-language models perform across 40+ established benchmarks without writing evaluation code.

USE CASE 2

Test a custom vision-language model against UniBench's standard benchmark suite to see how it stacks up.

USE CASE 3

Add a new custom dataset as a benchmark and evaluate all existing models against it automatically.

USE CASE 4

Download and analyze precomputed results from researchers to compare models without running new tests.

What is it built with?

Jupyter NotebookPython

How does it compare?

facebookresearch/unibenchfacebookresearch/sparshkrishnaik06/text-summarization-nlp-project
Stars228228198
LanguageJupyter NotebookJupyter NotebookJupyter Notebook
Last pushed2026-06-182025-02-272024-08-17
MaintenanceActiveStaleStale
Setup difficultymoderatehardhard
Complexity3/54/54/5
Audienceresearcherresearcherdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 1h+

Full evaluation suite requires downloading multiple pretrained models and benchmark datasets.

Not specified in the explanation.

Copy-paste prompts

Prompt 1
How do I install UniBench and run its full benchmark suite against a pretrained CLIP-style model?
Prompt 2
Show me how to add my own image-text dataset as a new benchmark in UniBench and evaluate existing models on it.
Prompt 3
How do I download UniBench's precomputed results to compare two vision-language models without running new evaluations?
Prompt 4
Explain how UniBench compares image and text embeddings to score a model's classification accuracy.

Frequently asked questions

What is unibench?

A benchmarking toolkit that tests vision-language AI models against 40+ tasks and 60+ pre-built models, letting you compare results or evaluate your own models and datasets.

What language is unibench written in?

Mainly Jupyter Notebook. The stack also includes Jupyter Notebook, Python.

Is unibench actively maintained?

Active — commit in last 30 days (last push 2026-06-18).

What license does unibench use?

Not specified in the explanation.

How hard is unibench to set up?

Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.

Who is unibench for?

Mainly researcher.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.

What are similar repos?