What is neuraltalk2?

karpathy/neuraltalk2 — explained in plain English

Analysis updated 2026-06-26

★ 5,579Jupyter NotebookAudience · researcherComplexity · 4/5Setup · hard

Mindmap

mindmap
  root((neuraltalk2))
    What It Does
      Image to text
      Caption generation
      Sentence output
    Tech Stack
      Lua and Torch
      CUDA GPU
      Python scripts
    Use Cases
      Photo captioning
      Research reference
      Custom training
    Audience
      ML researchers
      Students
    Setup
      Pretrained model
      GPU required
      2-3 days training

mindmap root((neuraltalk2)) What It Does Image to text Caption generation Sentence output Tech Stack Lua and Torch CUDA GPU Python scripts Use Cases Photo captioning Research reference Custom training Audience ML researchers Students Setup Pretrained model GPU required 2-3 days training

Click or tap to explore — scroll the page freely

What do people build with it?

USE CASE 1

Generate automatic text captions for a photo collection using the included pretrained model

USE CASE 2

Study how a CNN and RNN are combined for image-to-text tasks as a research reference

USE CASE 3

Fine-tune the image captioning model on a custom dataset of labeled photos

USE CASE 4

Understand early deep-learning image captioning through a readable Torch implementation

What is it built with?

LuaTorchCUDAPythonJupyter Notebook

How does it compare?

	karpathy/neuraltalk2	dibgerge/ml-coursera-python-assignments	probml/pml-book
Stars	5,579	5,566	5,563
Language	Jupyter Notebook	Jupyter Notebook	Jupyter Notebook
Setup difficulty	hard	easy	easy
Complexity	4/5	1/5	1/5
Audience	researcher	researcher	researcher

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1day+

Requires an NVIDIA GPU with CUDA and the Lua Torch framework, non-trivial to install on modern systems, author recommends newer alternatives for production use.

So what is it?

NeuralTalk2 is a research project that automatically generates text captions for images. You give it a photo and it produces a sentence describing what is in the picture, for example "a dog sitting on a grass field" or "two people riding bicycles on a street." This kind of task is called image captioning. The system works by combining two components. A convolutional neural network looks at the image and extracts a compact summary of its visual content. A recurrent neural network then takes that summary and generates a sentence word by word. The two components are trained together on large datasets of images paired with human-written captions. This release was written by Andrej Karpathy and is significantly faster than an earlier version called NeuralTalk. It runs on a graphics card, processes images in batches, and can fine-tune the image recognition component during training. According to the README, good models can be trained in two to three days with this setup, and the pretrained model scored competitively on a public image captioning benchmark. The code is written in Lua using a framework called Torch, which was a popular research tool at the time. It requires a compatible NVIDIA graphics card for training. A pretrained model is available for people who want to run captions on their own images without training from scratch. The author notes that Google Brain later released a similar model that performs better, and suggests using that for production purposes. This repository is kept up for educational reference and as a Torch implementation of the approach.

Copy-paste prompts

Prompt 1

I have a folder of JPEG images and want to run the neuraltalk2 pretrained model on each one. Write the Lua command to process a directory of images and print a caption for each.

Prompt 2

Explain step by step how neuraltalk2 combines a CNN and an RNN to produce image captions, what does the CNN output, and how does the RNN turn that into a sentence?

Prompt 3

I want to train neuraltalk2 on my own dataset of images with human-written captions. What format does the data need to be in, and what is the rough training command?

Prompt 4

Walk me through setting up the Torch environment on Ubuntu so I can run the neuraltalk2 pretrained model on my own photos.

Frequently asked questions