What is ng-video-lecture?

karpathy/ng-video-lecture — explained in plain English

Analysis updated 2026-06-26

★ 4,703PythonAudience · researcherComplexity · 2/5Setup · moderate

Mindmap

mindmap
  root((ng-video-lecture))
    What it does
      Lecture companion code
      GPT implementation
      Step by step history
    Tech stack
      Python
      PyTorch
    Concepts covered
      Language models
      Transformers
      Training loop
    Audience
      ML learners
      Students
      Curious developers

mindmap root((ng-video-lecture)) What it does Lecture companion code GPT implementation Step by step history Tech stack Python PyTorch Concepts covered Language models Transformers Training loop Audience ML learners Students Curious developers

Click or tap to explore — scroll the page freely

What do people build with it?

USE CASE 1

Follow along with Karpathy's Zero To Hero lecture series by running the code cells while watching the video to see each piece of GPT built live.

USE CASE 2

Walk through the git commit history to replay exactly how the GPT model was assembled step by step during the lecture.

USE CASE 3

Use the code as a minimal, readable GPT reference to understand the core architecture before studying larger implementations like nanoGPT.

What is it built with?

PythonPyTorch

How does it compare?

	karpathy/ng-video-lecture	googlefonts/noto-emoji	ronreiter/interactive-tutorials
Stars	4,703	4,707	4,698
Language	Python	Python	Python
Setup difficulty	moderate	easy	moderate
Complexity	2/5	2/5	3/5
Audience	researcher	designer	writer

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 30min

Requires PyTorch, GPU recommended for training but not required for small experiments, intended as a video lecture companion, not a standalone tutorial.

No explicit license is stated in the repository.

So what is it?

This repository holds the code written during Andrej Karpathy's Neural Networks: Zero To Hero video lecture series, specifically the first lecture covering nanoGPT. NanoGPT is a small, readable implementation of the GPT language model architecture, the same general type of model that powers ChatGPT. The repo is published so viewers can follow along, run the code themselves, and walk through the git history to see how the code was built step by step during the lecture. It is intended as a companion to the video, not a standalone tutorial. The README notes that model weight initialization was not covered in depth during the video, and the current code trains correctly but converges more slowly as a result. The author plans to address this in a future supplementary video and update the code at that time. For now the code remains close to what was shown on screen during the lecture.

Copy-paste prompts

Prompt 1

I'm following Karpathy's Zero To Hero lecture and want to run the nanoGPT training code. Walk me through the Python file structure, what data to use, and how to start training on my machine.

Prompt 2

Looking at the ng-video-lecture code, explain how the self-attention mechanism is implemented and how it differs from a simple feedforward layer.

Prompt 3

How do I use the git log history of ng-video-lecture to follow the code as it was written during the lecture, step by step?

Prompt 4

The README says weight initialization wasn't covered in depth and training converges slowly. What PyTorch weight initialization change would fix this for the transformer blocks in this code?

Prompt 5

I want to train the nanoGPT model from this repo on a custom text dataset. What do I need to change to use my own data instead of the default?

Frequently asked questions