What is jacobian-lens?

The jacobian-lens project (also called jlens) is a research tool that lets you peek inside a language model and see what it is "thinking" at each individual layer. When a language model processes text, it builds up an understanding step by step, but that internal state is usually a black box. This tool takes a snapshot of the model's internal activations at any point and translates them into a ranked list of actual words the model is most inclined to say next. For example, if the model is processing an ASCII drawing of a face, this tool can show that at a certain layer and position, the model's internal state is already representing the concept of a "nose" even if that word never appears in the original text prompt. At a high level, the tool works by creating a bridge between an early or middle layer of the model and its final output layer. It studies how the model transforms information across its layers by analyzing a large batch of example texts. Once it learns this transformation pattern, it can take any internal signal from the model, fast-forward it through the remaining layers, and decode it into human-readable words. This creates an interactive, layer-by-layer heatmap showing exactly when and where specific concepts form inside the model's architecture. This tool is designed for AI researchers and engineers who study how language models represent information internally. A concrete use case would be a researcher trying to understand at which layer a model first "realizes" that Italy is the country shaped like a boot, and at which layer it connects that to the concept of the Euro. The included walkthrough notebook lets these users load a model, apply the lens, and interactively click through a visual grid to track how word representations evolve across layers and positions. A notable aspect of this project is that it is explicitly a reference implementation for an Anthropic research paper on global workspace interpretability. The README states upfront that the code is not maintained and is not accepting contributions. It is also not optimized for performance, training the lens is heavily dependent on the model's own processing speed, though the documentation notes you can parallelize the process. It works with open-weights models available on HuggingFace, using Qwen as its primary example, and it requires relatively little data to work well, with quality saturating after about a thousand text sequences.