What is gpt2-jspace?

gpt2-jspace is a research project that tests a specific scientific question: does a newer interpretability tool called the Jacobian lens, introduced by Anthropic and only tested so far on their production Claude models, actually work better than an older, simpler tool called the logit lens when applied to much smaller, open source GPT-2 models. Interpretability tools like these are used by researchers to peek inside a language model and understand what it is predicting and why at each internal layer. The project builds its own from-scratch version of the Jacobian lens in Python, because the official version requires a newer version of a library that will not run on the author's older Intel Mac. This custom version was checked against the official implementation to make sure it produces matching results. The author then ran a series of experiments at two model sizes, 124 million and 355 million parameters, testing whether the Jacobian lens outperforms the logit lens at predicting the model's own next word, predicting several words ahead, and steering the model's output. The headline finding is that it does not. The Jacobian lens does not beat the simpler logit lens at this smaller scale, and the gap does not shrink as the model gets slightly bigger. Rather than stopping there, the author tested several possible explanations for this result, such as whether the comparison was unfair due to limited data, and ruled most of them out one by one, with the full process documented as it happened in a research log file. The repository also includes an interactive HTML visualizer that lets someone compare the two lenses side by side across a grid of model layers and text positions in a web browser, with no server needed since the data is bundled into the page itself. Everything runs on a regular computer's CPU except for one optional script meant to be run in Google Colab. The project is released under the MIT License and is not affiliated with Anthropic.