whatisgithub

What is token-print?

sudharsanselvaraj/token-print — explained in plain English

Analysis updated 2026-05-18

54TypeScriptAudience · researcherComplexity · 4/5LicenseSetup · hard

In one sentence

TokenPrint is a browser-based 3D tool that visualizes a real language model's tensors, live token generation, and internals for hands-on exploration and debugging.

Mindmap

mindmap
  root((tokenprint))
    What it does
      3D model visualization
      Real tensor data
      Live token generation
      Guided walkthrough
    Tech stack
      Next.js
      React Three Fiber
      FastAPI
      PyTorch
    Use cases
      Learn model internals
      Debug attention layers
      Compare quantizations
    Modes
      Architecture
      Generation
      Walkthrough
      Debugger

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Explore a language model's real tensors and architecture visually instead of reading raw config files.

USE CASE 2

Watch a live token-by-token generation rendered in 3D to understand attention heads and layers.

USE CASE 3

Debug model internals with breakpoints, a flame graph, and before-and-after ablation comparisons.

What is it built with?

TypeScriptNext.jsReact Three FiberPythonFastAPIPyTorch

How does it compare?

sudharsanselvaraj/token-printavijit07x/git-switchcryptosaras/tokenward
Stars545454
LanguageTypeScriptTypeScriptTypeScript
Setup difficultyhardmoderateeasy
Complexity4/54/52/5
Audienceresearcherdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1h+

Needs a Python backend for live models plus a Node.js frontend, GPU helps but CPU fallback works.

MIT license: use freely for any purpose, including commercial use, as long as you keep the copyright notice.

So what is it?

TokenPrint is a browser tool that lets you see the inner workings of a language model in interactive 3D. You can load a live small model or drop in your own model file, and every number shown is real, either parsed directly from the file or produced by an actual run of the model, nothing is faked or illustrative. The tool has four modes. Architecture shows a 3D point cloud of every real tensor in the model along with an overview of its size and structure, plus a searchable list and a view for comparing different quantized versions of the same model. Generation streams a real token by token run of the model over a live connection, drawing a distinct 3D shape for each part of the model such as one blade per attention head, while a side panel shows the matching math formula and live probability scores. Walkthrough is a guided, chaptered tour covering topics from tokenization through the final output layer, timed to real data rather than placeholder animations, and includes prediction games using sample prompts in different languages. Debugger is a more technical dashboard with breakpoints, a flame graph of operations, detection of unusual activity, and the ability to compare a model before and after specific parts are disabled. The visuals are built directly from real numbers: the count of shapes for attention matches the real number of attention heads, and the size of certain shapes matches real internal ratios of the model. The project runs a Python and FastAPI backend for the live model and generation, alongside a Next.js frontend using React Three Fiber for the 3D graphics, and it automatically recovers if the browser's 3D context is lost.

Copy-paste prompts

Prompt 1
Set up TokenPrint's backend and frontend locally and load the live Qwen2.5-0.5B model to explore its architecture.
Prompt 2
Drop a local .gguf file into TokenPrint and walk me through its tensor layout in the Architecture view.
Prompt 3
Use TokenPrint's Walkthrough mode to explain tokenization through the output layer with real data.
Prompt 4
Run an ablation comparison in TokenPrint's Debugger mode and explain what changed in the output.

Frequently asked questions

What is token-print?

TokenPrint is a browser-based 3D tool that visualizes a real language model's tensors, live token generation, and internals for hands-on exploration and debugging.

What language is token-print written in?

Mainly TypeScript. The stack also includes TypeScript, Next.js, React Three Fiber.

What license does token-print use?

MIT license: use freely for any purpose, including commercial use, as long as you keep the copyright notice.

How hard is token-print to set up?

Setup difficulty is rated hard, with roughly 1h+ to a first successful run.

Who is token-print for?

Mainly researcher.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.