Transformer explainer LLM simulator

Transformer Explainer was designed to provide an interactive exploration of the inner workings of a text-generating Transformer.

It is a useful tool for students and computer science scientists who wants to understand the mechanisms underlying artificial intelligence as applied to text generation.
In particular, the model provides a visual representation of the steps through which a transformer generates the word to be inserted at the end of the entered sentence.

Transformer Explainer features a live GPT-2 (small) model running directly in the browser. This model is derived from the PyTorch implementation of GPT by Andrej Karpathy's nanoGPT project and has been converted to ONNX Runtime for seamless in-browser execution. The interface is built using JavaScript, with Svelte as a front-end framework and D3.js for creating dynamic visualizations. Numerical values are updated live following the user input.

Here are some of the interactive features you can play with:

• Input your own text sequence to see how the model processes it and predicts the next word. Explore attention weights, intermediate computations, and see how the final output probabilities are calculated.


• Use temperature slider to control the randomness of the model’s predictions. Explore how you can make the model output more deterministic or more creative by changing the temperature value.


• Select top-k and top-p sampling methods to adjust sampling behavior during inference. Experiment with different values and see how the probability distribution changes and influences the model's predictions.


• Interact with attention maps to see how the model focuses on different tokens in the input sequence. Hover over tokens to highlight their attention weights and explore how the model captures context and relationships between words.