AI explainer

Interactive Transformer Explainer

Explore step-by-step how text is tokenized, projected through Self-Attention, and converted into next-token predictions.

Step 1 Interactive Tokenizer & Vector Embeddings

Type a prompt or choose a sample to see subword tokenization and token IDs:

Presets:

Step 2 Self-Attention Mechanism (Query × Key)

Shows calculated attention weights A = softmax(Q · K^T / √d_k). Hover over any square to see how strongly each token attends to others:

Hover over a matrix cell to inspect Query-Key compatibility.

Step 3 Next-Token Probabilities (Logits & Softmax)

Probabilities output by the final classification head after LayerNorm and Feed-Forward expansion:

Temperature: 0.7

Now that you've explored the individual components above, interact with the full 124-million-parameter GPT-2 pipeline below in real-time WebGL: