How Large Language Models (LLMs) Actually Work: An Interactive Guide

Large Language Models (LLMs) like GPT-4, Claude, and LLaMA appear to understand reasoning, humor, code, and nuance. Yet under the hood, every LLM operates on a single core objective: given a sequence of tokens, predict the probability distribution for the very next token.

In this guide, we demystify how LLMs transform human language into math, process context through the Transformer architecture, and generate coherent text. You can interact with the live demos below to see the math in action.

1. Step One: Tokenization

Computers cannot process raw characters or words directly. Before text enters a neural network, it is split into chunks called tokens using algorithms like Byte-Pair Encoding (BPE) or WordPiece.

A token can be an entire word, a subword, a punctuation mark, or even a single byte. On average in English, 1 token is roughly 4 characters or 0.75 words.

🎮 Interactive Tokenizer Simulator

Type any sentence below to see how an LLM breaks it down into discrete tokens:

Once tokenized, each token is mapped to a unique integer ID from a fixed vocabulary (typically between 32,000 and 128,000 distinct tokens).

2. From Token IDs to Vector Embeddings

An integer ID like 4215 conveys no geometric meaning to a neural network. To capture semantic relationships, the model maps each token ID into a high-dimensional vector space called an Embedding Vector:

$$\vec{e} \in \mathbb{R}^{d_{\text{model}}}$$

In models like LLaMA-3 (70B), $d_{\text{model}} = 8192$. In this space:

• Similar concepts point in similar directions (e.g., $\vec{\text{king}} - \vec{\text{man}} + \vec{\text{woman}} \approx \vec{\text{queen}}$).
• Words with multiple meanings get positioned in rich semantic sub-spaces.

Positional Encoding

Because Transformers process all tokens simultaneously (unlike older sequential RNNs), the model needs to know where each word appears in the sentence. We add Positional Embeddings (such as Sinusoidal encodings or RoPE — Rotary Position Embeddings) directly to the token embeddings:

$$\vec{x}_i = \vec{e}_i + \vec{p}_i$$

3. The Engine Room: Multi-Head Self-Attention

Self-attention allows the model to connect different words in a sentence, resolving pronouns, ambiguity, and long-range dependencies.

When reading the word “it” in:

“The animal didn’t cross the street because it was too tired.”

The self-attention mechanism computes high affinity between “it” and “animal”. If the sentence ended with “it was too wide”, the attention would shift to “street”.

The Query, Key, and Value ($Q, K, V$) Formulation

For each token vector, the model creates three distinct vectors by multiplying with learned weight matrices:

1. Query ($Q$): “What am I looking for?”
2. Key ($K$): “What kind of information do I offer?”
3. Value ($V$): “What is my actual content?”

The attention score between query $i$ and key $j$ is calculated via dot product:

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

🎮 Interactive Attention Matrix Explorer

Click any word below to see which other words it pays attention to in the context:

4. Inside the Transformer Layer

A single Transformer block consists of two primary modules:

1. Multi-Head Self-Attention (MHA): Lets tokens exchange information across the entire sequence.
2. Feed-Forward Network (FFN / MLP): Allows each token to process and retrieve factual knowledge stored within the model’s weights.

Surrounding each module are two critical architectural innovations:

• Residual (Skip) Connections: $\vec{x}_{\text{out}} = \vec{x} + \text{SubLayer}(\vec{x})$. This prevents gradients from vanishing during backpropagation across dozens of layers.
• Layer Normalization (RMSNorm): Stabilizes activations across features.

Modern LLMs stack dozens of these identical layers (e.g. 32 layers for a 7B model, 80+ layers for 70B+ models).

5. From Logits to Next Token: Sampling & Temperature

After passing through all Transformer layers, the final vector is projected onto the vocabulary dimension via the language modeling head (an unembedding matrix). This produces raw numerical scores called Logits ($z_1, z_2, \dots, z_V$).

To convert logits into a probability distribution, we apply the Softmax function with Temperature ($T$):

$$P(w_i) = \frac{\exp(z_i / T)}{\sum_{j} \exp(z_j / T)}$$

How Temperature Shapes Creativity

• Low Temperature ($T \to 0$): Sharpens the distribution. The highest logit dominates. Responses become deterministic, precise, and repetitive.
• High Temperature ($T > 1.0$): Flattens the distribution. Lower-probability tokens have a higher chance of being picked. Responses become diverse, creative, or chaotic.
• Top-$p$ (Nucleus Sampling): Selects only the smallest set of tokens whose cumulative probability exceeds threshold $p$.

🎮 Interactive Next-Token Probability Simulator

Adjust the Temperature and Top-$p$ sliders below in real time to observe how the candidate probability distribution shifts:

<div class="controls-row">
  <div class="control-col">
    <div class="slider-label">
      <span>Temperature ($T$): <strong id="temp-val">0.70</strong></span>
      <span class="tip" id="temp-desc">Balanced & Fluent</span>
    </div>
    <input type="range" id="temp-slider" min="0.05" max="1.8" step="0.05" value="0.70" oninput="updateSamplingSim()" />
  </div>
  <div class="control-col">
    <div class="slider-label">
      <span>Top-P (Nucleus): <strong id="topp-val">0.90</strong></span>
      <span class="tip">Cumulative probability cutoff</span>
    </div>
    <input type="range" id="topp-slider" min="0.1" max="1.0" step="0.05" value="0.90" oninput="updateSamplingSim()" />
  </div>
</div>

<div class="distribution-view">
  <div class="dist-header">
    <span>Candidate Token</span>
    <span>Raw Logit</span>
    <span>Probability</span>
  </div>
  <div id="candidates-container"></div>
</div>

<div class="sampling-actions">
  <button class="action-btn" onclick="sampleAndAppendToken()">🎲 Sample &amp; Append Next Token</button>
  <button class="action-btn secondary" onclick="resetSamplingPrompt()">↺ Reset Prompt</button>
</div>

6. How Are LLMs Trained?

Training modern foundation models happens in three distinct phases:

Key Takeaways

1. Tokens, not words: Text is divided into subword tokens and mapped into high-dimensional geometric embedding vectors.
2. Self-Attention is relational: It allows tokens to dynamic route contextual information regardless of distance.
3. Generative auto-regression: Text generation is an iterative loop where each newly sampled token is appended to the input context to predict the next one.
4. Sampling parameters control behavior: Temperature and Top-$p$ steer the trade-off between deterministic precision and exploratory creativity.