How LLMs actually work

Most engineers can use a language model competently without knowing what happens inside it. That works right up until it doesn’t: until the model invents an API that doesn’t exist, or answers correctly at the top of a conversation and badly at the bottom, and you have no model of the machine to reason about why.

This post is the mechanism. Not the maths in full, but enough of it that the failure modes stop being mysterious.

The unit of work is a token, not a word

A model doesn’t see text. Before anything happens, the input is run through a tokeniser, which splits it into pieces drawn from a fixed vocabulary — typically tens of thousands of entries — and maps each piece to an integer.

Tokens are usually subword fragments. Common words are a single token; rarer ones get split. understanding might survive whole, while an unusual proper noun or a long identifier is chopped into several pieces. Whitespace and punctuation are part of the tokens too.

This is not a detail. It’s why models are unreliable at character-level tasks like counting letters in a word — the word may never exist as characters inside the model at all, only as two or three opaque integers. It’s also why every limit and price you’ll ever see is quoted in tokens rather than words: tokens are the only unit the system actually has.

Embeddings: numbers with useful geometry

Each token id is used to look up a vector — a list of numbers, some hundreds or thousands of them long. That’s an embedding. The lookup table is learned during training.

It’s tempting to say embeddings “capture meaning”. They don’t, and the distinction matters. What training produces is a space whose geometry reflects the statistics of usage: tokens that appear in similar contexts end up in similar directions, because that arrangement makes the prediction task easier. The resulting structure is genuinely useful — it’s what makes semantic search work — but it is a compressed record of how language is used, not a representation of facts about the world.

Position matters too, and a bag of vectors has no order, so positional information is injected as well (modern models generally encode relative position rather than absolute). Without it, “the cat ate the fish” and “the fish ate the cat” would be identical inputs.

Attention: how tokens read each other

The vectors then pass through a stack of identical blocks. Each block does two things: attention, which moves information between positions, and a feed-forward network, which does computation at each position. Everything is wired with residual connections, so each block edits a running representation rather than replacing it.

Attention is the part worth understanding properly. For each token, the model projects its current vector into three others: a query, a key and a value.

• The query is what this token is looking for.
• The key is what this token offers to others looking.
• The value is what it hands over if selected.

Every token’s query is compared against every token’s key with a dot product. The scores are scaled and passed through a softmax, which turns them into weights that sum to one. The token’s output is the weighted sum of the value vectors:

Attention(Q, K, V) = softmax( Q · Kᵀ / √d ) · V

That’s it. “Attention” is a soft, learned lookup: a weighted average over other positions, where the weights are computed from the content itself. Nothing about it is symbolic or rule-based.

Two consequences fall straight out of the formula. First, it’s quadratic — every token compares against every other, so doubling the input roughly quadruples the attention work. That’s the source of a lot of cost and latency behaviour. Second, in a generative model the scores are causally masked: a token may only attend to positions before it, never after. That’s what makes generation left-to-right possible.

Models run many attention heads in parallel per block, each with its own projections, so different heads can specialise in different relationships.

Next-token prediction, all the way down

At the top of the stack, the final vector for the last position is projected back out to one score per vocabulary entry. Those are the logits. A softmax turns them into a probability distribution over every possible next token.

Then the system samples one — temperature and top-p shape how much of the tail gets considered — appends it to the input, and runs the whole thing again.

That is the entire behaviour of the model. Pretraining optimises one objective across an enormous corpus: predict the next token. Everything that looks like knowledge, style, arithmetic or code is a side effect of getting extremely good at that. Post-training (instruction tuning, preference optimisation) then shapes how it answers — it makes the model a helpful assistant rather than a document continuer — but it doesn’t install a new mechanism underneath.

A useful thing to hold onto: the model is stateless. It has no memory of your last message. What feels like a conversation is the entire transcript being re-submitted on every turn. Which is why the context window ends up being the central engineering constraint — the subject of the next post.

Why it hallucinates

Given the above, hallucination stops being surprising and starts being structural.

There is always a distribution. The model cannot decline to produce a next token; it can only produce one whose text happens to read as a refusal. There is no internal flag that says “I have no data for this”.

Facts are stored lossily. Knowledge lives smeared across billions of weights as a by-product of compression. Frequently-attested facts survive that compression well. Rare ones — a specific function signature, a particular citation, a middle name — are exactly the things a lossy encoder blurs, and the blur is filled in with whatever is most plausible.

Plausibility is the objective. The training signal rewards likely continuations, and a confidently-worded wrong answer is a very likely continuation of a confident question. Fluency and accuracy are produced by the same machinery, so fluency carries no information about accuracy. This is the single most expensive intuition to lack.

The practical response is not to hope for a better model. It’s to change what’s in the context: give the model the source text instead of asking it to recall one, and give it tools that return ground truth.

What “reasoning” is, and what it isn’t

When a model works through a problem step by step and gets a better answer, the mechanism is less mysterious than the word suggests.

Each token it generates is appended to its own input. So the intermediate steps become context the model can attend over when producing the next steps. Two real things follow: the model gets more serial computation for a hard problem (one forward pass is a fixed amount of compute; a hundred tokens of working is a hundred passes), and it gets externalised working memory it can refer back to. Reasoning-trained models are trained to do this well — often with reinforcement learning against problems whose answers can be checked automatically.

So it’s real, and it measurably helps on problems with structure. But note what it isn’t. There is no separate deliberation module, no internal search tree, no verification step unless one is built around the model. And the trace is not guaranteed to be a faithful description of how the answer was actually computed — it’s a generated artefact, subject to the same pressures as any other output. A plausible-looking chain of thought can accompany a wrong answer, and confidently justify it.

Treat a reasoning trace as a useful scratchpad the model can condition on, not as a proof.

The mental model worth keeping

A large language model is a very good next-token predictor with a learned soft lookup over its own input, no state, no ground truth, and no ability to abstain.

Almost every practical technique in this field — retrieval, tool use, structured outputs, evaluation harnesses, context management — exists to compensate for one of those four properties. Once you can name which one you’re fighting, the fix usually picks itself.