What Is a Large Language Model?

A large language model (LLM) is a neural network trained to predict the next piece of text. That sounds modest, yet the same mechanism writes code, summarises contracts, translates, and plans multi-step tasks. This article explains how that works, one layer at a time.

The one-sentence definition

An LLM is a function that takes a sequence of tokens and returns a probability for every token that could come next. Everything else follows from that: chat, coding, agents, even "reasoning". Generate a token, append it, run the function again, and you get text.

Three ideas make this function powerful:

• Scale. Modern models have billions to hundreds of billions of learned parameters and are trained on trillions of tokens of text and code.
• The transformer architecture. It lets every token look at every earlier token in the context, so the model can track meaning across long passages.
• Post-training. After learning to continue text, the model is tuned to follow instructions, refuse harmful requests and prefer helpful answers.

From text to tokens

Models do not read characters or words. They read tokens: chunks of text drawn from a fixed vocabulary, usually between 30,000 and 200,000 entries. Common words become a single token, rare words are split into pieces, and spaces are typically glued to the start of the following word.

The vocabulary is learned before training with algorithms such as byte-pair encoding (BPE), which repeatedly merges the most frequent adjacent pairs of bytes. Here is a real example using OpenAI's open-source tiktoken library:

import tiktoken

enc = tiktoken.get_encoding("o200k_base")
ids = enc.encode("Tokenization isn't magic.")

print(ids)
# [4421, 2860, 12471, 19745, 13]
print([enc.decode([i]) for i in ids])
# ['Token', 'ization', " isn't", ' magic', '.']

Five tokens for four words. As a rule of thumb, one English token is about four characters, or three quarters of a word. Other languages and source code usually need more tokens per word, which matters because everything about an LLM is metered in tokens: price, speed, and the size of the context window.

Inside the transformer

Each token ID is first mapped to an embedding: a vector of a few thousand numbers that the model learns during training. Words used in similar ways end up with similar vectors. Information about position is added so the model knows the order of the tokens.

The vectors then flow through a stack of identical transformer layers. Each layer refines every token's vector using two sub-blocks: attention, which mixes in information from other tokens, and a feed-forward network (MLP), which transforms each token's vector on its own. After the last layer, a final projection turns the vector of the last position into one score per vocabulary entry. Those scores are the logits.

Attention: deciding what to look at

Attention is the mechanism that made transformers win. For every token, the layer computes three vectors: a query ("what am I looking for?"), a key ("what do I contain?") and a value ("what do I pass on?"). The query of one token is compared with the keys of all earlier tokens; the closest matches get the largest weights, and the token absorbs a weighted mix of their values.

In the sentence "The trophy did not fit in the suitcase because it was too big", attention lets the vector for "it" pull information from "trophy" rather than "suitcase". Layers run many attention "heads" in parallel, each free to learn a different kind of relationship: syntax, coreference, or which bracket closes which function call.

In a language model, attention is causal: a token can only look backwards. That is what makes the model a next-token predictor and allows the whole sequence to be trained in parallel.

Layers, parameters and scale

A model's parameters are the numbers inside its embedding table, attention projections and MLPs. More layers and wider vectors mean more parameters and more capacity to store patterns. Many recent models use a mixture of experts (MoE) design: each layer contains many MLP "experts" and a router activates only a few of them per token. The model can then hold a very large number of parameters while spending far less compute on each token.

Generating text, one token at a time

The logits are converted into probabilities with a softmax. The application then chooses the next token. Picking the most likely token every time (greedy decoding) is predictable but repetitive; sampling from the distribution produces more varied text. The temperature setting controls how adventurous sampling is: low values sharpen the distribution, high values flatten it.

tokens = tokenizer.encode(prompt)

while len(tokens) < max_tokens:
    logits = model(tokens)[-1]              # one score per vocabulary entry
    probs = softmax(logits / temperature)   # temperature reshapes the distribution
    next_id = sample(probs)                 # or argmax(probs) for greedy decoding
    if next_id == tokenizer.eos_id:         # the model can emit "end of sequence"
        break
    tokens.append(next_id)

print(tokenizer.decode(tokens))

Because generation is autoregressive, output tokens are produced one after another, and long answers take longer. Inference engines avoid recomputing the whole prefix for each new token by storing intermediate attention results in a KV cache. That cache is also why long contexts consume so much GPU memory.

How an LLM is trained

Turning a randomly initialised network into an assistant happens in stages. The exact recipe differs between labs, but the shape is consistent.

Pretraining

During pretraining the model reads an enormous corpus and, at every position, is penalised for assigning low probability to the token that actually came next. Nobody labels the data: the text supervises itself. To predict well across the whole internet, the network has to absorb grammar, facts, styles, programming languages and a surprising amount of reasoning. The result is a base model: excellent at continuing documents, but not yet inclined to answer questions or follow instructions.

Post-training: SFT, preferences and RL

Supervised fine-tuning (SFT) trains the base model on thousands to millions of high-quality examples of instructions and ideal responses. The model learns the format of a conversation and the role of an assistant.

Preference tuning then teaches judgement. In classic reinforcement learning from human feedback (RLHF), people compare pairs of answers, a reward model learns to predict their preferences, and the LLM is optimised against that reward. Variants replace humans with AI feedback or skip the reward model entirely, as in Direct Preference Optimization (DPO).

The newest ingredient is reinforcement learning with verifiable rewards: the model attempts maths problems, coding tasks or puzzles whose answers can be checked automatically and is rewarded for getting them right. This is the main technique behind reasoning models.

Context windows and the illusion of memory

An LLM has no memory between calls. Everything it knows about your conversation must be inside the context window: the system prompt, the chat history, attached documents and tool results. Chat applications create the illusion of memory by re-sending the whole conversation on every turn.

Context windows have grown from a few thousand tokens to hundreds of thousands or more, but bigger is not automatically better. Cost grows with context length, and models attend less reliably to details buried in the middle of very long inputs. That is why practical systems select what goes into the window: retrieval-augmented generation fetches only relevant passages, and harnesses summarise or prune history. Choosing what the model sees is now a discipline of its own, often called context engineering.

What LLMs are good and bad at

LLMs are strong at transforming and generating language: drafting, rewriting, summarising, translating, extracting structured data from messy text, writing and explaining code. They are also good general-purpose "glue" between tools when given the right instructions.

Their weaknesses follow directly from how they work:

• They predict plausible text, not verified truth. When knowledge is missing, the most plausible continuation can be a confident fabrication: a hallucination.
• Their knowledge has a cutoff. A model knows nothing about events after its training data ends unless you supply that information in the context.
• Exact computation is not native. Long arithmetic, counting and precise lookups are better delegated to tools such as a calculator, code interpreter or database.
• Behaviour is probabilistic. The same prompt can produce different outputs. Production systems need validation, retries and tests, not hope.

The model landscape in brief

As of 2026, models differ along a few axes that matter more than leaderboard positions:

• Closed vs open-weight. Closed models are available only through an API. Open-weight models can be downloaded, fine-tuned and run on your own hardware; see running models locally.
• Size and cost. Small models answer quickly and cheaply and run on laptops or phones; frontier models are slower and pricier but handle harder tasks.
• Modalities. Many LLMs now accept images, audio or documents as input. Specialised models still lead in tasks such as speech and document reading.
• Reasoning effort. Some models can "think" longer before answering, trading latency for accuracy.

Not every problem needs a generative model at all. When the answer is a label, a yes/no or a score, a decision model can be faster, cheaper and better calibrated.