Glossary

A system in which a language model decides which actions to take, calls tools, observes the results and repeats until a goal is met. The loop around the model, not the model alone, is what makes it an agent.

Used in: AI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the ProductModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

The transformer mechanism that lets each token weigh information from other tokens. A token's query is compared with other tokens' keys to decide how much of their values to mix in.

Used in: What Is a Large Language Model?

Producing output one token at a time, where each new token is conditioned on everything generated before it. It is the reason long answers take longer to produce.

Used in: What Is a Large Language Model?

A tokenization algorithm that builds a vocabulary by repeatedly merging the most frequent adjacent pairs of bytes or characters in a training corpus.

Used in: What Is a Large Language Model?

How well a model's stated confidence matches reality. A well-calibrated model that says 80% is right about 80% of the time, so its probabilities can be used as thresholds.

Used in: Decision Models: AI That Answers With ProbabilitiesModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

Intermediate reasoning steps a model writes before its final answer. Prompting for it, or training models to produce it, improves accuracy on multi-step problems.

Used in: Reasoning Models: How LLMs Think Before They Answer

Splitting documents into smaller passages before embedding them for retrieval. Chunk size, overlap and boundaries have a large effect on RAG quality.

Used in: Embeddings, Vector Databases and RAG

The practice of deciding what goes into a model's context window (instructions, retrieved documents, tool results, memory) and in what form and order.

Used in: What Is a Large Language Model?Harnesses: Why the Model Is Not the Product

The maximum number of tokens a model can consider in a single call, covering both the input and the generated output.

Used in: What Is a Large Language Model?Reasoning Models: How LLMs Think Before They AnswerEmbeddings, Vector Databases and RAGHarnesses: Why the Model Is Not the ProductRunning Models Locally: Quantization, vLLM, Ollama and LoRA

A model that answers typed questions (yes/no, choice, score) with calibrated probabilities in a single forward pass, without generating text. Examples include TypeSafe Jev and Liquid AI d1.

Used in: Decision Models: AI That Answers With ProbabilitiesModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

Training a smaller student model to imitate the outputs of a larger teacher model, keeping much of its quality at a fraction of the cost.

Used in: Reasoning Models: How LLMs Think Before They Answer

A vector of numbers that represents a piece of content so that similar meanings land close together. Used inside LLMs and for semantic search.

Used in: What Is a Large Language Model?Speech AI: How STT and TTS Models WorkEmbeddings, Vector Databases and RAG

Repeatable test suites that measure how well a model or AI system performs on the tasks you care about: the AI equivalent of unit and regression tests.

Used in: Reasoning Models: How LLMs Think Before They AnswerOCR and Document AI: From Tesseract to Vision-Language ModelsDecision Models: AI That Answers With ProbabilitiesEmbeddings, Vector Databases and RAGHarnesses: Why the Model Is Not the ProductModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

Continuing to train a pretrained model on a smaller, task-specific dataset to adapt its behaviour, style or domain knowledge.

Used in: Embeddings, Vector Databases and RAGRunning Models Locally: Quantization, vLLM, Ollama and LoRA

A single-file format for storing model weights, usually quantized, together with tokenizer and metadata. Used by llama.cpp and tools built on it, such as Ollama.

Used in: Running Models Locally: Quantization, vLLM, Ollama and LoRA

Checks around a model that validate its inputs and outputs: schema validation, content filters, policy rules, or a second model acting as a judge.

Used in: OCR and Document AI: From Tesseract to Vision-Language ModelsDecision Models: AI That Answers With ProbabilitiesAI Agents, Tool Calling and MCPModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

A fluent, confident output that is not supported by the input or by facts. A natural side effect of a model trained to produce plausible text.

Used in: What Is a Large Language Model?Reasoning Models: How LLMs Think Before They AnswerSpeech AI: How STT and TTS Models WorkOCR and Document AI: From Tesseract to Vision-Language ModelsEmbeddings, Vector Databases and RAGModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

The software around a model that turns it into a working system: the agent loop, prompts, tool definitions, context management, permissions and sandboxing.

Used in: Decision Models: AI That Answers With ProbabilitiesAI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the Product

A design in which a person reviews, approves or corrects an AI system's actions at defined checkpoints, typically before irreversible or high-impact steps.

Used in: OCR and Document AI: From Tesseract to Vision-Language ModelsDecision Models: AI That Answers With ProbabilitiesAI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the ProductModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

Running a trained model to produce outputs, as opposed to training it. Inference cost and latency dominate most production AI budgets.

Used in: Running Models Locally: Quantization, vLLM, Ollama and LoRA

Stored attention keys and values for tokens that have already been processed, so each new token does not recompute the whole prefix. It grows with context length and batch size.

Used in: What Is a Large Language Model?Running Models Locally: Quantization, vLLM, Ollama and LoRA

The raw, unnormalised scores a model outputs for each possible next token. A softmax turns them into probabilities.

Used in: What Is a Large Language Model?Decision Models: AI That Answers With Probabilities

A parameter-efficient fine-tuning method that freezes the base model and trains small low-rank matrices added to selected layers. QLoRA applies it on top of a quantized model.

Used in: Running Models Locally: Quantization, vLLM, Ollama and LoRA

An architecture in which each layer contains many expert sub-networks and a router activates only a few of them per token, adding capacity without a proportional increase in compute.

Used in: What Is a Large Language Model?Running Models Locally: Quantization, vLLM, Ollama and LoRA

An open protocol that standardises how AI applications connect to tools and data. A capability exposed once as an MCP server can be used by any MCP-compatible client.

Used in: AI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the Product

Converting images of text, such as scans, photos or screenshots, into machine-readable text.

Used in: OCR and Document AI: From Tesseract to Vision-Language Models

The first and most expensive training stage, in which a model learns to predict the next token across a very large unlabeled corpus.

Used in: What Is a Large Language Model?

An attack in which instructions hidden in untrusted content, such as a web page, email or document, hijack a model's behaviour. A primary security risk for agents with tools.

Used in: AI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the ProductModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

Storing model weights, and sometimes activations, with fewer bits (for example 4-bit instead of 16-bit) to cut memory use and speed up inference at a small quality cost.

Used in: Running Models Locally: Quantization, vLLM, Ollama and LoRA

An LLM trained, usually with reinforcement learning, to produce an extended chain of thought before answering, trading extra tokens and latency for accuracy.

Used in: Reasoning Models: How LLMs Think Before They AnswerDecision Models: AI That Answers With Probabilities

A model that re-scores retrieved candidates against the query more accurately than first-stage vector search, so only the best few reach the prompt.

Used in: Embeddings, Vector Databases and RAG

Retrieving relevant documents at query time and inserting them into the prompt, so the model answers from supplied sources rather than from memory alone.

Used in: Embeddings, Vector Databases and RAGRunning Models Locally: Quantization, vLLM, Ollama and LoRA

Tuning a model against a reward signal learned from human preference comparisons, to make its outputs more helpful and safe.

Used in: What Is a Large Language Model?Reasoning Models: How LLMs Think Before They Answer

Working out who spoke when in a recording by splitting the audio into segments and assigning each segment to a speaker.

Used in: Speech AI: How STT and TTS Models Work

Automatic speech recognition: converting spoken audio into text.

Used in: Speech AI: How STT and TTS Models Work

Training a pretrained model on curated examples of prompts and ideal responses so that it learns to follow instructions.

Used in: What Is a Large Language Model?

Instructions placed at the start of the context by the application rather than the user, defining the model's role, rules and available tools.

Used in: AI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the Product

A parameter that rescales logits before sampling. Lower values make output more deterministic; higher values make it more varied.

Used in: What Is a Large Language Model?

Compute spent while answering, for example on longer reasoning or on sampling several candidates and picking the best, rather than during training.

Used in: Reasoning Models: How LLMs Think Before They Answer

Generating natural-sounding speech audio from text.

Used in: Speech AI: How STT and TTS Models Work

The unit of text a language model reads and writes: a word, part of a word, punctuation mark or byte sequence from the model's fixed vocabulary.

Used in: What Is a Large Language Model?Reasoning Models: How LLMs Think Before They AnswerSpeech AI: How STT and TTS Models WorkDecision Models: AI That Answers With Probabilities

A model's ability to emit a structured request (a tool name plus JSON arguments) that the application executes, returning the result to the model.

Used in: Reasoning Models: How LLMs Think Before They AnswerEmbeddings, Vector Databases and RAGAI Agents, Tool Calling and MCPHarnesses: Why the Model Is Not the ProductModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop

The neural network architecture behind modern LLMs, built from stacked layers of attention and feed-forward blocks. Introduced in 2017.

Used in: What Is a Large Language Model?Speech AI: How STT and TTS Models WorkOCR and Document AI: From Tesseract to Vision-Language Models

A database optimised for storing embeddings and finding the nearest vectors to a query, usually with approximate nearest-neighbour indexes such as HNSW.

Used in: Embeddings, Vector Databases and RAG

A model that takes images and text as input and produces text. Used for visual question answering, document understanding and OCR.

Used in: OCR and Document AI: From Tesseract to Vision-Language Models

Detecting which parts of an audio stream contain speech. Used to segment audio and to decide when a speaker has finished a turn.

Used in: Speech AI: How STT and TTS Models Work

A predefined sequence of steps, some of which call an LLM, where the developer rather than the model controls the flow. Contrast with an agent.

Used in: AI Agents, Tool Calling and MCPModern Automation with LLMs: Workflows, Tools and Human-in-the-Loop