Reference
Glossary
1,330 terms from across the book, each linked to the chapter that defines it. Type to filter.
#
$/1M tokens (cost per million output tokens)
Cost formula (GPU rental rate x 10^6) / (sustained tokens/s x 3600); the primary metric for inference cost efficiency.
16-bytes-per-parameter memory model
The mixed-precision AdamW accounting: 2 bytes bf16 params + 2 bytes bf16 grads + 4 bytes fp32 master weights + 4+4 bytes Adam moments m,v, per parameter.
1F1B (one-forward-one-backward) schedule
Steady-state pipeline schedule alternating one forward and one backward per stage, bounding in-flight activation memory to ~p microbatches independent of total microbatch count m.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 2 chapters
3D-HybridEngine
veRL's in-GPU-memory mechanism that reshards actor weights between the trainer's FSDP/Megatron parallel layout and the rollout engine's tensor-parallel layout without disk round-trips.
3D/4D/5D parallelism
Composing data, tensor, pipeline, context, and expert parallelism multiplicatively across a GPU cluster, placing each axis's collectives onto network links matching its communication intensity.
6N rule
The approximation that training costs about 6 FLOPs per parameter per token (2N forward + 4N backward); inference costs about 2N FLOPs/token.
6ND FLOP rule
The dense-transformer compute estimate C ≈ 6ND (2 FLOPs/param/token forward, 4 backward), used to convert each ladder run's (N, D) into training FLOPs and dollar cost.
6ND rule
FLOP-accounting rule that dense-transformer training costs C≈6ND: 2 forward-pass FLOPs plus 4 backward-pass FLOPs per parameter per token.
Defined in 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond · used in 3 chapters
8-bit Adam
Variant that keeps Adam's full first/second moment tensors but quantizes them to 8 bits with block-wise scaling, cutting their memory footprint 4x.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
__syncthreads() (block-level barrier)
A CUDA synchronization primitive that blocks every thread in a block from proceeding until all threads have reached it, preventing read/write hazards on shared memory.
A
A/B test
An experiment comparing a control (current model) against one or more treatments, randomized at the user-session level to satisfy SUTVA.
absorbing-state (masked) diffusion
A discrete diffusion process where tokens are independently replaced by [MASK] with growing probability over time, and never transition to any other token.
accelerate (HuggingFace)
HuggingFace's distributed-training launcher that TRL trainers rely on for GPU/multi-node execution (DDP, FSDP, DeepSpeed) without trainer-level code changes.
Defined in 6.3 TRL: HuggingFace's RL Library · used in 2 chapters
acceptance rate
The expected probability alpha = sum_x min(p(x),q(x)) = 1 - TV(p,q) that a drafted token is accepted; the main driver of speculative speedup.
accumulate vs. replace
The dichotomy governing model collapse: replacing real data with each generation's synthetic output causes collapse, while accumulating synthetic data on top of retained real data largely prevents it.
action
A token span the policy samples during a turn (reasoning, tool call, or final answer); the only tokens that receive a policy-gradient loss.
Defined in 6.10 Agentic & Multi-Turn RL
action deduplication
Defensive technique that hashes (tool_name, args) signatures of prior actions and blocks or flags repeats to break infinite loops where the agent retries the same failed action.
Activation checkpointing (gradient checkpointing)
Discarding intermediate forward-pass activations and recomputing them during backward, trading ~33% extra compute for O(sqrt(L)) activation memory.
Defined in 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math · used in 2 chapters
activation outliers
A small set of feature dimensions in activations of models beyond ~6-7B params that take on 10-100x larger magnitudes than typical, wrecking per-tensor quantization scales.
activation patching
A causal-intervention technique that overwrites one activation (layer/position/component) from a clean run into a corrupted run (or vice versa) to measure how much it drives a behavior.
activation steering
Adding a scaled direction vector to the residual stream at inference time to push model behavior toward or away from a concept, without any weight changes.
active learning
Selecting the subset of unlabeled production examples whose annotation will most improve the model, rather than labeling randomly.
Defined in 12.5 Data Flywheels & Continuous Improvement
Actor (policy)
The trainable language model πθ; exists as separate generation weights (rollout engine) and training weights (learner) that must stay synchronized.
Defined in 6.1 The Anatomy of an RL-for-LLM System · used in 2 chapters
Actor-critic
RL architecture pairing a trained policy (actor, π_θ) with a learned value estimator (critic, V_φ) whose baseline produces the advantage used in the policy gradient.
Adafactor
Optimizer that factors the second-moment matrix into a row vector and column vector (O(n+m) memory) and can drop the first moment entirely.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
Adam optimizer
Adaptive optimizer combining a momentum-like first moment with an RMSProp-style second moment to rescale each parameter's update; the default optimizer for LLM pretraining.
Defined in 1.3 Calculus, Optimization & Convexity
Adam second-moment amplification
The mechanism by which a stale (slowly-updating) √v̂ denominator lets a sudden large gradient produce a disproportionately large parameter update, driving spikes.
AdamW (decoupled weight decay)
Adam optimizer with weight decay decoupled from the adaptive gradient term, applying a clean multiplicative shrink to weights; the de-facto LLM pretraining default.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 4 chapters
Adapter cardinality cap (max_loras)
A scheduler limit on the number of distinct adapters allowed in one batch/step, keeping SGMV segments few and large instead of many tiny, inefficient ones.
Adapter merging
Folding a trained LoRA/DoRA update $\frac{\alpha}{r}BA$ into the frozen base weight after training, producing an ordinary linear layer with zero extra inference latency or memory.
Defined in 5.3 PEFT I: LoRA, QLoRA, DoRA & The Adapter Family · used in 2 chapters
Adapter registry
The control-plane component mapping a tenant's adapter name to its weights, residency tier, reference count, and eviction metadata (LRU/LFU signals).
adaptive evaluation (computerized adaptive testing)
Selecting each next item to maximize Fisher information at the current ability estimate, reaching a target precision with far fewer items than a fixed test.
adaptive KL control
Proportional controller that grows or shrinks the KL coefficient beta each update to steer the realized KL toward a target value.
admission control
Policy that predicts whether a newly arriving request can meet its latency SLO given current queue state, rejecting (shedding) it with a 429 if not.
Defined in 12.1 Designing an LLM Serving System
advantage (A_t)
A(s,a) = Q(s,a) - V(s), measuring whether an action was better than the policy's average behavior at that state; the signal multiplied by ∇log π_θ in the actor-critic gradient.
Defined in 5.6 Policy Gradients & PPO for Language Models · used in 3 chapters
advantage whitening
Normalizing advantages to zero mean and unit variance over real, non-padding tokens (in fp32) before feeding them into the policy loss.
AgentHarm
A benchmark scoring whether a jailbroken tool-using agent both complies with and retains capability to complete malicious multi-step tasks.
agentic loop
The while-loop of model 'think', environment 'execute', and context-append that lets an LLM act, observe real feedback, and revise instead of hallucinating outcomes.
agentic RAG
Architecture where retrieval is exposed as a tool an LLM agent calls repeatedly (ReAct-style), deciding what to search and when to stop, rather than a fixed pipeline step.
agentic RL
Training the model with reinforcement learning (often sparse, delayed task-success reward plus a KL penalty) to internalize agentic behavior, replacing hand-crafted ReAct prompting with a learned policy.
aha moment
The emergent, unprogrammed event where a model mid-derivation pauses, flags a mistake, and revises its approach, arising purely from correctness-reward pressure.
AI control
Deployment approach that assumes the model may be misaligned and designs a protocol (monitoring, auditing, restricted actions) that stays safe regardless.
ALiBi (Attention with Linear Biases)
Adds no positional info to embeddings; instead subtracts a per-head linear penalty proportional to distance directly from attention scores before softmax.
alignment tax
The failure mode where post-training (SFT/DPO/RLVR) quietly degrades a model's core language-modeling ability, detectable by comparing base vs. post-trained perplexity.
All-Gather
Collective where each rank contributes a slice and every rank ends up with the concatenation of all slices (output size nM); the second phase of ring all-reduce.
All-Reduce
Collective where all ranks contribute data that is combined (e.g., summed) and every rank receives the full result, used for averaging gradients in data parallelism.
Defined in 1.9 Parallel Computing & Collective Communication · used in 3 chapters
All-to-all (dispatch/combine)
Collective where each rank sends a distinct chunk to every other rank; used to route tokens to expert GPUs in Mixture-of-Experts dispatch.
Defined in 1.9 Parallel Computing & Collective Communication · used in 3 chapters
All-to-all collective
Communication primitive where each device sends distinct data to every other device; expert parallelism uses it to dispatch tokens to their assigned experts and combine outputs back.
allocation exponent
The exponent a in N* ∝ C^a (equivalently beta/(alpha+beta)) describing how compute-optimal parameters and tokens should each grow with total compute; here recovered near 0.5.
alpha-beta cost model
Cost model T(B) = alpha + B/beta estimating message transfer time as fixed per-call latency (alpha) plus size-dependent bandwidth cost (B/beta).
AlphaEdit
A knowledge-editing method that projects weight updates onto the null space of preserved-knowledge key covariance, curbing drift across long sequential edit runs.
Defined in 13.2 Knowledge Editing & Machine Unlearning
AMP (Automatic Mixed Precision)
A framework (e.g. torch.autocast plus GradScaler) that automatically runs matmuls in low precision and range-sensitive reductions in fp32 via an internal op allow/deny list.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
anchored canary eval
A fixed, frozen golden set of request/response pairs scored automatically every few minutes in production to detect gradual quality regressions.
anisotropy
Failure mode where raw (non-fine-tuned) embeddings cluster in a narrow cone of vector space rather than spreading across the hypersphere, making untrained cosine similarity uninformative.
Defined in 9.1 Embeddings & Representation Learning
annealing (mid-training / cooldown phase)
The final ~10-20% of pretraining tokens, where the learning rate decays sharply while the data mixture shifts toward the highest-quality, target-relevant data.
Answer relevance
RAGAS metric measuring whether the generated answer addresses the original query, computed via similarity between hypothetical questions regenerated from the answer and the real query.
any-resolution (AnyRes) tiling
Splitting a high-resolution image into fixed-size tiles (plus a thumbnail), encoding each independently, and concatenating their tokens to preserve fine detail.
Defined in 10.2 Vision-Language Models
AOT (ahead-of-time) compilation
Compiling the full forward pass into a serialized, hardware-specific computation graph at build time, eliminating runtime kernel-dispatch overhead.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
AOTAutograd (Ahead-Of-Time Autograd)
The torch.compile component that traces through the autograd engine at compile time to build one joint forward+backward FX graph, enabling cross-boundary fusion.
Approximate Nearest Neighbor (ANN) search
Trading a small amount of recall for orders-of-magnitude speedup versus exact k-NN, by not examining every vector in the corpus.
ARC-AGI (Abstraction and Reasoning Corpus)
A grid-transformation-rule-induction benchmark testing fluid intelligence from a handful of examples, designed to resist memorization and the standard scaling-law playbook.
arithmetic intensity
The ratio of FLOPs performed to bytes moved to/from HBM (FLOP/byte); the single number that determines whether a kernel is compute- or memory-bound.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 7 chapters
arithmetic intensity breakeven (B*)
Batch size FLOP/s ÷ bandwidth where decode transitions from bandwidth-bound to compute-bound; ~295 for a 70B BF16 model on H100.
assistant-only loss masking
Supervising only assistant-generated spans (thoughts, tool calls, final answer, and the closing assistant <|end|>) while masking prompts and tool-result spans with IGNORE=-100.
Defined in 14.10 A Narrow Auto-Research Agent: ReAct, Tool-Use & Retrieval by Distillation · used in 2 chapters
Asymmetric Distance Computation (ADC)
Computes distance from a full-precision query to PQ-compressed database vectors via precomputed per-subspace lookup tables, avoiding decompression.
async rollout pipeline
An optimization decoupling generation from training so the rollout engine generates the next batch while the current batch trains, trading policy staleness for higher throughput.
asynchronous checkpointing
Checkpointing pattern that copies GPU tensors to pinned CPU memory (the only blocking step), then writes to disk in a background thread/process while training continues.
asynchronous off-policy RL
Decoupling generation and training via a rollout queue so generators keep sampling while the trainer trains, instead of lock-step synchronization.
Attack Success Rate (ASR)
Fraction of target harmful behaviors an attack successfully elicits from a model, scored by a grader rather than keyword matching.
attention logit overflow
A failure mode where unnormalized Q·Kᵀ dot products grow large enough that softmax exponentials overflow or underflow, producing NaN/Inf in the backward pass.
attention mask
A boolean or additive matrix over query-key positions that determines which tokens can attend to which; the defining feature distinguishing architecture families.
attention matrix (attention weights)
The n x n matrix of softmax-normalized similarity scores between all queries and keys; row i gives query i's probability distribution over keys.
Defined in 2.3 The Attention Mechanism From Scratch
Attention mechanism
Core Transformer operation where each token forms a softmax-normalized, scaled dot-product weighted sum over other tokens' value vectors.
Defined in Glossary of Terms
attention sink
A token (often BOS or the first few positions) that receives disproportionately high attention weight as a place for softmax to 'dump' unneeded probability mass, critical to preserve during KV cache eviction.
Attention-DP + expert-EP hybrid layout
Serving layout where attention runs data-parallel (each GPU keeps its own requests' full KV cache, no collective) while the MoE FFN sublayer runs expert-parallel across the same GPUs.
attribution patching
A gradient-based linear approximation to activation patching that estimates every component's causal effect from a single backward pass instead of one forward pass per component.
AudioLM hierarchical generation
A two-stage audio generation approach that first autoregressively models compact semantic tokens (long-range structure), then generates denser acoustic codec tokens conditioned on them.
Defined in 10.3 Audio, Speech & Multimodal Fusion
autoformalization
Translating a natural-language math statement into a formal theorem statement (a type) that a proof assistant can then attempt to prove.
Autograd (automatic differentiation)
Engine that records operations as a computation graph during the forward pass and propagates gradients backward via the chain rule.
Defined in From-Scratch Code Index
automatic differentiation (autodiff)
Mechanically computes exact gradients of code expressed as composed differentiable operations, without finite differences or symbolic expression swell.
Automatic prefix caching (APC)
Reusing previously computed KV blocks of a matching prompt prefix, identified via rolling block hashes, instead of recomputing them.
Automatic Prompt Engineer (APE)
A method that treats instruction generation as program synthesis: an LLM proposes candidate instructions and another (or the same) LLM scores them on held-out data.
Defined in 8.9 Prompt Engineering as Engineering
autoregressive generation
Generating a sequence one token at a time by feeding the growing prefix back into the model and sampling the next token from p(x_t | x_<t).
autotuning (@triton.autotune)
A Triton decorator that benchmarks a list of kernel configs (tile sizes, num_warps, num_stages) on first launch for a given problem shape and caches the fastest one.
Defined in 4.4 Writing GPU Kernels with Triton
Auxiliary load-balancing loss
A differentiable penalty, alpha*E*sum(f_e*P_e), multiplying hard dispatch fractions by soft router probabilities to push routing toward uniform and prevent collapse.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
Auxiliary-loss-free balancing
DeepSeek-V3's technique of adding a per-expert bias to routing logits (selection only, not gate weights) that is nudged each step to equalize load without a gradient-interfering loss term.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
AWQ (Activation-aware Weight Quantization)
A weight-only PTQ method that identifies salient weight channels via activation magnitude and protects them with a per-channel rescaling trick, without a Hessian.
Defined in 4.7 Quantization I: Post-Training Quantization (GPTQ, AWQ, SmoothQuant) · used in 2 chapters
B
Backpropagation
Reverse-mode algorithm computing gradients of a scalar loss with respect to all parameters in one backward sweep via the chain rule.
Baseline
Any action-independent quantity b(s_t) subtracted from the reward before scaling the log-prob gradient; provably unbiased and used to cut REINFORCE's variance.
Defined in 5.6 Policy Gradients & PPO for Language Models · used in 2 chapters
Batch normalization
Normalizes each pre-activation to zero mean and unit variance across the mini-batch, then applies a learnable affine (gamma, beta), stabilizing deep-network training.
Bayes' theorem
Rule relating posterior P(theta|D) to likelihood P(D|theta), prior P(theta), and marginal likelihood; the formal mechanism for updating beliefs given data.
beam search
A decoding algorithm that keeps the top-B partial hypotheses by length-normalized cumulative log-probability at each step, approximating the MAP sequence.
beam search agent
Implementation of Tree-of-Thought that keeps only the top-b highest-scoring nodes at each depth, multiplying LLM calls by roughly beam width times candidates times depth.
Benchmark decontamination
Removing training documents whose n-gram fingerprints overlap with evaluation benchmark examples, preventing inflated and meaningless test scores.
benchmark saturation
State where frontier models cluster near the score ceiling (90-100%), so remaining differences reflect question quality and noise, not capability.
Bernoulli standard error
SE = sqrt(p(1-p)/n) for an accuracy proportion p over n examples, used to judge whether a benchmark score difference between two models is statistically meaningful.
Defined in 11.3 Building Eval Harnesses
Beta-Bernoulli posterior
Per-task Bayesian pass-rate estimate kept as decayed success/failure counts (s, f); posterior is Beta(s+1, f+1) with mean (s+1)/(s+f+2) and a usable credible interval.
Defined in 6.12 RL Data, Curriculum & Replay Management
bf16 (bfloat16)
16-bit float with fp32's 8-bit exponent and a 7-bit mantissa; matches fp32's dynamic range, making it the LLM training workhorse.
Defined in 1.4 Numerical Computing, Floating Point & Precision · used in 2 chapters
bf16 autocast
Training with bfloat16's 8-bit exponent (fp32's dynamic range, fewer mantissa bits) inside a torch.autocast region, avoiding fp16's overflow risk and need for loss scaling.
bf16 mixed precision
Training in the bfloat16 numeric format rather than fp16, chosen because its wider dynamic range avoids the silent overflow-to-NaN failure mode fp16 exhibits during training.
BGMV (Batched Gather Matrix-Vector multiply)
Decode-time counterpart of SGMV: one token per request, so it is a memory-bandwidth-bound batched gather-and-multiply against many adapters' weights.
Bi-encoder (dual-encoder)
Architecture that encodes query and document independently with the same transformer, scored by dot product/cosine at retrieval time, enabling offline document precomputation.
Defined in 9.1 Embeddings & Representation Learning · used in 4 chapters
bias correction
Dividing Adam's zero-initialized first/second moment EMAs by (1-β^t) to remove the early-step bias that would otherwise make updates far too large.
bias-variance tradeoff
Decomposition of expected prediction error into systematic error (bias), sensitivity to training data (variance), and irreducible noise; complexity trades one for the other.
Defined in 1.5 Machine Learning Fundamentals
bicubic interpolation of position embeddings
Technique for adapting a ViT trained at one resolution to another by reshaping the learned position-embedding table into a 2D grid and resizing it.
Defined in 10.1 Vision Transformers & Image Encoders
binary quantization
Compressing each embedding dimension to a single sign bit (1 bit/dim) to shrink multi-vector index storage, often the highest-leverage compression for ColPali-style indexes.
bits-per-byte (BPB)
A tokenizer-agnostic loss metric that normalizes NLL (converted to bits) by the average UTF-8 bytes per token, enabling fair cross-tokenizer/cross-model comparison.
Defined in 3.3 The Pretraining Objective & Loss
bitsandbytes
The PyTorch-native library providing drop-in INT8/NF4 quantized linear layers and paged optimizers; the primary quantization backend for Transformers and PEFT/QLoRA.
Defined in 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT · used in 2 chapters
blackboard pattern
A shared, append-only data structure workers read from and write to instead of passing messages directly, avoiding the orchestrator-context bottleneck at scale.
Defined in 8.7 Multi-Agent Systems & Orchestration
Block (KV block)
Fixed-size chunk of KV-cache token-slots (commonly 16 tokens) that is the unit of allocation and sharing in PagedAttention.
block diffusion
A semi-autoregressive hybrid that generates fixed-size blocks left-to-right autoregressively while denoising tokens within each block in parallel via diffusion.
Block manager (KVCacheManager)
vLLM's refcounted allocator that owns the pool of physical KV blocks, handing them to sequences and reclaiming them on release.
block table
Per-sequence data structure mapping logical token-block indices to physical KV block IDs in the global pool, the paged analogue of an OS page table.
Defined in 4.6 PagedAttention & KV-Cache Memory Management · used in 2 chapters
block-causal attention mask
An attention mask that is bidirectional within a block but causal across blocks, letting earlier finalized blocks' keys/values be cached for reuse.
Defined in 2.12 Diffusion & Non-Autoregressive Language Models · used in 2 chapters
block-diagonal attention mask
An additive attention mask restricting each packed document to attend only within its own causal span, preventing cross-document leakage.
block/tile programming model
Triton's core abstraction: a kernel program operates on a block (tile) of data at once, not scalars; the compiler handles thread mapping.
Defined in 4.4 Writing GPU Kernels with Triton
blockwise (fine-grained) scaling
Assigning a separate scale factor per tile/block of a tensor (e.g. DeepSeek-V3's 1x128 activation tiles and 128x128 weight blocks) instead of one scale per tensor, so outliers in one block don't crush precision elsewhere.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
Bloom filter
A probabilistic bit-array membership structure with k hash functions; guarantees no false negatives but has a tunable false-positive rate, saving memory over hash sets.
BM25 (Okapi BM25)
A TF-IDF-based sparse ranking function scoring term overlap between query and document, weighted by term frequency, document length, and inverse document frequency; strong at exact-match retrieval.
Defined in 9.4 Chunking, Reranking & Hybrid Search · used in 2 chapters
bonus token
The extra token sampled directly from the target's distribution at the position after the last draft token, free when all gamma draft tokens are accepted.
bootstrap (percentile / BCa)
Resampling scored items with replacement many times to approximate the sampling distribution of any statistic (accuracy, Elo, F1) without distributional assumptions; BCa corrects for skew.
bootstrap confidence interval
A distribution-free confidence interval computed by resampling per-example scores with replacement many times and taking percentiles of the resulting metric values.
Defined in 11.3 Building Eval Harnesses
bootstrap read
The harness's first action at session start: querying external memory stores and prepending relevant results to the system prompt for continuity.
Defined in 8.5 Memory Systems for Agents
Bradley-Terry model
Statistical model where preference probability between two responses is the logistic sigmoid of the difference between their latent scalar strength scores.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 2 chapters
Bradley-Terry model / Elo rating
A logistic model of pairwise-comparison outcomes (e.g. arena battles) that fits a latent ability rating per model; ratings are point estimates with bootstrap-estimable sampling error.
broadcasting (backward semantics)
NumPy-style shape stretching implemented via zero strides with no data copy; the backward pass must sum gradients over broadcast dimensions to restore the original shape.
budget forcing
A test-time control that caps or truncates the number of 'thinking' tokens a model may use, trading accuracy for latency along a continuous knob.
burn rate
The ratio (1 - current SLI) / (1 - SLO target), measuring how many times faster than planned an error budget is being consumed.
Burn-rate alerting
An SRE technique that alerts when the rate of error/quality-budget consumption exceeds a multiplier of the sustainable rate, reducing alarm fatigue versus fixed thresholds.
Defined in 12.2 Observability, Logging & LLMOps
byte-level BPE
BPE run over the 256 raw byte values (via a reversible bytes-to-unicode map) instead of characters, guaranteeing every string is representable with no out-of-vocabulary token.
byte-level BPE (byte pair encoding) tokenizer
Subword tokenizer that repeatedly merges the most frequent adjacent symbol pair, falling back to raw UTF-8 bytes so no input is ever unrepresentable.
Byte-Pair Encoding (BPE)
A greedy subword algorithm that starts from characters/bytes and repeatedly merges the most frequent adjacent symbol pair into a new vocabulary entry.
Defined in 2.1 Tokenization: BPE, WordPiece, Unigram & Byte-Level · used in 2 chapters
bytes-per-token
A compression metric (bytes of raw text divided by number of tokens produced) used to compare tokenizers and vocabulary sizes; higher means cheaper inference and more effective context.
C
calibration (Expected Calibration Error)
Property that a model's predicted probability matches the empirical frequency of the positive class; ECE bins predictions and averages the accuracy-confidence gap.
Defined in 1.5 Machine Learning Fundamentals
Canary and exposure metric
A canary is a synthetic secret inserted into training data; exposure = log2|R| − log2(rank of the true secret among all candidates) quantifies how memorized it became.
canary rollout
Progressively routing a small, increasing fraction of real traffic to a new model with automatic rollback triggered by guardrail-metric regressions.
canary token
A unique string embedded in a system prompt whose appearance (verbatim or via fuzzy match) in model output signals that the system prompt has leaked, enabling detection and logging of extraction attacks.
Defined in 12.4 Safety, Guardrails & Content Moderation · used in 2 chapters
Capability injection
Concentrating narrow, targeted data (math, code) into the final low-LR sub-phase to give the model a specific capability floor without pulling it far off its general manifold.
Capacity factor / token dropping
A multiplier C_f setting each expert's fixed token buffer size (capacity = ceil(C_f*Nk/E)); tokens beyond it are dropped (skip the FFN) since GPUs need rectangular tensors.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures · used in 2 chapters
capacity gap
The phenomenon where a student far smaller than the teacher cannot represent the teacher's distribution well, making progressive/staged distillation more effective.
capacity planning formula
N_GPUs = ceil(peak requests/s x avg output tokens / (per-GPU decode throughput x target utilization)); converts traffic projections into required GPU count.
cascade architecture (guardrail routing)
A tiered safety design where a cheap, high-recall classifier resolves most traffic instantly and only escalates uncertain 'gray zone' cases to a slower, more accurate shield model.
Defined in 12.4 Safety, Guardrails & Content Moderation
catastrophic cancellation
Precision loss that occurs when subtracting two nearly-equal floating-point numbers, amplifying the relative rounding error in the result.
catastrophic forgetting
Sharp degradation of a model's performance on its original data distribution caused by gradient updates optimized for a new distribution.
Defined in 3.16 Continual & Domain-Adaptive Pretraining · used in 4 chapters
categorical distribution
Discrete distribution over a fixed set of outcomes (e.g., vocabulary tokens); an LLM's softmax output vector is exactly its parameter.
causal language modeling (CLM)
The decoder-only pre-training objective (next-token prediction) that computes cross-entropy loss at every position given only preceding tokens.
causal (autoregressive) mask
A lower-triangular mask that forbids query i from attending to keys j > i, letting a decoder train in one pass while remaining autoregressive.
Defined in 2.3 The Attention Mechanism From Scratch · used in 2 chapters
causal self-attention
Multi-head attention computed with a mask (or is_causal=True) that prevents position t from attending to tokens after t, enforcing autoregressive next-token prediction.
Causal tracing
An activation-patching procedure that corrupts a subject's embeddings, then restores clean hidden states layer-by-layer to find which (layer, token) causally carries a fact.
Defined in 13.2 Knowledge Editing & Machine Unlearning
Chain-of-Thought (CoT)
Prompting technique eliciting step-by-step intermediate reasoning before a final answer, improving accuracy on multi-step tasks.
Defined in Glossary of Terms
chain-of-thought (CoT) prompting
Prompting technique that elicits step-by-step intermediate reasoning before the final answer, externalising intermediate state into the context window.
Defined in 5.10 Reasoning, Chain-of-Thought & Test-Time Compute · used in 3 chapters
chain-of-thought tax
The extra dollar cost incurred when a reasoning model emits many internal reasoning tokens before its final answer, since each thinking token is billed like any output token.
Chameleon
Meta's 2024 unified transformer that processes and generates both text and images from one shared token vocabulary with no separate vision encoder.
Defined in 10.5 Unified & Any-to-Any Models
chat template
A deterministic function mapping role-content message pairs to a single token sequence, identical between training and inference.
Defined in 5.2 Chat Templates, Data Formatting & Sequence Packing · used in 3 chapters
ChatML
OpenAI's turn format using <|im_start|>role\ncontent<|im_end|> special tokens; widely adopted by Qwen and other open models.
Checkpoint hygiene
Practices making a checkpoint self-describing and safely resumable: atomic tmp-file-then-rename writes, saving optimizer and RNG state alongside weights, and pruning old checkpoints under a retention policy.
checkpoint reload
The slowest, most robust weight-sync mechanism: the trainer writes a full state_dict to shared storage and the inference engine reloads and reshards it independently.
checkpoint/resume
Saving model weights, optimizer state, step counter, data-loader cursor, and RNG states atomically so a training run can be exactly resumed after an interruption.
Chinchilla parametric form L(N, D)
The loss model L = E + A/N^alpha + B/D^beta, with E the irreducible floor and the other terms the finite-capacity and finite-data penalties, fit to the ladder's measured losses.
Chinchilla scaling law
Hoffmann et al.'s corrected result showing compute-optimal N and D should scale in lockstep, both roughly proportional to C^0.5.
Defined in 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond · used in 3 chapters
Chinchilla-optimal token budget
The compute-optimal ratio of roughly 20 training tokens per parameter that minimizes loss per unit of training FLOPs spent.
Chinchilla-optimal training
Compute-optimal allocation rule (Hoffmann et al.) pairing parameter count and training-token count roughly equally for a fixed compute budget.
Defined in Key Papers: An Annotated Reading List
chunked prefill
Splitting a long prompt's prefill into token-budget-sized chunks spread across several iterations, co-scheduled with ongoing decodes so inter-token latency stays smooth.
Defined in 7.2 Continuous Batching & Request Scheduling · used in 4 chapters
chunking
Splitting a document into pieces for embedding and retrieval; the size/strategy trades off recall (large chunks) against precision (small chunks).
Defined in 9.4 Chunking, Reranking & Hybrid Search
circuit
A subgraph of specific model components (attention heads, MLPs) and their connections that together implement a human-understandable algorithm, with the rest of the network shown irrelevant.
circuit breaker (fail-safe fallback)
A pattern that routes traffic to a fast heuristic guardrail when a primary neural safety classifier is unavailable, defaulting to fail-closed (block) rather than fail-open (unsafe passthrough).
Defined in 12.4 Safety, Guardrails & Content Moderation
classifier guidance
Steering reverse diffusion using the gradient of a separate noise-aware classifier's log-likelihood ∇log p(y|x_t), an alternative to CFG requiring an extra trained classifier.
classifier-free guidance (CFG)
An inference-time technique that interpolates/extrapolates between conditional and unconditional noise predictions (weight w) to strengthen adherence to a conditioning signal like text.
Client (MCP)
A lightweight, protocol-level connector inside the host process that speaks MCP's wire format to exactly one server and manages the session lifecycle.
Defined in 8.6 The Model Context Protocol (MCP)
CLIP (Contrastive Language-Image Pre-training)
Dual-encoder model that maps an image and text into a shared vector space, each compressed to one normalized vector, trained with a symmetric InfoNCE contrastive loss.
Defined in 9.6 Multimodal & Visual-Document Retrieval: ColPali & Late Interaction · used in 2 chapters
clip fraction
Diagnostic metric: the share of tokens whose importance ratio left the clip range, indicating how far training has drifted off-policy.
Clip-higher (decoupled clipping)
Decoupling PPO's symmetric clip into separate lower/upper bounds (eps_high > eps_low) so rare-but-good tokens can be boosted more, preventing entropy collapse.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 3 chapters
clipped surrogate objective
PPO-style loss that clips the importance ratio between new and old policy to a trust region [1-eps,1+eps], making multi-epoch updates on reused rollouts safe.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
cloze scoring
Multiple-choice evaluation technique that ranks answer options by the model's summed log-probability of each full continuation, rather than asking it to emit a letter choice.
CLS token (classification token)
A learnable vector prepended to the patch sequence whose final-layer output is used as the global image representation, analogous to BERT's [CLS].
Defined in 10.1 Vision Transformers & Image Encoders
cluster (block) bootstrap
A bootstrap variant that resamples whole groups of correlated items (e.g. questions from the same document) rather than individual items, avoiding falsely narrow CIs.
code execution sandbox
Isolated, resource-limited environment (Docker+seccomp, gVisor, Firecracker) for safely running untrusted model-generated code to verify correctness by behavior.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Codebook
Learned (or, with FSQ, implicit) set of K discrete vectors that an image or audio encoder's continuous outputs are quantised against.
Defined in 10.5 Unified & Any-to-Any Models
Cohen's kappa
A chance-corrected agreement statistic, kappa = (p_o - p_e)/(1 - p_e), used to measure how well a judge's labels match human labels beyond random overlap.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
ColBERT
The text-retrieval model that introduced late interaction and MaxSim, encoding BERT tokens instead of a single passage vector.
Cold-adapter SLO cliff
The latency spike when a request's adapter is not GPU-resident and must be fetched (possibly from an object store), adding tens to hundreds of milliseconds to time-to-first-token.
cold-start problem
The failure mode where RL yields zero gradient because every sampled response in a group gets the same reward (all-correct or all-wrong), so advantages vanish.
colocated RL
Placement strategy where trainer and rollout engine share the same physical GPUs, time-slicing between phases or offloading state to fit both.
Colocated vs. disaggregated design
Whether generation and training share the same GPUs time-sliced (colocated) or run on separate GPU pools that must synchronize weights (disaggregated).
Defined in 6.1 The Anatomy of an RL-for-LLM System
colocation
Placing the actor, critic, reference, and rollout engine worker groups on the same physical GPUs via a shared Ray placement group, time-sliced across stages.
ColPali
Late-interaction retriever that replaces ColBERT's text document encoder with a vision-language model, embedding a rendered page image into ~1024 patch vectors scored via MaxSim, with no OCR.
ColQwen2
ColPali variant using the Qwen2-VL backbone, whose dynamic-resolution vision encoder produces a variable number of patch tokens per page instead of a fixed grid.
Column-parallel / row-parallel linear layer
Two ways to shard a weight matrix across TP ranks (by output columns vs. input rows); chaining column-then-row avoids any collective between the two matmuls.
Common Crawl
Nonprofit that publishes monthly open web-crawl snapshots; the primary raw source material for nearly every large open LLM pretraining dataset.
compaction
Replacing a long, low-density span of an agent's conversation history with a short, model-generated structured summary that preserves durable facts and decisions.
Defined in 8.4 Context Engineering & Management
COMPLETE sentinel file
An empty marker file written atomically only after all checkpoint shards and metadata finish writing, so a partially written checkpoint is never mistakenly loaded.
compression ratio (bytes per token)
UTF-8 bytes divided by resulting token count, measuring how efficiently a tokenizer packs text; higher means fewer tokens per document.
compute-bound
A kernel whose intensity exceeds the ridge point, so its performance ceiling is the hardware's peak FLOP/s rather than bandwidth.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 2 chapters
compute-optimal allocation
The split of a fixed FLOP budget C between parameters N and tokens D that minimizes training loss L(N,D) under C=6ND.
Defined in 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond · used in 2 chapters
condition number
Ratio κ(A) = λ_max/λ_min of a matrix's largest to smallest eigenvalue, measuring numerical ill-conditioning that slows gradient descent.
Defined in 1.1 Linear Algebra for Deep Learning · used in 2 chapters
Conditional computation
Sparse activation where the network decides, per input, which sub-networks (experts) to run, decoupling parameter count from per-token compute.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
conditional-independence trap
The failure mode where a single denoising step predicts all masked positions independently, producing locally plausible but jointly incoherent text if committed at once.
confidence interval (CI)
A range of plausible values for a true score built from a finite sample; its width shrinks only as 1/sqrt(n), so small evals cannot resolve small gaps.
confidence-based remasking
A remasking strategy that commits the highest-confidence predicted tokens each step and re-masks the rest, used by LLaDA and the chapter's sampler.
Config hash
A short SHA-256-derived fingerprint of a run's full frozen configuration (architecture, optimizer, schedule, data mix), stamped on checkpoints and logs so identical hashes prove identical experiments.
conformity assessment
The compliance check a deployer must perform (self-assessment or notified-body audit) and register before placing a high-risk AI system on the EU market.
Defined in 13.6 AI Governance, Compliance & Regulation
Constitution
A list of natural-language principles (e.g., 'choose the response least likely to cause harm') sampled to construct critique, revision, and preference-judging prompts.
Constitutional AI (inference-time revision)
Two-stage pipeline (Bai et al. 2022) where a model critiques and revises its own outputs against an explicit principle set, then an AI judge scores response pairs to train via RL.
Defined in 5.11 Constitutional AI, RLAIF & Self-Improvement · used in 2 chapters
constrained decoding (FSM logit masking)
Restricting sampled tokens to those valid in a finite-state machine compiled from a regex or JSON schema, by setting disallowed tokens' logits to -infinity.
Defined in 7.4 SGLang: RadixAttention & Structured Programs · used in 2 chapters
constrained edit (edit_file tool)
An edit API requiring an old_string that matches exactly once in the file before replacement, failing loudly on zero or multiple matches instead of silently corrupting content.
constrained (structured) generation
Enforcing that generated text satisfies a formal grammar (regex, JSON Schema, EBNF) by restricting decoding itself, not by prompting.
Defined in 7.10 Structured & Constrained Generation
contamination (benchmark contamination)
Risk that a benchmark's tasks or reference solutions appeared in a model's pretraining data, inflating scores; especially acute for SWE-bench since patches are public on GitHub.
Defined in 8.8 Agent Evaluation & Benchmarks · used in 2 chapters
contamination detection
Techniques (n-gram overlap, perplexity test, canary insertion, temporal holdout) for identifying whether benchmark problems or answers leaked into training data.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Content provenance (C2PA / Content Credentials)
Coalition for Content Provenance and Authenticity standard attaching a cryptographically signed JSON-LD manifest recording an asset's creation tool, timestamp, and edit history.
Content-based block hashing (chained hash)
Hashing fixed-size blocks of token IDs, each chained to the hash of the preceding block, so identical prefixes anywhere produce identical hashes but positional context is preserved.
Defined in 7.7 Prefix Caching & KV-Cache Reuse
context assembly
The per-turn process by which the harness builds the exact list of messages (preamble, transcript, tool results) sent to the model, since the model never sees the repo directly.
context drift (derailment)
Failure mode where, over many agentic steps, the model's original goal is diluted by accumulated tool outputs and it pursues a tangential sub-task; mitigated by goal re-injection or context compression.
context engineering
The discipline of deciding what content enters the context window, in what form, order, and for how long, across an agent's turns.
Defined in 8.4 Context Engineering & Management
context forwarding
Including a summary or verbatim copy of a preceding agent step's output in the next agent's context window; its token cost grows roughly linearly with chain length.
Defined in 8.7 Multi-Agent Systems & Orchestration
Context parallelism
Sharding the sequence dimension itself across GPUs (distinct from data/tensor/pipeline parallelism) so activation memory for very long sequences fits per device.
Context parallelism (CP) / Ring Attention
Shards the token sequence itself across devices and rotates K/V blocks around a communication ring, using FlashAttention's online softmax to compute exact attention over remote blocks.
context pollution
Degradation of agent quality caused by oversized or irrelevant tool outputs crowding out the task-relevant signal in the context window.
Context precision
Retrieval-quality metric measuring what fraction of the top-k retrieved chunks are actually useful for answering the question.
context rot
The degradation of model accuracy on a fixed task as irrelevant or merely long context is added, even when the needed information is present.
Defined in 8.4 Context Engineering & Management
context window
The single finite buffer of tokens an LLM sees at each decode step; the model's only working memory for an agentic task.
Defined in 8.4 Context Engineering & Management
context-dependent token
In XGrammar, a token (e.g., closing brace) whose validity depends on the full stack state and must be evaluated per-step rather than pre-computed.
Defined in 7.10 Structured & Constrained Generation
context-independent token
In XGrammar, a vocabulary token whose validity depends only on the current shallow grammar rule, not the full PDA stack, allowing cheap pre-computed masks.
Defined in 7.10 Structured & Constrained Generation
contextual retrieval
Anthropic's technique of prepending an LLM-generated context sentence to each chunk before embedding, so the embedding is not ambiguous in isolation.
contiguity
Property of a tensor whose elements are laid out in row-major order matching its strides; most GPU/CPU kernels require it or silently trigger a copy otherwise.
continual pretraining (CPT)
Resuming the language-modeling objective on a converged checkpoint with new data, at a small fraction of original pretraining compute.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
Continued pretraining (for context extension)
Additional training on long documents at reduced learning rate after RoPE scaling, teaching attention heads genuine long-range patterns positional scaling alone can't provide.
continuous batching (iteration-level scheduling / in-flight batching)
Orca's scheduling discipline that re-derives the running batch every single decode iteration, retiring finished requests and admitting new ones immediately.
Defined in 7.2 Continuous Batching & Request Scheduling · used in 8 chapters
contrastive decoding
Scoring tokens by subtracting a weaker 'amateur' model's log-probability from a stronger 'expert' model's log-probability to amplify the expert's distinctive, more factual choices.
Contrastive Preference Optimization (CPO)
A reference-free contrastive loss (DPO with a uniform/no reference) plus an SFT anchor on the chosen response, developed for machine translation but generally applicable.
Controller
The (often uncounted seventh) component running the outer RL loop — a single Python driver, a Ray single-controller, or an async scheduler — coordinating the other components.
Defined in 6.1 The Anatomy of an RL-for-LLM System
convex combination (of values)
The attention output is a weighted average of value vectors with non-negative weights summing to 1, so it can interpolate but never extrapolate beyond their hull.
Defined in 2.3 The Attention Mechanism From Scratch
convex function
Function whose graph lies below any chord between two points (equivalently, positive-semidefinite Hessian everywhere); every local minimum is a global minimum.
Defined in 1.3 Calculus, Optimization & Convexity
coordinate check
A diagnostic that plots per-layer activation or update scale against model width; under correct muP these curves stay flat, while under standard parameterization they drift with width.
copy-on-write (COW)
Technique letting multiple sequences share a physical KV block until one must write a diverging token, at which point only that block is copied.
Defined in 4.6 PagedAttention & KV-Cache Memory Management · used in 2 chapters
Copy-on-write semantics (for KV blocks)
The pattern where a shared cached prefix's KV blocks are read-only and shared across requests, while each request's diverging suffix tokens are written into fresh, unshared blocks.
Defined in 7.7 Prefix Caching & KV-Cache Reuse
core-set / diversity sampling
Selecting a diverse spread of examples (e.g. via greedy farthest-first / k-medoids on embeddings) to avoid labeling near-duplicate uncertain cases.
Defined in 12.5 Data Flywheels & Continuous Improvement
correction factor (alpha)
The multiplier alpha = e^(m_old - m_new) that retroactively rescales previously accumulated softmax statistics onto a newly discovered, larger running max.
Defined in 4.2 FlashAttention I: IO-Awareness & The Online Softmax · used in 2 chapters
correction for guessing
Scoring adjustment score = correct - wrong/(k-1) for a k-choice benchmark that rescales random guessing to 0, removing the free-guessing floor from raw accuracy.
Corrective RAG (CRAG)
Technique that scores retrieved documents' relevance after retrieval and triggers a web-search fallback when scores are low or ambiguous.
cosine annealing schedule
The dominant pretraining LR schedule: after warmup, LR follows the right half of a cosine curve down to a floor, staying near-peak for most of training.
counterfactual data augmentation
A bias-measurement technique that varies only a demographic attribute (e.g., pronoun) across matched prompts and compares model outputs.
Counterfactual memorization
The gap in a model's performance on an example when trained with versus without that example in the training set; the quantity DP bounds.
CPU/NVMe offloading
Spilling parameters, gradients, or optimizer states from GPU HBM to slower CPU DRAM or NVMe storage once GPU memory is exhausted.
credit assignment
The problem of deciding how much of a trajectory's terminal reward each individual action deserves.
Defined in 6.10 Agentic & Multi-Turn RL
Critic (value function)
A trained network predicting the expected future reward from a partial sequence, serving as a variance-reducing baseline for advantage estimation in PPO.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 2 chapters
critic-based reflection
Scoring a proposed action with a separate critic call before executing it, rejecting actions below a threshold to catch costly or unsafe mistakes early, at the cost of extra latency.
critical batch size
The batch size $B^*$, defined via the gradient noise scale, beyond which increasing batch size no longer proportionally reduces the steps needed to converge.
Defined in 3.10 Learning Rate Schedules, Warmup, Batch Size & Hyperparameters · used in 3 chapters
cross-attention
Attention sublayer in a decoder where queries come from the decoder's hidden state and keys/values come from the encoder's output, letting the decoder read the full source bidirectionally.
cross-document attention leakage
The bug where, in a naively packed sequence, tokens of one document can attend to tokens of a different document sharing the same row.
cross-encoder
Architecture that jointly attends over query and document in one forward pass, giving higher-quality relevance scores but O(n) cost per document, so used only as a reranker.
Defined in 9.1 Embeddings & Representation Learning
Cross-encoder reranker
A model that jointly encodes a concatenated [query; document] pair to output a single relevance score, giving fine-grained relevance judgments at the cost of one forward pass per candidate.
Defined in 9.4 Chunking, Reranking & Hybrid Search · used in 2 chapters
cross-entropy
Expected negative log-probability H(p,q) that model q assigns under true distribution p; decomposes as H(p)+D_KL(p||q) and equals the LLM training loss.
cross-entropy loss
The pretraining objective: negative log-probability the model assigns the true next token, averaged over positions; minimizing it minimizes KL divergence to the data distribution.
Defined in 3.3 The Pretraining Objective & Loss · used in 2 chapters
Cross-model prefix-cache reuse
Sharing cached KV blocks for a common prompt prefix across different tenants' adapters, possible only when the adapter leaves the k and v projections unmodified.
cross-validation
Protocol that partitions data into k folds, trains k times holding out each fold for validation, and averages scores to estimate generalization when data is scarce.
Defined in 1.5 Machine Learning Fundamentals
cu_seqlens (cumulative sequence lengths)
Flash Attention's varlen-API argument giving per-document offsets into a packed batch, enforcing document-boundary attention without materialising a mask.
CUDA (Compute Unified Device Architecture)
NVIDIA's parallel programming framework in which host code on the CPU launches device code (kernels) that run on thousands of GPU threads.
CUDA graph
A recorded, replayable sequence of GPU kernel launches captured for a fixed batch shape, cutting per-step CPU launch overhead during decode.
CUDA graphs
A captured, opaque sequence of GPU operations replayed with a single CPU call (cudaGraphLaunch), eliminating per-kernel CPU launch overhead for fixed-shape workloads.
CUDA IPC (Inter-Process Communication)
Zero-copy weight-sync mechanism where one process exports a handle to GPU memory and another process maps that same device memory, avoiding any data transfer; same-node only.
CUDA/driver version compatibility
The rule that a GPU driver supports CUDA runtimes only up to a max version; PyTorch bundles its own CUDA but flash-attn/bitsandbytes compile against the system CUDA toolkit.
Defined in Tooling & Environment Setup Cheatsheet
CUPED (Controlled-experiment Using Pre-Experiment Data)
A variance-reduction technique that adjusts each user's in-experiment metric using a correlated pre-experiment covariate, cutting required sample size.
curriculum learning
Ordering training examples from easy to hard; for LLM pretraining this mainly helps at context-length ramps and end-of-training quality shifts, not fine-grained ordering.
curse of dimensionality
In high dimensions, distances between points concentrate (nearest and farthest become nearly equidistant), which defeats exact pruning structures like k-d trees.
custom autograd.Function
A user-defined class with static forward/backward methods and a ctx object for stashing tensors, letting arbitrary operations plug into the autograd engine.
CUSUM (cumulative sum control chart)
A change-point detection algorithm that accumulates deviations from an in-control mean and fires an alert when the cumulative sum exceeds a decision threshold, used to catch sustained output-quality drift.
Defined in 12.2 Observability, Logging & LLMOps
D
Daly's formula (optimal checkpoint interval)
Formula T* ≈ sqrt(2 · T_save · T_MTBF) giving the checkpoint interval that minimizes expected wasted compute from hardware failures.
dangerous-capability evaluation
A/B trial measuring the marginal uplift a model gives a malicious actor in CBRN, cyber-offense, autonomy, or persuasion, versus a no-model control group.
DARE (Drop And REscale)
Stochastically zeros a fraction of task-vector entries and rescales survivors by 1/(1-p) to reduce merge interference, like dropout on delta weights.
data flywheel
Self-reinforcing production loop: a better model attracts more users, generating more interaction data that trains the next, better model.
Defined in 12.5 Data Flywheels & Continuous Improvement
data leakage
Information from validation/test data (or the future) contaminating training or preprocessing, making reported performance look better than it truly generalizes.
Defined in 1.5 Machine Learning Fundamentals
Data manifest
A content-addressed record listing the SHA-256, byte size, and token count of every tokenized data shard, plus a corpus-level hash, proving the training data is byte-identical across reruns.
data mixture (mixing weights)
The set of per-domain sampling weights w_i, summing to 1, that determine what fraction of training tokens come from each source corpus.
Defined in 3.1 Pretraining Data: Sources, Crawling & The Data Pipeline · used in 2 chapters
Data parallel (DP) replicas
Independent, identical (possibly TP/PP-sharded) copies of the model each serving separate requests; scales throughput linearly with replica count and adds no communication overhead.
Defined in 7.11 Multi-GPU & Multi-Node Inference
Data parallelism (DP)
Replicating the full model on every worker, splitting the mini-batch across workers, and averaging per-worker gradients (via all-reduce) so replicas stay in sync.
data wall (data-constrained scaling)
The point at which finite high-quality data forces repeated epochs, where each repeated token yields less loss reduction than a fresh one.
Defined in 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond · used in 2 chapters
DataProto
veRL's typed batch container (named tensors plus metadata) that crosses the single-/multi-controller boundary, supporting chunk, concat, and union operations.
datasheet for datasets
A structured dataset-documentation framework (Gebru et al., 2021) covering motivation, composition, collection, preprocessing, uses, and maintenance of a dataset.
Defined in 13.6 AI Governance, Compliance & Regulation
DDIM (Denoising Diffusion Implicit Models)
A non-Markovian, deterministic (ODE-based) sampler sharing DDPM's marginals that allows skipping timesteps, cutting sampling from ~1000 steps to ~20-50.
DDPM (Denoising Diffusion Probabilistic Model)
Ho et al.'s formulation training a network to predict the noise added at each step via a simplified MSE loss, making training stable regression.
dead group
A sampled group where all responses receive equal reward, making every advantage zero and contributing no gradient signal; a key GRPO training pathology to monitor.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
Dead ReLU
A neuron whose pre-activation is negative on every training example, so its ReLU gradient is permanently zero and it never updates again.
debate (AI safety via debate)
Two copies of a model argue opposing answers before a human judge, betting that lies are harder to defend than truths.
debate topology
A peer-to-peer pattern where multiple agents independently argue or solve a problem and a judge (or reconciliation step) reduces overconfidence and variance, at risk of convergence to a shared wrong answer.
Defined in 8.7 Multi-Agent Systems & Orchestration
Decay-phase annealing
Feeding a higher-quality, denser data mix specifically during the WSD decay window, since that window is where most committed loss reduction happens.
Deceptive alignment (alignment faking)
Failure mode where a sufficiently capable model behaves well when it infers it is being trained or evaluated but would behave differently at deployment.
decode
The inference phase generating one token at a time; a matrix-vector product with almost no weight reuse, making it memory-bound.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 3 chapters
decode phase
The memory-bandwidth-bound stage that generates one token at a time via vector-matrix multiplies, streaming model weights from HBM each step.
decode speed ceiling
Upper bound on single-stream tokens/sec, equal to memory bandwidth divided by (model parameter count times bytes per parameter).
decoder-only model
A transformer stack using only causal (lower-triangular) self-attention, with no encoder or cross-attention, predicting each token from prior tokens (e.g., GPT).
decontamination
Filtering synthetic records that n-gram-overlap with evaluation benchmark items, preventing a generator from leaking memorized test questions into training data.
Decoupled RoPE (rotary position embedding)
MLA's fix for RoPE's incompatibility with weight absorption: splitting each key into an absorbable content part (no RoPE) and a small separately cached positional part that carries RoPE.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
decoupled weight decay (AdamW)
Applying θ←(1-ηλ)θ directly to weights rather than adding λθ to the gradient, avoiding Adam's adaptive denominator under-decaying high-gradient parameters.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
Deduplication
Removing (near-)duplicate documents from training data; since memorization scales with duplication count, dedup is the cheapest, highest-ROI privacy defense.
deep-and-thin architecture
At a fixed parameter budget, using many narrow layers (Stack-100M: 30 layers, d_model=512) rather than fewer wide ones, following MobileLLM's finding that depth beats width.
deep-and-thin aspect ratio
Trading network width (d_model) for depth (n_layers) at a fixed parameter budget; sub-billion models favor many narrow layers over few wide ones (MobileLLM, 2024).
DeepEP
DeepSeek's open-sourced expert-parallel communication library providing high-throughput (NVLink+RDMA forwarding) kernels for prefill and low-latency (pure RDMA, hook-overlap) kernels for decode.
defense-in-depth
A safety design where model alignment, input guardrails, output guardrails, and PII redaction operate as independent layers so no single bypassed layer causes a catastrophic failure.
Defined in 12.4 Safety, Guardrails & Content Moderation · used in 2 chapters
deferred rescaling (deferred normalization)
FA2's technique of keeping the output accumulator unnormalized during the inner loop and dividing by the running softmax denominator only once at the end, cutting per-iteration division work.
degradation ladder
A sequence of automatically triggered fallback states (full RAG to retrieval-off to smaller model to cache-only to graceful error) instead of a binary up/down system.
Delay pattern
A codec-token interleaving scheme (used in AudioLM, VALL-E) that shifts each RVQ level right by k steps so a causal model conditions each level on coarser levels of the same and prior frames.
Defined in 10.3 Audio, Speech & Multimodal Fusion
delayed scaling
NVIDIA Transformer Engine technique that picks each step's FP8 scale factor from a rolling history of past tensor amax values, avoiding a stalling reduction on the current tensor before casting.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
Deliberate over-training
Training far past the Chinchilla-optimal token count (here ~200 tokens/param, ~10x optimal) to shrink a served model's inference cost at the price of extra training compute.
dense retrieval
Retrieval paradigm that projects queries and documents into a low-dimensional continuous vector space trained so semantically related pairs are close, contrasted with sparse (BM25) retrieval.
Defined in 9.1 Embeddings & Representation Learning
Dense-equivalent throughput efficiency (η)
Ratio of an MoE deployment's achieved tokens/s to the tokens/s of a dense model with the same active parameter count on the same GPUs; measures how much of MoE's FLOP advantage survives serving overhead.
DetectGPT
Post-hoc detector that flags text as AI-generated when its log-probability under a model is a local maximum relative to small perturbations, a property expected of model-sampled text.
Differential privacy (DP)
A training algorithm is (ε,δ)-differentially private if its output distribution changes by at most a factor e^ε (plus δ) when one training record is added or removed, bounding every membership-inference adversary.
difficulty-targeted selection
Choosing, before generation, which prompts to roll out based on their estimated pass rate landing near a target band (e.g. [0.2, 0.8]) to raise survival under dynamic sampling.
Defined in 6.12 RL Data, Curriculum & Replay Management
diffusion language model (masked/absorbing diffusion)
A text-generation approach that replaces autoregressive left-to-right decoding with a masking-based noise process, predicting all masked tokens in parallel over a small fixed number of passes.
diffusion model
A generative model that learns to reverse a fixed process of gradually adding Gaussian noise to data, sampling by iterative denoising from pure noise.
digit tokenization / place-value scrambling
The problem where a tokenizer chunks multi-digit numbers inconsistently across similar numbers, obscuring positional place value and impairing arithmetic; fixed by per-digit or fixed-size digit grouping.
DINOv2
Self-supervised ViT trained via student-teacher distillation on images alone (no text/labels), producing dense spatial features useful for segmentation and depth.
Defined in 10.1 Vision Transformers & Image Encoders
Direct Preference Optimization (DPO)
A supervised classification loss over preference pairs, derived from the closed-form RLHF optimum, that aligns a policy without a reward model, sampling, or RL.
Defined in 5.7 Direct Preference Optimization & Its Variants · used in 3 chapters
Disaggregated base/adapter execution
An architecture separating shared base-model compute from per-adapter LoRA correction across different machines or pools, used only at extreme adapter counts or memory pressure.
disaggregated prefill/decode
An architecture that runs prefill and decode on separate GPU pools, transferring the KV cache between them to eliminate phase interference.
disaggregated RL
Placement strategy using separate GPU pools for training and rollout, sized independently and connected by a network and weight-sync channel.
dispatch decorator (@register(dispatch=...))
veRL decorator on worker methods declaring how driver-call arguments are split across ranks and how per-rank outputs are recombined, e.g. DP_COMPUTE_PROTO.
dispatcher
PyTorch's routing table that sends each operation to the correct backend implementation based on dispatch keys (device, dtype, layout); torch.compile and vmap hook in at this layer.
Distortion-free watermarking
Kuditipudi et al. construction that samples tokens via an inverse-CDF transform of a key-derived random sequence, preserving the model's exact marginal token distribution while remaining detectable.
Distributed Data Parallel (DDP)
PyTorch's data-parallel wrapper that makes DP fast via gradient bucketing and autograd-hook-driven overlap of communication with backward compute.
Defined in 3.5 Distributed Training I: Data Parallelism, DDP, ZeRO & FSDP · used in 2 chapters
distributional narrowing
A subtler collapse-adjacent failure where synthetic data over-represents common topics/phrasings and under-represents rare entities, dialects, and long-tail facts.
DistServe
The landmark 2024 paper/system that formally quantified prefill-decode interference and posed prefill/decode replica allocation as a goodput-optimization problem.
Document attention mask (packing)
Block-diagonal causal mask (or FlashAttention's varlen/cu_seqlens API) ensuring tokens in a packed training sequence attend only within their own document.
document packing
Concatenating multiple short documents end-to-end (separated by a SEP token) up to the context length to eliminate padding waste in pretraining batches.
Defined in 3.3 The Pretraining Objective & Loss
document-aware attention masking
A causal mask that also forbids attention across packed-document boundaries, so a token in one packed document can never attend into another document sharing its training window.
DoLa (Decoding by Contrasting Layers)
A single-model factuality method that subtracts an intermediate 'premature' transformer layer's log-probabilities from the final layer's, amplifying knowledge acquired in later layers.
Dolma
AI2's 3-trillion-token, extensively documented open pretraining corpus with per-document license tags and un-removed quality-signal taggers, used to train OLMo.
domain (data domain)
A partition of the training corpus, e.g. web, code, math, books, treated as a unit for weighting, upsampling, and evaluation.
domain-adaptive pretraining (DAPT)
Continual pretraining where the new data is concentrated in one domain (e.g., medical, legal, code) rather than a general mix.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
domain-routed quality filtering
Applying a different heuristic filter per source domain (prose vs. code vs. math) rather than one generic filter, since code/math have character distributions the generic prose filter would wrongly reject.
DoRA (Weight-Decomposed Low-Rank Adaptation)
LoRA refinement that decomposes each weight matrix into a magnitude vector and a direction matrix, training magnitude directly while adapting direction with a low-rank update, matching full fine-tuning more closely.
DoReMi (Domain Reweighting with Minimax Optimization)
A method that learns mixture weights automatically via a small reference model and a proxy model trained with online Group-DRO on excess loss.
Double quantization
Quantizing the per-group scale factors themselves (e.g., FP32 to FP8 with a second-level meta-scale) to shave further overhead off the compressed model's memory.
Defined in 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT · used in 2 chapters
DP-SGD (differentially private stochastic gradient descent)
SGD variant that clips each per-example gradient to norm C and adds calibrated Gaussian noise before averaging, giving a formal (ε,δ) privacy guarantee tracked by a privacy accountant.
DPO (Direct Preference Optimization)
A logistic loss over preference pairs (chosen, rejected) that raises the policy's log-probability of the winner relative to a frozen reference, with no reward model or rollouts.
Defined in 14.9 Post-Training: SFT, DPO, and Narrow RLVR (GRPO) That Works at 100M · used in 2 chapters
DPO degeneration
A failure mode where DPO's margin improves while both chosen and rejected log-probabilities fall, since the loss constrains only their difference, pushing mass onto unseen outputs.
DPOTrainer
TRL trainer implementing Direct Preference Optimization, which reparameterizes reward as a policy/reference log-probability ratio, avoiding a separate reward model.
Defined in 6.3 TRL: HuggingFace's RL Library
Dr. GRPO
2025 fix removing GRPO's two length/scale biases: switches to token-level loss aggregation and drops the std-normalization, leaving a mean-only group baseline.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
draft model (drafter)
A small, cheap model that proposes candidate continuation tokens for the target model to verify; only affects speed, never correctness.
dropout
Regularization that randomly zeroes activations with probability p during training, forcing the network to learn redundant, non-co-adapted representations.
Defined in 1.5 Machine Learning Fundamentals
DSPy
A framework that expresses LLM pipelines as typed-signature programs and 'compiles' them against a metric on train/dev splits, auto-tuning instructions and few-shot demos.
Defined in 8.9 Prompt Engineering as Engineering
dual-clip PPO
Adds a lower floor c*A_t on the surrogate objective for negative-advantage tokens with large ratios, preventing exploding negative gradients from drifted policies.
dual-LLM pattern
An architecture splitting a privileged orchestrator from an unprivileged reader model that processes untrusted content and returns only structured, schema-constrained output.
dynamic batching
Waits a short window to accumulate arriving requests before dispatching one frozen batch; bounds queueing delay but still freezes membership for the whole generation.
Defined in 7.2 Continuous Batching & Request Scheduling
Dynamic sampling (DAPO)
Discarding prompt-groups whose rewards are all identical (zero gradient) and oversampling new prompts until a full batch of informative, non-degenerate groups is collected.
Defined in 6.11 Scaling RL: Throughput, Load Balancing & The Latest Tricks · used in 2 chapters
E
EAGLE
A self-drafting method that autoregresses in the target's continuous feature space (penultimate-layer hidden states) rather than token space, giving much higher acceptance than Medusa.
Earley parser
An incremental parsing algorithm used by llguidance to compute valid next tokens on the fly, avoiding DFA state explosion for complex grammars.
Defined in 7.10 Structured & Constrained Generation
early fusion
A multimodal design where image and text become one joint token stream (e.g. via a discrete visual codebook) attended over from the first layer, with no separate vision tower at inference.
Defined in 10.2 Vision-Language Models
early stopping
Regularization method that halts training when validation loss stops improving, restoring the best checkpoint instead of adding a loss penalty.
Defined in 1.5 Machine Learning Fundamentals
Eckart-Young theorem
Theorem stating the best rank-k approximation of a matrix in Frobenius norm is obtained by truncating its SVD to the top k singular components.
Defined in 1.1 Linear Algebra for Deep Learning
effective epochs
How many times a domain's unique tokens are revisited during training, e_i = w_i D / n_i; the core unit for reasoning about repetition.
effective window
The context length at which a model's accuracy on a task is still acceptable, measured empirically; typically much smaller than the advertised maximum window.
Defined in 8.4 Context Engineering & Management
eigendecomposition
Factorization A = QΛQ^T of a symmetric matrix into orthogonal eigenvectors Q and a diagonal matrix Λ of eigenvalues, revealing stretch directions.
Defined in 1.1 Linear Algebra for Deep Learning
einsum (Einstein summation notation)
A notation (`torch.einsum`) expressing matrix multiplication, batch matmul, outer products, and traces as one general tensor-contraction operation.
Defined in 1.1 Linear Algebra for Deep Learning
elastic training
Resuming or continuing a distributed training job on a different number of nodes/GPUs than it was running on when it crashed or paused.
Elo rating system
A logistic rating model (from chess) that converts pairwise win/loss/tie outcomes into a global skill rating per model, used to build leaderboards like Chatbot Arena.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
embarrassingly parallel processing
Pipeline architecture in which each WARC/WET file is an independent unit a worker can process alone, needing no cross-worker coordination until dedup.
embedding dimension (d_model)
The width of each token vector and of the residual stream throughout the Transformer; a central architectural hyperparameter.
Defined in 2.2 Embeddings & The Input Pipeline
embedding lookup (index select)
Implementing token-to-vector mapping as an O(Td) row copy from the embedding table rather than an O(TVd) one-hot matrix multiply.
Defined in 2.2 Embeddings & The Input Pipeline
embedding matrix
Learned weight tensor W_E of shape [V, d] whose rows (or columns) are dense vectors representing each vocabulary token.
Defined in 2.2 Embeddings & The Input Pipeline
embedding norm clipping
Projecting embedding row vectors back to a bounded norm ball after each optimizer step, preventing rare-token embeddings from drifting unboundedly.
embedding table
The V×d_model parameter matrix mapping token ids to vectors; its size scales linearly with vocab_size and, when tied, is also the output projection.
emergent abilities
Capabilities that appear to jump abruptly at scale under discontinuous, all-or-nothing metrics, though underlying per-token loss improves smoothly with scale.
emergent curriculum
The training band's contents drifting automatically from easy to hard prompts as online difficulty tracking follows the improving policy, rather than a hand-authored stage schedule.
Defined in 6.12 RL Data, Curriculum & Replay Management
Emergent misalignment
Phenomenon where narrow finetuning on one harmful behavior (e.g., writing insecure code) induces broadly misaligned outputs on entirely unrelated prompts.
encoder-decoder model
An architecture pairing a bidirectional encoder stack with an autoregressive decoder stack connected via cross-attention (e.g., T5, BART).
encoder-only model
A transformer stack using fully bidirectional attention (no causal masking), producing contextual representations but unable to generate autoregressively (e.g., BERT).
engine-mismatch bias
The systematic disagreement between log-probs computed by the inference engine (e.g., vLLM in fp8) and the training engine (e.g., FSDP in bf16) for identical weights and tokens, biasing the importance ratio even at zero staleness.
Environment fingerprint
A captured snapshot of Python/PyTorch/CUDA/cuDNN versions, GPU model, and git commit/dirty status, recorded so non-code sources of divergence (library or hardware changes) can be ruled out.
EP small-batch trap
The failure mode where a small decode batch under expert parallelism leaves most expert GPUs idle while the all-to-all cost is still paid, wasting the cluster.
Defined in 7.11 Multi-GPU & Multi-Node Inference
episodic memory
Timestamped, situated record of a specific past event or interaction; append-only, grows monotonically, queried with temporal or situational cues.
Defined in 8.5 Memory Systems for Agents
epoch count
Number of times a domain's corpus is repeated during training, computed as N·w_i / |D_i|; repetition beyond ~4 epochs degrades generalization.
error budget
The allowed amount of SLO violation (e.g., 43.2 minutes of downtime or a bad-response quota) before triggering escalation or feature freezes.
error-as-observation
The design pattern of feeding tool failures back into the context as normal (is_error=True) tool results so the model can self-correct, instead of crashing the loop.
error-as-tool-result pattern
Feeding parse, validation, or execution errors back into the conversation as if they were normal tool results, so the model can self-correct on the next turn.
Defined in 8.1 Tool Use & Function Calling
EU AI Act
Regulation (EU) 2024/1689; the world's most comprehensive AI law, phasing in risk-tiered obligations and fines from 2025-2027.
Defined in 13.6 AI Governance, Compliance & Regulation
EU AI Act Article 50
EU AI Act transparency provision requiring machine-readable labeling of AI-generated content and robust watermarking for AI text used in public communication, effective 2026.
Eval harness (four-layer eval stack)
A repeatable measurement framework of layered checks (cheap regression tests up through human review) used to verify prompt changes actually improve quality.
Defined in 8.9 Prompt Engineering as Engineering · used in 2 chapters
eval-gated deployment
An automated pass/fail decision function (win rate, regression suite, safety thresholds) that a candidate model must clear before it can deploy.
Defined in 12.5 Data Flywheels & Continuous Improvement
Evidence Lower Bound (ELBO) for masked diffusion
The mask-rate-weighted (by 1/(1-alpha_t)) cross-entropy loss on masked positions, shown to be a valid variational bound on the true data log-likelihood.
exact caching
Storing a response keyed by a hash of the full request (model, messages, sampling params) and returning it verbatim on a byte-identical repeat request.
Exact deduplication
Removing documents or paragraphs whose normalized byte sequence (or content hash, e.g. SHA-256) is identical to one already seen.
Defined in 3.2 Data Cleaning, Deduplication & Quality Filtering · used in 2 chapters
exact resumability
Property that a training run interrupted and resumed from checkpoint produces (bit-for-bit or statistically) the same weight trajectory as an uninterrupted run.
excess loss
A proxy model's per-domain loss minus a reference model's loss on that domain, clamped at zero; measures closeable, not raw, loss.
Experience buffer
The in-memory, short-lived dataset of one iteration's rollouts (tokens, behavior log-probs, rewards, advantages) that the learner draws minibatches from.
Defined in 6.1 The Anatomy of an RL-for-LLM System
Expert offloading
Keeping hot/shared parameters resident in HBM while streaming cold routed experts from CPU RAM or NVMe on demand, trading latency for memory capacity.
Expert parallelism (EP)
Places different experts of a Mixture-of-Experts layer on different devices, routing tokens to their chosen experts via dispatch and combine all-to-all collectives.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 5 chapters
Expert-choice routing
A routing scheme where each expert picks its top-C tokens rather than tokens picking experts, guaranteeing perfect load balance but breaking causal autoregressive decoding.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
exponential moving average (EMA) difficulty
Online running estimate p̄ ← (1-α)p̄ + α·p̂_t updated after each rollout group, tracking a task's pass rate as the policy improves (non-stationary difficulty).
Defined in 6.12 RL Data, Curriculum & Replay Management
exposure bias
The training/inference mismatch where a model conditions on ground-truth prefixes during teacher-forced training but on its own generated tokens at inference, letting small errors compound into degenerate repetition.
external fragmentation
Free GPU memory that is unusable because it is split into scattered holes, none individually large enough to satisfy a new contiguous request.
F
Faithfulness
RAGAS metric measuring the fraction of a generated answer's atomic claims that are entailed by the retrieved context; low scores signal hallucination.
false discovery rate (FDR) / Bonferroni correction
Multiple-testing corrections (Benjamini-Hochberg FDR, Bonferroni) that control the elevated false-positive probability when running many simultaneous experiments.
feed-forward network (FFN) / MLP sublayer
The per-token, position-independent nonlinear sublayer of a transformer block, typically the largest parameter block (~2/3 of total).
feedback-loop bias
Distortion where model outputs become future training or usage inputs, so metrics improve because the world adapted to the model rather than the model improving.
Few-shot learning (in-context learning)
Providing input-output example pairs in the prompt that shift the model's output prior toward a format, register, and rubric without any weight update.
Defined in 8.9 Prompt Engineering as Engineering
file-as-memory pattern
Using a structured file-system directory (markdown/JSON files, indexed by grep or a vector index) as the agent's persistent, human-auditable memory store.
Defined in 8.5 Memory Systems for Agents
filtered ANN
Nearest-neighbor search constrained by structured predicates (e.g., tenant_id, date); naive pre-filter or post-filter strategies can break recall or waste work.
final norm
An extra normalization layer (e.g. ln_f, norm) after the last block in pre-norm architectures, needed since pre-norm never normalizes the final block's output.
Fine-grained and shared experts
DeepSeek-MoE refinements: splitting experts into many smaller ones (more combinable specialization at fixed active cost) plus always-on shared experts absorbing common knowledge.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
FineWeb / FineWeb-Edu
15-trillion-token Common Crawl-derived dataset showing aggressive quality filtering alone rivals curated mixes; the Edu subset filters for educational value.
FineWeb-Edu educational classifier
A cheap regression head trained on frozen embeddings, distilled from LLM-assigned 'educational value' labels, used to filter the web for textbook-quality pages.
finite-state machine (FSM)
A 5-tuple (states, alphabet, transition function, start state, accepting states) that recognizes regular languages; used to enforce regex-like constraints.
Defined in 7.10 Structured & Constrained Generation
first-fit-decreasing bin packing
A packing algorithm that sorts examples longest-first and greedily places each into the first bin with enough remaining room.
FitNet-style feature distillation
Distillation that matches intermediate hidden states or attention patterns between teacher and student, via a projector head, rather than only output distributions.
flash-attn (FlashAttention library)
A pip package implementing FlashAttention's tiled, IO-aware attention kernel that avoids materializing the full N×N attention matrix; must be installed with --no-build-isolation after torch.
Defined in Tooling & Environment Setup Cheatsheet
FlashAttention (online softmax)
An exact attention algorithm that fuses QK^T, softmax, and the value multiply into one tiled kernel, never materializing the full N x N score matrix in HBM.
Defined in 4.2 FlashAttention I: IO-Awareness & The Online Softmax · used in 4 chapters
FlashDecoding
An attention kernel strategy for single-token decode that parallelizes over the KV sequence (split-KV) rather than queries, keeping SMs busy when the query-parallel axis has no width.
FlatParameter
FSDP's data structure: all parameters within an FSDP unit (e.g. one transformer block) flattened into a single 1-D buffer, then sharded evenly across ranks.
Flow matching
Generative training objective that regresses a constant velocity field along the linear interpolation path between noise and data, used by Transfusion's image head.
Defined in 10.5 Unified & Any-to-Any Models
flow matching / rectified flow
A generalization of diffusion that trains a velocity field along a straight-line path between noise and data, removing the Markov chain and needing far fewer ODE integration steps.
fork/join (SGLang frontend primitives)
DSL operations that split a program's state into k branches sharing the parent's KV prefix (fork), running in parallel, then recombine their results (join).
formal-proof benchmark
Math eval (e.g. miniF2F, ProofNet, PutnamBench) where the answer is a complete proof in a proof assistant like Lean 4, verified by its type-checker with no equivalence heuristic.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Format reward
A small auxiliary reward for producing the expected output structure (e.g., a boxed answer), added to fix sparse-reward cold-start problems.
forward process (noise schedule)
The fixed Markov chain q(x_t|x_{t-1}) that adds Gaussian noise per step, governed by a schedule {β_t} so x_T approaches pure noise.
four-root-cause diagnosis tree
A triage guide checking model/provider regression, prompt change, retrieval drift, and upstream data regression in parallel when a quality or latency alert fires.
fp16 (IEEE half precision)
16-bit float with a 5-bit exponent and 10-bit mantissa; overflows above ~65504, which forces loss scaling during training.
Defined in 1.4 Numerical Computing, Floating Point & Precision · used in 2 chapters
FP8 (8-bit floating point)
8-bit format (E4M3 for activations/weights, E5M2 for gradients) needing per-tensor or per-block dynamic scaling due to only 2-3 mantissa bits.
Defined in 1.4 Numerical Computing, Floating Point & Precision · used in 4 chapters
Frobenius norm
Matrix norm equal to the square root of the sum of squared entries (equivalently sqrt of summed squared singular values); the natural l2 norm on matrices.
Defined in 1.1 Linear Algebra for Deep Learning
FSDP (Fully Sharded Data Parallel)
PyTorch-native implementation of the ZeRO-3 idea, sharding a per-unit FlatParameter across ranks and all-gathering/freeing it just-in-time for forward and backward.
FSDP FULL_SHARD (ZeRO-3)
A Fully Sharded Data Parallel strategy that shards model parameters, gradients, and optimizer state across all ranks, used for the 8-GPU and multi-node scales in this chapter.
Full-duplex spoken dialogue
A real-time conversational mode (as in Moshi) where the model continuously listens and speaks simultaneously via parallel audio token streams and a hierarchical temporal architecture.
Defined in 10.3 Audio, Speech & Multimodal Fusion
Fully Sharded Data Parallel (FSDP)
Distributed training strategy where each rank holds only a disjoint shard of every parameter, complicating naive whole-model checkpoint save/load.
Defined in 3.12 Checkpointing, Fault Tolerance & Long-Running Jobs · used in 2 chapters
function-calling fine-tune
SFT (optionally plus RL) on multi-turn conversations containing tool calls and results, teaching a model when and how to invoke tools.
Defined in 8.1 Tool Use & Function Calling
function-preserving growth
Architecture-expansion technique (depth or width) that initializes new parameters so the grown model computes exactly the same function as the original at step 0.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
FX graph
PyTorch's traced intermediate representation of tensor operations, built by TorchDynamo as it walks Python bytecode and encounters PyTorch ops.
G
G-Eval
An evaluation framework (Liu et al., 2023) that has the judge produce chain-of-thought rationale before emitting a score, improving calibration versus a bare score.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
GAE (Generalized Advantage Estimation)
Exponentially-weighted sum of TD residuals from a learned critic, computed by a backward recurrence; the gamma-lambda knob trades bias against variance.
GAIA (General AI Assistants benchmark)
Benchmark of exact-match, multi-step reasoning-plus-tool-use questions across three difficulty levels, testing general assistant capability rather than coding.
Defined in 8.8 Agent Evaluation & Benchmarks
gang scheduling
Launching all resource bundles of a multi-GPU actor group atomically and simultaneously (e.g., via Ray's STRICT_PACK placement group) rather than acquiring them incrementally.
gated cross-attention (Flamingo-style)
New cross-attention layers inserted into a frozen LLM that let hidden states attend to vision features, scaled by a learnable gate initialized to zero.
Defined in 10.2 Vision-Language Models
Gated Linear Attention (GLA)
A unifying recurrence S_t = G_t ⊙ S_{t-1} + k_t^T v_t whose gate matrix G_t choice recovers RetNet, RWKV, Mamba, or plain linear attention as special cases.
Gaudi
Intel's accelerator line pairing programmable TPC vector cores with systolic MME matmul blocks and on-die RoCE Ethernet ports for standard-fabric scale-out.
GeGLU
Gated activation identical to SwiGLU but using GELU instead of Swish as the gate function; used in Gemma 2.
GELU (Gaussian Error Linear Unit)
Smooth activation x·Φ(x) using the standard normal CDF; used in BERT, GPT-2, and GPT-3 in place of ReLU.
GEMV (general matrix-vector multiply)
Multiplying a weight matrix by a single activation vector, streaming the whole matrix from HBM for one output; intensity ≈1 FLOP/B, as in single-stream decode.
Generalized Advantage Estimation (GAE)
A (γ,λ)-weighted sum of TD residuals δ_t, computed by a single backward recursion, that interpolates between low-variance one-step TD and unbiased Monte Carlo advantage estimates.
Generation efficiency (η_gen)
Mean sequence length over max sequence length in a generation batch; the fraction of fleet token-time that is useful work rather than tail-waiting.
generation prompt
The template suffix (e.g. <|im_start|>assistant\n) appended after the last user turn to signal the model should generate next.
generation scoring (generate_until)
A harness scoring mode that freely generates text with stop strings, then extracts and compares an answer via a process_results function, used for tasks like GSM8K.
Defined in 11.3 Building Eval Harnesses
GGUF (GPT-Generated Unified Format)
The self-describing binary container format used by llama.cpp, bundling quantized weights with tokenizer and architecture metadata in a single portable file.
Defined in 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT · used in 2 chapters
Gibbs/Boltzmann distribution (partition function Z(x))
The closed-form optimal policy for the KL-regularized RLHF objective: the reference policy reweighted by exponentiated reward, normalized by an intractable sum Z(x) over all responses.
glitch token
A vocabulary entry so rare in pretraining data that its embedding stays near random initialization, causing bizarre model behavior when invoked (e.g. SolidGoldMagikarp in GPT-2/3).
global gradient norm clipping
Rescaling the gradient vector across all parameters jointly (not per-tensor) so its L2 norm does not exceed a threshold, catching model-wide gradient explosions before the optimizer step.
global memory coalescing
The hardware's combining of a warp's global-memory accesses into as few 128-byte transactions as possible, achieved when threads read consecutive addresses.
Goal misgeneralization
Failure where a model learns a superficial correlate of the intended goal in training distribution A and generalizes the correlate, not the goal, to distribution B.
Goodhart's law
Principle that optimizing a proxy measure causes it to diverge from the true target; manifests in LLMs as regressional, extremal, causal, and adversarial modes.
Defined in 5.13 Reward Hacking, Over-Optimization & Alignment Failures · used in 2 chapters
goodput
The rate of requests served while meeting latency SLOs, as opposed to raw throughput that ignores whether responses arrive fast enough.
Gopher-style quality heuristics
Document-level filters (length, terminal-punctuation lines, word-repetition ratio, symbol density) used to strip low-quality or spam text from web crawls.
GPAI (general-purpose AI) model
An AI model trained on broad data via self-supervision at scale, capable of many downstream tasks; every large pre-trained LLM qualifies under Article 53.
Defined in 13.6 AI Governance, Compliance & Regulation
GPQA (Graduate-Level Google-Proof Q&A)
A ~450-question science benchmark designed to be hard for non-experts even with internet access, giving a meaningful non-trivial human-expert ceiling (60-70%).
GPTConfig
A dataclass naming every architectural hyperparameter (block_size, vocab_size, n_layer, n_head, n_embd, dropout, bias) used to construct and serialize a GPT model.
GPTQ
A post-training weight-quantization algorithm that minimizes layer output error using the input Hessian, quantizing columns one at a time and compensating remaining weights via H^{-1} (Cholesky-stabilized).
Defined in 4.7 Quantization I: Post-Training Quantization (GPTQ, AWQ, SmoothQuant) · used in 2 chapters
GPU utilization (training-capable)
The fraction of iteration wall-clock time training GPUs spend on compute-bound work versus idling or doing memory-bound decode, a key metric for comparing placement strategies.
GQA (Grouped Query Attention)
An attention variant where multiple query heads share a smaller number of key/value heads, shrinking KV cache memory by the grouping ratio with minimal quality loss.
Defined in 2.10 Modern Architecture Improvements & Design Choices · used in 4 chapters
GQA/MQA (grouped-query attention / multi-query attention)
Attention variants where multiple query heads share fewer key/value heads, shrinking the per-token KV cache footprint independent of query-head count.
GRACE
A memory-adapter editor that stores edits as (key, value) pairs in a discrete codebook, replacing activations that fall within a learned epsilon-ball, base weights untouched.
Defined in 13.2 Knowledge Editing & Machine Unlearning
grad_fn / computation graph
The attribute on a non-leaf tensor pointing to the Function node that produced it; PyTorch builds this graph dynamically (define-by-run) during the forward pass.
gradient
Vector of partial derivatives of a scalar function pointing in the direction of steepest increase; negating it gives the steepest-descent direction used by gradient descent.
Defined in 1.3 Calculus, Optimization & Convexity
gradient accumulation
Simulating a larger effective batch by summing (loss-averaged) gradients over several micro-batches before calling optimizer.step(), used when the target batch doesn't fit in memory.
Defined in 3.10 Learning Rate Schedules, Warmup, Batch Size & Hyperparameters · used in 5 chapters
gradient bucketing
Technique of grouping small gradient tensors into larger buffers (e.g., 25 MB) before issuing all-reduce, amortizing per-call latency and enabling overlap with backward computation.
Defined in 1.9 Parallel Computing & Collective Communication · used in 2 chapters
Gradient checking
Verifying a hand-written analytic gradient by comparing it to the central-difference numerical gradient; near-zero relative error confirms correctness.
gradient checkpointing
Technique that recomputes activations during backward instead of storing them, trading one extra forward pass for roughly halved activation memory.
Defined in 1.7 Automatic Differentiation & PyTorch Internals · used in 2 chapters
gradient clipping (global-norm clip)
Rescaling the gradient vector so its norm never exceeds a fixed threshold, preventing one abnormally large update from destabilizing training.
Defined in 1.3 Calculus, Optimization & Convexity · used in 2 chapters
gradient highway
The identity term I in the backward-pass product (I + ∂F/∂x) contributed by residual connections, preventing vanishing gradients in deep stacks.
gradient noise scale
The ratio tr(Σ)/‖G‖² of per-sample gradient covariance to squared mean gradient, used to locate the critical batch size boundary between noise-limited and saturated training regimes.
gradient norm clipping
Rescaling all gradients by min(1, τ/‖g‖₂) so updates never exceed a threshold τ, preserving gradient direction while capping spike-driven update size.
GradScaler (dynamic loss scaling)
PyTorch's AIMD controller that adapts the loss-scale constant: halves it and skips the optimizer step on detected inf/NaN gradients, doubles it after many clean steps.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
gradual silent quality collapse
A failure mode where quality degrades slowly over days from provider drift, prompt edits, or retrieval staleness, with no HTTP errors or latency signal.
graph break
A point where TorchDynamo cannot trace further (e.g., print, data-dependent control flow, tensor.item()) and falls back to eager execution, fragmenting compilation.
Graph of Thoughts (GoT)
Generalization of Tree of Thoughts where reasoning states form a DAG and independently-derived branches can merge into a single node.
GraphRAG
RAG variant that replaces flat chunk indexes with an LLM-extracted entity/relationship graph, clustered into communities and summarized for local or global search.
Greedy Coordinate Gradient (GCG)
A gradient-based algorithm that searches for adversarial token suffixes minimizing the loss toward a compliant response, transferring across models.
Greedy Coordinate Gradient (GCG) attack
A gradient-based jailbreak that optimizes a token suffix appended to a prompt to maximize probability of a compliant target response.
greedy decoding
Picking the argmax token at every step; deterministic and fast but locally optimal, causing repetitive low-entropy text like 'mat mat mat'.
Green list / red list
At each decoding step, a pseudorandom hash of prior context and a secret key partitions the vocabulary into a green subset (boosted) and red subset (unboosted).
Group Distributionally Robust Optimization (Group-DRO)
An optimization framework minimizing the worst-case loss over an adversarially reweighted set of domains, so no domain is left underperforming.
Group Relative Policy Optimization (GRPO)
Critic-free RL algorithm that samples a group of completions per prompt and normalizes advantages by the group's mean and standard deviation instead of using a value network.
Defined in Key Papers: An Annotated Reading List · used in 2 chapters
Group-relative advantage (GRPO)
Advantage computed by normalizing each response's reward against the mean and standard deviation of rewards within its sampled group, requiring no critic.
Defined in 6.1 The Anatomy of an RL-for-LLM System · used in 3 chapters
group-relative advantage / z-score
GRPO's advantage: (reward − group mean) / (group std + epsilon), computed per prompt across G sampled responses and applied uniformly to all their tokens.
Defined in 5.8 GRPO, RLOO & Critic-Free RL
group-size quantization
A quantization layout that assigns one shared scale (and zero-point) to each contiguous block of a fixed number of weights, trading finer granularity for more overhead as the group shrinks.
Grouped GEMM
A single fused kernel that runs each local expert's matmul over its assigned (ragged) group of tokens in one launch, the compute step of EP after dispatch.
Grouped-Query Attention (GQA)
Interpolation between MHA and MQA where h query heads are partitioned into g KV groups, each group sharing one key/value head; cache scales with g instead of h.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA · used in 6 chapters
GRPO (Group Relative Policy Optimization)
Critic-free RL method that assigns every token the group-standardized (mean-subtracted, std-divided) reward as advantage, optimized via PPO's clipped surrogate for multi-epoch updates.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 5 chapters
GRPOTrainer
TRL trainer implementing Group Relative Policy Optimization, which drops the value-head critic and computes advantages from the mean reward of a sampled group of completions per prompt.
Defined in 6.3 TRL: HuggingFace's RL Library
GSPO (Group Sequence Policy Optimization)
Sequence-level variant of GRPO that clips whole-sequence rather than per-token importance ratios, used for extra stability on MoE policies (e.g., Qwen3).
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
guardrail
A classifier, heuristic, or policy layer that inspects requests before the LLM sees them or responses before the user sees them, independent of the model's own alignment.
Defined in 12.4 Safety, Guardrails & Content Moderation
guardrail metric
A do-no-harm metric (e.g. latency, safety, cost) that must not regress beyond a threshold; violating it halts a rollout regardless of primary-metric gains.
H
Hadamard transform
A structured orthogonal matrix (random-sign Hadamard, computed in O(d log d)) used as the practical, cheap rotation that implements incoherent processing in FA3.
hallucinated observation
Failure mode where the model generates a fabricated Observation line in the same completion as its Action, bypassing real tool execution while looking correct in logs; prevented by a stop sequence at 'Observation:'.
hard negatives
Passages that are superficially relevant but incorrect, mined via BM25, an earlier dense-model checkpoint (ANN-mining), or a cross-encoder filter, to give richer training gradients than random negatives.
Defined in 9.1 Embeddings & Representation Learning
Hard-binding / soft-binding
C2PA techniques binding a manifest to a file: hard-binding uses an exact cryptographic hash (breaks on any edit); soft-binding uses a perceptual hash that survives lossy transformations.
hard-negative mining
Automatically identifying examples where the model confidently produced a verifiably wrong answer (via a test oracle) and routing them straight to training.
Defined in 12.5 Data Flywheels & Continuous Improvement
Hardware FLOP Utilization (HFU)
Fraction of peak FLOP/s spent on all FLOPs actually issued to the GPU, including wasted recomputation from activation checkpointing; always HFU >= MFU.
HarmBench
A standardized red-teaming benchmark of 510 harmful behaviors with fixed attack methods and a grading model for cross-model ASR comparison.
harness
The deterministic program (prompt, tools, context assembly, control loop, permissions, verification) wrapping a model to turn it into a working coding agent.
harness effects
The systematic swing in benchmark scores (often 10-20 percentage points) caused by scaffold choices such as file localization strategy, iteration budget, tool suite, and context truncation policy, independent of the underlying model.
Defined in 8.8 Agent Evaluation & Benchmarks
hashing trick (feature hashing)
Mapping tokens to fixed-width vector indices via a hash function to build a dependency-free 'embedding-lite' retriever with no trained encoder.
HBM (high-bandwidth memory)
The GPU's large, stacked DRAM off the compute die; roofline 'bytes moved' counts traffic between HBM and on-chip registers/SRAM.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 2 chapters
Head dimension
The width d_h = d_model / h of each attention head's Q/K/V subspace; the KV cache and MHA/GQA/MQA formulas are all expressed in terms of it.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
head-of-line (HOL) blocking
A newly arrived request must wait for an entire in-flight batch to drain before being admitted, inflating time-to-first-token under request-level batching.
Defined in 7.2 Continuous Batching & Request Scheduling
HELM (Holistic Evaluation of Language Models)
An evaluation framework organized as scenario x adaptation x metric, reporting multiple axes (accuracy, calibration, fairness, toxicity, efficiency) rather than one aggregate score.
Defined in 11.3 Building Eval Harnesses
Hessian (in GPTQ)
Matrix of all second-order partial derivatives of a function; its eigenvalues determine whether a critical point is a minimum, maximum, or saddle.
Defined in 1.3 Calculus, Optimization & Convexity · used in 2 chapters
HFU (Hardware FLOPs Utilization)
Like MFU but also counts activation-recomputation FLOPs, so HFU is always at least MFU; the gap reveals checkpointing overhead.
Hierarchical Navigable Small World (HNSW)
Graph-based ANN index with layered proximity graphs enabling greedy, logarithmic-hop search; typically the best recall/latency for in-memory data.
high-bandwidth memory (HBM)
The GPU's off-chip stacked DRAM (e.g. 80 GB on H100) holding weights, activations, and KV cache; its capacity and bandwidth bound model size and decode speed.
Defined in 1.8 GPU Architecture & The Memory Hierarchy
high-risk AI system
An AI system listed in Annex III of the EU AI Act (e.g., employment, credit scoring, biometric ID) subject to logging, human-oversight, and conformity requirements.
Defined in 13.6 AI Governance, Compliance & Regulation
HIP (Heterogeneous-compute Interface for Portability)
AMD's CUDA-like kernel language whose API calls mechanically rename cuda* to hip*, letting most CUDA source compile for AMD with minimal changes.
HiPPO matrix
The structured initialization of an SSM's state matrix A (used in S4) that projects input history onto orthogonal polynomials for near-optimal long-range memory compression.
HippoRAG
Method combining dense retrieval with an entity graph, using Personalized PageRank seeded at query-matched entities to propagate relevance transitively for multi-hop retrieval.
Host
The end-user application (e.g. Claude Desktop) that owns the conversation loop, invokes the LLM, and decides when to call tools via one or more MCP clients.
Defined in 8.6 The Model Context Protocol (MCP)
Hot-expert skew / imbalance factor (IF)
Uneven routing load across GPUs, quantified as max per-rank token load over mean load; since the all-to-all is a barrier, effective compute time scales with IF.
HTTP transport (SSE / Streamable HTTP)
MCP transport for network-hosted, multi-tenant servers, using Server-Sent Events or the newer unified Streamable HTTP endpoint with OAuth 2.1 auth.
Defined in 8.6 The Model Context Protocol (MCP)
Huber loss
A loss function that is quadratic near zero and linear for large residuals, used here on log-space fit residuals so a few noisy ladder runs cannot dominate the scaling-law fit.
HuBERT
A BERT-style self-supervised speech model that predicts k-means cluster assignments (pseudo-labels) of masked audio frames, yielding discrete semantic units used as targets for speech LMs.
Defined in 10.3 Audio, Speech & Multimodal Fusion
HumanEval
Seminal 164-problem Python coding benchmark (Chen et al. 2021) with unit tests, which introduced the unbiased Pass@k estimator.
Defined in 11.4 Reasoning, Coding & Agentic Evals
hybrid architecture (attention + SSM)
A language model interleaving a small fraction of full-attention layers with many SSM/linear-attention layers to combine exact retrieval with cheap long-context memory, e.g. Jamba, Nemotron-H.
hybrid engine
veRL's colocated pattern where a single set of workers flips between a training role (FSDP/Megatron) and a generation role (vLLM/SGLang) via sleep/wake.
Hybrid search
Combining sparse lexical retrieval (BM25) with dense embedding retrieval so exact-match and paraphrase queries are both handled well.
Defined in 9.4 Chunking, Reranking & Hybrid Search · used in 2 chapters
HYBRID_SHARD
An FSDP sharding strategy that shards parameters within a node (fast NVLink) but replicates the sharded model across nodes, avoiding slow inter-node all-gathers.
Defined in 3.5 Distributed Training I: Data Parallelism, DDP, ZeRO & FSDP · used in 2 chapters
HybridFlow
The programming model (from Sheng et al., 2024) combining single-controller dataflow between RL stages with multi-controller SPMD computation within each stage.
HyDE (Hypothetical Document Embeddings)
Query-rewriting technique where an LLM generates a hypothetical answer document, which is embedded and used as the retrieval query instead of the raw question.
Defined in 9.3 Retrieval-Augmented Generation Architectures · used in 2 chapters
I
IA3 (Infused Adapter by Inhibiting and Amplifying Inner Activations)
Rescales key, value, and FFN activations with learned per-channel vectors; can be folded into weights for zero inference overhead.
ICI (Inter-Chip Interconnect)
TPU's dedicated interconnect wiring thousands of chips into a 2D/3D torus so collectives flow ring-to-ring without a central switch.
Identity Preference Optimization (IPO)
A DPO variant replacing the log-sigmoid loss with a squared-error loss targeting a finite margin $1/(2\beta)$, preventing unbounded margin growth on near-deterministic preference labels.
IEEE 754
Standard defining how floating-point numbers are encoded (sign, exponent, mantissa) and rounded; nearly all ML accelerators implement it.
implicit reward
The quantity $\beta\log(\pi_\theta/\pi_{\text{ref}})$, recovered by inverting the closed-form RLHF policy; a reward computed from two log-probabilities, no reward network needed.
importance ratio (importance sampling)
The ratio πθ/πθ_old of current- to behavior-policy probabilities for sampled tokens, used with clipping to correct for off-policy drift.
Defined in 6.1 The Anatomy of an RL-for-LLM System · used in 2 chapters
importance ratio / PPO clipping
Ratio rho_t of current to old policy probability on a token, clamped by PPO's clipped surrogate so a single update cannot move too far from the rollout policy.
Importance sampling (importance ratio)
Reweighting rollouts from an old behavior policy by r_t(θ)=π_θ/π_old so gradients computed on stale data remain valid estimates for the current policy, enabling multiple update epochs per batch.
importance sampling ratio
Per-token ratio pi_theta/pi_behavior correcting for the mismatch between the training policy and the (possibly older) policy that generated the data.
in-batch negatives
Using the other N-1 (query, document) pairs in a training mini-batch as negatives for each query, so larger batches give more negatives per query.
Defined in 9.1 Embeddings & Representation Learning
In-flight batching
TensorRT-LLM's term for continuous batching: inserting new requests into an active GPU batch at token boundaries without stopping generation.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
in-memory checkpointing
Fault-tolerance technique keeping the most recent checkpoint in CPU DRAM across nodes for near-instant recovery, falling back to disk only for full node failures.
incoherent processing
Multiplying Q and K by a random orthogonal matrix before FP8 quantization so QK^T is unchanged but outliers are spread across coordinates, keeping the FP8 scale tight.
indirect injection
Injection where the attacker plants malicious instructions in data the model later reads (web pages, emails, docs), not in the user's own message.
induction head
An attention head, working with a previous-token head one layer earlier, that implements the pattern-copying rule '[A][B]...[A] → predict [B]', underlying much in-context learning.
inference-aware over-training
Deliberately training a smaller model on far more than 20 tokens/parameter to minimize total lifetime (train + inference) compute when serving heavily.
InfiniBand
High-speed inter-node network fabric (HDR/NDR/XDR) connecting GPU nodes, roughly 10-50x slower than intra-node NVLink, often the bottleneck in multi-node training.
InfoNCE loss (NT-Xent)
Contrastive cross-entropy loss over a similarity matrix that pulls a query's embedding toward its positive document and away from all other in-batch documents, scaled by temperature.
Defined in 9.1 Embeddings & Representation Learning · used in 2 chapters
instruction embeddings
Embedding models that prepend a natural-language task description to the query at inference time so the same frozen backbone adapts its representation to different retrieval tasks.
Defined in 9.1 Embeddings & Representation Learning
instruction hierarchy
A training-level (and reinforceable prompt-level) defense that makes the model treat system-prompt instructions as higher priority than user-turn instructions, resisting override attempts.
Defined in 12.4 Safety, Guardrails & Content Moderation
instruction-augmented pretraining
Injecting grounded reading-comprehension QA pairs synthesized from raw passages directly into the pretraining mix so the base model arrives at SFT already instruction-fluent.
inter-token latency (ITL)
The time between successive generated tokens for a decoding request; spikes when a large prefill is admitted into the same iteration as ongoing decodes.
Defined in 7.2 Continuous Batching & Request Scheduling · used in 2 chapters
interleaving
An experiment design showing both models' outputs within the same user session to measure preference (win rate), eliminating within-user variance for far smaller samples than A/B.
internal fragmentation
Wasted memory inside a reserved allocation, caused by reserving space for the worst-case `max_seq_len` when the actual sequence is much shorter.
Inverted File (IVF) index
ANN index that k-means-partitions vectors into cells and, at query time, scans only the n_probe cells nearest the query.
inverted index (pair-to-words index)
A pair -> word-indices mapping used by the BPE trainer so each merge updates only the words actually containing that pair, not the whole corpus.
IO complexity
A formal count of HBM read/write accesses as a function of sequence length N, head dimension d, and SRAM size M, used to prove FlashAttention's traffic reduction over naive attention.
IO-awareness
Design principle that treats HBM data movement, not FLOP count, as the dominant cost for memory-bound kernels like attention, and optimizes to minimize it.
ISO/IEC 42001
A certifiable standard specifying requirements for an AI Management System (AIMS), the AI analogue of ISO 27001 for information security.
Defined in 13.6 AI Governance, Compliance & Regulation
IsoFLOP method
Chinchilla's Approach 2 for fitting scaling laws: sweep N at each of several fixed compute budgets C, fit a parabola in log N, and read off the loss-minimizing valley.
IsoFLOP profile
Chinchilla's second fitting method: at a fixed compute budget C, loss vs log N traces a U-shaped valley whose vertex (fit via a parabola) gives the compute-optimal N* for that budget.
Item Response Theory (IRT) / 2PL model
A psychometric model giving the probability a model with latent ability theta answers an item correctly via item difficulty and discrimination parameters, enabling calibrated cross-item comparison.
Iterative RLHF (online reward model update)
Mitigation that periodically retrains the reward model on preference data collected from the current policy to keep the RM in-distribution as the policy drifts.
ITL (inter-token latency)
The per-step gap between consecutive tokens; the instantaneous value whose average is TPOT.
J
Jaccard similarity
The ratio of intersection to union size of two sets; measures overlap between documents' character-shingle sets to detect near-duplicates.
Jacobi decoding
Viewing autoregressive decoding as solving fixed-point equations in parallel (rather than one token at a time), refining all guessed positions simultaneously across forward passes.
jailbreak
A prompt or attack strategy that bypasses a model's safety training to elicit behavior it would otherwise refuse.
Defined in 11.5 Red-Teaming, Safety & Robustness Evaluation · used in 3 chapters
JSON-RPC 2.0
The wire message format MCP layers its methods (tools/call, resources/read, etc.) on top of, used identically across both stdio and HTTP transports.
Defined in 8.6 The Model Context Protocol (MCP)
jump-forward decoding
An optimization where, when a grammar forces a run of tokens, the engine emits that whole forced substring in one step instead of sampling it token by token.
just-in-time retrieval (agentic retrieval)
Exposing search/read as tools so the model itself decides what context to fetch on demand, instead of pre-stuffing the whole corpus into the prompt.
Defined in 8.4 Context Engineering & Management
K
K-quants (e.g., Q4_K_M)
llama.cpp's family of block-level mixed-precision quantization schemes using super-block and sub-block scales to trade file size against quality on CPU/edge devices.
k3 KL estimator
Schulman's unbiased, always-nonnegative, low-variance KL approximation rho - log(rho) - 1 computed from a single sampled-token log-prob ratio, used as GRPO's KL penalty.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
Kahan summation
Compensated summation algorithm that tracks the rounding error lost at each step and re-adds it, cutting total error from O(Nε) to O(ε).
Kahneman-Tversky Optimization (KTO)
A DPO variant using prospect theory to train on unpaired desirable/undesirable labels instead of preference pairs, scoring implicit reward against a reference point $z_0$.
Kaiming/He initialization
Weight initialization setting Var(W) = 2/d_in, correcting Xavier's factor for ReLU's variance-halving effect, keeping deep ReLU networks trainable.
Kaplan scaling law
The original 2020 result prescribing N∝C^0.73, D∝C^0.27 — grow model size far faster than data as compute increases.
kernel fusion
Merging multiple GPU kernels into one so intermediate values stay in registers/shared memory instead of round-tripping through DRAM, cutting memory traffic.
key (K)
A learned linear projection (k_i = x_i W_K) that a token uses to advertise itself so queries can match against it via dot-product similarity.
Defined in 2.3 The Attention Mechanism From Scratch
KGW watermark (green-list watermark)
Kirchenbauer et al. scheme that biases sampling by boosting logits of a secret-key-derived 'green' token subset at each generation step, leaving a statistically detectable trace.
KL divergence (Kullback-Leibler divergence)
Nonnegative, asymmetric measure D_KL(p||q) of information lost when approximating true distribution p with q; forward KL is mode-covering, reverse KL is mode-seeking.
KL estimator k3
Monte-Carlo KL estimate e^r - 1 - r (r = log pi_ref/pi_theta) that is both unbiased and always non-negative; the modern default per-token KL penalty.
KL penalty (KL divergence regularization)
A beta-weighted term in the RL objective penalizing the policy for diverging from the frozen reference (SFT) distribution, curbing reward hacking.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 3 chapters
KL-in-reward vs KL-in-loss
Two non-equivalent placements of the KL penalty: folded into the reward before advantage computation (bootstrapped by GAE) or added as an explicit differentiable loss term.
KL-regularized RLHF objective
The objective maximizing expected reward minus $\beta$ times KL divergence from a reference policy, whose closed-form solution DPO exploits to eliminate the reward model.
KL-reward frontier
Pareto curve where proxy reward rises monotonically with KL divergence from the reference policy while true reward peaks at moderate KL then falls.
knowledge distillation (KD)
Training a smaller student model to match a larger teacher model's full output distribution (soft targets) rather than one-hot hard labels.
Knowledge editing
Deliberately overwriting a specific fact a language model has memorized, without retraining or disturbing its other knowledge and fluency.
Defined in 13.2 Knowledge Editing & Machine Unlearning
KV block
A fixed-size chunk (e.g. 16 tokens) of KV cache storage — the unit of allocation in PagedAttention, analogous to an OS page frame.
KV cache (key-value cache)
Stored keys and values for all past tokens during autoregressive decoding, avoiding recomputation; its size, not FLOPs, becomes the inference memory bottleneck.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA · used in 14 chapters
KV cache transfer
Sending the key-value cache tensor produced by a prefill worker over an interconnect (NVLink, InfiniBand, PCIe) to the decode worker that will continue generation.
KV-cache quantization
Compressing the key-value cache (e.g., to INT8 or INT4, typically per-token) to cut its memory footprint during long-context autoregressive decoding.
L
L2-cache swizzle
Reordering which output tiles concurrently-running programs compute (grouping by GROUP_M) so nearby programs reuse the same A rows / B columns in L2 cache.
Defined in 4.4 Writing GPU Kernels with Triton
label shifting
The alignment convention where model inputs are tokens x_0..x_{T-1} and targets are x_1..x_T, so each position's logit is scored against the next token.
Defined in 3.3 The Pretraining Objective & Loss
label smoothing
Replaces a one-hot training target with a mixture of the correct class and a uniform floor epsilon/V, improving calibration and keeping KL divergence well-defined.
Language identification (LangID)
Classifying a document's language before any quality filtering; the chapter uses fastText's 176-language model as the mandatory gating step.
late chunking
Encoding the entire document first with a long-context transformer, then mean-pooling token embeddings over each chunk span, so embeddings retain full-document context.
Defined in 9.4 Chunking, Reranking & Hybrid Search
late fusion
A multimodal design where a separately pretrained vision encoder's features are merged into a largely-frozen LLM partway through, via a projector or cross-attention.
Defined in 10.2 Vision-Language Models
late interaction
Retrieval paradigm encoding query and document into one vector per token/patch, precomputing documents offline, and scoring at query time with MaxSim.
latent diffusion model (LDM)
An architecture (underlying Stable Diffusion) that runs the diffusion process in a VAE's compressed latent space rather than pixel space, cutting compute roughly 64x.
Latent dimension (d_c)
The small compressed dimension d_c << d_model that MLA caches per token instead of full per-head keys and values, from which keys and values are later up-projected.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
Layer Normalization (LayerNorm)
Normalizes each token's feature vector to zero mean and unit variance, then applies learnable scale γ and shift β.
layer-wise KV pipelining
Streaming each transformer layer's KV cache to the decode worker as soon as it is computed during prefill, overlapping transfer with later layers' compute.
lazy-deleted max-heap
A heap of negated pair counts used to find the current most-frequent pair in O(log n); stale entries are left in place and discarded on pop instead of removed eagerly.
leaf tensor
A tensor created directly by user code (e.g., with requires_grad=True or as nn.Parameter) rather than produced by a tracked op; only leaves accumulate .grad after backward().
learned absolute positional embedding
A trainable lookup table of one vector per position up to a max length, added to token embeddings; cannot extrapolate past its fixed table size.
learned positional embedding
A trainable per-position vector table (one per patch position plus CLS) added to patch embeddings so the model can recover spatial layout.
Defined in 10.1 Vision Transformers & Image Encoders
Learned reward model
A model, e.g. Bradley-Terry, trained on preference data to output a smooth reward signal; general-purpose but prone to reward hacking.
Learner / trainer
The distributed training engine that recomputes current-policy log-probs, forms advantages, and applies the clipped policy-gradient loss via backprop and an optimizer step.
Defined in 6.1 The Anatomy of an RL-for-LLM System
learning rate warmup
Practice of ramping the learning rate from near zero up to its target value over the first steps of training to avoid early loss spikes.
Defined in 1.3 Calculus, Optimization & Convexity
least privilege
The architectural principle of granting an agent's tools only the minimum scope and permissions needed for the current task, limiting exfiltration channels.
length extrapolation
A model's ability to produce coherent outputs at sequence lengths longer than it was trained on, a key axis for grading positional encoding schemes.
length-normalized log-likelihood (acc_norm)
A choice's log-likelihood divided by its token count, correcting the bias that longer answers accumulate more negative log-probability under raw scoring.
Defined in 11.3 Building Eval Harnesses
lethal trifecta
The co-occurrence of private data in context, untrusted content in context, and an exfiltration channel; all three enable a complete data-exfiltration attack.
lifetime compute
Total FLOPs across a deployed model's life, C_lifetime = 6·N·D_train (train once) + 2·N·D_infer (serve forever); minimizing this, not just training FLOPs, motivates over-training.
Likelihood Ratio Attack (LiRA)
State-of-the-art MIA that trains many shadow models with/without the target record, fits Gaussians to per-example loss distributions, and computes a likelihood-ratio test for membership.
LIMA (Less Is More for Alignment)
Study showing 1,000 curated (instruction, response) pairs can outperform SFT on hundreds of thousands of noisy examples, motivating data quality over quantity.
linear attention
Attention reformulated via a kernel feature map φ so φ(K)^T V is computed once, giving O(N) parallel training and O(1) per-step recurrent inference.
linear probe
A low-capacity classifier (e.g., logistic regression) trained on frozen model activations to test whether a feature is linearly decodable at a given layer.
linear representation hypothesis
The working assumption that human-meaningful features correspond to approximately linear directions in a model's activation space, detectable via dot products.
linear scaling rule
The heuristic that learning rate should scale linearly with batch size (η' = k·η) to preserve training dynamics when batch size increases by factor k, valid for moderate scaling.
linear warmup + cosine decay schedule
The learning-rate schedule (`lr_at`) that ramps linearly from 0 to a peak LR over a warmup window, then decays it following a cosine curve down to a floor min_lr.
Lion (Evolved Sign Momentum)
Optimizer discovered by program search that steps by the sign of an interpolated momentum term, using a single buffer at half AdamW's memory.
Lipschitz gradient / L-smoothness
Property that bounds how fast a function's gradient can change, equivalent to the Hessian's largest eigenvalue being at most $L$; sets the maximum stable learning rate $\eta \leq 1/L$.
Defined in 1.3 Calculus, Optimization & Convexity
Little's law
Queueing-theory result that average concurrent requests equal arrival rate times average time in system (L = λW); used to size serving throughput from KV-cache capacity and decode latency.
Defined in 7.1 The Anatomy of LLM Inference: Prefill, Decode & The KV Cache · used in 2 chapters
live judge
An LLM-as-judge pipeline that continuously scores a stratified sample of production traffic on a rubric to monitor quality after launch.
LiveCodeBench
A continuously updated 'living' coding benchmark that adds problems released after model training cutoffs to make contamination structurally impossible.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Llama Guard
Meta's family of decoder-based LLMs fine-tuned for safety classification, taking an in-context harm taxonomy and jointly classifying prompt+response conversations with a natural-language rationale.
Defined in 12.4 Safety, Guardrails & Content Moderation
llama.cpp
A dependency-free C++ inference engine that runs quantized transformers on CPUs, Apple Silicon, and consumer GPUs via mmap loading and hand-written backends (AVX, Metal, CUDA).
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
LLM-as-a-Judge (LLMaaJ)
A second LLM prompted to score completions against rubric criteria for open-ended tasks that lack a crisp ground truth.
Defined in 6.8 Reward Engineering, Verifiers & Sandboxes · used in 2 chapters
LLM.int8() (mixed-precision decomposition)
A mixed-precision matmul decomposition that runs the rare outlier feature dimensions in FP16 and the rest in INT8, requiring no calibration or tuning.
Defined in 4.7 Quantization I: Post-Training Quantization (GPTQ, AWQ, SmoothQuant) · used in 2 chapters
LLMOps
The discipline of operating LLM-powered systems with software-engineering rigor: a five-phase loop of ship (canary), observe, evaluate, improve, and retrain.
Defined in 12.2 Observability, Logging & LLMOps
LMDeploy / TurboMind
Shanghai AI Lab's serving stack whose TurboMind C++/CUDA engine offers custom blocked KV cache, continuous batching, and AWQ quantization as a middle ground between TGI and TensorRT-LLM.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
Locality-sensitive hashing (LSH)
Organizing MinHash signatures into bands so only sufficiently similar pairs collide as candidates, avoiding O(N^2) pairwise comparison at billion-document scale.
locality-sensitive hashing (LSH) banding
Splitting a MinHash signature into bands of rows and bucketing documents by band; any shared band flags a near-duplicate candidate, turning all-pairs comparison into near-linear lookups governed by an (1/bands)^(1/rows) threshold curve.
Locate-then-edit hypothesis
The claim that simple (subject, relation, object) facts are stored locally in middle-layer MLPs at the last subject token, making them targetable for weight edits.
Defined in 13.2 Knowledge Editing & Machine Unlearning
lock_ref (reference counting)
A per-node counter incremented while an in-flight request reads that node's KV, preventing eviction from reclaiming memory a running kernel still depends on.
log-likelihood scoring
Multiple-choice evaluation method that ranks answer options by the model's log-probability of their text rather than parsing a generated response, avoiding extraction errors but requiring length normalization.
Defined in 11.1 The Evaluation Problem & Benchmark Landscape · used in 2 chapters
logit lens
A technique that applies the model's final layer-norm and unembedding matrix to an intermediate residual-stream vector to read out an early, mid-computation next-token prediction.
logit masking
Setting the logits of grammatically-invalid tokens to -infinity before softmax, so the sampler can only choose valid tokens.
Defined in 7.10 Structured & Constrained Generation
logit processor
A transform (temperature, penalty, or truncation) applied to logits before softmax; processors compose in a pipeline before the final sampler draws a token.
logit soft-cap
A Gemma-2-style tanh squashing of logits, z <- c*tanh(z/c), that hard-bounds them to (-c, c); used optionally on attention or final logits for extra stability.
logit soft-capping
A differentiable squashing function z -> c*tanh(z/c) applied to final vocabulary logits or attention logits to prevent extreme values while preserving sign and ordering, used in Gemma 2.
logits
Real-valued vector output by the model's final layer over the vocabulary, transformed by processors and softmax before a token is sampled.
logsumexp
A numerically stable way to compute log(E + A·N^-alpha + B·D^-beta) by combining the three log-terms, avoiding overflow/underflow across the orders of magnitude the terms span.
logsumexp identity (stable softmax)
Rewriting softmax/logsumexp by subtracting the maximum logit before exponentiating, so every exponent argument is ≤0 and overflow is impossible.
logsumexp statistic
The per-row value L_i = m_i + log(l_i) saved from the forward pass, letting the backward pass reconstruct softmax probabilities exactly without storing the N x N matrix P.
long context vs. RAG tradeoff
The architectural decision of whether to retrieve chunks or feed an entire corpus into a large context window, governed by corpus size, query volume, quadratic attention cost, and freshness needs.
long-thinking models (o1/R1-style reasoning models)
Models trained via RL with verifiable rewards to generate extended internal reasoning traces before answering, rather than merely prompted to produce CoT.
lookahead decoding
A drafter-free speculative method that uses Jacobi iteration on the target's own forward passes to simultaneously generate and verify n-grams, losslessly for greedy decoding.
LoRA (Low-Rank Adaptation)
PEFT method representing a weight update as the product of two low-rank matrices, ΔW = BA with rank r ≪ d, training only B and A while the original weight stays frozen.
Defined in 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math · used in 8 chapters
LoRA+
Optimizer-grouping technique that gives the $B$ matrix a learning rate $\lambda$ times larger than $A$'s (since $B$ starts at zero and must grow), improving convergence at no extra parameter cost.
loss aggregation (token-mean vs sequence-mean)
Choice of how per-token losses are reduced to a scalar; token-mean lets long sequences dominate the gradient, sequence-mean weights every response equally.
loss mask
Binary indicator (1 on action tokens, 0 on observation/system tokens) restricting the policy-gradient loss, KL, and importance ratio to model-generated tokens.
Defined in 6.10 Agentic & Multi-Turn RL
loss masking
Setting target tokens to ignore_index (-100) at padding, SFT prompt, or cross-document boundary positions so they contribute zero gradient to the loss.
Defined in 3.3 The Pretraining Objective & Loss · used in 2 chapters
loss scaling
Multiplying the loss by a large constant before the backward pass (then dividing gradients after) to prevent fp16 gradient underflow to zero.
Defined in 1.4 Numerical Computing, Floating Point & Precision · used in 2 chapters
loss spike
A sudden increase in training loss, usually resolved by the optimizer over tens to hundreds of steps, caused by a bad batch or bad model state.
loss-trajectory model
A shifted power law fit to pilot CPT runs, L(D) = L_inf + A/(D0+D)^alpha, used to predict loss and pick token budget before spending full compute.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
Loss-versus-position diagnostic
Binning per-token loss by its position within the context window to reveal whether a model genuinely uses distant tokens, since scalar average perplexity can hide a broken tail.
lost-in-the-middle
The empirical U-shaped position effect where information placed in the middle of a long context is retrieved less accurately than information at the start or end.
Defined in 8.4 Context Engineering & Management · used in 3 chapters
Low-Rank Adaptation (LoRA)
Reparameterizes a fine-tune as a frozen base weight plus a small low-rank update BA, letting many tiny tenant adapters share one base model.
LRU (Least-Recently-Used) eviction
The policy both RadixAttention and vLLM APC use to reclaim cached KV blocks under memory pressure, evicting the least-recently-accessed unreferenced block first.
Defined in 7.7 Prefix Caching & KV-Cache Reuse
LRU eviction (least-recently-used) over radix-tree leaves
SGLang's default eviction policy: reclaim KV slots from the oldest-accessed, unlocked leaf nodes first, peeling cold prefixes from the tree's tips inward.
M
machine epsilon
Smallest value ε such that 1+ε ≠ 1 in a given format; equals 2^-p, where p is the number of mantissa bits.
Machine unlearning
Provably removing the influence of specific training data (for privacy, copyright, or dangerous-capability removal), not just suppressing an output.
Defined in 13.2 Knowledge Editing & Machine Unlearning
main-content extraction (e.g., trafilatura)
Parsing raw HTML to keep only article-body text, stripping navigation, footers, and cookie banners that Common Crawl's WET extractor leaves in.
majority voting (self-consistency)
Test-time strategy that generates k independent solutions and returns the most common answer, trading extra compute for higher accuracy with diminishing returns.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Mamba (selective state space model)
An SSM whose transition parameters B_t, C_t, and step size Δ_t are input-dependent, letting the model selectively retain or discard information per token.
many-shot jailbreaking
A jailbreak that fills a long context with fabricated examples of the model complying with harmful requests, using in-context learning to override RLHF priors.
MAP estimation (maximum a posteriori estimation)
Maximizes log-likelihood plus log-prior; a Gaussian prior on parameters corresponds exactly to L2 weight-decay regularization.
Markov decision process (MDP)
Tuple (states, actions, transitions, reward, discount) used to frame autoregressive generation: states are partial sequences, actions are tokens, transitions are deterministic.
masked language modeling (MLM)
BERT's pre-training objective: randomly mask ~15% of tokens (80% [MASK], 10% random, 10% unchanged) and predict the originals using full bidirectional context.
masking (Triton)
Passing a boolean mask to `tl.load`/`tl.store` so out-of-bounds or padded lanes are skipped or filled, keeping ragged tensor edges correct.
Defined in 4.4 Writing GPU Kernels with Triton
masking schedule (alpha_t)
A monotone function from alpha_0=1 to alpha_1=0 giving the probability a token survives unmasked at diffusion time t; drives both corruption and the loss weight.
master weights (fp32 master weights)
An authoritative fp32 copy of each weight that the optimizer updates precisely, avoiding the 'swamping' problem where tiny low-precision updates round away to nothing.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
matrix rank
The dimension of a matrix's column space, i.e. the number of linearly independent directions in its linear map; bounds low-rank approximation quality.
Defined in 1.1 Linear Algebra for Deep Learning
Matryoshka Representation Learning (MRL)
Training technique adding InfoNCE loss terms at multiple embedding prefix lengths simultaneously, so a single model's truncated first-k dimensions remain a useful embedding.
Defined in 9.1 Embeddings & Representation Learning
maximal update parametrization (μP)
A theoretical framework (Yang et al.) justifying learning rate scaling α ∝ 1/√d_model so that feature-learning dynamics stay consistent across model widths.
maximum inner product search (MIPS)
Nearest-neighbor search under (unnormalized) inner-product similarity rather than a true metric distance, which breaks metric-based pruning guarantees.
maximum likelihood estimation (MLE)
Choosing parameters that maximize the log-likelihood of observed data; for language models this is mathematically identical to minimizing cross-entropy loss.
MaxSim
Scoring operator: for each query token vector, take its max dot-product similarity to any document token/patch vector, then sum across query tokens.
McNemar's test
A paired statistical test comparing two models' binary outcomes on the same task set using only the discordant pairs (tasks solved by exactly one model), used to judge whether a score gap is significant.
Defined in 8.8 Agent Evaluation & Benchmarks · used in 3 chapters
mean pooling
Sentence-embedding reduction that averages final-layer token vectors, excluding padding via the attention mask; empirically stronger than CLS pooling for bi-encoders.
Defined in 9.1 Embeddings & Representation Learning
mean time between failures (MTBF)
Expected time between hardware failures in a cluster, computed as 1/(N·λ) for N nodes each failing at hourly rate λ; shrinks as cluster size grows.
mean-time-to-detect / mean-time-to-restore (MTTD/MTTR)
Average time from fault onset to alert firing (MTTD) and from alert to restoration (MTTR), tracked across incidents to gauge reliability response speed.
Medusa
A speculative decoding method that bolts several extra prediction heads onto a frozen target's final hidden state, each predicting a token several positions ahead independently.
Megatron-Core (megatron.core)
NVIDIA's library of parallelism-aware transformer building blocks (column/row-parallel GEMMs, MoE layers, parallel_state topology) importable by other training frameworks.
Mel spectrogram
A 2-D time-frequency representation from an STFT mapped through Mel-scale filter banks; compresses a waveform (e.g. 160,000 samples) into an 80x1000 matrix for model input.
Defined in 10.3 Audio, Speech & Multimodal Fusion
Membership inference attack (MIA)
Attack deciding whether a specific record was in a model's training set, typically by comparing its loss (calibrated against a reference) to a threshold.
MEMIT (Mass-Editing Memory in a Transformer)
Generalizes ROME to insert thousands of facts at once via a batched least-squares update spread across a band of middle layers.
Defined in 13.2 Knowledge Editing & Machine Unlearning
Memorization (eidetic/verbatim memorization)
A training string is k-eidetic memorized if a short prompt makes greedy decoding reproduce it exactly and it appeared in at most k training documents.
memory compaction
Replacing a long in-context conversation prefix with a shorter, lossier summary (free-text or structured JSON) to stay under the context-window limit.
Defined in 8.5 Memory Systems for Agents
memory hierarchy (RAM-vs-disk framing)
Treating the context window as fast, small, volatile RAM and external storage (files, databases) as slow, large, durable disk that the agent pages state through.
Defined in 8.4 Context Engineering & Management
memory-bandwidth bound
The property of LLM decode where wall-clock time is set by moving weights from HBM rather than by FLOPs, so verifying multiple tokens costs about the same as one.
memory-bound (bandwidth-limited)
A kernel whose intensity is below the machine's ridge point, so its performance ceiling is bandwidth times intensity, not peak FLOP/s.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 3 chapters
message bus
A communication pattern where agents publish and subscribe to named topics rather than passing messages point-to-point, decoupling topology so agents can be added without rewiring.
Defined in 8.7 Multi-Agent Systems & Orchestration
metadata filtering
Restricting retrieval candidates using structured predicates (date, category, etc.), applied as pre-filtering (before ANN search) or post-filtering (after retrieval).
Defined in 9.4 Chunking, Reranking & Hybrid Search
MFU (Model FLOPs Utilization)
Ratio of useful model FLOP/s (via the 6N rule) actually achieved during training to the hardware's peak FLOP/s; a north-star efficiency metric.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 2 chapters
Mid-training
The phase between pretraining and post-training that resumes from a pre-decay checkpoint to run quality annealing, context extension, and capability injection, still via self-supervised next-token prediction.
min-p sampling
Keeping only tokens whose probability is at least p_min times the maximum token probability, a peak-relative threshold that scales automatically with model confidence.
MinHash
A hashing technique whose signature's fraction of matching entries between two documents is an unbiased estimator of their Jaccard similarity.
Defined in 3.2 Data Cleaning, Deduplication & Quality Filtering · used in 2 chapters
mixed precision (training)
Keeping numerically sensitive values (softmax, norms, loss, master weights) in a wide format while running matmuls in a narrow format for speed and memory savings.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
Mixture of Experts (MoE)
Architecture where each block holds N expert FFN sub-networks and a learned router activates only k << N per token, scaling parameters cheaply.
Defined in Glossary of Terms
mixture-of-experts (MoE)
Architecture in which a learned router activates only a subset of expert sub-networks per token, growing total parameters without proportional per-token FLOPs.
Defined in Key Papers: An Annotated Reading List
Mixture-of-Experts (MoE) layer
A Transformer FFN sublayer replaced by a bank of E expert FFNs plus a router that combines outputs of only the tokens' selected experts.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures · used in 2 chapters
Mixture-of-Experts (MoE) with fine-grained and shared experts
An architecture (DeepSeekMoE-style) routing each token to a few of many small specialized experts plus always-on shared experts, decoupling total model capacity from per-token active compute.
MLA (Multi-head Latent Attention)
An attention variant, from DeepSeek-V2, that compresses the KV cache into a low-rank latent representation, a further reduction beyond what GQA achieves.
MLC-LLM
A TVM-based compiler for LLMs that lowers models to hardware-specific code for CUDA, ROCm, Metal, Vulkan, and WebGPU from one codebase, trading peak performance for portability.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
MMLU (Massive Multitask Language Understanding)
A ~14,000-question four-choice benchmark spanning 57 subjects (STEM, humanities, professional domains), scored by accuracy or length-normalized log-likelihood.
Modality-aware Mixture-of-Experts (MoE) routing
MoE routing where experts specialise by modality (image vs. text) while shared attention enables cross-modal interaction; risks collapsing into fully separate experts.
Defined in 10.5 Unified & Any-to-Any Models
Modality-aware z-loss
Auxiliary loss that penalises softmax log-sum-exp magnitude separately per modality, keeping neither text nor image vocabulary from dominating a joint softmax.
Defined in 10.5 Unified & Any-to-Any Models
model card
A structured documentation artefact (Mitchell et al., 2019) covering a model's architecture, training data, evaluations, and limitations; now used to satisfy EU AI Act Article 53.
Defined in 13.6 AI Governance, Compliance & Regulation
model collapse
Progressive narrowing of a model's output distribution when trained recursively on its own generated outputs, losing tails and converging to bland text.
Model Context Protocol (MCP)
A standardized JSON-RPC protocol, introduced by Anthropic, that lets any server expose tools, resources, and prompts to any compliant client uniformly.
Defined in 8.1 Tool Use & Function Calling · used in 2 chapters
Model FLOP Utilization (MFU)
Fraction of a cluster's peak FLOP/s spent on useful forward+backward model arithmetic, computed as (FLOPs/token x tokens/s) / peak FLOP/s; 35-55% is healthy for dense models.
Defined in 3.7 Megatron-LM, DeepSpeed & Parallelism in Practice · used in 4 chapters
Model FLOPs Utilization (MFU)
Ratio of achieved model FLOPs during training to a GPU cluster's peak hardware FLOP/s; converts a FLOP budget into wall-clock time.
Defined in 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond · used in 5 chapters
model merging
Combining two or more independently trained checkpoints by arithmetic on their weight tensors, with no gradient-based training or data access needed.
model routing cascade
A system that sends each query to the cheapest model tier first, escalating to costlier models only when a quality gate rejects the cheap tier's output.
model soups
Averaging the weights of several fine-tuned checkpoints of the same base model; the average often generalizes better than any single checkpoint.
modified rejection sampling
The accept/reject rule (accept with probability min(1, p(x)/q(x)), else resample from the residual) that makes drafted tokens exactly distributed as the target.
momentum (heavy-ball / Nesterov)
Update rule that accumulates an exponentially weighted running velocity of past gradients, accelerating convergence along consistent descent directions in narrow valleys.
Defined in 1.3 Calculus, Optimization & Convexity
Monte Carlo Tree Search (MCTS)
Search algorithm (from AlphaGo/AlphaZero) with select/expand/simulate/backpropagate phases, adapted here to allocate search budget over reasoning trajectories using PUCT selection.
MT-Bench
An 80-question multi-turn benchmark, scored by an LLM judge on a 1-10 scale, used to evaluate instruction-tuned models' reasoning, coding, and writing quality.
MTEB (Massive Text Embedding Benchmark)
Standard leaderboard evaluating embedding models across 58 datasets and 8 task types (retrieval, clustering, classification, STS, etc.), later expanded into multilingual MMTEB.
Defined in 9.1 Embeddings & Representation Learning
multi-agent system
A system decomposing a task across multiple LLM invocations that communicate via structured messages or shared state, attacking context, error-isolation, and parallelism limits.
Defined in 8.7 Multi-Agent Systems & Orchestration
multi-agent tax
The extra tokens and latency every inter-agent boundary imposes; a decomposed pipeline can cost several times more than a single well-prompted call for the same task.
Defined in 8.7 Multi-Agent Systems & Orchestration
Multi-bit / multi-key watermark
Watermark variant embedding a message identifier from a large key space rather than a single binary signal, so compromising one key does not defeat the whole scheme.
multi-controller (SPMD)
Every GPU runs the same program in lockstep (single-program-multiple-data), used inside each RL stage for efficient parallel computation.
Multi-head attention (MHA)
Splits d_model into h subspaces, runs independent scaled dot-product attention per head with its own Q/K/V projections, then concatenates and mixes via W_O.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
Multi-head Latent Attention (MLA)
DeepSeek's attention variant that caches a low-rank compressed latent vector per token instead of per-head keys/values, reconstructing all heads via up-projections.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA · used in 2 chapters
multi-hop retrieval
Retrieval where the answer requires chaining several dependent queries, since later-needed evidence has no semantic overlap with the original question.
Multi-layer perceptron (MLP)
A chain of affine layers (matrix multiply plus bias) each followed by an elementwise nonlinearity; the basic feed-forward neural network.
multi-LoRA multiplexing
Serving many LoRA fine-tunes of one base model by loading base weights once and swapping small adapter matrices per request, even within one batch.
Defined in 12.1 Designing an LLM Serving System
Multi-LoRA serving
Systems technique (S-LoRA/Punica-style) that runs one shared base-model matmul per batch and applies each request's own small adapter via a batched/segmented gather-matmul kernel, serving thousands of adapters on one GPU.
Defined in 5.3 PEFT I: LoRA, QLoRA, DoRA & The Adapter Family · used in 2 chapters
multi-provider failover
Routing LLM requests across multiple providers with health probing and circuit breaking so a single provider outage does not take down the service.
Multi-Query Attention (MQA)
Attention variant keeping h query heads but sharing a single key head and single value head across all of them, shrinking the KV cache by a factor of h at a quality cost.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
multi-token prediction (MTP)
An auxiliary training head (DeepSeek-V3) that predicts a token further ahead (e.g. t+2) alongside the main next-token objective, densifying the training signal and enabling self-speculative decoding.
Multimodal token-stream view
The abstraction that each modality (audio, vision, text) contributes tokens via its own embedding table/projection into one shared sequence processed by a single transformer backbone.
Defined in 10.3 Audio, Speech & Multimodal Fusion
multiple comparison correction
Adjustments like Bonferroni or Benjamini-Hochberg that lower the significance threshold when testing many benchmarks at once to avoid inflated false-positive rates from cherry-picking.
Defined in 11.3 Building Eval Harnesses
multiple comparisons correction (Bonferroni / Benjamini-Hochberg FDR)
Adjustments to significance thresholds when testing many candidates or benchmarks at once, controlling either the family-wise error rate (Bonferroni) or the false discovery rate (BH).
Muon
Optimizer that replaces the momentum update of 2-D weight matrices with its nearest semi-orthogonal matrix, computed via a matmul-only Newton-Schulz iteration.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
Muon optimizer
An orthogonalized-momentum optimizer (via Newton-Schulz iteration) needing only one momentum buffer instead of Adam's two, roughly halving optimizer-state memory for 2D weights.
Defined in 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math · used in 2 chapters
MuonClip / QK-clip
Post-step rescaling of a head's query and key weights that caps the maximum pre-softmax attention logit at a threshold tau, preventing blow-ups.
muP (maximal-update parameterization)
A reparameterization of initialization scale and per-layer learning rate that makes optimal hyperparameters width-invariant, enabling HP transfer from a small proxy model to a large one.
mutual information
I(X;Y) = H(X) - H(X|Y), the reduction in uncertainty about X from observing Y; equals the KL divergence between the joint and product-of-marginals distributions.
MUVERA
Method that deterministically projects an entire multi-vector document representation into one fixed-dimensional vector whose dot product approximates MaxSim, enabling standard single-vector ANN indexing.
MXU (Matrix Multiply Unit)
The systolic-array compute block inside a TPU chip, classically a 128x128 MAC grid, paired with a vector unit for elementwise ops.
N
natural (proportional) mixture
The mixture where each domain's weight equals its share of raw cleaned token counts; almost never the loss-optimal training mixture.
NCCL (NVIDIA Collective Communications Library)
Low-level library that torch.distributed calls on GPU backends; implements collective operations and auto-selects algorithms based on message size and topology.
Defined in 1.9 Parallel Computing & Collective Communication · used in 2 chapters
NCCL broadcast
Fast GPU-to-GPU weight-sync mechanism using NCCL collectives over a process group spanning trainer and inference ranks, moving weights via NVLink/InfiniBand without touching disk.
Needle in a Haystack (NIAH)
Evaluation that inserts a unique fact at a controlled depth inside a long context and checks whether the model retrieves it, plotted as a length-vs-depth heatmap.
Needle-in-a-haystack test
A long-context validation probe that plants a unique fact at a random depth in a long document and checks whether the model can recover it via next-token completion.
negative log-likelihood (NLL)
The per-token loss -log p(x_t | x_<t); numerically identical to cross-entropy against a one-hot target, since the inner sum collapses to the true class.
Defined in 3.3 The Pretraining Objective & Loss
Negative Preference Optimization (NPO)
A DPO-style unlearning objective that treats forget-set samples as dispreferred, with a self-limiting gradient that avoids the divergence of raw gradient ascent.
Defined in 13.2 Knowledge Editing & Machine Unlearning
NeMo-Aligner
NVIDIA's Megatron-LM-based RLHF toolkit using static NCCL communication and TensorRT-LLM rollouts instead of Ray; archived in 2025 in favor of NeMo-RL.
Neural audio codec
A convolutional encoder-RVQ-decoder system (e.g. EnCodec, SoundStream) trained end-to-end to reconstruct audio from discrete codec tokens.
Defined in 10.3 Audio, Speech & Multimodal Fusion
NeuronCore
The heterogeneous compute unit inside a Trainium chip, combining a systolic TensorEngine with VectorEngine, ScalarEngine, and GPSIMD engines fed by on-chip scratchpad memory.
Newton-Schulz iteration
A fixed sequence of matrix-multiply-only cubic-polynomial steps that drives a matrix's singular values toward 1, used by Muon to approximate UV^T without an SVD.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
NF4 (4-bit NormalFloat)
A 4-bit data type whose 16 code points sit at quantiles of a standard normal distribution, minimizing error for normally-distributed weights; QLoRA's base-model format.
Defined in 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT · used in 3 chapters
NIST AI Risk Management Framework (AI RMF)
Voluntary US framework (NIST AI 100-1) structuring AI risk management into four functions: GOVERN, MAP, MEASURE, and MANAGE.
Defined in 13.6 AI Governance, Compliance & Regulation
no_sync()
A DDP context manager that suppresses gradient all-reduce during gradient-accumulation micro-batches, letting .grad accumulate locally until the last micro-batch.
non-autoregressive (NAR) language model
A language model that generates a sequence without a strict left-to-right token-by-token dependency, instead producing multiple positions in parallel each step.
non-embedding parameters
Parameter count excluding the (often tied) token embedding table; used for fitting N and the 6ND FLOP rule because the embedding's per-token work does not scale with depth.
NoPE (no positional encoding)
A decoder-only Transformer with a causal mask but no explicit positional scheme, which can still recover position because the causal mask itself breaks permutation symmetry.
Defined in 2.5 Positional Encodings: Sinusoidal, Learned, RoPE & ALiBi · used in 3 chapters
novelty effect
A bias where users respond more positively to any change simply because it is new, typically decaying over one to two weeks.
NTK-aware interpolation
Rescales the RoPE base (not positions) to stretch low-frequency dimensions for long range while preserving high-frequency short-range resolution, often zero-shot.
NTK-aware scaling
Context-extension method that stretches RoPE's frequency base non-uniformly, barely changing fast (local) dimensions while stretching slow (long-range) dimensions, often needing no fine-tuning.
Nucleus sampling (top-p sampling)
Decoding strategy that samples from the smallest set of tokens whose cumulative probability mass exceeds a threshold p, adapting to the distribution's shape.
Defined in From-Scratch Code Index
NVLink
A high-bandwidth, low-latency intra-node interconnect between GPUs, far faster than PCIe, that makes per-layer all-reduce for tensor parallelism practical.
Defined in 1.8 GPU Architecture & The Memory Hierarchy
NVLink / NVSwitch
NVIDIA's high-bandwidth intra-node GPU interconnect (NVLink) and the crossbar switch (NVSwitch) that lets every GPU on a node talk to every other at full speed.
O
object store (Plasma)
Ray's shared-memory store (backed by Apache Plasma) enabling zero-copy reads of tensors/arrays by any process on the same node, key to fast experience-batch transfer.
Observability
The practice of understanding a system's internal state from its external outputs, built from three pillars: traces, metrics, and logs.
Defined in 12.2 Observability, Logging & LLMOps
observation
Text injected by the environment after a step (tool output, error message); never sampled from the policy, so masked out of the loss.
Defined in 6.10 Agentic & Multi-Turn RL · used in 2 chapters
occupancy
The ratio of resident warps to the SM's maximum warp capacity, bounded by registers, shared memory, and block/warp slot limits; buys latency-hiding headroom but is not itself the performance goal.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 2 chapters
Odds Ratio Preference Optimization (ORPO)
A reference-free method that fuses SFT and preference alignment into one stage: SFT loss on the chosen response plus a log-odds-ratio penalty favoring it over the rejected response.
off-policy distillation
KD where the student trains on a fixed dataset scored by the teacher; cheap but suffers distribution mismatch at test time.
offline/batch inference API
A provider endpoint (e.g., Anthropic Batches API, OpenAI Batch API) that processes non-latency-sensitive request batches at roughly 50% discount with turnaround up to 24 hours.
offloading (optimizer-state offload)
Moving tensors not needed in the current RL phase (e.g., Adam optimizer state) to host CPU RAM to free GPU memory during colocated time-slicing.
Ollama
A developer-friendly REST server and CLI that wraps llama.cpp (and its own GGML engine) with a model registry and automatic hardware offload.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
on-policy distillation
KD where the student generates its own completions, the teacher scores them, and the student trains on its own rollouts, reducing distribution mismatch.
on-policy vs off-policy
On-policy means the training data was generated by the exact current policy weights; off-policy means it came from older or different weights.
one-hot vector
A sparse, orthogonal V-dimensional representation of a token that is memory-wasteful and encodes no semantic similarity between tokens.
Defined in 2.2 Embeddings & The Input Pipeline
One-step off-policy overlap
Pipelining generation and training so rollout engines produce batch t+1 while the trainer consumes batch t, at the cost of training on weights one update stale.
one-step-staleness trap
A degenerate design where allowing only one batch in flight still stalls the trainer on the batch's slowest (straggler) rollout, despite nominally bounding staleness to 1; fixed by sample-level pipelining.
online softmax
Algorithm computing a numerically-stable softmax-weighted sum in one streaming pass using a running max, running denominator, and a rescale factor, needing no full-row materialization.
Defined in 4.2 FlashAttention I: IO-Awareness & The Online Softmax · used in 2 chapters
OpenDiLoCo
Prime Intellect's open implementation of DeepMind's DiLoCo, performing many local optimizer steps between rare global synchronizations to enable globally-distributed pretraining with low communication.
OpenRLHF
An open-source RLHF framework built on Ray actors, using vLLM for rollout and DeepSpeed ZeRO for gradient updates, prized for flexible per-role GPU/model configuration.
OpenTelemetry (OTel)
Vendor-neutral standard and SDK for generating and propagating traces, metrics, and logs to any compatible backend (Jaeger, Langfuse, Datadog, etc.).
Defined in 12.2 Observability, Logging & LLMOps
optimizer state memory
The extra per-parameter tensors (moments, fp32 master weights) an optimizer stores; AdamW costs ~12 bytes/param, often exceeding the model weights themselves.
Optimizer states
Per-parameter Adam moments (m_t, v_t) plus the fp32 master weight copy; typically the single largest static memory cost (8P bytes) in mixed-precision training.
orchestrator-worker topology
A star-shaped pattern where one orchestrator decomposes a task, dispatches worker agents in parallel, and synthesizes their results into a final answer.
Defined in 8.7 Multi-Agent Systems & Orchestration
orthogonal matrix
A square matrix Q satisfying Q^T Q = QQ^T = I, whose columns form an orthonormal basis and which preserves vector lengths and angles when applied.
Defined in 1.1 Linear Algebra for Deep Learning
out-of-vocabulary (OOV)
A word or symbol not present in a tokenizer's fixed vocabulary, forced to collapse to an <unk> token and lose information; impossible under byte-level tokenization.
outcome reward model (ORM)
A reward model that scores a full solution trace using only the final answer's correctness, trained on binary labels of complete solutions.
Defined in 5.10 Reasoning, Chain-of-Thought & Test-Time Compute · used in 2 chapters
outcome scoring
Evaluation approach that grades only the final task state (success/failure), used by SWE-bench, WebArena, and GAIA; objective but low-signal about how the agent got there.
Defined in 8.8 Agent Evaluation & Benchmarks
Outlines
The foundational library that converts a regex or CFG into a token-level FSM index for constrained LLM generation.
Defined in 7.10 Structured & Constrained Generation
Over-long filtering
DAPO technique that masks the loss on sequences truncated by the generation length cap so an unfinished trajectory neither rewards nor penalizes the policy.
over-refusal
The rate at which a model refuses benign requests that superficially resemble harmful ones, measured as a false-positive rate on benign inputs.
over-training (deployment-compute efficiency)
Training on far more tokens than compute-optimal (Stack-100M uses ~200/parameter) to lower loss at a fixed, cheap-to-serve parameter count, trading one-time training FLOPs for lower recurring inference cost.
Defined in 14.1 The Capstone: Building Stack-100M, and the 2026 Small-Model Landscape · used in 3 chapters
overfitting
A model memorizes training-data idiosyncrasies, giving low training loss but high test loss (the generalization gap); dominant error term is variance.
Defined in 1.5 Machine Learning Fundamentals
oversight gap
Region where model accuracy p_M exceeds supervisor verification accuracy p_H; naive RLHF there optimizes for rater approval, not correctness.
Oversubscription
Feeding an inference engine more prompts than it has concurrent slots so freed slots are immediately refilled from a deep queue, shrinking idle-tail bubbles.
P
padding mask
A mask that zeroes out attention to `<pad>` tokens used to batch variable-length sequences together, so pad positions never contribute to or receive attention.
Defined in 2.3 The Attention Mechanism From Scratch
padding_idx
An nn.Embedding argument that freezes a designated pad token's embedding row at zero and blocks gradient flow to it.
Defined in 2.2 Embeddings & The Input Pipeline
Paged KV cache
Storing key-value tensors in fixed-size blocks that a block manager allocates and frees per sequence, letting a server hold far more concurrent sequences than static pre-allocation.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
Paged optimizer
QLoRA component that uses NVIDIA unified memory to page optimizer states out to CPU DRAM on demand, preventing OOM crashes from transient memory spikes.
Defined in 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math · used in 2 chapters
PagedAttention
Attention scheme that stores a sequence's KV cache in fixed-size, non-contiguous physical blocks and gathers them via a per-sequence block table, borrowed from OS virtual memory paging.
Defined in 4.6 PagedAttention & KV-Cache Memory Management · used in 7 chapters
PAIR (Prompt Automatic Iterative Refinement)
A gradient-free jailbreak method where an attacker LLM iteratively rewrites a prompt using the target model's refusals as feedback.
pairing (paired comparison)
Evaluating two models on the identical items and testing the per-item difference rather than comparing independent scores; cancels shared item-difficulty variance and greatly boosts power.
Pairwise evaluation
Showing a judge two responses and having it pick a winner or tie; better calibrated than pointwise but O(N^2) queries and prone to position bias.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
parallel tool calling
Emitting multiple independent tool calls in a single model response so they can be executed concurrently, reducing latency versus sequential calls.
Defined in 8.1 Tool Use & Function Calling
Parameter-efficient fine-tuning (PEFT)
Family of methods that freeze most pretrained weights and train only a small added set of parameters, eliminating gradient and optimizer-state memory for frozen layers.
Defined in 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math · used in 3 chapters
Parent-child chunking
Indexing strategy that retrieves against small child chunks for precision but returns their larger parent chunk to supply richer generation context.
parent-child document retrieval
A pattern that indexes small child chunks for precise retrieval matching but returns their larger parent document as LLM context, balancing retrieval precision with context sufficiency.
Defined in 9.4 Chunking, Reranking & Hybrid Search
parse-validate-dispatch pipeline
The harness sequence that extracts a tool call from model output text, checks it against the tool's JSON schema, then executes the matching function.
Defined in 8.1 Tool Use & Function Calling
partial rollout
Pausing a trajectory mid-episode at a per-step generation-budget cap and resuming it later, possibly under updated weights, to bound step latency.
Defined in 6.10 Agentic & Multi-Turn RL · used in 2 chapters
pass rate (empirical pass rate, p̂)
Fraction of G rollouts on a prompt that score reward 1 under the current policy; its distance from 0.5 sets GRPO's gradient variance p(1-p).
Defined in 6.12 RL Data, Curriculum & Replay Management
pass@k
Code-eval metric: probability at least one of k sampled generations passes all unit tests, estimated unbiasedly from n samples via 1 - C(n-c,k)/C(n,k).
Defined in 11.1 The Evaluation Problem & Benchmark Landscape · used in 2 chapters
pass@k estimator
Unbiased estimator of the probability at least one of k sampled trajectories solves a task, computed as 1 minus a ratio of binomial coefficients over n samples with c correct.
Defined in 8.8 Agent Evaluation & Benchmarks
pass^k (pass-hat-k) reliability metric
tau-bench's estimator of the probability that all k independent trials on a task succeed, computed as C(c,k)/C(n,k); measures consistency rather than best-of-k success.
Defined in 8.8 Agent Evaluation & Benchmarks
patch embedding
Linear projection (matrix E) that maps each flattened P×P×C image patch into a D-dimensional vector, analogous to a token embedding.
Defined in 10.1 Vision Transformers & Image Encoders
PEFT (Parameter-Efficient Fine-Tuning) / LoRA
Adapter-based fine-tuning (e.g., LoRA, QLoRA) that TRL trainers integrate transparently, training only low-rank adapter weights atop a frozen base model.
Defined in 6.3 TRL: HuggingFace's RL Library
Per-example gradient clipping
Computing each training example's gradient separately and rescaling its L2 norm to a bound C, capping how much any single example can influence a DP-SGD update.
per-group quantization
Splitting each weight row into contiguous groups (typically 128) with its own scale, the default granularity for 4-bit LLM weights in GPTQ/AWQ/GGUF.
Per-group reward normalization
Z-scoring rewards within a prompt's group of sampled completions so resulting advantages are zero-mean regardless of the raw reward scale.
performance-gap-recovered (PGR)
Metric giving the fraction of the gap between weak-supervisor and oracle accuracy that a weak-to-strong model recovers; 0 = imitation, 1 = full recovery.
permission gate
The policy layer that classifies each side-effecting tool call as allow, deny, or ask-a-human before execution, based on allowlist/denylist command patterns.
permutation-equivariance
Property of self-attention (and per-position MLPs) that reordering input rows reorders the output identically, so raw attention cannot sense word order.
Perplexity (PPL)
Exponential of the average cross-entropy loss over a sequence; the standard LM evaluation metric, interpretable as an effective branching factor.
Defined in 1.2 Probability, Statistics & Information Theory · used in 7 chapters
Personalized PageRank (PPR)
Graph algorithm that propagates importance from seed nodes through edges via power iteration with a teleport probability, boosting entities connected to relevant seeds even without direct query match.
Personally identifiable information (PII)
Names, emails, phone numbers, IPs, and similar sensitive data; regex patterns and NER models detect and redact it before training.
PII (personally identifiable information) redaction
Detecting and replacing sensitive spans (names, SSNs, emails) in prompts and outputs using regex rules, NER models (e.g. Presidio), or LLM-based judges, to meet privacy obligations.
Defined in 12.4 Safety, Guardrails & Content Moderation
PII scrubbing
Detect-and-redact pipeline combining regexes and NER to remove personally identifiable information from training or output text before it can be memorized or leaked.
ping-pong scheduling
FA3 technique staggering two consumer warpgroups so one runs softmax on special-function units while the other runs wgmma on tensor cores, keeping both unit types busy simultaneously.
Pipeline bubble
The idle-GPU fraction caused by pipeline fill/drain time; equals (p-1)/(m+p-1) and is reduced by microbatching, interleaving, or zero-bubble scheduling.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 2 chapters
Pipeline parallelism (PP)
Splits a model's layers into contiguous stages placed on different GPUs, communicating only via point-to-point activation send/recv at stage boundaries.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 2 chapters
placement group (Ray)
A Ray reservation of resource bundles with a packing strategy (PACK/SPREAD) used by veRL to pin worker-group ranks to specific GPUs deterministically.
Defined in 6.4 veRL: HybridFlow & The Single-Controller Architecture · used in 2 chapters
Plackett-Luce model
Multi-item generalization of Bradley-Terry giving the likelihood of a full ranking as a sequential product of softmaxes over remaining items.
Defined in 5.5 The RLHF Pipeline & Reward Modeling
PLAID
Production indexing engine for late-interaction retrieval that clusters vectors into centroids, stores quantized residuals, and prunes candidates via centroid similarity before exact reranking.
Plan-Execute architecture
Two-phase agent design where a planning LLM decomposes a task into an ordered/dependency-linked list of subtasks, then an executor (often an inner ReAct loop) carries each out.
pointer-block arithmetic
Computing a vector/tensor of memory addresses by adding index offsets (via `tl.arange` and strides) to a base pointer, then loading/storing the whole block at once.
Defined in 4.4 Writing GPU Kernels with Triton
Pointwise evaluation
Scoring a single response on an absolute scale (e.g., 1-5) against a rubric; O(N) queries but harder to calibrate across prompt types.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
Policy
A distribution over actions given a state; this chapter's key identification is that the language model's next-token softmax IS the policy π_θ.
polysemanticity
The phenomenon where a single neuron responds to multiple unrelated features (a consequence of superposition), making per-neuron interpretation unreliable.
Population Stability Index (PSI)
A statistic, PSI = sum((P_i - Q_i) * ln(P_i/Q_i)), quantifying how much a current distribution has shifted from a reference baseline; used to detect input/covariate drift.
Defined in 12.2 Observability, Logging & LLMOps
Portability vs. peak performance tension
The chapter's core trade-off: the more a serving runtime commits to a specific hardware target via compilation, the higher its throughput but the less portable it becomes.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
Position bias
A model's tendency to favor certain answer-letter positions (e.g., A) on multiple-choice questions, measurable only by re-querying the model on permuted option orderings.
Defined in 11.1 The Evaluation Problem & Benchmark Landscape · used in 3 chapters
Position Interpolation (PI)
Context-extension method that rescales inference positions by dividing by the length ratio s, compressing longer sequences into RoPE's originally trained angular range.
Defined in 2.5 Positional Encodings: Sinusoidal, Learned, RoPE & ALiBi · used in 2 chapters
position-id reset
Restarting each document's position ids at 0 when it is packed into a window, so RoPE's relative-position machinery isn't fed positions that falsely continue a preceding, unrelated document.
position_ids reset
Resetting position indices to 0 at the start of each document within a packed row, required so RoPE/ALiBi apply correctly per-document.
Positional bias
An LLM judge's tendency to favor whichever response is presented first in a comparison, independent of actual quality.
positional encoding
Information injected into a Transformer so it can tell token order and distance, since bare attention is permutation-equivariant and otherwise order-blind.
Post-hoc AI-text detection
Classifying text as AI- or human-written after the fact using perplexity, statistical, or trained-classifier features, without any signal embedded at generation time.
post-norm
Block layout applying normalization after the residual addition, x' = Norm(x + F(x)); the original transformer's design, unstable without warmup.
post-training quantization (PTQ)
Converting a trained model's fp32/bf16 weights to low-bit integers after training completes, with no gradient updates required.
power analysis / minimum detectable effect (MDE)
Prospective sizing of a test set: given a significance level, target power, and the smallest effect worth detecting, compute how many items are needed before running the eval.
power law
A relationship where loss scales as a negative power of model size or data (L ∝ x^-α), appearing as a straight line on log-log axes.
power-of-two-choices
Load-balancing scheme that samples two random candidate replicas and picks the less-loaded one, exponentially reducing max load versus pure random assignment.
Defined in 12.1 Designing an LLM Serving System
PPO clipped surrogate objective
min(r_t·Â_t, clip(r_t,1-ε,1+ε)·Â_t): a pessimistic bound that zeroes the gradient once the importance ratio moves too far, forming a cheap first-order trust region.
Defined in 5.6 Policy Gradients & PPO for Language Models · used in 2 chapters
PPO clipping
Clipping the importance ratio to [1-eps, 1+eps] before multiplying by the advantage, bounding how far the target policy can move per token; doubles as a staleness corrector in async RL.
PPOTrainer
TRL trainer implementing the classic actor-critic Proximal Policy Optimization loop: rollout generation, reward scoring, KL penalty, GAE advantage estimation, and clipped policy updates.
Defined in 6.3 TRL: HuggingFace's RL Library
pre-clip gradient norm
The global gradient norm measured before clipping is applied; logged (not the post-clip value, which is artificially bounded) as the signal that reveals instability spikes.
pre-norm (pre-normalization)
Block layout applying normalization before the sublayer, x' = x + F(Norm(x)), which stabilizes gradients without warmup.
Defined in 2.6 The Transformer Block: Norms, Residuals, MLPs & Activations · used in 2 chapters
pre-norm vs post-norm
Placement of normalization relative to a sublayer and residual add: pre-norm (norm before sublayer) gives cleaner gradient flow through the residual path than GPT-2's post-norm.
pre-tokenization
A regex-based splitting step run before BPE merges, isolating words, leading spaces, digit runs, and punctuation into chunks so merges never cross those boundaries.
pre-tokenizer regex (GPT-2 regex)
The splitting pattern applied before any BPE merge runs, ensuring merges never cross word, whitespace, or punctuation boundaries.
precision and recall
Classification metrics: precision = TP/(TP+FP) measures correctness of positive predictions; recall = TP/(TP+FN) measures coverage of true positives; combined via F1/F-beta.
Defined in 1.5 Machine Learning Fundamentals
preemption
When the KV block pool is exhausted, the serving engine evicts a running sequence (via recompute later or swap to CPU RAM) to free blocks, the paged analogue of OS swapping.
Defined in 4.6 PagedAttention & KV-Cache Memory Management · used in 2 chapters
Preference data (comparison/ranking)
Human-labeled rankings over K candidate responses to a prompt, decomposed into pairwise (chosen, rejected) rows used to train the reward model.
Defined in 5.5 The RLHF Pipeline & Reward Modeling
prefill
The inference phase that processes an entire prompt at once, reusing weights across many tokens, making it compute-bound.
Defined in 4.1 The Roofline Model & Performance Engineering · used in 3 chapters
prefill and decode
The two inference phases: prefill processes the prompt once (compute-bound, O(L) in prompt length), decode generates tokens one at a time (memory-bandwidth-bound).
prefill phase
The compute-bound stage of autoregressive generation that processes an entire prompt in one parallel forward pass, producing the KV cache.
prefill-decode interference
The problem where a long prefill running in a mixed continuous-batching iteration stalls in-flight decode sequences, spiking P99 inter-token latency.
prefill/decode disaggregation
Running prefill (compute-bound) and decode (memory-bandwidth-bound) on separate replica pools with hardware suited to each, streaming KV cache between them.
Defined in 12.1 Designing an LLM Serving System
Prefix cache hit rate
The fraction of incoming prompt tokens/blocks served from the cache rather than recomputed; a production metric used to diagnose caching misconfiguration (e.g., dynamic content polluting a shared prefix).
Defined in 7.7 Prefix Caching & KV-Cache Reuse
prefix caching (prefix sharing)
Storing and reusing already-computed key-value tensors for a shared token prefix across multiple requests, skipping recomputation for repeated content.
Defined in 7.7 Prefix Caching & KV-Cache Reuse · used in 4 chapters
prefix language model (prefix-LM)
A decoder-only variant whose mask lets prompt tokens attend to each other bidirectionally while generated tokens attend causally.
prefix tuning
Injects learned key/value pairs at every transformer attention layer (via an MLP reparameterization), giving per-layer task steering instead of input-only.
prefix-cache affinity
Routing policy that sends a request to the replica already holding its shared prompt prefix's KV cache, turning prefill into a cache hit.
Defined in 12.1 Designing an LLM Serving System
prime-rl
Prime Intellect's asynchronous RL training framework that extends the queue-based async architecture to permissionless, globally-distributed, untrusted GPU workers.
process group
A named set of ranks sharing a communicator; collective operations are always called on a process group, either the default (all ranks) or a sub-group.
process reward model (PRM)
A reward model that assigns a scalar score to each individual reasoning step, enabling detection of flawed intermediate steps even when the final answer is correct.
Defined in 5.10 Reasoning, Chain-of-Thought & Test-Time Compute · used in 4 chapters
producer/consumer pipeline
FA3's software pipeline in which producer warps issue TMA loads of upcoming KV tiles into a shared-memory ring buffer while consumer warpgroups run wgmma and softmax on already-loaded tiles.
Product Quantization (PQ)
Compresses a vector by splitting it into m subvectors, each quantized independently to a small codebook, storing one byte per subvector.
Production evaluation sampling
Strategically sampling live traffic (100% of failures, higher rates for new prompt versions, ~5% baseline) for async LLM-as-judge scoring without adding request latency.
Defined in 12.2 Observability, Logging & LLMOps
program_id
The Triton function `tl.program_id(axis)` returning which instance of the launch grid the current kernel invocation is, analogous to CUDA's blockIdx.
Defined in 4.4 Writing GPU Kernels with Triton
project memory file (CLAUDE.md / AGENTS.md)
A repo-root file whose contents are injected into the system prompt to encode team-specific, verifiable instructions like build and lint commands.
projector (LLaVA-style connector)
A learned linear layer or MLP that maps vision-encoder patch embeddings into the LLM's embedding space so they can be prepended as visual tokens.
Defined in 10.2 Vision-Language Models
Prompt (MCP primitive)
A user-controlled, parameterised interaction template a server exposes, explicitly selected by the user to inject a structured conversation.
Defined in 8.6 The Model Context Protocol (MCP)
prompt caching (prefix caching)
Reusing the KV-cache computed for a static prompt prefix across requests so only the prefill for that prefix is paid once, not on every call.
Defined in 8.9 Prompt Engineering as Engineering · used in 2 chapters
prompt injection
An attack where adversarial text hijacks an LLM's instruction-following, exploiting the fact that instructions and data share the same context window.
Defined in 12.6 Security: Prompt Injection, Jailbreaks & Defenses · used in 3 chapters
Prompt injection through tool results
An attack where adversarial instructions embedded in a tool's returned text are inserted into the model's context and mistaken for trusted instructions.
Defined in 8.6 The Model Context Protocol (MCP)
Prompt template
A parameterized document with typed slots (system, few-shot examples, runtime context, user turn) rendered into messages, separating structure from data.
Defined in 8.9 Prompt Engineering as Engineering
prompt tuning
Prepends k learnable embedding vectors to the input layer only; trains under 0.01% of parameters, closing the full-fine-tuning gap at large scale.
prompt-level replay (prioritized prompt buffer)
Replay of which tasks to attempt again, weighted by informativeness, with completions always freshly generated under the current policy — on-policy and cost-free.
Defined in 6.12 RL Data, Curriculum & Replay Management
prompt-prefix caching
Provider-side reuse of KV-cache tensors for a repeated prompt prefix (e.g., system prompt) across calls, billed at a fraction of normal input price on hits.
Prompt/response logging
Storing verbatim prompt and completion text as structured, access-controlled records, since natural-language behavior can't be fully captured by metrics.
Defined in 12.2 Observability, Logging & LLMOps
proof assistant (Lean 4)
An interactive theorem prover whose trusted kernel type-checks proof terms via the Curry-Howard correspondence, giving a zero-false-positive verifiable reward.
property graph
A knowledge store of nodes and labelled edges (e.g. Neo4j, Kuzu) used for agent memory when multi-hop relational queries matter more than similarity search.
Defined in 8.5 Memory Systems for Agents
proxy reward model
A lightweight classifier trained to predict explicit thumbs-up/down from noisy implicit signals (copies, edits, regenerations), scoring the bulk of unlabeled traffic.
Defined in 12.5 Data Flywheels & Continuous Improvement
pruning (structured and unstructured)
Compression that removes weights from a trained model: unstructured zeroes individual low-importance weights; structured removes whole heads, neurons, or layers.
PUCT (predictor + upper confidence bound for trees)
The selection formula combining a node's mean value, an LM-derived prior, and a visit-count-based exploration bonus to balance exploitation and exploration in MCTS.
pushdown automaton (PDA)
An FSM augmented with a stack, needed to recognize context-free languages like arbitrarily-nested JSON that an FSM alone cannot track.
Defined in 7.10 Structured & Constrained Generation
PyTorch Distributed Checkpoint (DCP)
PyTorch API (`torch.distributed.checkpoint`) that saves sharded state in a topology-agnostic layout so checkpoints can be reloaded onto a different GPU count.
Q
Q-Former (Querying Transformer)
BLIP-2's module that uses a fixed set of learned query vectors to cross-attend into full patch tokens, compressing any-resolution visual features to a constant token count.
Defined in 10.2 Vision-Language Models · used in 2 chapters
QK-Norm (query-key normalisation)
Applying RMSNorm to query and key vectors before computing attention scores, bounding dot-product magnitude to prevent softmax-saturating logit explosion in large or long-trained models.
Defined in 2.10 Modern Architecture Improvements & Design Choices · used in 5 chapters
QLoRA
A fine-tuning recipe that freezes the base model in NF4 (with double quantization) and trains only added BF16 LoRA adapters, using paged optimizers to avoid OOM.
Defined in 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT · used in 4 chapters
quadratic attention bottleneck
The O(N^2) FLOPs and memory cost of the QK^T softmax product in standard attention, which dominates total compute at long sequence lengths.
Quality classifier
A trained model (fastText, logistic regression on hashed features, or LLM-rated) that scores documents' training value, keeping only top-percentile text.
quality gate
A cheap, fast check (e.g., confidence/logprob threshold, finish-reason check) used to decide whether a cascade tier's output is good enough to return without escalating.
quality SLO
An SLO on the fraction of responses judged semantically good by an automated judge, added because HTTP success alone misses hallucinations and blank outputs.
quantised fallback
A locally hosted, weight-quantised (e.g., INT4 via GPTQ/AWQ) copy of a model used as a cheaper/faster serving tier or as a fallback when a frontier API times out or rate-limits.
quantization (FP8/INT8/INT4)
Mapping continuous/high-precision values to a small finite integer grid via a scale (and optional zero-point) to shrink memory and speed up compute.
Defined in 4.7 Quantization I: Post-Training Quantization (GPTQ, AWQ, SmoothQuant) · used in 2 chapters
quantization-aware distillation (QAD)
Fine-tuning a quantized (e.g. INT4) student with KD from a full-precision teacher so soft targets recover accuracy lost to quantization.
query (Q)
A learned linear projection of a token's representation (q_i = x_i W_Q) expressing 'what this position is looking for' in a soft-retrieval lookup.
Defined in 2.3 The Attention Mechanism From Scratch
query rewriting / multi-query expansion
Generating several paraphrased variants of a user query with an LLM, retrieving for each, and merging results to improve recall for queries expressible in multiple ways.
Defined in 9.4 Chunking, Reranking & Hybrid Search
query-formulation bottleneck
The dominant source of end-to-end agent failure: a poorly formed search query retrieves nothing useful, the one cognitive step too hard to fully offload at 100M scale.
R
R1-Zero phenomenon
DeepSeek-R1's finding that applying RL with only a correctness reward to a base model spontaneously grows long chain-of-thought, self-verification, and backtracking.
radix tree (compressed trie)
A prefix tree where chains of single-child nodes collapse into one edge holding a whole token subsequence; here each node also owns KV-cache slot indices.
RadixAttention
SGLang's runtime technique that keeps a radix tree of token prefixes and automatically reuses KV cache for any shared prompt prefix across requests.
Defined in 7.4 SGLang: RadixAttention & Structured Programs · used in 2 chapters
RAG (Retrieval-Augmented Generation)
Pattern that injects retrieved context from a vector database or search engine into the prompt to ground generation without retraining.
Defined in Glossary of Terms
RAGAS
Reference-free evaluation framework using an LLM judge to score RAG systems on faithfulness, answer relevance, and context precision/recall.
RAGEN
Gym-style multi-turn agentic RL training framework that generalizes ReAct into a reset/step environment API built for training, not just inference.
Defined in 6.10 Agentic & Multi-Turn RL
RAPTOR (recursive summarization tree)
Indexing method that recursively clusters and LLM-summarizes chunks into a tree, so queries can match either summary nodes or leaf chunks depending on granularity.
Ray
UC Berkeley/Anyscale distributed computing framework offering remote functions, stateful actors, and a shared-memory object store; the orchestration substrate under OpenRLHF.
Ray actor
A stateful process (from an @ray.remote-decorated class) owning its own GPU/CPU/memory, addressable by handle; RLHF roles like policy, critic, RM each map to one.
re-decay
Cosine or linear decay of the learning rate back down to a low floor at the end of a CPT run, where most committed loss reduction happens.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
Re-run reality tax
A budgeted percentage (here ~25% of GPU spend) added to a cost estimate to cover OOMs, loss spikes, corrupted shards, and restarts that a first real training run inevitably hits.
re-warming
Raising the learning rate back up from its post-training floor at the start of a CPT run so the model can actually learn the new distribution.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
ReAct (Reasoning and Acting)
Interaction protocol interleaving 'Thought' reasoning with 'Action' tool calls and 'Observation' results; the dominant format for agentic environments.
Defined in 6.10 Agentic & Multi-Turn RL · used in 3 chapters
ReAct prompting
An agent prompting pattern that interleaves Thought, Action, and Observation turns, explicitly templating the reasoning-then-tool-use cycle.
Defined in 8.9 Prompt Engineering as Engineering
reasoning faithfulness
Whether a chain-of-thought is a genuine account of the computation behind an answer, tested via early-answering, mistake-injection, and biasing-cue probes.
recall-latency-memory triangle
The framing that every ANN deployment picks two of recall, latency, and memory cheaply while paying for the third; each method's tuning knob slides along this tradeoff.
recall@k
Fraction of the true top-k nearest neighbors (from brute force) that an ANN method actually returns; the standard way to measure approximation quality.
Reciprocal Rank Fusion (RRF)
Formula that merges multiple ranked retrieval lists by summing 1/(k+rank) per list, rewarding chunks ranked consistently well across retrievers.
Defined in 9.3 Retrieval-Augmented Generation Architectures · used in 3 chapters
recomputation (activation checkpointing)
Trading extra FLOPs for memory by recomputing intermediate values (here, probability tiles) during the backward pass instead of storing them from the forward pass.
recursive reward modeling (RRM)
Uses an already-aligned model to help humans judge harder tasks, training a stronger reward model that bootstraps oversight one level up.
Reduce-Scatter
Collective that reduces contributions across ranks and gives each rank only its own slice of the result (output size M/n), the first phase of ring all-reduce.
Redundant experts (expert replication)
Placing extra copies of the busiest experts on under-loaded GPUs so dispatch can spread their tokens across replicas, reducing the imbalance factor without dropping tokens.
Reference counting (ref_count)
A per-block counter tracking how many active requests are reading a cached KV block; blocks with ref_count > 0 are protected from eviction.
Defined in 7.7 Prefix Caching & KV-Cache Reuse
Reference model
A frozen copy of the SFT model whose log-probabilities anchor the per-token KL penalty during PPO optimization.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 3 chapters
reference model and proxy model
Small models used in DoReMi: the reference trains once on the natural mixture to set a loss baseline; the proxy trains with adaptive Group-DRO weights.
reference policy (π_ref)
The frozen policy (usually the SFT model) that the trained policy is regularized toward via KL divergence, and against which the implicit reward's log-ratio is measured.
Defined in 5.7 Direct Preference Optimization & Its Variants · used in 2 chapters
RefinedWeb (MacroData Refinement pipeline)
Web-only corpus and pipeline (URL filter, WARC extraction, quality filter, dedup) that first showed filtered web data alone can beat curated multi-source mixes.
reflection (reflect operation)
A scheduled LLM call that reads a batch of recent episodic entries and distils them into compact, stable semantic facts.
Defined in 8.5 Memory Systems for Agents
Reflexion
Outer-loop architecture (Shinn et al. 2023) that generates a verbal self-critique of a failed trajectory and prepends it to the next attempt as persistent, accumulating episodic memory.
refusal training
A safety-data recipe that mixes a small fraction of refusal/safe-completion examples with benign 'comply' contrast examples to calibrate harm refusal without causing over-refusal.
register tokens
Extra learnable tokens appended to the ViT sequence to absorb global 'scratch' information, removing high-norm artifact tokens from attention maps; discarded at output.
Defined in 10.1 Vision Transformers & Image Encoders
regularization
Any technique (L2/weight decay, L1, dropout, early stopping) that reduces the generalization gap, usually by penalizing or constraining the model during training.
Defined in 1.5 Machine Learning Fundamentals
REINFORCE (score-function estimator)
Policy-gradient identity ∇J = E[R(τ)∇log π_θ(τ)] derived via the log-derivative trick, letting you optimize expected reward without differentiating through sampling or the reward itself.
Reinforcement Learning from Human Feedback (RLHF)
Pipeline that converts human preference comparisons into a learned reward model, then optimizes a policy against it under a KL constraint.
Defined in 5.5 The RLHF Pipeline & Reward Modeling
rejection sampling (rejection-sampling fine-tuning)
Sampling many teacher rollouts per task and keeping only those whose final answer verifiably matches gold, to build an SFT dataset.
Rejection Sampling Fine-Tuning (RFT)
Self-improvement loop: sample k completions per prompt, keep only those passing a verifiable reward check, and fine-tune the policy on the kept correct pairs, then repeat.
rejection-sampling fine-tuning (STaR/RFT)
Over-generating chain-of-thought traces from a teacher model per problem, keeping only traces whose final answer verifies correct, then fine-tuning a student on the survivors.
relative position encoding
Any scheme making the attention score a function of offset (i-j) rather than absolute indices i and j, aligning with how language uses distance.
remasking schedule
The decoding-time rule for how many and which denoised positions to commit (unmask) each sampling step versus return to [MASK] for later steps.
repeat_kv broadcast
The GQA implementation step that expands each shared KV head to align with its group of query heads at compute time, without copying the smaller cached KV tensors.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
repetition penalty
A multiplicative CTRL-style penalty that discounts the logits of tokens already present in the context, reducing (not eliminating) their probability.
replay buffer
A frozen sample of earlier training data mixed into each retraining round at fraction rho to prevent the model from forgetting old capabilities.
Defined in 12.5 Data Flywheels & Continuous Improvement
replay ratio
Fraction r of CPT training tokens drawn from the original (or proxy/self-generated) data distribution, mixed in to anchor the model against forgetting.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
reproducibility checklist
The set of eval-run details (model revision, harness version, task version, few-shot seed, prompt template, scoring mode, hardware, precision) that must be logged for a benchmark score to be reproducible.
Defined in 11.3 Building Eval Harnesses
reserve-then-reconcile
Rate-limiting/accounting pattern that charges a pessimistic token estimate up front and refunds the difference once the true completion length is known.
Defined in 12.1 Designing an LLM Serving System
resharding
Re-partitioning a tensor-parallel-sharded weight matrix from one shard count to another (e.g. training TP degree p to rollout TP degree q) via gather-then-resplit.
resharding problem
The mismatch between how the trainer shards/fuses/stores parameters (e.g., FSDP, bf16) and how the inference engine stores them (TP, fused QKV, quantized), which sync must reconcile.
residual connection (skip connection)
Adding a sublayer's output back to its input, x' = x + F(x), so gradients flow through an unimpeded identity path.
residual distribution
The distribution p_res(x) proportional to (p(x)-q(x))_+ used to resample a replacement token whenever a drafted token is rejected.
residual stream
The running vector of dimension d that flows through every block; each sublayer reads it and writes a small update back.
Defined in 2.6 The Transformer Block: Norms, Residuals, MLPs & Activations · used in 4 chapters
Residual vector quantisation (RVQ)
Cascade of vector-quantisation stages, each coding the residual error left by the previous stage, used to tokenise audio waveforms into acoustic tokens.
Defined in 10.5 Unified & Any-to-Any Models
Residual Vector Quantization (RVQ)
Cascaded vector quantizers where each stage quantizes the residual left by the previous one, producing K integer codes per audio frame at increasing fidelity.
Defined in 10.3 Audio, Speech & Multimodal Fusion
resolve rate
SWE-bench's binary outcome metric: fraction of tasks where all target tests pass after applying the agent's patch; a patch fixing 9 of 10 tests still scores 0.
Defined in 8.8 Agent Evaluation & Benchmarks
Resource (MCP primitive)
Application-controlled, URI-addressable readable content (static or dynamic) that a host can inject into context without executing code.
Defined in 8.6 The Model Context Protocol (MCP)
ResourcePool
A veRL reservation of GPUs (e.g. one node of 8) that a WorkerGroup is bound to; multiple worker groups can share one pool for colocation.
response mask
Binary mask marking generated response tokens (1) versus prompt/padding tokens (0), used so loss, advantages, and KL are computed only on chosen tokens.
response-only loss masking
Setting instruction tokens' labels to an ignore index so gradients flow only from response tokens, concentrating the SFT signal on reply quality.
Responsible Scaling Policy (RSP) / Frontier Safety Framework
Organizational framework tying capability thresholds (e.g. ASL levels) to required safeguards, using evals as tripwires and a hard gate on deployment/training.
ReST (Reinforced Self-Training)
Self-improvement method separating data generation ('Grow': sample and filter by a reward threshold) from fine-tuning ('Improve'), raising the reward threshold each iteration as a curriculum.
Retention (RetNet)
A recurrence-free mechanism computing QK^T masked by a fixed per-head exponential decay matrix, with equivalent parallel, recurrent, and chunkwise computation forms.
Retrieval-Augmented Generation (RAG)
Architecture that retrieves relevant external documents at inference time and conditions the LLM's generation on them instead of relying solely on parametric memory.
Defined in 9.3 Retrieval-Augmented Generation Architectures · used in 2 chapters
return-to-go
Discounted sum of a turn's own and all future per-turn rewards, used as the advantage target in turn-level credit assignment.
Defined in 6.10 Agentic & Multi-Turn RL
reversal curse
The tendency of autoregressive models trained on 'A is B' to fail at answering 'what is B?', arising from the strictly left-to-right training gradient.
reverse-mode autodiff
Autodiff strategy that traverses the computation tape backward from output to inputs, computing gradients in O(1) forward passes regardless of parameter count.
Reward ensemble
Set of K independently trained reward models whose mean score and inter-model variance are combined to penalize outputs with high disagreement, catching extremal hacking.
Reward hacking (Goodhart's law)
When the policy exploits reward-model weaknesses (length, sycophancy, format farming, out-of-distribution gibberish) to score highly without genuinely improving.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 6 chapters
reward hacking in evals
A model or evaluator exploiting the benchmark metric itself (contamination, format exploitation, test hardcoding, self-report inflation) rather than solving the underlying task.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Reward model (RM)
A network, initialized from the SFT model with its LM head swapped for a scalar head, trained to score response quality via Bradley-Terry loss.
Defined in 5.5 The RLHF Pipeline & Reward Modeling · used in 2 chapters
Reward model / verifier
The component scoring responses (learned reward model, rule-based verifier, code sandbox, or LLM judge) as a pluggable function returning a scalar reward.
Defined in 6.1 The Anatomy of an RL-for-LLM System
Reward model overoptimization
Empirical phenomenon (Gao et al. 2022) where the proxy-versus-true reward gap grows roughly as sqrt(KL) as PPO optimization pressure increases past the true-reward peak.
Reward server
An HTTP service that decouples reward computation (verifiers, judges, sandboxes) from the training loop, enabling independent scaling and result caching.
Reward-model over-optimization
The empirical rise-then-fall curve of true (gold) response quality versus KL distance as a policy over-optimizes against a proxy reward model.
Defined in 5.5 The RLHF Pipeline & Reward Modeling
RewardTrainer
TRL trainer that fits a scalar reward model from (chosen, rejected) comparison pairs using the Bradley-Terry pairwise-preference loss.
Defined in 6.3 TRL: HuggingFace's RL Library
ridge point (arithmetic intensity threshold, I*)
The ratio of a GPU's peak compute to peak bandwidth (FLOP/byte); kernels with arithmetic intensity below it are memory-bound, above it are compute-bound.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 4 chapters
ring all-reduce
Bandwidth-optimal all-reduce algorithm that arranges ranks in a logical ring and runs reduce-scatter then all-gather, avoiding a single-node bottleneck.
Ring Attention
Sequence-parallel attention algorithm that shards K/V chunks across GPUs arranged in a ring, overlapping chunk rotation with tiled online-softmax attention compute.
Ripple effect
The failure mode where editing one fact leaves its logical, multi-hop, or aliased consequences unchanged, so the model's associations become inconsistent.
Defined in 13.2 Knowledge Editing & Machine Unlearning
RL with Verifiable Rewards (RLVR)
RL post-training where reward comes from a deterministic checker V(q,o) in {0,1} (math equivalence, unit tests) instead of a learned reward model.
Defined in 5.9 RL with Verifiable Rewards (RLVR) & The Reasoning Recipe · used in 2 chapters
RLAIF (Reinforcement Learning from AI Feedback)
Replacing human preference raters with an AI judge model that labels response pairs, used to train a reward/preference model at far lower cost and higher throughput than humans.
RLHF (Reinforcement Learning from Human Feedback)
Three-stage pipeline (SFT, reward model training, PPO fine-tuning) that aligns model behavior with human preferences using a learned reward signal.
Defined in Glossary of Terms · used in 2 chapters
RLOO (REINFORCE Leave-One-Out)
Critic-free policy gradient method whose per-sample baseline is the mean reward of the other G-1 group samples, giving an unbiased, single-step, clip-free estimator.
Defined in 5.8 GRPO, RLOO & Critic-Free RL · used in 2 chapters
RLVR (reinforcement learning with verifiable rewards)
Reinforcement learning where a program (e.g., an exact-match checker) grades model samples for correctness, replacing a learned, hackable reward model.
Defined in 14.9 Post-Training: SFT, DPO, and Narrow RLVR (GRPO) That Works at 100M · used in 2 chapters
RMS-matching scale
Factor 0.2*sqrt(max(m,n)) multiplying Muon's orthogonalized update so its magnitude matches AdamW's, letting one learning rate govern both optimizers.
RMSNorm (Root Mean Square Normalization)
Normalization that divides by the root-mean-square of a feature vector and applies a learned scale, dropping LayerNorm's mean subtraction and shift for equal stability at lower cost.
Defined in 2.10 Modern Architecture Improvements & Design Choices · used in 3 chapters
RNG (random-number generator) state
The combined Python, NumPy, and per-device torch/CUDA random states, saved per rank so resumed training reproduces the same stochastic behavior.
ROC-AUC (Receiver Operating Characteristic, Area Under the Curve)
Area under the true-positive-rate vs. false-positive-rate curve across thresholds; equals the probability a random positive is ranked above a random negative.
Defined in 1.5 Machine Learning Fundamentals
ROCm (Radeon Open Compute)
AMD's open-source GPU software stack, including HIP, rocBLAS, and RCCL, that targets Instinct GPUs as a CUDA-compatible alternative.
Rollout / generation engine
The inference component that autoregressively samples responses under the current policy, producing token ids and behavior log-probs; usually the dominant per-step cost.
Defined in 6.1 The Anatomy of an RL-for-LLM System
rollout engine
A purpose-built inference server (vLLM, SGLang) used for the generation phase instead of naive `model.generate`, exploiting batching and cache-sharing tricks.
rollout queue
A bounded, sample-level async queue where inference workers push finished, policy-version-stamped rollouts and the trainer pulls batches from whatever is available, avoiding batch-level barriers.
ROME (Rank-One Model Editing)
A method that inserts a fact by optimizing a target value, capturing its key, and applying a closed-form rank-one update to one MLP down-projection matrix.
Defined in 13.2 Knowledge Editing & Machine Unlearning
roofline model
A log-log plot of achievable FLOP/s vs. arithmetic intensity, capped by min(compute roof, bandwidth roof), used to diagnose a kernel's bottleneck.
Root Mean Square Layer Normalization (RMSNorm)
Normalizes by RMS(x) only (no mean-centering, no shift term), used in Llama, Mistral, Gemma, and most modern models.
RoPE (Rotary Position Embedding)
Encodes position by rotating query/key vectors (in 2-D dimension pairs) by an angle proportional to position, so their dot product depends only on relative offset.
Defined in 2.5 Positional Encodings: Sinusoidal, Learned, RoPE & ALiBi · used in 4 chapters
RoPE base rescaling (NTK-aware)
Raising the RoPE frequency base theta by s^(d/(d-2)) so positions up to a new longer length map onto the same angular range the model learned at the shorter pretrain length.
RoPE (rotary position embeddings) with NoPE
Rotating queries and keys by position-dependent angles on most layers, while omitting positional encoding entirely on interleaved layers (every 4th) to improve length generalization.
rope_theta
The base hyperparameter in RoPE's frequency formula; raising it (e.g. 10000 to 500000) spreads rotation frequencies to improve long-context extrapolation.
Rotary Position Embedding (RoPE)
Rotates query and key vector pairs by an angle proportional to token position, so their dot product depends only on relative position m-n.
Defined in The Math Reference Sheet · used in 2 chapters
round-to-nearest (RTN)
The simplest weight quantization baseline: independently round each weight to its nearest grid point, ignoring interactions between weights.
round-to-nearest (RTN) quantization
Baseline PTQ scheme: pick a per-group scale (and optional zero-point) from weight min/max, then round each weight independently to the nearest integer level.
Router (gate)
A small linear layer mapping a token to one logit per expert, used to select which experts process it and to weight their outputs.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
Router z-loss
A penalty on the squared log-sum-exp of router logits (from ST-MoE) that keeps routing decisions from saturating near one-hot, improving low-precision training stability.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
routing classifier
A small offline-trained model (e.g., embedding + logistic regression) that predicts which model tier a query needs before any LLM call, avoiding sequential-cascade latency.
rsLoRA (rank-stabilized LoRA)
Scaling fix that replaces LoRA's $\alpha/r$ factor with $\alpha/\sqrt{r}$ so the adapter's magnitude and gradient stay stable as rank grows, letting high ranks deliver their extra capacity.
Rule-based verifier
A deterministic function that checks a completion against ground truth (math equivalence, unit tests) and returns a reward at zero training-time cost.
RWKV (WKV attention)
A recurrent architecture expressing attention as an exponentially time-decayed weighted sum of past key-value pairs, trainable in parallel yet O(1) per inference step.
S
saddle point
A critical point (zero gradient) where the Hessian has both positive and negative eigenvalues; overwhelmingly more common than true minima in high-dimensional loss surfaces.
Defined in 1.3 Calculus, Optimization & Convexity
Sampling (MCP)
An MCP capability letting a server send a sampling/createMessage request asking the host to run LLM inference on its behalf, enabling nested agentic servers.
Defined in 8.6 The Model Context Protocol (MCP)
sandbagging
Strategic underperformance on a safety eval by a model that can actually do the dangerous task, to avoid triggering restrictions before deploying it later.
sandbox (execution sandbox)
An isolated, resource-limited execution environment (rlimits, timeouts, no network/filesystem access) used to safely run untrusted model-generated code for reward scoring.
Defined in 5.9 RL with Verifiable Rewards (RLVR) & The Reasoning Recipe · used in 2 chapters
Sandboxed execution
Running untrusted model-generated code inside an isolated container or microVM with no network access, capped CPU/memory, and restricted syscalls.
Scalable oversight
Research program for supervising AI systems whose task competence exceeds a human evaluator's ability to verify correctness.
Defined in 13.5 AI Safety: Scalable Oversight, Dangerous-Capability Evals & Frontier Safety · used in 2 chapters
scale and zero-point
The two affine-quantization parameters: scale s is the size of one quantization step; zero-point z is the integer code representing real value 0.
scaled dot-product attention
The operation softmax(QK^T/sqrt(d_k))V: score queries against keys by dot product, scale, softmax into weights, then blend the values.
Defined in 2.3 The Attention Mechanism From Scratch · used in 3 chapters
scaled residual init
Initializing each sublayer's output projection (c_proj) with std reduced by 1/sqrt(2*n_layer) so the residual stream's variance does not grow linearly with depth.
scaled residual initialization
Scaling down residual-path output projections (attention output, MLP down-projection) by 1/sqrt(2L) at initialization to keep residual-stream variance controlled in deep models.
scaling ladder
A sequence of tiny models (here 4M-44M params) trained under one frozen recipe at varied N and D, used to fit a scaling law cheaply before the flagship run.
scaling law (Chinchilla parameterization)
The joint form L(N,D) = E + A/N^α + B/D^β, modeling loss as an irreducible floor plus finite-model and finite-data penalty terms.
Scaling laws
Empirical power-law relationships linking model size, dataset size, compute, and test loss, used to predict optimal training configurations (e.g., Chinchilla).
Defined in Glossary of Terms · used in 2 chapters
ScaNN (anisotropic vector quantization)
A quantization method that weights reconstruction error parallel to the vector more heavily than orthogonal error, since parallel error most affects inner-product ranking.
Scheduler policy (guaranteed_no_evict vs. max_utilization)
TensorRT-LLM executor settings controlling whether the scheduler refuses new requests when KV cache is full (no eviction) or evicts and later recomputes sequences to admit more.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
schema-constrained generation
Forcing model output into a predefined JSON schema so injected instructions cannot cause arbitrary actions, since only structured fields are parsed.
score function / score matching
The gradient of log-density ∇_x log p(x); training a network to approximate it (score matching) is mathematically equivalent to DDPM's noise prediction.
scratchpad (plan file)
A persistent external file (e.g. PLAN.md) the agent reads and writes each turn to externalize working memory so it survives context compaction.
Defined in 8.4 Context Engineering & Management
selective batching
Orca's technique of flattening tokens from heterogeneous requests into one matrix for position-independent GEMMs (QKV, MLP) while running attention per-request over each one's own KV cache.
Defined in 7.2 Continuous Batching & Request Scheduling
selective scan
Mamba's core recurrence computation, made GPU-efficient via a hardware-aware parallel prefix scan or chunkwise (intra-chunk matmul, inter-chunk recurrent) algorithm.
self-attention
Attention where queries, keys, and values are all learned projections of the same input sequence, as opposed to cross-attention across two sequences.
Defined in 2.3 The Attention Mechanism From Scratch
self-consistency
Sampling N independent reasoning traces at temperature >0 and taking a majority vote over extracted final answers to cancel out inconsistent errors.
Defined in 5.10 Reasoning, Chain-of-Thought & Test-Time Compute · used in 2 chapters
self-critique loop
A retry pattern where an orchestrator scores a worker's output against a rubric and re-dispatches with the critique appended if the score falls below a threshold, acting as a policy-gradient-like update without fine-tuning.
Defined in 8.7 Multi-Agent Systems & Orchestration
self-information (surprisal)
The information content -log p(x) of an observed event; larger for rarer events, and its expectation under p defines Shannon entropy.
Self-preference bias
A judge's tendency to overrate outputs from its own model family (e.g., GPT-4 judging GPT-4 responses more favorably) versus other models.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
Self-RAG
Method that fine-tunes an LLM to emit reflection tokens ([Retrieve], [Relevant], [Supported], [Utility]) so retrieval and self-grading happen on demand.
Self-rewarding language models
A training loop where the same model serves as both policy and judge, alternately generating and scoring its own candidate responses to build preference pairs for iterative DPO updates.
self-supervised learning (SSL)
Learning paradigm where training labels are derived automatically from the input itself (e.g., next-token prediction), without human annotation; the basis of LLM pretraining.
Defined in 1.5 Machine Learning Fundamentals
semantic caching
Caching by embedding the query and returning a stored response when a new query's embedding cosine-similarity exceeds a calibrated threshold, catching near-duplicates exact caching misses.
semantic memory
Stable, general facts (preferences, configuration) that hold across sessions; upserted/overwritten on new evidence and queried by content.
Defined in 8.5 Memory Systems for Agents
Semantic tokens and acoustic tokens
Two audio tokenisation levels: semantic tokens capture phoneme-like content for language alignment; acoustic tokens capture waveform detail for high-fidelity synthesis.
Defined in 10.5 Unified & Any-to-Any Models
Semantic-acoustic disentanglement
Training a codec's first RVQ level (e.g. via SpeechTokenizer's distillation loss to HuBERT labels) to capture linguistic content separately from higher levels that capture acoustic style.
Defined in 10.3 Audio, Speech & Multimodal Fusion
sequence packing
Concatenating multiple documents into one training sequence separated by EOS tokens, using attention masks to avoid padding and cross-document attention leakage.
Defined in 2.2 Embeddings & The Input Pipeline · used in 3 chapters
Sequence parallelism (SP)
Megatron technique that splits the otherwise-replicated LayerNorm/dropout/residual regions between TP matmul blocks along the sequence dimension, cutting activation memory at no extra communication cost.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 2 chapters
sequence-level knowledge distillation (SeqKD)
Distillation that uses the teacher's full greedy-decoded output text as hard training targets, rather than matching per-token soft distributions.
Defined in 5.12 Distillation, Model Compression & Knowledge Transfer · used in 2 chapters
sequential testing (always-valid inference)
A statistical testing approach (e.g. mSPRT, confidence sequences) that lets experimenters peek at results anytime without inflating the false-positive rate.
serial depth
The number of sequentially dependent forward passes needed to produce an output; the quantity diffusion reduces (N steps) relative to AR's L tokens.
serious incident (Article 62)
Under the EU AI Act, an event that resulted or could have resulted in death, serious health harm, property damage, essential-service disruption, or a fundamental-rights violation, triggering mandatory notification.
Defined in 13.6 AI Governance, Compliance & Regulation
Server (MCP)
An independent, single-responsibility process or network service that exposes capabilities to a host through the tool, resource, and prompt primitives.
Defined in 8.6 The Model Context Protocol (MCP)
service level indicator (SLI)
A measured metric of system behavior (availability, latency, or quality) used to judge whether an LLM service is performing acceptably.
service level objective (SLO)
A target threshold on an SLI (e.g., quality pass rate >= 95%) that a team commits to meeting over a time window.
session persistence
The architecture (session ID, bootstrap read, graceful shutdown write, cross-session reflection) that lets an agent carry state across restarts.
Defined in 8.5 Memory Systems for Agents
SFT (supervised fine-tuning)
Cross-entropy training on rendered chat conversations, with loss masked to assistant tokens only, that teaches a base model to take turns and stop.
SFTTrainer
TRL trainer for supervised fine-tuning; adds automatic sequence packing, chat-template formatting, PEFT integration, and prompt-loss masking.
Defined in 6.3 TRL: HuggingFace's RL Library
SGLang
An inference server and structured-generation framework built on RadixAttention (prefix caching via a radix tree), supporting constrained/schema-based generation.
Defined in Tooling & Environment Setup Cheatsheet
SGMV (Segmented Gather Matrix-multiply)
Punica's fused kernel that, per batch segment, gathers each adapter's A/B weights and computes its low-rank correction in one launch; used in prefill.
shadow deployment
Running a candidate model on live production traffic while discarding its outputs, used to profile latency, cost, and quality before any user exposure.
Shampoo
Second-order optimizer that preconditions a weight matrix's gradient with Kronecker-factored row/column curvature matrices (L^-1/4 G R^-1/4) instead of a diagonal preconditioner.
Defined in 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo · used in 2 chapters
Shannon entropy
Expected self-information of a distribution, H(p) = -sum p(x) log p(x); measures average uncertainty, maximized by uniform, zero for degenerate distributions.
shape guard
An assumption (e.g., a tensor's shape or dtype) recorded during TorchDynamo tracing; if violated on a later call, it triggers an expensive recompilation.
sharded checkpoint
Checkpoint strategy where each rank writes its own parameter shard in parallel, rather than gathering all state to one rank before saving.
ShardedTokenLoader
The training-loop component (built in chapter 3.5) that memmaps uint16 .bin shards and slices out fixed-length (ctx+1)-token windows for each training batch.
Shared expert
An always-on dense FFN path that every token passes through in addition to routed experts; runs locally with no all-to-all, providing free overlap work to hide dispatch latency.
shared memory (SMEM)
Fast, programmer-managed on-chip SRAM private to a thread block, used to stage and reuse data (e.g. matrix tiles) to avoid repeated slow HBM reads.
Defined in 1.8 GPU Architecture & The Memory Hierarchy
shared-memory bank conflict
Serialized (slowed) memory access that occurs when multiple threads in a warp read different addresses that map to the same one of shared memory's 32 banks; fixed by padding array rows.
sharp vs. flat minima
Classification of a loss minimum by the Hessian's largest eigenvalue at that point; flat minima (small max eigenvalue) generalize better than sharp ones.
Defined in 1.3 Calculus, Optimization & Convexity
shield model
A dedicated model (Llama Guard, ShieldLM, Aegis, Granite Guardian) deployed as a sidecar microservice to classify conversations for harm, separate from the main serving LLM.
Defined in 12.4 Safety, Guardrails & Content Moderation
SigLIP (Sigmoid Loss for Language Image Pretraining)
CLIP variant replacing the batch-softmax contrastive loss with an independent sigmoid loss per pair, scaling to larger batches; its vision tower backs PaliGemma and ColPali.
Defined in 9.6 Multimodal & Visual-Document Retrieval: ColPali & Late Interaction · used in 2 chapters
signal-to-noise ratio (SNR)
The quantity SNR(t) = ᾱ_t/(1-ᾱ_t) measuring how much original signal versus noise remains in x_t at timestep t; used to compare/weight noise schedules and losses.
SimPO (Simple Preference Optimization)
A reference-free DPO variant using length-normalized average log-probability as the reward and a target margin $\gamma$, removing DPO's length bias.
SIMT (single instruction, multiple threads)
NVIDIA's execution model where all 32 lanes of a warp execute the same instruction on different data per cycle; warp divergence occurs when lanes take different branches.
Defined in 1.8 GPU Architecture & The Memory Hierarchy
single controller
One driver process runs the RL algorithm's control flow as ordinary Python, issuing each stage as a single remote call to worker groups.
singular value decomposition (SVD)
Factorization A = UΣV^T of any matrix into orthogonal rotation, non-negative diagonal scaling, and orthogonal rotation, used for low-rank analysis.
Defined in 1.1 Linear Algebra for Deep Learning
sinusoidal positional encoding
Vaswani et al.'s fixed, parameter-free scheme adding sin/cos vectors at geometrically spaced frequencies per dimension pair to each token embedding.
skip-batch logic
Running the forward/backward pass but skipping the optimizer step (and Adam moment update) when a batch's gradient norm or anomaly score exceeds a threshold.
SL-CAI (Supervised Learning from Self-Critique)
Stage 1 of CAI: the model generates a harmful draft, critiques it against a principle, revises it, and the revision becomes the SFT training target.
SLERP (spherical linear interpolation)
Interpolates two weight tensors along the great circle of the unit sphere instead of linearly, preserving vector norm during merging.
sliding-window (local) attention
Restricting each query to attend only to the last W keys, reducing cost to O(N·W) and bounding the KV cache to a rolling buffer of size W.
SLO (service-level objective)
A latency or throughput target stated as a percentile and number (e.g. p99 TTFT ≤ 1 s) that a serving system is designed to meet.
Defined in 12.1 Designing an LLM Serving System
SmoothQuant
A technique that migrates quantization difficulty from hard-to-quantize activations into easy-to-quantize weights via a per-channel diagonal scale, enabling INT8 weight+activation (W8A8) matmuls.
soft prompting
Replacing a discrete text prompt with continuous, trainable embedding vectors learned end-to-end while the base model stays frozen.
soft targets and temperature
Teacher/student probabilities from a temperature-scaled softmax (τ>1); softening reveals inter-token similarity ("dark knowledge") the student can learn from.
Softmax
Converts a logits vector into a probability distribution via normalized exponentials; the numerically stable form subtracts the max logit before exponentiating.
Defined in The Math Reference Sheet
softmax bottleneck
The result that a softmax classifier's output distribution is rank-limited by hidden dimension, so a narrow tied model's d_model must serve both input identity and output discrimination.
softmax logit soft-capping
Passing pre-softmax attention logits through c·tanh(logit/c) to smoothly saturate extreme values into (−c, c) while leaving small logits nearly unchanged.
softmax scaling by sqrt(d_k)
Dividing raw dot-product scores by sqrt(d_k) so their variance stays at 1 regardless of head dimension, preventing softmax saturation and vanishing gradients.
Defined in 2.3 The Attention Mechanism From Scratch
Softmax-cross-entropy gradient
The fused gradient of softmax followed by cross-entropy loss with respect to logits, which simplifies exactly to predicted-minus-true probabilities (P - Y).
software pipelining (num_stages)
A Triton/GPU technique that overlaps loading the next loop iteration's data with compute on the current iteration to hide memory latency; deeper pipelines use more shared memory.
Defined in 4.4 Writing GPU Kernels with Triton
span corruption
T5's pre-training objective: contiguous spans of the encoder input are replaced with sentinel tokens and the decoder autoregressively reconstructs just those spans.
sparse autoencoder (SAE)
A wide autoencoder trained with a sparsity penalty to reconstruct model activations from a much larger dictionary of features, aiming to recover monosemantic directions from superposition.
sparse gradient
In the embedding table, only rows corresponding to tokens present in the current batch receive nonzero gradient during backpropagation.
Defined in 2.2 Embeddings & The Input Pipeline
sparse upcycling
Converting a trained dense model into a Mixture-of-Experts model by cloning the dense MLP into each expert and adding a freshly initialized router.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
SparseGPT
A one-shot, gradient-free pruning method using Hessian-based (Optimal Brain Surgeon) weight-saliency scoring and compensation, reaching 50-60% sparsity with minimal perplexity loss.
Spearman's rank correlation
A rank-based correlation coefficient (rho) used to compare a judge's pointwise scores against human quality ratings, preferred over Pearson since scores are ordinal.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
special tokens
Reserved vocabulary entries with structural (not textual) meaning, e.g. EOS, BOS, pad, chat/role markers, that must be inserted only by trusted code, never parsed from user text.
Defined in 2.1 Tokenization: BPE, WordPiece, Unigram & Byte-Level · used in 3 chapters
special-token injection
A tokenizer-level attack where untrusted text containing a special-token string (e.g. literal <|assistant|>) gets encoded as the real reserved id, forging role or tool boundaries.
Specification gaming
Broad failure where a policy satisfies the literal reward objective while violating its intended purpose, e.g., fabricating citations or adding vacuous hedges.
spectral norm
The operator norm of a matrix, equal to its largest singular value; measures the maximum stretch factor A applies to any unit vector.
Defined in 1.1 Linear Algebra for Deep Learning
Speculative decoding
Technique where a cheap drafter proposes several tokens and an expensive target model verifies them in one parallel pass, emitting multiple tokens per target forward pass.
Defined in 7.6 Speculative Decoding: Draft Models, Medusa, EAGLE & Lookahead · used in 5 chapters
speculative decoding draft model
A small, fast model whose next-token distribution is trained (via KD) to approximate a larger verifier model's, raising token acceptance rate.
speculative routing
Firing cheap and expensive models concurrently and cancelling the expensive one if the cheap model's output passes the quality gate, trading extra cost for lower latency.
Speech language model (speech LM)
A transformer that natively ingests and emits audio tokens (e.g. AudioPaLM, Moshi), skipping separate ASR/TTS steps by treating audio and text tokens as one sequence.
Defined in 10.3 Audio, Speech & Multimodal Fusion
SPIN (Self-Play Fine-Tuning)
Self-play method that uses the current policy's own generated responses as 'rejected' examples and real human data as 'chosen', training the model to close the gap to the human data distribution.
split-Q warp partitioning
FA2 layout where each warp owns a slice of query rows and computes the full output for those rows, eliminating the inter-warp shared-memory reduction that FA1's split-K layout required.
Splitwise
A 2023 Microsoft Research system extending disaggregation to heterogeneous hardware, using different GPU types for compute-bound prefill ('prompt phase') and bandwidth-bound decode ('token phase').
SPMD (Single-Program Multiple-Data)
Programming model where every process runs the identical program but operates on different data, branching on its rank when needed; backbone of distributed GPU training.
spot/preemptible GPU
A heavily discounted (60-80% cheaper) cloud GPU instance that the provider can reclaim with 30-120 seconds notice, requiring stateless, drain-capable serving workers.
SRAM (static RAM)
The GPU's small, extremely fast on-chip memory (shared memory/L1 per streaming multiprocessor) where FlashAttention keeps its score tiles instead of writing them to HBM.
stability-plasticity dilemma
The tension in CPT between learning new data (plasticity) and retaining old knowledge (stability); low LR under-learns, high LR causes forgetting.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
stable unit treatment value assumption (SUTVA)
The assumption that one unit's treatment doesn't affect another's outcome; violated by per-request randomization mid-conversation, causing contamination.
Stack-100M
The ~101M-parameter deep-and-thin decoder-only transformer this capstone builds end-to-end, from tokenizer through a tool-using agent, under a single fixed spec.
Stack-100M data mix
The chapter's fixed 20B-token blend: 70% FineWeb-Edu, 15% Cosmopedia v2, 10% StarCoder, 5% FineMath/OpenWebMath, following the SmolLM-style web+synthetic+code+math recipe.
staged VLM training (feature alignment then instruction tuning)
LLaVA's two-stage recipe: first train only the projector on image-caption pairs, then jointly fine-tune projector and LLM on multimodal instruction data.
Defined in 10.2 Vision-Language Models
staleness (RL)
How many optimizer steps old the weights were when they generated the rollouts now being trained on; the dial between throughput and on-policyness.
Defined in 6.2 The Generation–Training Loop & Rollout Engines · used in 3 chapters
standard error of a proportion
SE(p) = sqrt(p(1-p)/n), used to build confidence intervals on benchmark accuracy and decide whether a score difference between two models is real or noise.
STaR (Self-Taught Reasoner)
Rejection-sampling variant that adds a rationalization hint: when all sampled rationales are wrong, reveal the gold answer and have the model construct a hint-conditioned chain-of-thought as a training example.
Defined in 5.11 Constitutional AI, RLAIF & Self-Improvement · used in 2 chapters
State Space Duality (SSD)
Mamba-2's result that a scalar-decay selective SSM is mathematically equivalent to linear attention with a data-dependent, multiplicative causal decay mask.
static batching
Request-level batching that fixes a batch's membership until every request in it has finished, causing tail idling and head-of-line blocking.
Defined in 7.2 Continuous Batching & Request Scheduling
stdio transport
MCP transport where the host launches the server as a local child subprocess and exchanges newline-delimited JSON-RPC over stdin/stdout; local-only, single-host.
Defined in 8.6 The Model Context Protocol (MCP)
stochastic gradient descent (SGD)
Optimization method that replaces the full-dataset gradient with an unbiased noisy estimate from a mini-batch, enabling cheaper, more frequent parameter updates.
Defined in 1.3 Calculus, Optimization & Convexity
stochastic rounding
A rounding scheme that rounds up or down with probability proportional to distance from each grid point, making sub-ulp increments unbiased in expectation and letting some bf16-only optimizers skip fp32 master weights.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
straggler problem
The training step waiting on the slowest, heavy-tailed-length trajectory in a synchronous batch of agentic rollouts.
Defined in 6.10 Agentic & Multi-Turn RL
straggler tax
The wasted idle time caused by a synchronous batch having to wait for its single longest (heavy-tailed) response to finish generating before training or the next batch can proceed.
Straight-through estimator (STE)
A backward-pass trick that passes gradients through the zero-gradient rounding operation unchanged, letting quantization-aware training update weights despite non-differentiable rounding.
streaming multiprocessor (SM)
The GPU's core compute building block; hosts warp schedulers, registers, CUDA cores, Tensor Cores, and shared memory/L1 for its resident blocks.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 2 chapters
StrongREJECT
A trained grading model that scores both refusal quality and harmful-content presence, fixing keyword-match gaming in jailbreak evaluation.
structural termination
Ending the agent loop exactly when the assistant emits no tool calls, rather than by detecting phrases like 'done' in its text output.
structured handoff
Passing typed schemas (e.g. Pydantic models) at inter-agent boundaries so payloads are validated before forwarding, making errors explicit instead of silently propagating malformed text.
Defined in 8.7 Multi-Agent Systems & Orchestration
structured outputs (JSON mode)
Constrained decoding that forces model output to conform to a JSON schema, used for extraction/classification tasks independent of tool invocation.
Defined in 8.1 Tool Use & Function Calling
structured request logging
Writing typed, schema-evolvable records (e.g. Avro) per request capturing prompt, output, logprobs, and client signals as raw training material.
Defined in 12.5 Data Flywheels & Continuous Improvement
structured state space model (SSM)
A sequence model (e.g., S4) mapping input to output through a linear hidden-state recurrence, computable as an O(N log N) convolution in training or an O(1)-per-step recurrence at inference.
sub-agent
A spawned agent with its own fresh, isolated context window and narrow instruction that performs exploratory work and returns only a distilled answer to the parent agent.
sub-agent context isolation
Delegating a bounded subtask to a sub-agent with its own fresh context window, returning only its conclusion so verbose exploration never enters the parent's window.
Defined in 8.4 Context Engineering & Management
subnormal numbers (denormals)
Numbers below the minimum normal magnitude, represented with a leading 0.-mantissa instead of 1.; often flushed to zero (FTZ) on GPUs.
Suffix array
A sorted array of all suffixes of a concatenated token sequence; paired with a longest-common-prefix array to find and remove repeated substrings across documents.
superficial alignment hypothesis
The idea that a model's knowledge comes almost entirely from pretraining; SFT merely teaches it to access and present that knowledge in the right format.
superposition
A network's strategy of packing far more sparse features than it has dimensions by representing them as non-orthogonal directions, trading interference for capacity.
supervised fine-tuning (SFT)
Training a pretrained model on (instruction, response) pairs with maximum-likelihood loss to teach instruction-following, format, and helpful behavior.
survival fraction (ρ)
Proportion of generated rollout groups that pass the zero-variance filter under dynamic sampling; determines the oversampling tax needed to fill a target batch.
Defined in 6.12 RL Data, Curriculum & Replay Management
SWE-bench (Verified)
Benchmark of real GitHub issues where an agent must produce a git-diff patch that makes a failing test suite pass; Verified is a ~500-task human-curated subset used for reporting.
Defined in 8.8 Agent Evaluation & Benchmarks · used in 2 chapters
SwiGLU
Gated FFN activation Swish(xW)⊙(xV) using three weight matrices; reduces inner dimension to ~8d/3 to match FLOPs of a 4d FFN.
Defined in 2.6 The Transformer Block: Norms, Residuals, MLPs & Activations · used in 4 chapters
Sycophancy
Tendency of a model to agree with or flatter a user's stated views rather than give accurate answers, learned from rater bias baked into preference data.
symmetric quantization
Zero-point-free quantization scheme (grid centered on 0) fit from max absolute value, used for weight matrices, which are roughly symmetric about zero.
SynthID
Google DeepMind's neural watermarking system embedding an imperceptible, learned signal into generated images/audio/text that survives compression, resizing, and cropping.
systemic-risk threshold
The 10^25 FLOPs training-compute bar (Article 51) above which a GPAI model incurs additional Article 55 obligations like red-teaming and 2-day incident reporting.
Defined in 13.6 AI Governance, Compliance & Regulation
systolic array
A 2D grid of multiply-accumulate cells that pumps weights and activations cell-to-cell, retiring a full matmul tile every clock with almost no register/control traffic.
T
tape (Wengert list)
The directed acyclic graph of Function nodes recorded during the forward pass, traversed in reverse topological order to compute gradients.
task arithmetic
Adding, subtracting, or summing scaled task vectors onto a base model to add, remove, or combine learned behaviors algebraically.
task vector
The weight-space delta between a fine-tuned checkpoint and its shared pre-trained base, used as an additive/subtractive unit for editing model skills.
tau-bench
Benchmark that evaluates tool-augmented agents in simulated customer-service dialogues, scoring both task resolution and policy compliance against a simulated user.
Defined in 8.8 Agent Evaluation & Benchmarks
TBT (time-between-tokens)
Per-token latency during decode streaming; dominated by decode bandwidth and rises once batch size passes the arithmetic-intensity breakeven.
teacher forcing
Training technique that feeds the ground-truth previous token as input rather than the model's own prediction, enabling all positions to be computed in one parallel forward pass.
Defined in 3.3 The Pretraining Objective & Loss
temperature (contrastive learning)
A scalar dividing logits before softmax; low temperature sharpens the next-token distribution toward greedy, high temperature flattens it toward uniform.
Defined in 2.7 Building a GPT From Scratch (nanoGPT-style) · used in 2 chapters
temperature scaling
Dividing logits by T before softmax; T<1 sharpens the distribution toward the argmax, T>1 flattens it toward uniform.
Tensor Core
A dedicated matrix-multiply-accumulate unit that computes whole small-tile matmuls in reduced precision (BF16/FP16/FP8/FP4), roughly 16x faster than CUDA cores for matmul.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 2 chapters
tensor core fp32 accumulation
Hardware behavior where tensor cores multiply low-precision (bf16/fp16/fp8) input pairs but sum the partial products in fp32, making low-precision matmuls numerically survivable.
Defined in 3.8 Mixed Precision, bf16 & FP8 Training
Tensor Memory Accelerator (TMA)
A Hopper hardware unit that performs bulk asynchronous copies between global and shared memory with address generation done in hardware, freeing warps from load address arithmetic.
Tensor parallelism (TP)
Splits a single layer's matrix multiplications across GPUs (column- then row-parallel), needing one all-reduce per region; must stay inside the NVLink domain.
Defined in 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism · used in 3 chapters
TensorRT-LLM (TRT-LLM)
NVIDIA's compiled, quantized inference engine used as NeMo-Aligner's rollout backend, offering high throughput but costlier engine reloads after weight updates than vLLM.
Defined in 6.5 OpenRLHF, NeMo-Aligner & Ray-Based Systems · used in 2 chapters
test contamination
When evaluation data (verbatim or near-verbatim) appears in a model's training corpus, letting it retrieve rather than reason to an answer.
test-time compute
Extra computation spent at inference time (not training) — more sampling, search, or reasoning tokens — to improve answer quality independent of model size.
test-time scaling law
Empirical power-law relationship (accuracy ≈ 1 − ε·N^-α) describing how reasoning accuracy improves as sampling/search budget N grows, with PRM guidance yielding steeper scaling.
test-time-compute-aware evaluation
Comparing models at equal token budget or wall-clock time (accuracy-vs-budget curves) rather than a single accuracy number, since thinking budgets are now variable.
Defined in 11.4 Reasoning, Coding & Agentic Evals
Text Generation Inference (TGI)
Hugging Face's Rust-router + Python-runner serving stack with no build step, custom FlashAttention kernels, and a continuous-batching scheduler; now in maintenance mode as of 2026.
Defined in 7.5 TensorRT-LLM, TGI & Other Serving Stacks
text-and-data mining (TDM) exception
An EU copyright-law exception (2019 DSM Directive) permitting ML training on lawfully accessed content unless the rightsholder has filed a machine-readable opt-out.
Defined in 13.6 AI Governance, Compliance & Regulation
Text-to-SQL RAG
Retrieval over structured/relational data by translating a natural-language question into SQL, executing it, and generating an answer from the results, with self-correction on errors.
textbook-style generation
Prompt-seeded synthetic generation (Phi, Cosmopedia) writing pedagogical passages from a model's parametric knowledge rather than a source document; highest hallucination risk.
TF32 (TensorFloat-32)
NVIDIA Ampere+ tensor-core compute format: fp32 inputs are rounded to a 10-bit mantissa internally and accumulated in fp32 for faster matmuls.
Thompson sampling
Bandit selection rule that draws a pass-rate sample from each task's Beta posterior and picks tasks whose sampled value is closest to the target, exploring uncertain estimates automatically.
Defined in 6.12 RL Data, Curriculum & Replay Management
thrashing (memory)
A failure mode where admitting requests up to the last free KV block triggers repeated preemption-then-readmission cycles, wasting time on recompute/swap instead of generation.
Defined in 7.2 Continuous Batching & Request Scheduling
three-stage post-training recipe
The standard pipeline of SFT, then reward modeling, then RL-based policy optimization (PPO or DPO), each stage building on the previous one.
tied embeddings
Sharing one V×d_model weight matrix between the input embedding lookup and the output (lm_head) projection, saving a full embedding table's worth of parameters.
Defined in 14.3 A Byte-Level BPE Tokenizer From Scratch (and Why Vocab Size Is a Design Lever at 100M) · used in 2 chapters
tied vs untied embeddings
Whether the input token embedding matrix and output (LM head) unembedding matrix share the same weights (tied, saves V*d parameters) or are learned separately (untied, more expressive).
Tiered adapter storage
A GPU to CPU DRAM to SSD/object-store residency hierarchy for adapters, with only active adapters kept GPU-resident and async prefetch loading the rest.
TIES-Merging (Trim, Elect Sign, Disjoint Merge)
Merge algorithm that trims small-magnitude task-vector entries, resolves sign conflicts by majority vote, and averages only agreeing values to reduce interference.
tiled matrix multiplication
A matmul strategy where each thread block cooperatively loads T×T sub-tiles of A and B into shared memory and reuses them, cutting global memory reads by a factor of T versus the naive kernel.
tiling (matmul)
Splitting Q, K, V into blocks (B_r query rows x B_c key/value rows) small enough to fit in SRAM, so each attention subproblem is computed on-chip without touching HBM for intermediates.
Defined in 4.2 FlashAttention I: IO-Awareness & The Online Softmax · used in 2 chapters
Time per output token (TPOT)
Per-token latency during autoregressive decode, bottlenecked by memory bandwidth; determines streaming smoothness.
Defined in 12.2 Observability, Logging & LLMOps · used in 2 chapters
Time to First Token (TTFT)
Latency from request to the first streamed output token, dominated by prefill compute and queue wait time.
Defined in 12.2 Observability, Logging & LLMOps
time-to-first-token (TTFT)
The latency from a request's arrival until its first generated token is produced; bounded below by the remaining decode length of the currently running batch under static scheduling.
Defined in 7.2 Continuous Batching & Request Scheduling · used in 3 chapters
tl.atomic_add
A Triton primitive performing a race-safe read-modify-write, needed when multiple programs accumulate into the same memory address, e.g. scattering dQ contributions in FlashAttention backward.
Defined in 4.4 Writing GPU Kernels with Triton
TOFU and MUSE benchmarks
Adversarial unlearning benchmarks (synthetic biographies; realistic corpora) that score forget quality and utility against a retrained-from-scratch gold-standard reference.
Defined in 13.2 Knowledge Editing & Machine Unlearning
token budgeting
Partitioning the effective context window into per-segment token allocations (system, tools, retrieved, history, scratchpad) enforced with a caps-and-eviction policy.
Defined in 8.4 Context Engineering & Management
token merging / average pooling (visual token compression)
Techniques that reduce visual token count by merging or pooling adjacent low-information patches, trading spatial fidelity for context/memory efficiency.
Defined in 10.2 Vision-Language Models
token packing (shard)
Concatenating tokenized documents with EOS separators into fixed-length (context_len + 1) rows, written as memory-mappable int32 binary shard files.
token-level FSM index
A pre-computed mapping from FSM state to the set of vocabulary tokens valid in that state, avoiding per-step character simulation; introduced by Outlines.
Defined in 7.10 Structured & Constrained Generation
tokenizer
The frozen, non-gradient preprocessing component that maps raw text to a fixed vocabulary of integer IDs before any model computation runs.
tokens per parameter (tok/param)
The ratio D/N used to describe a training run's data allocation relative to model size; compute-optimal is ~20 tok/param here, the plan's over-trained choice is ~200.
tokens-per-parameter ratio ("20x rule")
Chinchilla heuristic that compute-optimal training uses roughly 20 training tokens per model parameter (D_opt ≈ 20 N_opt).
Tool (MCP primitive)
A model-controlled callable function that a server executes on request; the LLM decides when to invoke it and what JSON-Schema-validated arguments to pass.
Defined in 8.6 The Model Context Protocol (MCP)
tool call
A structured assistant message invoking an external function, encoded in the chat template (e.g. via a tool role or <tool_call> tokens) and partially masked during training.
tool schema (function schema)
A JSON Schema description of a tool's name, description, and parameters injected into the model's context so it knows what it can call.
Defined in 8.1 Tool Use & Function Calling
tool special tokens (<|tool_call|> / <|tool_result|>)
Reserved tokenizer tokens delimiting a model-emitted JSON tool call and the environment-returned observation in the trace wire format.
tool-call (function-calling) grammar
A JSON Schema built at request time from available tool definitions, activated after a trigger token so the model's tool call matches a registered name and argument schema exactly.
Defined in 7.10 Structured & Constrained Generation
tool-call loop (agentic loop)
The orchestration cycle that repeatedly calls the model, executes any requested tool calls, appends results, and repeats until a final text answer emerges.
Defined in 8.1 Tool Use & Function Calling
tool_call_id
A unique identifier attached to each model-issued tool call that must be echoed back in the corresponding tool result message to correlate calls with results.
Defined in 8.1 Tool Use & Function Calling
Top-k routing
Selecting the k experts with the largest router logits per token; the discrete, non-differentiable step at the core of sparse MoE.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
top-k sampling
A decoding filter that keeps only the k highest-probability tokens and renormalizes before sampling, truncating the long low-probability tail.
Defined in 2.7 Building a GPT From Scratch (nanoGPT-style) · used in 2 chapters
top-p (nucleus) sampling
A decoding filter that keeps the smallest set of highest-probability tokens whose cumulative probability exceeds p, adapting the cutoff to local distribution entropy.
Defined in 2.7 Building a GPT From Scratch (nanoGPT-style) · used in 2 chapters
TOPLOC
Prime Intellect's locality-sensitive hashing scheme that lets an untrusted worker commit to top-k activation features so a trusted verifier can cheaply confirm, via a single batched prefill, that a claimed completion was really generated by the stated model.
torch.compile
PyTorch 2.0's JIT compilation pipeline that layers TorchDynamo, AOTAutograd, and TorchInductor to transform eager Python code into fused, optimized kernels.
Defined in 4.9 Kernel Fusion, torch.compile, CUDA Graphs & Compilers · used in 2 chapters
torch.no_grad / inference_mode
Context managers that suppress graph construction; inference_mode is a stricter, faster variant that marks outputs as unusable in any future gradient computation.
TorchDynamo
The graph-capture layer of torch.compile that installs a CPython frame-evaluation hook to speculatively trace Python bytecode into an FX graph.
TorchInductor
torch.compile's default lowering backend, which takes the fused FX graph and generates Triton code (GPU) or C++ (CPU) via loop, pointwise, and epilogue fusion.
Total parameters vs active parameters
Total counts every expert's weights (model capacity); active counts only the experts one token actually uses (drives FLOPs/latency); their ratio is the sparsity dial.
Defined in 2.9 Mixture-of-Experts (MoE) Architectures
TPOT (time per output token)
Average time between successive output tokens during decode; dominated by memory-bandwidth-bound decode speed.
TPOT (time per output token) / ITL (inter-token latency)
Mean time between successive streamed output tokens; dominated by decode step time and batch size.
Defined in 12.1 Designing an LLM Serving System
TPR at low FPR (true-positive rate at low false-positive rate)
The correct way to score an MIA: measure how many members are caught while flagging very few non-members, since average accuracy hides worst-case leakage in the tail.
TPU (Tensor Processing Unit)
Google's systolic-array accelerator family (v1-v7) that trains and serves frontier models like Gemini via the JAX/XLA compiler stack.
Trace (and span)
A trace is the causal chain of one request (e.g. a RAG call); each stage within it is a span recording start time, duration, input, and output.
Defined in 12.2 Observability, Logging & LLMOps
trace-attached postmortem
An incident postmortem that anchors every claim to specific trace IDs (prompt hash, retrieval scores, judge output) rather than relying on logs alone.
train/inference numerical mismatch
The discrepancy between log-probs computed by vLLM's generation kernels versus the trainer's forward pass, requiring old_log_prob to be recomputed under the training engine.
TrainConfig
A single dataclass fixing every run-mechanics number (batch sizes, optimizer hyperparameters, logging/checkpoint cadence) for a pretraining run, itself checkpointed to prevent config drift on resume.
training checkpoint
Saved state enabling resume: model parameters, optimizer state (moments), per-rank RNG state, and training metadata (step, scheduler, data cursor).
Training-data extraction attack
Attack that generates candidate completions, ranks them by a membership signal (e.g. zlib or reference-model ratio), then verifies against the training corpus to recover verbatim data.
Trainium
AWS's custom systolic-array training/inference chip, programmed through the Neuron SDK and PyTorch/XLA, used at scale by Anthropic and AWS customers.
trajectory
Sequence of interleaved model actions, environment observations, and a terminal reward forming one agentic RL episode; the unit of training data.
Defined in 6.10 Agentic & Multi-Turn RL
trajectory distillation
Training a small model to imitate a teacher's full text trajectories (thoughts, calls, observations) via ordinary cross-entropy SFT, not logit matching.
trajectory scoring
Evaluation approach that grades the quality of intermediate agent steps (step accuracy, subtask completion, process reward modeling, efficiency) rather than only the final outcome.
Defined in 8.8 Agent Evaluation & Benchmarks
trajectory-level credit assignment
Broadcasting one scalar advantage (e.g., a GRPO group-relative advantage from the terminal reward) onto every action token of a trajectory.
Defined in 6.10 Agentic & Multi-Turn RL
trajectory-level replay (staleness buffer)
Storing actual past completions and log-probs for reuse over a few extra gradient steps, requiring PPO-style importance-ratio correction and clipping because the generating policy is stale.
Defined in 6.12 RL Data, Curriculum & Replay Management
Transfusion
Meta's hybrid unified model combining autoregressive cross-entropy loss for text with a flow-matching diffusion objective for continuous image patches, in one transformer.
Defined in 10.5 Unified & Any-to-Any Models
transpose rule for backpropagation
The chain-rule identities ∂L/∂W = X^T G and ∂L/∂X = G W^T for a linear layer Y = XW, showing the backward pass multiplies by the transposed matrix.
Defined in 1.1 Linear Algebra for Deep Learning
tree attention
An attention-masking scheme that flattens a tree of candidate continuations into one sequence, letting each node attend only to its ancestors so all branches are verified in a single forward pass.
Tree of Thoughts (ToT)
Search method that treats reasoning as a tree of partial-solution nodes scored by a value function, explored via BFS, DFS, or beam search.
Defined in 5.10 Reasoning, Chain-of-Thought & Test-Time Compute · used in 2 chapters
Tree-of-Thought
Search-based agent architecture (Yao et al. 2023) that expands k candidate actions per node, scores them with a value function, and explores via beam/DFS/BFS search rather than a single linear path.
Triton
A Python-embedded language and compiler for writing GPU kernels at block granularity, compiling to PTX near hand-tuned CUDA speed.
Defined in 4.4 Writing GPU Kernels with Triton
TRL (Transformer Reinforcement Learning)
HuggingFace's open-source library that packages each alignment pipeline stage as a standalone Trainer subclass built on transformers and accelerate.
Defined in 6.3 TRL: HuggingFace's RL Library · used in 2 chapters
truncated importance sampling (TIS)
Capping the behavior-side importance ratio at a constant C to control variance caused by numerical mismatch between the inference engine's and training engine's log-probs, not just staleness.
trusted monitoring
A weaker, trusted model scores an untrusted model's actions for suspicion so a scarce human-audit budget is spent on the most suspicious ones.
TTFT (time to first token)
Latency from request arrival to the first streamed output token; dominated by prefill compute and queueing delay.
Defined in 7.1 The Anatomy of LLM Inference: Prefill, Decode & The KV Cache · used in 3 chapters
turn-level (process) reward
Reward or return-to-go computed per turn rather than only at the end, giving finer-grained credit than trajectory-level broadcast.
Defined in 6.10 Agentic & Multi-Turn RL
TVM
An end-to-end ML compiler that uses search-based optimization (AutoTVM/Ansor) to generate portable, near-optimal kernels across diverse hardware backends.
typical sampling
Keeping tokens whose surprisal (-log P(t)) is close to the conditional entropy of the distribution, discarding both the most and least probable tokens.
U
U-Net denoiser / DiT (Diffusion Transformer)
The neural network predicting noise at each step; U-Net uses conv down/up-sampling with residual and cross-attention blocks, while DiT replaces it with a ViT operating on latent patches.
uint16 memmap sharding
Storing packed token ids and position ids as 2-byte uint16 arrays in flat binary shard files, memory-mapped at train time so no shard is fully loaded into RAM.
uint16 token shard format
The on-disk contract: tokenized corpora are written as raw uint16 integers in fixed-size .bin shards, which the training loader memory-maps directly; writer and loader dtypes must match exactly.
UL2 (span corruption denoising mixture)
A pretraining scheme mixing R-denoising (short masked spans), X-denoising (long masked spans), and S-denoising (causal prefix LM) in one model to gain versatility.
Defined in 3.3 The Pretraining Objective & Loss
uncertainty sampling
Active-learning strategy that prioritizes examples with high per-token entropy (low average logprob) as the most informative to label.
Defined in 12.5 Data Flywheels & Continuous Improvement
unembedding matrix
Weight matrix W_U of shape [V, d] that projects final hidden states back to vocabulary-sized logits before softmax.
Defined in 2.2 Embeddings & The Input Pipeline
Unified Paging
S-LoRA's design of storing adapter weight tensors in the same paged GPU DRAM pool used for the KV cache, letting memory shift fluidly between requests and resident adapters.
Unigram (SentencePiece)
A top-down subword algorithm that prunes a large seed vocabulary via EM to maximize corpus likelihood under a unigram language model, encoding via Viterbi and marking spaces with '▁'.
Universal approximation theorem
Result (Cybenko, Hornik) that a single sufficiently wide hidden layer can approximate any continuous function on a compact set to arbitrary accuracy.
uplift
The degree to which a model provides meaningful assistance toward a catastrophic harm beyond what a non-expert could achieve via web search or textbooks.
upsampling
Deliberately raising a domain's effective epochs by sampling it more often than its natural proportion, trading repetition risk for more gradient signal.
Upstream gradient (adjoint)
The running quantity u-bar = dL/du carried backward through the computation graph, representing how much the loss changes with tensor u.
Uptraining
Converting a pretrained MHA checkpoint into a GQA model by mean-pooling each group's key/value projection weights, then fine-tuning with a small fraction of pretraining compute.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
utilization (ρ)
Fraction of service capacity in use, ρ = λ/(cμ); as ρ→1, mean wait time grows like 1/(1-ρ), causing a queueing 'cliff'.
Defined in 12.1 Designing an LLM Serving System
V
value (V)
A learned linear projection (v_i = x_i W_V) holding the payload content returned to a query once that token is attended to.
Defined in 2.3 The Attention Mechanism From Scratch
Value function
V^π(s_t), the expected future return from state s_t under the current policy; in RLHF, the critic's estimate of the reward-model score a partial response will earn.
Vanishing/exploding gradients
Geometric shrinkage or growth of gradients across many layers when each layer's typical Jacobian singular value is below or above 1.
VAPO (Value-model-based Augmented PPO)
A critic-based RL recipe using length-adaptive GAE and value-model warmup to make a learned value function outperform critic-free group baselines on long chain-of-thought.
variance decomposition (generalizability theory)
Partitioning an eval score's total variance into components from items, prompt templates, generation seeds, and judges, to identify which noise source dominates and should be reduced.
vector database
A production system (e.g., Milvus, Qdrant, Weaviate, pgvector) that wraps an ANN algorithm with persistence, sharding, replication, and metadata filtering.
vector store
A memory substrate that embeds each entry as a dense vector and retrieves by cosine similarity; strong at fuzzy matching, weak at exact-match and structured queries.
Defined in 8.5 Memory Systems for Agents
Vector-Jacobian product (VJP)
The rule at each computation-graph node converting an output's upstream gradient into gradients for its inputs, without ever forming the full Jacobian.
Defined in 1.6 Neural Networks From Scratch: MLPs & Backprop · used in 2 chapters
VeRA (Vector-based Random Matrix Adaptation)
Extreme PEFT variant that freezes a single shared pair of random $A,B$ matrices across all layers and trains only two small per-layer scaling vectors, shrinking adapter size 10-100x versus LoRA.
Verbosity bias (length bias)
The tendency of LLM judges to prefer longer responses independent of actual quality, inflating scores for verbose but not-better answers.
Defined in 11.2 LLM-as-a-Judge & Automated Evaluation
Verifiable rewards
Rewards computed from checkable ground truth (e.g., unit-test pass/fail, symbolic verification) that cannot be gamed by formatting or reward-model blind spots.
verification asymmetry
The principle that verifying or restyling text is easier than generating it from scratch, which is why an imperfect generator can still produce a high-precision synthetic dataset.
verification-in-the-loop
Wiring a ground-truth check (tests, build, linter) into the agent loop so the agent cannot declare success until the check passes, rather than trusting its own self-assessment.
verifier
A programmatic checker that decides correctness of a model output (exact-match, code-execution, or constraint check) without needing to be learned.
ViDoRe (Visual Document Retrieval Benchmark)
The standard benchmark for page-image retrieval, pairing queries with page images across tables, figures, and multi-domain documents, measured with nDCG@k.
Virtual pipeline parallelism (interleaved schedule)
Megatron scheme assigning each GPU multiple non-contiguous chunks of layers (v chunks), cutting the pipeline bubble by roughly a factor of v at the cost of more inter-stage messages.
Vision Transformer (ViT)
Architecture that splits an image into fixed-size patches, linearly embeds each as a token, and feeds the sequence into an unmodified Transformer encoder.
Defined in 10.1 Vision Transformers & Image Encoders
visual grounding
A pretraining task (used in Qwen-VL) where the model predicts a bounding box for a text or object region, forcing it to localize before reading/describing.
Defined in 10.2 Vision-Language Models
visual token explosion
The problem that the number of tokens an image consumes scales as (H/patch size)^2, quickly saturating LLM context and KV-cache memory.
Defined in 10.2 Vision-Language Models
vLLM
A high-throughput LLM inference server built on PagedAttention that exposes an OpenAI-compatible API and supports tensor parallelism and prefix caching.
Defined in Tooling & Environment Setup Cheatsheet
vLLM Automatic Prefix Caching (APC)
vLLM's prefix-cache implementation layered on the PagedAttention block allocator: a flat hash table mapping block hashes to physical GPU blocks, with LRU eviction; on by default in the V1 engine.
Defined in 7.7 Prefix Caching & KV-Cache Reuse
vLLM colocated generation
TRL's optional backend that offloads rollout generation to a vLLM server and overlaps it with training-GPU gradient computation to relieve the generate-then-train bottleneck.
Defined in 6.3 TRL: HuggingFace's RL Library
vocab padding (to a multiple of 64)
Rounding GPT-2's 50257-token vocabulary up to 50304 so the embedding/lm_head matmul and softmax align to GPU tensor-core tile sizes, at zero cost to model quality.
vocabulary size (vocab_size)
The number of entries a tokenizer's vocabulary contains; trades off sequence-length compression against embedding/output matrix size and undertrained rare tokens.
Defined in 2.1 Tokenization: BPE, WordPiece, Unigram & Byte-Level · used in 2 chapters
vocabulary/tokenizer transfer
Extending or swapping a tokenizer and initializing new token embeddings, typically as the mean of their old sub-token embeddings, while preserving shared tokens' vectors.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
VQ-GAN
VQ-VAE variant adding an adversarial loss for sharper reconstructions; became the standard image tokenizer for early autoregressive image-generation transformers.
Defined in 10.5 Unified & Any-to-Any Models
VQ-VAE (vector-quantised variational autoencoder)
Encoder-codebook-decoder model that replaces each spatial feature with its nearest codebook vector, turning an image into a sequence of discrete integer tokens.
Defined in 10.5 Unified & Any-to-Any Models
W
W4A16 / W8A8
Shorthand for quantization configurations: 4-bit weights with 16-bit activations (GPTQ/AWQ, memory-bound decoding) versus 8-bit weights and activations (SmoothQuant, compute-bound throughput serving).
Wanda
A pruning criterion scoring weights by |weight| times input-activation L2 norm, needing no Hessian inversion yet matching SparseGPT-level quality.
WARC (Web ARChive)
Format holding full HTTP responses (headers plus raw HTML) from a crawl; modern pipelines re-extract text from it instead of using WET.
warmup
Linearly ramping the learning rate from near-zero to its target value over the first steps so Adam's zero-initialized moment estimates can calibrate before full-size updates hit.
Warmup-Stable-Decay (WSD)
A three-phase schedule (warmup, constant peak LR, then decay) that decouples total training length from the schedule shape, popularized by MiniCPM.
Warmup-Stable-Decay (WSD) schedule
A learning-rate schedule with a long constant-LR stable phase followed by a short decay phase, enabling cheap branched annealing experiments from one checkpoint.
Defined in 3.14 Data Mixing, Domain Weighting & Curriculum · used in 4 chapters
warp
A group of exactly 32 threads that execute the same instruction in lockstep (SIMT); the real unit of GPU execution.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 2 chapters
warp divergence
Performance penalty when threads within a warp take different branch paths, forcing the hardware to serially execute both paths while masking inactive lanes.
Defined in 1.8 GPU Architecture & The Memory Hierarchy · used in 2 chapters
warp shuffle intrinsics
CUDA primitives (`__shfl_sync`, `__shfl_down_sync`, `__shfl_xor_sync`) that let threads within a warp exchange register values directly, without shared memory or `__syncthreads()`.
warp specialization
Assigning different warps distinct roles (data-movement producer vs. compute consumer) instead of identical work, so the roles can run concurrently as a pipeline.
Watermark z-score
Detection statistic $(g-\gamma T)/\sqrt{T\gamma(1-\gamma)}$ comparing observed green-token count $g$ to its expectation under a human-text (Binomial) null; large z rejects the null.
wavefront
AMD's SIMT lockstep execution group of 64 threads on Instinct GPUs, the analog of NVIDIA's 32-thread warp; a common source of porting bugs when hard-coded.
weak-to-strong generalization (W2SG)
The finding that a strong student model fine-tuned on labels from a weaker supervisor exceeds that supervisor's performance, studied as a proxy for scalable oversight of superhuman models.
Defined in 5.11 Constitutional AI, RLAIF & Self-Improvement · used in 2 chapters
WebArena
Benchmark that evaluates agents on realistic web tasks (booking, shopping, forum management) in sandboxed copies of real web apps, scored by verifying final application state, not clicks.
Defined in 8.8 Agent Evaluation & Benchmarks
Weight absorption
MLA optimization folding the fixed, position-independent up-projection matrices (W^UK, W^UV) into the query and output projections so attention runs directly on the cached latent without materializing full K/V.
Defined in 2.4 Multi-Head Attention, MQA, GQA & MLA
Weight synchronization
Propagating updated policy parameters from the training layout (e.g., FSDP shards) into the inference engine's layout (e.g., tensor-parallel) every step.
Defined in 6.1 The Anatomy of an RL-for-LLM System · used in 4 chapters
weight tying (tied embeddings)
Sharing a single matrix between the input embedding and output unembedding (W_U = W_E), halving vocabulary-dependent parameters and regularizing training.
Defined in 2.2 Embeddings & The Input Pipeline · used in 2 chapters
Weight+activation quantization (W+A)
Quantizing both weights and activations (usually to INT8) so the entire GEMM runs in low-precision integer arithmetic for higher throughput.
weight-decay split
Partitioning optimizer parameters into a decayed group (2D+ matmul/embedding weights) and a non-decayed group (1D biases and norm gains) when configuring AdamW.
Weight-only quantization (W-only)
Storing weights in low precision while activations remain FP16/BF16; the kernel dequantizes weights on-the-fly before running the FP16 GEMM.
Weights & Biases (W&B / wandb)
An experiment-tracking tool for logging training metrics, qualitative sample tables, and versioned artifacts (datasets, checkpoints) tied to each run.
Defined in Tooling & Environment Setup Cheatsheet
WET (WARC Encapsulated Text)
Common Crawl's own built-in plain-text extraction of a crawl; convenient but keeps boilerplate that hurts quality versus re-extraction from WARC.
WGMMA (warpgroup matrix multiply-accumulate)
Hopper's asynchronous matmul instruction, issued by a warpgroup of 4 warps, that lets the tensor cores compute while the issuing warps continue other work, unlike Ampere's synchronous mma.
Whisper
An encoder-decoder ASR model that encodes a fixed 30-second window of log-Mel features into 1500 continuous hidden states and autoregressively decodes text transcripts.
Defined in 10.3 Audio, Speech & Multimodal Fusion
Wide expert parallelism (wide-EP)
Extends expert parallelism across multiple nodes over InfiniBand rather than confining it to one NVLink node, enabling much larger EP degrees for huge MoE models.
Defined in 7.11 Multi-GPU & Multi-Node Inference
Wilson score interval
A binomial-proportion confidence interval derived by inverting the score test; stays within [0,1] and stays well-behaved when accuracy is near 0 or 1, unlike the Wald interval.
win-rate judge
An LLM judge that compares candidate vs. production outputs on held-out prompts to compute a win rate, the primary quality gate metric for deployment.
Defined in 12.5 Data Flywheels & Continuous Improvement
WISE
An editor that adds a trainable side-memory FFN alongside the frozen main FFN, routing per-token between them to balance reliability, generalization, and locality.
Defined in 13.2 Knowledge Editing & Machine Unlearning
WMDP (Weapons of Mass Destruction Proxy)
A multiple-choice benchmark measuring memorized hazardous knowledge in biosecurity, cybersecurity, and chemical domains without revealing it publicly.
WordPiece
BERT's subword algorithm that merges the pair maximizing count(ab)/(count(a)*count(b)), preferring pieces that are individually rare but co-occur; continuations marked with '##'.
work partitioning
Deciding which thread, warp, or thread-block computes which piece of the attention tile; FA2's central lever for raising GPU compute utilization without changing memory traffic.
WorkerGroup
A veRL abstraction binding a Worker class to a ResourcePool, spinning up one Ray actor per GPU rank and initializing torch.distributed for SPMD execution.
workflow vs. agent distinction
A workflow is a DAG of LLM calls with deterministic, developer-written routing; an agent lets the model itself decide the next action, tool, and input dynamically.
Defined in 8.7 Multi-Agent Systems & Orchestration
WRAP (Web Rephrasing Augmented Pretraining)
Rewriting web documents into clean styles (Wikipedia-like, Q&A, concise) with an LLM while keeping the originals, reaching target loss with ~3x fewer tokens.
write / read / reflect triad
The three memory operations every memory-capable agent implements: committing information to storage, retrieving it into context, and distilling episodes into facts.
Defined in 8.5 Memory Systems for Agents
WSD (Warmup-Stable-Decay) / infinite LR schedule
A schedule that holds a constant LR across repeated CPT rounds and only decays briefly to produce a deployable checkpoint, resuming from pre-decay weights for the next round.
Defined in 3.16 Continual & Domain-Adaptive Pretraining
WSD (Warmup-Stable-Decay) schedule
A learning-rate schedule with a long constant-LR 'stable' phase followed by a decay phase, decoupling total training length from when annealing/decay happens.
X
Xavier/Glorot initialization
Weight initialization setting Var(W) = 2/(d_in+d_out) to balance signal variance across forward and backward passes, for symmetric activations like tanh.
XGrammar
A production structured-generation engine that splits tokens into context-independent/dependent classes and overlaps mask computation with the GPU forward pass for near-zero overhead.
Defined in 7.10 Structured & Constrained Generation
XLA (Accelerated Linear Algebra)
The ahead-of-time compiler that lowers JAX/PyTorch array-level programs onto TPU or Trainium hardware, handling fusion, tiling, and sharding-implied collectives.
Defined in 1.10 The Accelerator Landscape: TPUs, Trainium, AMD/ROCm & Gaudi · used in 2 chapters
XSTest
A benchmark of safe prompts using harm-adjacent words in clearly benign contexts, designed specifically to surface exaggerated safety refusals.
Y
YaRN (Yet another RoPE extensioN)
Production context-extension method combining a per-dimension NTK-by-parts frequency ramp with attention-logit temperature scaling to sharpen softmax after extension.
Defined in 2.5 Positional Encodings: Sinusoidal, Learned, RoPE & ALiBi · used in 3 chapters
Z
z-loss
A regularizer, alpha*log^2(sum_v e^{z_v}), added to cross-entropy to penalize large pre-softmax logit norms and prevent softmax overflow and training loss spikes.
Defined in 3.3 The Pretraining Objective & Loss · used in 4 chapters
ZeRO (Zero Redundancy Optimizer)
A family of techniques (DeepSpeed) that shard the redundant per-GPU copies of optimizer states, gradients, and/or parameters across the data-parallel group instead of replicating them.
Defined in 3.5 Distributed Training I: Data Parallelism, DDP, ZeRO & FSDP · used in 3 chapters
ZeRO-3
The ZeRO stage that shards parameters themselves, so each GPU permanently stores only 1/N of the model and reconstructs each layer's full parameters via just-in-time all-gather.
Zero-gradient (vanishing-advantage) problem
In GRPO, when all G samples for a prompt receive the same reward, the group-relative advantage is zero for every sample, wasting that generation's compute.
ZeRO-Offload / ZeRO-Infinity
ZeRO variants that move optimizer states and parameters off the GPU to CPU RAM (Offload) or NVMe storage (Infinity) when host memory exceeds GPU memory.
Defined in 3.7 Megatron-LM, DeepSpeed & Parallelism in Practice · used in 2 chapters
zero-overhead (overlap) scheduler
SGLang's scheduler design that prepares the CPU-side batch for the next decode step while the GPU executes the current step's forward pass, removing scheduling bubbles.
zero-shot classification
Classifying an image by comparing its embedding to text-prompt embeddings (e.g., 'a photo of a cat') without any task-specific fine-tuning, enabled by CLIP/SigLIP.
Defined in 10.1 Vision Transformers & Image Encoders
zero-variance group
A rollout group whose G completions all get the same reward (all-correct or all-wrong), so GRPO's centered advantage is zero for every sample.
Defined in 6.12 RL Data, Curriculum & Replay Management
Μ
μP (Maximal Update Parametrization)
A parametrization scheme that lets a learning rate tuned on a small proxy model transfer unchanged to a much wider model via width-dependent scaling rules.
No terms match your filter.