Navigate the book
Learning map
Guided reading tracks for specific goals, a topic index, and the prerequisite map for every chapter — so you can chart a path instead of reading front to back.
Guided reading tracks
ML engineers building a pretraining stack
Train an LLM From Scratch
For engineers who want to go from raw text to a trained, sampling checkpoint. You'll build every piece of the pipeline: tokenizer, transformer, pretraining objective, scaling plan, and the end-to-end training recipe.
- 1.6 Neural Networks From Scratch: MLPs & Backprop
- 1.7 Automatic Differentiation & PyTorch Internals
- 2.1 Tokenization: BPE, WordPiece, Unigram & Byte-Level
- 2.3 The Attention Mechanism From Scratch
- 2.4 Multi-Head Attention, MQA, GQA & MLA
- 2.5 Positional Encodings: Sinusoidal, Learned, RoPE & ALiBi
- 2.6 The Transformer Block: Norms, Residuals, MLPs & Activations
- 2.7 Building a GPT From Scratch (nanoGPT-style)
- 3.1 Pretraining Data: Sources, Crawling & The Data Pipeline
- 3.3 The Pretraining Objective & Loss
- 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond
- 3.9 Optimizers: SGD, Adam, Adafactor, Lion, Muon & Shampoo
- 3.17 Training an LLM From Scratch: The End-to-End Recipe
- 14.1 The Capstone: Building Stack-100M, and the 2026 Small-Model Landscape
Inference/serving engineers
Inference & Serving Optimization
For engineers serving LLMs in production who need low latency and high throughput. You'll master inference anatomy, continuous batching, KV-cache management, quantization, and the modern serving stacks.
- 2.4 Multi-Head Attention, MQA, GQA & MLA
- 7.1 The Anatomy of LLM Inference: Prefill, Decode & The KV Cache
- 7.2 Continuous Batching & Request Scheduling
- 4.6 PagedAttention & KV-Cache Memory Management
- 7.3 vLLM: Architecture, PagedAttention & Internals
- 7.7 Prefix Caching & KV-Cache Reuse
- 7.4 SGLang: RadixAttention & Structured Programs
- 7.6 Speculative Decoding: Draft Models, Medusa, EAGLE & Lookahead
- 4.7 Quantization I: Post-Training Quantization (GPTQ, AWQ, SmoothQuant)
- 4.8 Quantization II: INT4/INT8/FP8, GGUF, bitsandbytes & QAT
- 7.9 Sampling Strategies & Decoding Algorithms
- 7.5 TensorRT-LLM, TGI & Other Serving Stacks
- 7.12 Inference Economics: Latency, Throughput & Cost
- 12.1 Designing an LLM Serving System
Alignment & post-training engineers
Post-Training & Alignment
For engineers turning a base model into a helpful, aligned assistant. You'll work through SFT, reward modeling, PPO/DPO/GRPO, RL with verifiable rewards, the failure modes, and how to evaluate the result.
- 5.1 Supervised Fine-Tuning & Instruction Tuning
- 5.2 Chat Templates, Data Formatting & Sequence Packing
- 5.3 PEFT I: LoRA, QLoRA, DoRA & The Adapter Family
- 5.5 The RLHF Pipeline & Reward Modeling
- 5.6 Policy Gradients & PPO for Language Models
- 5.7 Direct Preference Optimization & Its Variants
- 5.8 GRPO, RLOO & Critic-Free RL
- 5.9 RL with Verifiable Rewards (RLVR) & The Reasoning Recipe
- 6.1 The Anatomy of an RL-for-LLM System
- 5.13 Reward Hacking, Over-Optimization & Alignment Failures
- 11.1 The Evaluation Problem & Benchmark Landscape
- 11.2 LLM-as-a-Judge & Automated Evaluation
Agent & applied-LLM engineers
Agents & Tool Use
For engineers building LLM agents that call tools, retrieve knowledge, and coordinate. You'll cover tool use, agentic loops, harness and context engineering, memory, RAG, and multi-agent orchestration.
- 8.1 Tool Use & Function Calling
- 8.2 The Agentic Loop: ReAct, Plan-Execute & Reflection
- 8.3 Harness Engineering: Building a Coding Agent
- 8.4 Context Engineering & Management
- 8.5 Memory Systems for Agents
- 8.6 The Model Context Protocol (MCP)
- 9.1 Embeddings & Representation Learning
- 9.2 Vector Databases & Approximate Nearest Neighbor Search
- 9.3 Retrieval-Augmented Generation Architectures
- 9.4 Chunking, Reranking & Hybrid Search
- 9.5 Advanced RAG: GraphRAG, Agentic RAG & Long-Context vs RAG
- 8.7 Multi-Agent Systems & Orchestration
- 8.8 Agent Evaluation & Benchmarks
GPU/performance & systems engineers
GPU Systems & Performance
For performance engineers who want to make LLMs go fast on real hardware. You'll cover GPU architecture, the roofline model, CUDA and Triton kernels, FlashAttention, compilers, and distributed training.
- 1.8 GPU Architecture & The Memory Hierarchy
- 4.1 The Roofline Model & Performance Engineering
- 4.5 CUDA Programming Essentials for ML Engineers
- 4.4 Writing GPU Kernels with Triton
- 4.2 FlashAttention I: IO-Awareness & The Online Softmax
- 4.3 FlashAttention 2 & 3: Work Partitioning, Warp Specialization & FP8
- 4.9 Kernel Fusion, torch.compile, CUDA Graphs & Compilers
- 4.10 Memory-Efficient Training: Checkpointing, Offloading & LoRA Math
- 1.9 Parallel Computing & Collective Communication
- 3.5 Distributed Training I: Data Parallelism, DDP, ZeRO & FSDP
- 3.6 Distributed Training II: Tensor, Pipeline, Sequence & Expert Parallelism
- 3.7 Megatron-LM, DeepSpeed & Parallelism in Practice
ML/LLM engineer interview candidates
The ML Interview Fast Path
For candidates prepping for an ML/LLM engineer interview. A high-yield sweep of the concepts most likely to come up: fundamentals, the transformer, pretraining and scaling, alignment, inference, kernels, RAG, agents, and evaluation.
- 1.5 Machine Learning Fundamentals
- 1.2 Probability, Statistics & Information Theory
- 2.3 The Attention Mechanism From Scratch
- 2.4 Multi-Head Attention, MQA, GQA & MLA
- 2.6 The Transformer Block: Norms, Residuals, MLPs & Activations
- 2.7 Building a GPT From Scratch (nanoGPT-style)
- 3.3 The Pretraining Objective & Loss
- 3.4 Scaling Laws: Kaplan, Chinchilla & Beyond
- 5.1 Supervised Fine-Tuning & Instruction Tuning
- 5.7 Direct Preference Optimization & Its Variants
- 7.1 The Anatomy of LLM Inference: Prefill, Decode & The KV Cache
- 4.2 FlashAttention I: IO-Awareness & The Online Softmax
- 9.3 Retrieval-Augmented Generation Architectures
- 8.2 The Agentic Loop: ReAct, Plan-Execute & Reflection
- 11.2 LLM-as-a-Judge & Automated Evaluation
Browse by topic
Prerequisite map
Each chapter with what it lets you do and the concepts to know first (linked to where they are taught).
Part I — Mathematical & Systems Foundations
Compute and reason with matmuls, SVD, norms, and the backprop transpose identities behind LoRA.
Prereqs: basic vector and matrix notation derivatives and the chain rule Python and PyTorch tensor basics big-O time-complexity notation
Compute entropy, KL divergence, cross-entropy, and perplexity, and justify cross-entropy as the LLM training loss.
Prereqs: logarithms and exponentials basic calculus (derivatives, integrals) summation and integral notation the softmax function vectors and basic linear algebra
Analyze loss landscapes via gradients and Hessians, and choose/tune optimizers like SGD and Adam.
Prereqs: single-variable calculus (derivatives) linear algebra basics (vectors, matrices, eigenvalues) matrix multiplication partial derivatives the concept of a neural network loss function
Reason about floating-point formats and apply stable-arithmetic tricks to keep LLM training numerically sound.
Prereqs: binary representation of numbers basic calculus (exponentials and logarithms) the softmax function gradients and backpropagation basics matrix multiplication basic Python/PyTorch tensor operations
Diagnose bias-variance tradeoffs, apply regularization, and choose correct metrics for any ML model.
Prereqs: gradient descent basic probability and expectation vector norms (L1/L2) loss functions and cross-entropy basic Python/PyTorch usage
Build and hand-verify backprop for an MLP, from scalar autograd to a vectorized NumPy classifier.
Prereqs: matrix multiplication and transposes partial derivatives and the chain rule gradient descent and convex/non-convex optimization basics vectors and linear maps (linear algebra) softmax and probability basics
Trace PyTorch's autograd tape, write custom Function backwards, and debug memory/precision bugs from strides to NaNs.
Prereqs: chain rule Jacobian matrix MLP forward and backward pass (backpropagation) ReLU activation function gradient descent basics
Classify any GPU kernel as compute- or memory-bound and reason about occupancy and hardware fit.
Prereqs: matrix multiplication basic linear algebra floating-point number formats (FP16/BF16/FP32) Python programming basic notion of parallel computing
Reason about GPU collective communication costs and choose the right collective for a parallelism strategy.
Prereqs: gradient descent and backpropagation GPU memory hierarchy (HBM) and CUDA basics Python and PyTorch tensor operations basic computer networking (bandwidth vs. latency) processes vs. threads / operating-system basics
Choose and port LLM workloads across TPU, Trainium, AMD ROCm, and Gaudi accelerators using roofline reasoning.
Prereqs: SIMT GPU architecture and the memory hierarchy floating-point formats (bf16, FP8/FP4) collective communication (all-reduce, all-gather, NVLink) arithmetic intensity and the roofline model the KV cache in LLM inference tensor-parallel sharding
Part II — The Transformer Architecture
Implement and choose a subword tokenizer (BPE, WordPiece, Unigram, byte-level) and reason about its cost/arithmetic tradeoffs.
Prereqs: basic Python programming UTF-8 and character encoding basics regular expressions the idea of an embedding table big-O complexity basic probability (likelihood, EM at an intuitive level)
Implement and size the token embedding, weight-tied unembedding, and input pipeline for a Transformer LM.
Prereqs: matrix multiplication softmax tokenization and token IDs backpropagation and gradients basic PyTorch nn.Module usage
Derive, implement, and mask scaled dot-product attention from scratch in NumPy and PyTorch.
Prereqs: matrix multiplication and transposes dot product as a similarity measure the softmax function backpropagation / automatic differentiation basic NumPy and PyTorch tensor operations
Derive KV-cache size for MHA/MQA/GQA/MLA and implement the MHA-to-GQA conversion in PyTorch.
Prereqs: single-head scaled dot-product attention softmax matrix multiplication and matrix reshaping autoregressive decoding GPU memory hierarchy and HBM bandwidth
Implement RoPE/ALiBi positional encodings and extend a model's context length via NTK/YaRN scaling.
Prereqs: self-attention / scaled dot-product attention softmax token embeddings KV cache basic trigonometry and rotation matrices multi-head attention
Assemble and implement a full pre-norm transformer block: norms, residuals, gated FFN, activations.
Prereqs: the attention mechanism multi-head attention matrix multiplication basic backpropagation / gradients sigmoid function
Implement, initialize, train, and sample from a complete decoder-only GPT in PyTorch from scratch.
Identify and implement encoder-only, encoder-decoder, decoder-only, and prefix-LM attention masks and their tradeoffs.
Prereqs: the attention mechanism multi-head attention the transformer block (norms, residuals, MLP) softmax KV cache basics next-token prediction / language modeling
Build a sparse MoE layer with routing, load balancing, and capacity, and reason about its systems cost.
Prereqs: the Transformer block (attention plus position-wise FFN) the FFN as two linear layers with a nonlinearity (GELU/SwiGLU) softmax gradient descent and backpropagation basic distributed/parallel computing concepts (optional, for the expert-parallelism preview)
Explain and implement the modern transformer recipe (RMSNorm, SwiGLU, RoPE, GQA, QK-norm) used in production LLMs.
Prereqs: the attention mechanism and scaled dot-product attention multi-head attention, MQA, GQA, MLA basics LayerNorm and residual connections positional encodings (sinusoidal, learned, RoPE basics) KV cache concept basic floating-point precision (bf16) considerations
Analyze and choose sub-quadratic attention alternatives (SSMs, Mamba, RWKV, RetNet, hybrids) for long-context LLMs.
Prereqs: the softmax attention mechanism (QK^T/sqrt(d) softmax V) KV cache recurrent neural networks (RNNs) Big-O complexity analysis for compute and memory FlashAttention / IO-aware kernels the transformer block (residuals, normalization, MLP)
Build and reason about masked-diffusion LLMs: derive the objective, implement a sampler, and design block diffusion.
Prereqs: autoregressive next-token prediction and the chain-rule factorization the attention mechanism and causal masking KV cache and memory-bandwidth-bound decode masked language modeling (BERT-style pretraining) softmax and cross-entropy loss basic diffusion models (forward/reverse noising process)
Part III — Pretraining at Scale
Build a pretraining data pipeline: source, extract, weight, and shard trillions of tokens.
Prereqs: BPE tokenization next-token prediction training objective basic HTTP/HTML concepts distributed/streaming systems basics (object storage, worker pools)
Build a production pipeline to filter, deduplicate, and decontaminate raw web text for pretraining.
Prereqs: tokenization web crawl data pipelines (CommonCrawl/WARC/WET) hash functions basic probability and set operations n-gram language models next-token prediction / language modeling objective
Compute, mask, and debug the causal-LM cross-entropy loss, including packing and tokenizer-agnostic BPB scoring.
Prereqs: the attention mechanism and causal masking softmax function chain rule of probability KL divergence / entropy tokenization and vocabulary (BPE)
Derive and apply scaling laws to compute-optimally plan a training run's model size and data.
Prereqs: cross-entropy loss for next-token prediction FLOP counting for matrix multiplications learning rate schedules (cosine decay and warmup) basic optimization (derivatives, Lagrange multipliers) GPU peak throughput and memory hierarchy basics pretraining data pipeline and tokenization
Implement DDP's bucketed gradient sync and choose/apply ZeRO-1/2/3 or FSDP sharding to fit large models.
Prereqs: collective communication primitives (all-reduce, reduce-scatter, all-gather) the alpha-beta communication cost model backpropagation and autograd mixed-precision (bf16/fp32) training the Adam/AdamW optimizer GPU memory basics
Partition huge transformer models via tensor, pipeline, sequence/context, and expert parallelism to train beyond single-GPU memory.
Prereqs: data parallelism and ZeRO/FSDP sharding GPU memory hierarchy and NVLink/interconnect bandwidth collective communication primitives (all-reduce, reduce-scatter, all-gather) the transformer block (attention and MLP sublayers) FlashAttention and the online-softmax trick Mixture-of-Experts architecture and routing
Configure Megatron-LM + DeepSpeed parallelism degrees for a multi-hundred-GPU run and measure hardware efficiency via MFU/HFU.
Prereqs: tensor parallelism pipeline parallelism data parallelism / DDP ZeRO optimizer state sharding mixed-precision (bf16/fp16) training NVLink vs InfiniBand networking bandwidth
Implement correct bf16/fp16 AMP training loops and scale FP8 GEMMs for production LLM training.
Prereqs: floating-point representation (exponent/mantissa, dynamic range vs precision) matrix multiplication and GEMMs backpropagation and gradient computation gradient-based optimizers (e.g. AdamW) GPU tensor cores / accelerator throughput basics
Pick, derive, and implement an optimizer (AdamW, Adafactor, Lion, Shampoo, Muon) for a memory/speed budget.
Prereqs: stochastic gradient descent and backpropagation the Hessian and condition number of a loss surface singular value decomposition (SVD) mixed-precision training (bf16/fp32 master weights) matrix multiplication and basic linear algebra
Choose, implement, and debug LR schedules, warmup, batch-size scaling, and muP hyperparameter transfer for pretraining.
Prereqs: the Adam optimizer and its moment estimates gradient descent and backpropagation weight initialization schemes hidden dimension / model width distributed data parallelism (DDP)
Diagnose, mitigate, and prevent loss spikes and divergence in large-scale LLM pretraining runs.
Prereqs: Adam/AdamW optimizer dynamics mixed-precision (bf16/fp16) training learning rate warmup and schedules the attention mechanism RMSNorm normalization distributed data parallelism (DDP, ZeRO, FSDP)
Design and implement fault-tolerant, resumable checkpointing for large-scale distributed LLM training.
Prereqs: Fully Sharded Data Parallel (FSDP) / ZeRO parameter sharding Adam optimizer and its moment estimates mixed-precision training (bf16/fp16) basic distributed training concepts (ranks, world size, process groups) PyTorch training loop fundamentals
Extend a pretrained model's context window via RoPE scaling, continued pretraining, and ring attention.
Design, weight, and schedule a pretraining data mixture using epoch budgets, DoReMi, and annealing.
Prereqs: cross-entropy loss data deduplication scaling laws for language models learning rate warmup and decay schedules power-law learning curves basic probability distributions (simplex, weighted average)
Design synthetic-data pipelines (rephrasing, textbook generation, reasoning distillation) that avoid model collapse.
Prereqs: Chinchilla scaling laws (compute-optimal token-to-parameter ratios) pretraining data sourcing and web crawling data deduplication and quality filtering supervised fine-tuning (SFT) basics chain-of-thought reasoning maximum-likelihood estimation and variance
Adapt a converged base model to new domains/languages/data via re-warm+replay CPT, model growth, and compute planning.
Prereqs: cosine learning rate schedules and warmup Adam optimizer and its moment estimates compute/scaling-law estimates (C ≈ 6ND) Mixture-of-Experts architecture and routing BPE tokenization embedding and unembedding matrices
Run a full pretrain pipeline end-to-end, raw corpus to sampled checkpoint, at four hardware scales.
Prereqs: GPT / transformer architecture BPE tokenization AdamW and gradient-based optimization cross-entropy loss distributed data-parallel training (DDP/FSDP) learning rate scheduling
Part IV — Kernels, Efficiency & Quantization
Classify any GPU kernel as compute- or memory-bound and pick the matching optimization.
Prereqs: GPU architecture and memory hierarchy (HBM, SM, L2) the transformer block (attention and FFN) matrix multiplication and its FLOP cost floating-point number formats (fp32, bf16, fp8) basic big-O style counting of operations
Derive, implement, and IO-analyze exact FlashAttention via the online softmax and tiling.
Prereqs: the attention mechanism (Q, K, V and softmax) matrix multiplication and its FLOP cost numerically-stable softmax (max-subtraction trick) GPU memory hierarchy (HBM vs SRAM) the roofline model / arithmetic intensity backpropagation and the chain rule
Analyze FlashAttention-2/3's work partitioning, warp specialization, and FP8 incoherent-processing tricks.
Prereqs: the online-softmax recurrence for attention FlashAttention-1's IO-aware tiling GPU architecture (SMs, warps, shared memory, register pressure) the roofline model and arithmetic intensity the attention mechanism and softmax matrix multiplication / GEMM basics
Write, tile, autotune, and debug fused GPU kernels (softmax, matmul, FlashAttention) in Triton.
Prereqs: GPU architecture and memory hierarchy (SMs, warps, shared memory, HBM vs SRAM) the roofline model (arithmetic intensity, memory- vs compute-bound) the online softmax / FlashAttention IO-awareness idea basic NumPy-style array broadcasting matrix multiplication and softmax mixed-precision (fp16/bf16 with fp32 accumulation)
Write, profile, and optimize custom CUDA kernels (tiled matmul, fused ops) for LLM workloads.
Prereqs: GPU memory hierarchy roofline model and arithmetic intensity matrix multiplication basic C/C++ programming PyTorch tensor operations
Compute KV-cache memory, diagnose fragmentation, and implement block-table paging with copy-on-write sharing.
Prereqs: the attention mechanism autoregressive decoding (prefill and decode phases) KV cache GQA/MQA virtual memory and paging (operating systems) FlashAttention and online softmax
Quantize a trained LLM's weights (and activations) to INT4/INT8 post-hoc using GPTQ, AWQ, or SmoothQuant.
Prereqs: floating-point number representation matrix multiplication / linear layers KV cache memory-bandwidth-bound vs compute-bound performance (roofline model) LayerNorm / RMSNorm affine parameters
Choose and implement INT4/INT8/FP8/GGUF quantization formats, plus QAT and QLoRA, for efficient deployment.
Prereqs: floating-point number formats (FP16/BF16) matrix multiplication / GEMM KV cache (key-value cache) the roofline model and memory-bandwidth-bound reasoning post-training quantization calibration algorithms (GPTQ, AWQ, SmoothQuant) LoRA (low-rank adaptation)
Fuse kernels, capture CUDA graphs, and use torch.compile to speed up PyTorch models.
Prereqs: GPU memory hierarchy (DRAM, shared memory, registers) roofline model of compute-bound vs. memory-bound performance PyTorch eager execution model backpropagation and autograd matrix multiplication / GEMM basics
Compute a transformer's full training memory budget and apply checkpointing, offloading, and LoRA/QLoRA to fit it on smaller GPUs.
Prereqs: GPU memory hierarchy (HBM, PCIe bandwidth) mixed-precision training (bf16/fp32) the Adam optimizer transformer architecture (attention, FFN, layer norm) ZeRO/FSDP distributed training basics
Part V — Post-Training & Alignment
Fine-tune a base model into an instruction-follower via response-only loss masking, with data curation and forgetting mitigation.
Prereqs: causal language modeling / next-token prediction objective cross-entropy loss gradient descent and backpropagation transformer architecture the AdamW optimizer mixed-precision (bf16) training
Build chat templates, loss masks, and packed batches for training conversational models.
Prereqs: causal language modelling loss tokenization and special tokens (BPE) the attention mechanism / causal mask RoPE positional encoding supervised fine-tuning basics Flash Attention
Fine-tune giant LLMs cheaply with LoRA/QLoRA adapters and serve thousands of them per GPU.
Adapt LLMs via soft prompts, prefix tuning, IA3, or by merging fine-tuned checkpoints without retraining.
Prereqs: the attention mechanism (query/key/value) full fine-tuning of neural networks gradient descent and backpropagation token embeddings low-rank adaptation (LoRA) and PEFT basics
Train a Bradley-Terry reward model and assemble RLHF's four-model PPO pipeline.
Prereqs: supervised fine-tuning (SFT) transformer decoder architecture logistic sigmoid function maximum likelihood estimation KL divergence basic policy-gradient reinforcement learning
Derive REINFORCE, GAE, and PPO's clipped objective, then implement the RLHF training loop.
Derive DPO's closed-form loss from RLHF, implement it, and pick among IPO/KTO/ORPO/SimPO/CPO variants.
Prereqs: the RLHF pipeline and reward modeling KL divergence policy gradients and PPO supervised fine-tuning (SFT) Bradley-Terry preference model sigmoid/logistic function
5.8 GRPO, RLOO & Critic-Free RLadvanced
Derive and implement critic-free RL (RLOO, GRPO) and apply the 2025 Dr. GRPO/DAPO fixes.
Prereqs: policy gradients / REINFORCE PPO and the clipped surrogate objective reward models and KL penalties in RLHF generalized advantage estimation (GAE) autoregressive language model sampling basic variance/bias tradeoffs in estimators
Build verifiable-reward checkers/sandboxes and explain how correctness-only RL grows emergent reasoning.
Implement and evaluate test-time reasoning strategies from chain-of-thought to MCTS and budget-forced long-thinking models.
Build AI-labeled alignment pipelines (CAI, RLAIF, STaR, ReST, SPIN) that scale beyond human annotation.
Implement knowledge distillation, pruning, and quantization-aware compression to shrink large models.
Prereqs: softmax and logits KL divergence and cross-entropy loss supervised fine-tuning (SFT) transformer architecture (attention heads, layers, residual stream) chain-of-thought reasoning gradient descent and backpropagation
Diagnose reward hacking via the KL-reward frontier and defend RLHF pipelines with KL control, reward ensembles, and verifiable rewards.
Prereqs: RLHF (reinforcement learning from human feedback) pipeline reward model training from preference data PPO / policy gradient methods for language models KL divergence basic RL objective structure (reward maximization plus penalty term)
Part VI — RL Infrastructure (Deep Dive)
Map the six components and dataflow of any RL-for-LLM system and diagnose its bottlenecks.
Prereqs: policy-gradient RL for LLMs (PPO) GRPO / RLOO critic-free RL autoregressive decoding and KV cache distributed training (FSDP / data parallelism) mixed precision training basics of LLM inference serving
Trace, profile, and optimize the five-phase RL rollout-to-gradient loop and its staleness tradeoffs.
Prereqs: PPO (Proximal Policy Optimization) objective GRPO / group-relative advantage estimation KV cache autoregressive decoding FSDP or other distributed training basics reward modeling / RLHF basics
Configure and debug TRL's SFT/Reward/DPO/PPO/GRPO trainers, including PEFT and vLLM integration, for real alignment runs.
Prereqs: policy gradient methods and PPO Direct Preference Optimization theory LoRA / PEFT adapter fine-tuning KL divergence distributed training (FSDP/DeepSpeed/DDP) reward models for RLHF
Explain and implement veRL's HybridFlow dispatch and 3D-HybridEngine resharding to scale RL-for-LLM training.
Prereqs: the RL-for-LLM training loop (policy, rollout, reward, trainer) PPO/GRPO and advantage estimation distributed training (FSDP, Megatron tensor/pipeline parallelism) vLLM/SGLang inference engines and PagedAttention Ray actors and remote calls GPU memory budgeting for weights, gradients, and optimizer state
Design and choose a Ray/Megatron-based RLHF system, laying out actors and weight sync for a given cluster.
Prereqs: the PPO objective vLLM's PagedAttention engine the four-role decomposition of an RL-for-LLM system (policy, critic, reward model, reference model) DeepSpeed ZeRO / distributed data parallelism Megatron-LM tensor/pipeline/data parallelism
Design async off-policy RL pipelines with staleness corrections and verify untrusted decentralized inference via TOPLOC.
Prereqs: the generation-training loop and rollout engines policy gradients and PPO GRPO / group-relative advantage estimation continuous batching in inference engines (vLLM/SGLang) KL control and reward verifiers in RL post-training basic distributed training concepts (weight synchronization, data centers/clusters)
Decide GPU placement (colocated vs disaggregated) and design/reason about RL weight-sync latency.
Prereqs: anatomy of an RL system / generation-training loop GRPO and PPO for language models FSDP/ZeRO and tensor/pipeline parallelism KV cache and prefill vs decode in LLM inference roofline model (compute-bound vs memory-bandwidth-bound) NCCL collective communication basics
Design rule-based verifiers, LLM judges, and sandboxes to compute reliable RL training rewards.
Prereqs: RLHF/RLVR post-training pipeline the generation-training rollout loop policy-gradient/GRPO advantage estimation KL penalty against a reference policy basic containerization (Docker) chain-of-thought reasoning format
Implement and debug production-grade advantage estimators, KL control, and PPO clipping tricks for LLM RL.
Prereqs: policy gradient theorem PPO's clipped surrogate objective GRPO and group-relative advantages value functions / critics in RL log-probabilities over a token vocabulary
6.10 Agentic & Multi-Turn RLadvanced
Build and train multi-turn agentic RL rollouts with correct action/observation masking and credit assignment.
Diagnose RL generation bubbles and apply oversubscription, partial rollout, and DAPO/Dr. GRPO tricks to fix them.
Select, curriculum-order, and replay RL prompts so every rollout carries gradient signal.
Prereqs: GRPO group-relative advantage estimation RL with verifiable rewards (RLVR) recipe PPO clipped objective and importance sampling reward verifiers and sandboxed checkers RL rollout generation throughput and cost
Part VII — Inference & Serving
Estimate LLM serving speed from model size and GPU bandwidth using prefill/decode physics.
Prereqs: the transformer forward pass causal self-attention mechanism the roofline performance model GPU memory hierarchy and HBM bandwidth softmax and matrix multiplication
Design and implement an iteration-level scheduler that keeps LLM-serving batches pinned near capacity.
Prereqs: autoregressive decoding KV cache memory-bound vs. compute-bound operations (roofline model) the attention mechanism GPU high-bandwidth memory (HBM) and weight streaming
Explain and tune vLLM's PagedAttention block manager, scheduler, prefix caching, and multi-LoRA internals.
Explain and implement radix-tree KV-cache reuse and write branching SGLang programs with constrained decoding.
Choose and configure the right serving stack (TensorRT-LLM, TGI, llama.cpp, LMDeploy, MLC-LLM) for a workload.
Derive, implement, and analyze speculative decoding (Medusa, EAGLE, lookahead) to speed up lossless LLM decoding.
Reuse cached KV blocks across requests via content hashing/radix tries to cut TTFT and prefill compute.
Diagnose prefill/decode interference and design chunked or disaggregated serving to fix P99 latency.
Choose and implement decoding algorithms (temperature, top-k/p, min-p, beam search, contrastive decoding) to control LLM output quality.
Prereqs: the autoregressive decoding loop / KV cache softmax function transformer output logits over a vocabulary basic probability (multinomial sampling, entropy) cumulative sum / cumulative probability mass
Implement and reason about FSM/PDA-based logit masking to guarantee LLM output matches a grammar or schema.
Prereqs: the decoding loop / autoregressive generation token logits and softmax temperature scaling and top-k/top-p sampling tokenization (BPE, multi-character tokens) regular expressions and context-free grammars batched/continuous inference serving
Size and configure multi-GPU/multi-node LLM serving using tensor, pipeline, expert, and data parallelism.
Prereqs: GPU architecture and interconnects (NVLink, PCIe, InfiniBand) collective communication primitives (all-reduce, all-to-all) prefill and decode phases of inference and the KV cache Mixture-of-Experts architecture and top-k routing Grouped-Query Attention (GQA) tensor and pipeline parallelism fundamentals from distributed training
Derive $/1M-token costs, size GPU clusters, and pick batching/quantization tradeoffs against latency SLOs.
Prereqs: prefill and decode mechanics continuous batching KV cache memory management (PagedAttention) the roofline model (FLOPs vs. bandwidth) GPU architecture basics (HBM bandwidth, tensor cores) basic quantization concepts
Design and diagnose large-EP MoE inference: overlap all-to-all, balance hot experts, size dense-equivalent throughput.
Prereqs: Mixture-of-Experts (MoE) architectures and top-k routing prefill vs. decode and the KV cache collective communication (all-reduce, all-to-all) tensor parallelism Multi-head Latent Attention (MLA) continuous batching and request scheduling
Design and build a system serving thousands of LoRA adapters over one shared base model.
Part VIII — Agents & Harness Engineering
Build a robust tool-calling loop: define schemas, parse/validate/dispatch calls, handle errors, run in parallel.
Prereqs: JSON and JSON Schema basics chat templates and system prompts supervised fine-tuning (SFT) context window limits basic Python and REST APIs
Build a working ReAct agent and apply plan-execute, reflection, and tree-search patterns with guardrails against loops and drift.
Prereqs: tool use / function calling API mechanics chain-of-thought / test-time reasoning LLM context window and token counting basic reinforcement learning reward signals (for the agentic RL section) JSON-mode / structured output from an LLM API
Build a coding-agent harness: tools, context assembly, loop, permissions, and verification.
Budget, retrieve, compact, and cache an LLM agent's context window to keep it accurate and cheap.
Architect and implement episodic, semantic, and vector agent memory with compaction and session persistence.
Prereqs: the context window and token budgets embeddings and cosine similarity the agentic loop (ReAct / plan-execute) vector databases and ANN search basics JSON / key-value data structures
Build, connect, and secure MCP servers exposing tools, resources, and prompts to agent hosts.
Prereqs: tool use / function calling for LLMs JSON Schema client-server architecture the agentic loop (ReAct) OAuth 2.0 authentication basics
Design, implement, and cost-benchmark multi-agent LLM systems across topologies and frameworks.
Prereqs: the agentic loop (ReAct, plan-execute, reflection) tool use and function calling context window limits system/user prompt roles in chat APIs basic probability (independent events) reinforcement learning basics
Select agent benchmarks and compute pass@k, confidence intervals, and harness-adjusted comparisons correctly.
Prereqs: tool use / function calling in agents the agent harness/scaffold concept basic probability and combinatorics software unit testing and test suites reward modeling / RL reward hacking basics
Design, optimize, and evaluate production prompts using CoT, few-shot, DSPy/APE, and caching.
Prereqs: instruction-tuned language models and chat message roles autoregressive next-token generation KV cache and prefill/decode inference basics basic conditional probability notation the agentic tool-use loop (calling tools, observing results)
Part IX — Retrieval & RAG
Train and evaluate bi-encoder dense embedding models with contrastive InfoNCE and hard negatives for RAG retrieval.
Build and choose ANN indexes (IVF, PQ, HNSW) to retrieve top-k vectors at scale within a latency budget.
Prereqs: dense vector embeddings cosine similarity and Euclidean distance k-means clustering matrix multiplication / GEMM GPU memory bandwidth vs. compute (memory-bound operations)
Design, build, and evaluate a production RAG pipeline from chunking through RAGAS metrics.
Prereqs: dense embeddings / vector representations cosine similarity approximate nearest neighbor search transformer-based language models BM25 sparse retrieval basics prompt construction for LLMs
Build a production RAG retrieval pipeline: chunk documents, fuse BM25/dense search, and rerank with cross-encoders.
Prereqs: embeddings and cosine similarity dense vector retrieval / ANN search the attention mechanism tokenization RAG system architecture basics TF-IDF / inverted index concepts
Diagnose naive RAG failure modes and choose among GraphRAG, agentic, self-correcting, or long-context architectures.
Prereqs: standard RAG architecture (embed-retrieve-generate) chunking and hybrid BM25/dense search embeddings and cosine similarity the ReAct agentic loop and tool use attention computation and context windows knowledge graphs / entity-relation extraction
Build OCR-free visual-document retrieval by indexing and scoring page images with ColPali-style late interaction.
Part X — Multimodal & Generative Frontiers
Patchify images into tokens and build ViT-based CLIP, SigLIP, and DINOv2 image encoders.
Prereqs: the attention mechanism the Transformer block (LayerNorm, residuals, MLP) softmax matrix multiplication / linear projection cross-entropy loss
Design and cost a VLM's vision-to-LLM connector, resolution strategy, and training recipe.
Convert audio to tokens/features and build ASR, TTS, and streaming speech LMs fused with text.
Prereqs: the attention mechanism / transformer architecture autoregressive next-token prediction and text tokenization (BPE) cross-attention and encoder-decoder architectures vector quantization / embedding lookup basics KV cache and inference latency (prefill/decode, TTFT) basic signal processing (sampling rate, Fourier transform)
Derive, train, and sample diffusion models (DDPM, DDIM, CFG, latent diffusion, flow matching, diffusion LMs).
Prereqs: Gaussian distributions and basic probability the attention mechanism (self- and cross-attention) variational autoencoders (VAEs) convolutional neural networks and U-Net basics the Transformer architecture gradient-based training and MSE regression
10.5 Unified & Any-to-Any Modelsadvanced
Design and train a single transformer that understands and generates text, images, audio, and video.
Prereqs: decoder-only transformer architecture autoregressive next-token prediction and cross-entropy loss causal self-attention masking vision-language models with frozen encoder plus connector basic diffusion/denoising generative models Mixture-of-Experts routing basics
Part XI — Evaluation
Critically read LLM benchmark claims: spot contamination, saturation, and statistically insignificant score gains.
Prereqs: next-token prediction / language modeling objective basic probability and confidence intervals chain-of-thought prompting reading and writing basic Python (regex, functions) multiple-choice question-answering setup
Build, debias, and calibrate an LLM-as-a-judge pipeline, including Elo-based arena ranking.
Prereqs: prompting and chat-completion APIs chain-of-thought prompting basic probability and statistics (correlation, expected value) reward modeling and the RLHF pipeline benchmark-style evaluation (e.g., MMLU, HumanEval)
Build, run, and statistically validate reproducible eval harnesses like lm-evaluation-harness and HELM.
Prereqs: next-token prediction and log-probabilities few-shot prompting the benchmark landscape (e.g. MMLU, HellaSwag) tokenization and special/BOS tokens basic statistics (standard error, hypothesis testing) chat/instruction-tuned model formatting
Build trustworthy, execution-verified evals for coding, math reasoning, and long-horizon agentic tasks.
Prereqs: basic probability and combinatorics chain-of-thought reasoning unit testing / test suites containerization and process isolation the broader benchmark landscape (accuracy, exact-match metrics) reinforcement learning reward signals
Design, run, and critique safety evals: jailbreak ASR, over-refusal, bias/toxicity, and dangerous-capability elicitation.
Attach confidence intervals, paired significance tests, and power analysis to any eval score.
Prereqs: benchmark evaluation basics (accuracy, pass/fail scoring) probability and statistics foundations (sampling distributions, standard error, hypothesis testing) binomial distribution LLM-as-a-judge evaluation eval harness mechanics
Part XII — Production, Systems & MLOps
Design and size a multi-replica LLM serving fleet: routing, SLOs, queueing, memory, batching, autoscaling.
Prereqs: prefill and decode phases of inference KV cache continuous batching basics grouped-query attention (GQA) PagedAttention basic queueing/probability intuition
Instrument, log, evaluate, and alert on production LLM systems, closing the ship-observe-improve loop.
Prereqs: the LLM serving pipeline (prefill, decode, KV cache) retrieval-augmented generation (RAG) basics LLM-as-a-judge evaluation basic statistics (mean, standard deviation, hypothesis testing) distributed systems / microservices concepts token-based API pricing
Design a layered cache-route-batch-spot stack that cuts LLM production API spend 50-90%.
Prereqs: KV cache token-based API pricing (input/output tokens) quantization (PTQ, INT4/INT8) continuous batching / request scheduling basics embeddings and cosine similarity basic distributed-systems concepts (load balancers, retries, TTL)
Design, tune, and deploy a layered guardrail stack (input/output classifiers, PII redaction, shield models) around a production LLM.
Prereqs: RLHF/DPO-based alignment precision, recall, and F-scores fine-tuning encoder classifiers (e.g. BERT-scale models) system prompts and chat templates retrieval-augmented generation (RAG) basic LLM serving/inference pipeline (prefill, time-to-first-token)
Build a production data flywheel that logs, labels, distills, retrains, and eval-gates model updates.
Design layered defenses (sandboxing, dual-LLM, structured output) against prompt injection and jailbreaks.
Design A/B tests, canaries, and guardrail metrics to evaluate and safely roll out LLM features in production.
Prereqs: hypothesis testing and p-values statistical power and sample size LLM-as-a-judge evaluation offline benchmark evaluation basic probability and variance
Define quality-aware SLOs, diagnose LLM failures, and build rollback, postmortem, and failover systems.
Prereqs: LLM inference basics (latency, tokens, streaming) retrieval-augmented generation (RAG) pipelines LLM-as-a-judge automated evaluation structured logging and distributed tracing A/B testing and canary deployments basic statistics (percentiles, standard deviation)
Part XIII — Interpretability, Safety & Governance
Reverse-engineer transformer internals using probing, logit lens, activation patching, circuits, and SAEs.
Prereqs: the transformer block (attention + MLP + residual connections) the attention mechanism layer normalization softmax and logits basic gradient-based training / backpropagation logistic regression / linear classifiers
Locate and surgically edit facts in model weights, and verify unlearning survives adversarial audits.
Prereqs: transformer MLP (feed-forward) sublayers residual stream activation patching / mechanistic interpretability basics gradient descent and backpropagation cross-entropy loss Direct Preference Optimization (DPO)
Measure and defend against LLM training-data leakage using canaries, MIA audits, and DP-SGD.
Prereqs: cross-entropy loss and maximum-likelihood training objective gradient descent / SGD basic probability (log-likelihood, Gaussian distributions, ROC curves) overfitting vs. generalization next-token prediction / autoregressive language modeling
Implement, evaluate, and choose among watermarking, C2PA provenance, and post-hoc AI-text detection.
Prereqs: next-token prediction and logits softmax and sampling (temperature/nucleus) cryptographic hash functions basic hypothesis testing (p-values, z-scores) LLM inference serving basics
Design scalable-oversight, dangerous-capability, and control protocols to gate frontier model deployment.
Prereqs: RLHF (reinforcement learning from human feedback) and reward modeling chain-of-thought reasoning reward hacking and alignment failure modes red-teaming and safety evaluation methodology basic statistical hypothesis testing (effect size, p-values)
Determine which AI-regulation obligations apply to a model and build the compliance artefacts to satisfy them.
Prereqs: training compute (FLOPs) and Chinchilla-style scaling approximations pretraining data pipelines and web crawling red-teaming and safety evaluation methodology watermarking and content-provenance techniques basic notions of model parameters and training tokens
Part XIV — Capstone: Build a 100M LLM End-to-End
Plan and budget a from-scratch 101M-parameter LLM build using 2026 small-model best practices.
Prereqs: the transformer architecture and attention mechanism KV cache Chinchilla scaling laws byte-level BPE tokenization mixed-precision (bf16) training basics GPU FLOP throughput and training compute (the 6ND rule)
Source, filter, deduplicate, tokenize, and pack a 20B-token corpus for a 100M-parameter model.
Prereqs: Chinchilla scaling law byte-pair encoding (BPE) tokenization RoPE (rotary positional encoding) causal self-attention masking training compute estimation (C≈6ND)
Build a fast from-scratch byte-level BPE trainer and choose vocab size as a 100M-parameter budget tradeoff.
Assemble and parameter-count a from-scratch 100M-parameter transformer from named, cited 2024-2026 SOTA components.
Prereqs: the transformer block (residuals, norms, MLP) multi-head attention and GQA/MLA basics RoPE and positional encodings parameter counting for linear layers scaling laws (6ND FLOPs rule) PyTorch nn.Module basics
Fit and validate your own scaling law from a cheap model ladder before committing training budget.
Prereqs: Kaplan/Chinchilla scaling laws for loss vs. parameters and tokens the 6ND FLOP-counting rule for transformers transformer architecture components (GQA, RoPE, SwiGLU, RMSNorm) learning rate schedules (WSD/cosine decay) tokenization and vocabulary size trade-offs nonlinear curve fitting / non-convex optimization
Implement a Muon+AdamW hybrid optimizer with MuonClip/QK-clip and a WSD schedule.
Write and run a complete, resumable single-GPU pretraining loop with real throughput monitoring.
Prereqs: transformer forward/backward pass the optimizer and WSD learning-rate schedule (Muon+AdamW) packed dataset and doc-aware masking floating-point precision (fp32/fp16/bf16) cross-entropy loss scaling laws and compute-optimal training
Anneal on premium data, extend RoPE context to 8192, and inject math/code capability across one continuous LR decay.
Prereqs: RoPE (rotary positional encoding) Warmup-Stable-Decay (WSD) learning rate schedule FlashAttention grouped-query attention (GQA) data mixing and curriculum design continual/domain-adaptive pretraining
Post-train a 100M base model with SFT, DPO, and narrow GRPO/RLVR into a stopping, preference-tuned, arithmetic-capable assistant.
Distill teacher ReAct trajectories to SFT a 100M model into a narrow tool-using research agent.
Build an honest small-model eval harness, then quantize and serve the model on a laptop CPU.
Prereqs: cross-entropy loss and training-time perplexity autoregressive decoding and the KV cache grouped-query attention (GQA) SFT/DPO/RLVR post-training floating-point vs. integer number formats (fp32/bf16/int8/int4) held-out validation data and train/test contamination
Itemize the ~$100 training bill, make a run bit-reproducible, and plan the 100M-to-1B scale-up.
Prereqs: scaling laws (Chinchilla-optimal compute allocation) distributed training (DDP, FSDP/ZeRO, tensor and pipeline parallelism) checkpointing and fault-tolerant training loops mixed-precision (bf16/fp8) training and optimizer memory footprint optimizers (AdamW, Muon) and learning-rate/batch-size scheduling the full Stack-100M pipeline (tokenizer, pretraining, alignment, quantization)
Appendix
Glossary of Termsintro
Look up a precise, cross-linked definition for any term used anywhere in the LLM stack.
Prereqs: the Transformer architecture gradient descent and backpropagation GPU and CUDA basics probability and information theory
The Math Reference Sheetadvanced
Look up and correctly implement the core LLM-stack equations: norms, attention, RoPE, AdamW, scaling laws, LoRA, and RL losses.
Navigate the field's canonical papers by topic and follow a six-week reading curriculum with textbook cross-links.
Prereqs: the Transformer architecture and self-attention gradient-based training and backpropagation the standard pretraining-then-fine-tuning paradigm GPU memory and compute basics the RLHF/alignment pipeline at a conceptual level familiarity with the book's core chapters this list cross-links to
Set up, pin, debug, and profile the CUDA/PyTorch/HF/DeepSpeed/vLLM stack for real LLM work.
Prereqs: GPU architecture and memory hierarchy PyTorch autodiff basics distributed data parallelism (DDP/FSDP) mixed-precision training (bf16/fp16/fp8) quantization concepts (INT4/INT8) the attention mechanism and KV cache
From-Scratch Code Indexadvanced
Locate any from-scratch code implementation in the book and trace what it depends on and enables.
Prereqs: backpropagation and gradient descent the transformer architecture the attention mechanism PyTorch tensors and nn.Module basics softmax and cross-entropy loss reinforcement learning from human feedback (RLHF) basics