1.8 The 10-Day Study Plan & Final Checklist¶

You have ten days. Maybe seven. Maybe the recruiter called yesterday and you have a long weekend. Whatever your window, the failure mode is always the same: you re-read the chapters you already know, panic-skim the ones you don’t, and walk into the interview having recognized a thousand concepts but retrieved none of them under pressure. Recognition is not retrieval. An interview is a retrieval test conducted by a stranger while you are nervous.

This chapter is the antidote. It is a concrete, day-by-day operating plan that maps directly onto the chapters of this book, a spaced-repetition flashcard set of the must-knows, a final-48-hours checklist, and a short page to read on the morning of the interview to reset your nerves. It is deliberately prescriptive. When you are stressed, decision-making is expensive; a plan you can follow on autopilot is worth more than a perfect plan you have to design each morning.

The plan assumes you have already worked through the earlier interview-prep chapters — particularly The Google ML Domain Interview: Format & Strategy for what the loop looks like, and ML Breadth: Rapid-Fire Concepts & Model Answers for the breadth bank. Here we sequence the whole book into a study cadence and turn it into muscle memory.

The Philosophy: Retrieval, Spacing, and Interleaving¶

Three findings from cognitive science should drive every hour of your prep, because they are the cheapest performance wins available to you.

The testing effect. Trying to recall an answer before you check it strengthens memory far more than re-reading the answer. The classic Roediger & Karpicke studies showed that learners who studied once and then tested themselves repeatedly outperformed learners who studied the material four times — and the gap widened at a one-week delay. So: do not re-read chapters. Quiz yourself on them, then re-read only the gaps the quiz exposes.

Spacing. The same total study time produces dramatically better long-term retention when spread across days rather than massed into one block. The forgetting curve is roughly exponential. If \(R(t)\) is the probability you still recall an item at time \(t\) (in days) since the last review, a simple model is

where \(S\) is the memory strength — a time constant. Each successful recall increases \(S\), so the curve flattens. That is why a card reviewed on days 1, 3, and 7 is far stickier than the same card hammered three times on day 9. The entire reason for a 10-day plan rather than a 10-hour cram is to let \(S\) grow between reviews.

Interleaving. Practicing related-but-different problems in a shuffled order — a coding problem, then a system-design prompt, then a breadth question — feels harder and slower than blocking, but produces better transfer. Interviews are interleaved by nature: you do not get to study one topic per interviewer in advance. Practice the way you will perform.

We will operationalize all three: the daily plan spaces topics, every day ends with self-testing, and the back half of the plan interleaves mock interviews across domains.

A Minimal Spaced-Repetition Engine You Can Actually Run¶

You can use Anki, and you probably should. But it is worth understanding the algorithm — both because the mechanism is itself a plausible interview topic (it is a tiny online scheduling problem) and because a 40-line script you control beats an app you fight. Below is a from-scratch implementation of a simplified SM-2 scheduler — the algorithm behind Anki — that you can load with the flashcards later in this chapter.

"""
A minimal spaced-repetition scheduler (simplified SM-2).

Each card has:
  - ease (E): how "easy" the card is; scales the interval. Starts at 2.5.
  - interval (I): days until the next review.
  - reps: number of consecutive successful recalls.

You grade each recall 0..5 (0 = blackout, 5 = perfect). A grade >= 3 is a pass.
On a pass, the interval grows multiplicatively by the ease factor, so well-known
cards rapidly drift to weeks apart while struggling cards stay daily.
"""
from dataclasses import dataclass, field
from datetime import date, timedelta
import json


@dataclass
class Card:
    front: str
    back: str
    ease: float = 2.5          # E-factor; SM-2 floor is 1.3
    interval: int = 0          # days
    reps: int = 0
    due: date = field(default_factory=date.today)

    def review(self, grade: int, today: date) -> None:
        """Update scheduling state given a recall grade in 0..5."""
        if grade < 3:
            # Failed: reset the streak, see it again tomorrow.
            self.reps = 0
            self.interval = 1
        else:
            # Passed: grow the interval.
            if self.reps == 0:
                self.interval = 1
            elif self.reps == 1:
                self.interval = 6
            else:
                self.interval = round(self.interval * self.ease)
            self.reps += 1

        # Update ease. Hard passes (grade 3) shrink ease; perfect passes (5) keep it.
        # This is the classic SM-2 update; ease never drops below 1.3.
        self.ease = max(1.3, self.ease + (0.1 - (5 - grade) * (0.08 + (5 - grade) * 0.02)))
        self.due = today + timedelta(days=self.interval)


def due_cards(deck, today):
    """Return cards due today or earlier, hardest-first (lowest ease)."""
    return sorted([c for c in deck if c.due <= today], key=lambda c: c.ease)


# --- Example session -------------------------------------------------------
if __name__ == "__main__":
    deck = [
        Card("Chinchilla optimal tokens-per-param?", "~20 tokens / parameter"),
        Card("Why sqrt(d_k) in attention?", "Keeps logit variance ~1 so softmax "
             "doesn't saturate; dot product of d_k unit-var dims has variance d_k."),
        Card("LoRA: what is delta W?", "B @ A, with A in R^{r x d}, B in R^{d x r}, "
             "r << d; only A,B are trained, W0 frozen."),
    ]
    today = date.today()
    for card in due_cards(deck, today):
        print("Q:", card.front)
        # In real use you'd prompt for input(); here we simulate a confident pass.
        card.review(grade=5, today=today)
        print("   ->", card.back, f"(next in {card.interval}d)\n")

    # Persist so the schedule survives across the 10 days.
    with open("deck.json", "w") as f:
        json.dump([{**c.__dict__, "due": c.due.isoformat()} for c in deck], f, indent=2)

Run this once a day, grade honestly, and the scheduler surfaces exactly the cards you are about to forget — which, by the forgetting-curve logic above, is precisely when reviewing them yields the most strength gain per minute.

The 10-Day Plan, Day by Day¶

The plan is built around four-hour study days (scale the hours to your situation — the sequence matters more than the volume). Each day has a theme, a set of chapters to actively recall (not re-read cover to cover — skim, then self-test), a build/code block, and a review block that runs your due flashcards. Days are front-loaded with the highest-leverage, most-frequently-asked material, so that if life eats your last days, you have already covered the topics most likely to appear.

The ordering principle: breadth and fundamentals first (they anchor everything and are the most common bar-raiser), the transformer and inference next (the densest cluster of LLM-specific questions), training and alignment in the middle, and system design plus behavioral last (they integrate everything, so they reward being studied after the parts are fresh).

Days 1–2: Foundations and Breadth¶

The goal of the first two days is to make your mathematical and classical-ML reflexes sharp, because a missed bias–variance or a botched cross-entropy derivation early in a loop colors every later interviewer’s read of you.

On day 1, the code block is the single highest-yield exercise in classical ML: derive and implement binary cross-entropy and its gradient. The loss for one example with label \(y \in \{0,1\}\) and predicted probability \(\hat{y} = \sigma(z)\) is

and the gradient with respect to the logit \(z\) collapses to the famously clean

Being able to derive that on a whiteboard — showing that the \(\sigma'(z) = \sigma(z)(1-\sigma(z))\) term cancels the \(1/\hat{y}\) from the log — is a classic warm-up interviewers love, because it separates people who memorized the result from people who understand it.

import numpy as np

def sigmoid(z):
    # Numerically stable sigmoid: avoid exp overflow for large |z|.
    return np.where(z >= 0, 1.0 / (1.0 + np.exp(-z)),
                    np.exp(z) / (1.0 + np.exp(z)))

def bce_loss_and_grad(X, y, w, b):
    z = X @ w + b
    p = sigmoid(z)
    eps = 1e-12                                   # guard log(0)
    loss = -np.mean(y * np.log(p + eps) + (1 - y) * np.log(1 - p + eps))
    # dL/dz = p - y  (the clean result you derive on the whiteboard)
    dz = (p - y) / len(y)
    grad_w = X.T @ dz
    grad_b = dz.sum()
    return loss, grad_w, grad_b

End each day by running your flashcard deck. On days 1–2 you are seeding the deck — most cards are brand new, so expect everything to be due tomorrow.

Days 3–4: The Transformer and Attention¶

This is the densest, most-asked cluster in any LLM interview. If you can derive scaled dot-product attention, explain the KV cache, and reason about MQA/GQA trade-offs cold, you have covered a large fraction of the LLM-specific surface area.

The must-build for day 3 is causal self-attention. Type it from memory; the goal is to reach the point where the shapes flow without thinking.

import torch
import torch.nn.functional as F

def causal_self_attention(x, W_q, W_k, W_v, W_o):
    """
    x: (B, T, d_model). One head, for clarity.
    Returns (B, T, d_model).
    """
    B, T, d = x.shape
    q = x @ W_q                       # (B, T, d_k)
    k = x @ W_k                       # (B, T, d_k)
    v = x @ W_v                       # (B, T, d_k)
    d_k = q.shape[-1]

    # Scaled scores. The 1/sqrt(d_k) keeps logit variance ~O(1) so softmax
    # doesn't saturate into a near-one-hot distribution with tiny gradients.
    scores = (q @ k.transpose(-2, -1)) / (d_k ** 0.5)   # (B, T, T)

    # Causal mask: position t may attend to <= t only. Set future to -inf
    # so softmax assigns them exactly zero weight.
    mask = torch.triu(torch.ones(T, T, device=x.device), diagonal=1).bool()
    scores = scores.masked_fill(mask, float("-inf"))

    attn = F.softmax(scores, dim=-1)  # (B, T, T)
    out = attn @ v                    # (B, T, d_k)
    return out @ W_o                  # (B, T, d_model)

Day 4’s pen-and-paper exercise — counting a GPT’s parameters — deserves its own worked example, because the magnitudes recur in every memory and cost question you will get.

Days 5–6: Pretraining, Scaling, and Efficiency¶

The day-5 reflex to drill is the Chinchilla compute relation. Training compute is approximately

FLOPs, where \(N\) is parameter count and \(D\) is the number of training tokens (the factor 6 is 2 for the forward multiply-add and 4 for the backward pass). Chinchilla’s headline finding is that compute-optimal training uses roughly \(D \approx 20 N\) — about 20 tokens per parameter. Memorize “6ND” and “20 tokens per param”; together they let you sanity-check any training proposal in your head. See Scaling Laws.

The day-6 reflex is arithmetic intensity: a kernel is compute-bound if its FLOPs-per-byte exceeds the hardware’s ratio of peak FLOP/s to peak memory bandwidth, and memory-bound otherwise. LLM decode is memory-bound — one token at a time streams the whole weight matrix — which is the root cause of why batching, speculative decoding, and quantization all help. This one idea unifies half of Part IV and Part VII.

Days 7–8: Alignment, Inference, and Agents¶

The day-7 build is the DPO objective: short, conceptually deep, and a magnet for follow-ups. Given a preferred completion \(y_w\) and a dispreferred \(y_l\) for prompt \(x\), with policy \(\pi_\theta\) and frozen reference \(\pi_\text{ref}\),

import torch.nn.functional as F

def dpo_loss(pi_logp_w, pi_logp_l, ref_logp_w, ref_logp_l, beta=0.1):
    """
    Each arg is a (batch,) tensor of *summed* token log-probs of a full
    completion under the policy (pi_) or frozen reference (ref_).
    The model implicitly *is* its own reward model: r(x,y) = beta * log(pi/ref).
    """
    pi_logratio  = pi_logp_w  - pi_logp_l     # how much the policy prefers w over l
    ref_logratio = ref_logp_w - ref_logp_l    # the reference's preference
    # Train the policy to prefer w more strongly than the reference does.
    logits = beta * (pi_logratio - ref_logratio)
    return -F.logsigmoid(logits).mean()

Be ready for the killer follow-up: why is there no separate reward model or PPO loop in DPO? The answer — that the optimal RLHF policy has a closed form which lets you re-parameterize the reward as a function of the policy’s own log-ratio, turning a two-stage RL problem into a single classification loss — is one of the cleanest “do you actually understand this” tests in alignment. See Direct Preference Optimization & Its Variants.

Days 9–10: Integration — System Design, Coding, and Behavioral¶

The last two days are pure interleaving and mock interviews. You stop learning new material — the deadline for new facts has passed, and cramming new content now displaces consolidation of what you already know — and instead integrate under realistic conditions.

Do the day-9 mocks out loud, ideally with a peer or by recording yourself. Speaking the design — “let me start by clarifying requirements: what is the QPS, the latency SLO, and the acceptable cost per query?” — exposes the gaps that silent reading hides. The single biggest difference between candidates who know system design and candidates who pass it is whether they have rehearsed talking through one. See ML System Design: A Framework.

The Spaced-Repetition Flashcard Set: The Must-Knows¶

Below is the core deck — the facts and one-liners that should be instant by interview day. Seed them into the scheduler on day 1. They are grouped by area, but you should review them shuffled (interleaving). Format: front → back.

Foundations & math

• Bias–variance decomposition? → \(\mathbb{E}[(\hat{f}-y)^2] = \text{Bias}^2 + \text{Variance} + \sigma^2_\text{noise}\); high bias = underfit, high variance = overfit.
• Cross-entropy gradient w.r.t. the logit? → \(\hat{y} - y\) (softmax/sigmoid + CE collapses cleanly).
• Why does Adam use \(\hat{m}, \hat{v}\) bias correction? → \(m, v\) start at 0, so early estimates are biased toward 0; dividing by \(1-\beta^t\) corrects it.
• L1 vs L2 regularization? → L1 induces sparsity (corners of the diamond constraint); L2 shrinks smoothly toward 0.

Transformer

• Why \(1/\sqrt{d_k}\) in attention? → A dot product of \(d_k\) unit-variance dims has variance \(d_k\); scaling restores variance \(\approx 1\) so softmax doesn’t saturate.
• MHA vs MQA vs GQA? → MQA shares one K,V across all heads (small cache, some quality loss); GQA shares K,V across groups — the practical middle ground.
• RoPE in one sentence? → Rotates Q,K by a position-dependent angle so the dot product depends only on relative position; extrapolates, and is the modern default.
• Pre-norm vs post-norm? → Pre-norm (LayerNorm/RMSNorm inside the residual branch) is far more stable at depth; the modern default.

Pretraining & efficiency

• Training FLOPs rule? → \(C \approx 6ND\) (\(N\) params, \(D\) tokens).
• Chinchilla optimal? → ~20 tokens per parameter for compute-optimal training.
• ZeRO stages? → Stage 1 shards optimizer states, 2 adds gradients, 3 adds parameters (= FSDP).
• Memory-bound vs compute-bound decode? → Decode is memory-bound (streams all weights per token); prefill is compute-bound.

Alignment & inference

• DPO vs PPO? → DPO removes the explicit reward model + RL loop by re-parameterizing reward as \(\beta\log(\pi_\theta/\pi_\text{ref})\); a single classification loss.
• KL term in RLHF, why? → Penalizes drift from the SFT reference so the policy doesn’t reward-hack into degenerate text.
• Continuous (in-flight) batching? → Add/evict requests at the token level instead of waiting for a whole batch to finish; a big throughput win.
• Speculative decoding correctness? → A cheap draft proposes \(k\) tokens; the target verifies in one pass and accepts via a rejection rule that preserves the target’s distribution exactly.

System design & production

• First move in any ML system-design question? → Clarify requirements: scale (QPS), latency SLO, cost budget, quality bar, and what “good” means.
• RAG vs long-context vs fine-tuning? → RAG for fresh/large knowledge + citations; long-context for whole-document reasoning; fine-tuning for behavior/format/style.

Interview Corner¶

The Final 48 Hours Checklist¶

The last two days are about consolidation and logistics, not new learning. Cramming new material in the final 48 hours is actively counterproductive: it raises anxiety, displaces sleep, and interferes with the consolidation of what you already know. Work this checklist instead.

T-minus 48 hours

• Do one full timed mock per remaining interview type (coding, ML design, breadth). Score yourself honestly; note the top 3 gaps only.
• Run your entire flashcard deck once, even cards not due. Mark every wince — these few cards, not the whole book, are your final-day review list.
• Re-read your own STAR stories (situation, task, action, result). Say each one out loud once. Three to five rehearsed stories cover most behavioral prompts.
• Confirm logistics: interview time in your timezone, video link, interviewer names/roles, and the coding environment (shared doc? IDE? which languages are allowed?).

T-minus 24 hours

• Skim only your wince-cards and the Key Takeaways boxes of the chapters you study from. No new topics.
• Prepare your environment: water, charged laptop, backup internet (phone hotspot), pen and blank paper, a quiet room, a “do not disturb” sign.
• Write your three clarifying-question openers on a sticky note: “What’s the scale? What’s the latency SLO? What does ‘good’ mean here?” — you’ll use these in design and ambiguous coding problems alike.
• Stop studying by early evening. Light exercise. Protect your sleep — see the next section on why this is non-negotiable.

Morning of

• No new material. Glance once at the must-know one-liners (FLOPs \(= 6ND\), 20 tokens/param, \(\hat{y}-y\), \(1/\sqrt{d_k}\), KV cache \(\approx\) 4 GB / 4k seq).
• Eat something with protein. Hydrate. Arrive or log in 10 minutes early.
• Re-read the “Reset Your Nerves” page below.
• Have your questions-for-the-interviewer ready. Asking good questions signals seniority and genuine interest.

Reset Your Nerves: The Morning-Of Page¶

Read this on the morning of the interview. It is short on purpose.

You are not being asked to know everything. No one does. The interviewer is not running a checklist of trivia; they are sampling how you think — how you decompose an ambiguous problem, how you reason about trade-offs, whether you say “I’m not sure, but here’s how I’d figure it out” instead of bluffing. A confident “I don’t know that specific number, but I can derive a bound” beats a confident wrong answer every time. Not knowing one thing does not sink the interview. Pretending to know it might.

Reframe the nerves. The racing heart and the adrenaline are not fear — they are your body mobilizing resources for a hard task. The physiological signature of “anxiety” and “excitement” is nearly identical; the difference is the label you attach. Tell yourself “I’m excited,” not “I’m nervous.” It sounds glib; it is supported by real research (Brooks, Get Excited, 2014) and it works.

Use the box-breath before you start. If your heart is pounding, do three rounds of: inhale 4 seconds, hold 4, exhale 4, hold 4. This down-regulates the sympathetic nervous system in under a minute and clears the cognitive fog that panic creates.

Think out loud — it is a gift to you, not just the interviewer. Narrating your reasoning slows you to a sustainable pace, surfaces partial credit even when you don’t finish, and lets the interviewer nudge you back before you waste ten minutes down a wrong path. Silence helps no one. When stuck, say what you’re considering and why; when you make an assumption, state it.

Clarify before you solve. The most common avoidable failure is solving the wrong problem fast. Spend the first two minutes asking questions. Interviewers want you to — designing the problem with you is part of the signal.

It is a conversation between two engineers, not an interrogation. The person across the table (or screen) was once where you are. Most interviewers are quietly rooting for you; a candidate who is a pleasure to think alongside is a candidate they want to hire. Be warm. Be curious. Treat their hints as collaboration, not as a verdict.

One bad question is not a bad loop. Interviews are scored holistically, often across multiple interviewers. If one segment goes sideways, mentally close that tab and start the next one fresh. Candidates routinely pass loops in which they bombed a question. The story you tell yourself between rounds matters more than any single round.

You have done the work. The ten days of spaced, interleaved, retrieval-based practice are in you now — that is exactly why we front-loaded the plan and protected your sleep. Trust the preparation, breathe, clarify, and think out loud. Go.

Further reading¶

• Roediger & Karpicke, Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention, 2006 — the foundational paper on the testing effect.
• Ebbinghaus, Memory: A Contribution to Experimental Psychology, 1885 — the original forgetting curve.
• Brown, Roediger & McDaniel, Make It Stick: The Science of Successful Learning, 2014 — the practitioner’s synthesis of spacing, interleaving, and retrieval practice.
• Brooks, Get Excited: Reappraising Pre-Performance Anxiety as Excitement, 2014 — the anxiety-reframing result.
• Wozniak, SuperMemo / SM-2 algorithm — the spaced-repetition scheduler reconstructed in this chapter, and the basis for Anki.
• Hoffmann et al., Training Compute-Optimal Large Language Models (Chinchilla), 2022 — the source of the “20 tokens per parameter” rule you will quote.
• Rafailov et al., Direct Preference Optimization, 2023 — the derivation behind the DPO loss you should be able to write from memory.