Reference, not curriculum · Phase 1 — Foundations

Math, just-in-time

This topic deliberately gets a ledger, not a simulator — because the roadmap's rule is that you never study math ahead of need. The trick: you've already used every concept below. The four labs smuggled them in. What follows is the receipt — each idea named, typeset, and linked back to the moment you felt it.

Math-first paralysis is the #1 way strong engineers waste months. You don't finish a linear algebra course; you raid one, with a specific question, then get back to the build.

§ 1 · The ledger

Math you already did

Calculus

raid: MML ch. 5 · 3Blue1Brown "Essence of Calculus"

derivative = local slope $f'(x) \approx \frac{f(x+\varepsilon) - f(x)}{\varepsilon}$: How much the output wiggles per unit of input wiggle, at the point where you're standing. Not a formula to memorize — a question to ask.you felt it: backprop §2 — the tangent on the loss curve →
partial derivative $\frac{\partial L}{\partial a}$: The same question with every other dial frozen. A billion-parameter model is just a billion of these, one per dial.you felt it: backprop §2 — "freeze every dial except a" →
the chain rule $\frac{\partial L}{\partial a} = \frac{\partial L}{\partial u} \cdot \frac{\partial u}{\partial a}$: Effects multiply through a pipeline: if a moves u and u moves L, then a moves L by the product. Backprop is this rule applied backward, once per node.you felt it: backprop §1 — every step of the ticker →
the gradient $\nabla L = \left(\tfrac{\partial L}{\partial w_1}, \ldots, \tfrac{\partial L}{\partial w_n}\right)$: All the partials stacked into one arrow that points uphill. Training is walking against it.you felt it: gradient descent §1 — the step rule →
local linear approximation $\Delta L \approx \nabla L \cdot \varepsilon$: The tangent's promise: valid near your point, breaking as you stray. This is why learning rates must be small, and why one giant step fails.you felt it: backprop §2 — the nudge that drifts off the curve →

Linear algebra

raid: MML ch. 2–4 · 3Blue1Brown "Essence of Linear Algebra"

matrix × vector = mixing $a^{(l)} = W a^{(l-1)} + b$: Each output is a weighted blend of all the inputs — a matrix is a recipe book of blends. A layer is exactly this, plus a bend.you felt it: neural net — the layer formula in the hero →
matrix products collapse $W^{(2)} W^{(1)} = W$: Two mixes in a row are one mix. This is a theorem AND a design warning: without nonlinearities, depth buys nothing.you felt it: neural net §3 — "activations bend" →
the transpose runs it backward $\delta^{(l-1)} = W^{\top} \delta^{(l)}$: The same mixer, driven in reverse, routes blame from outputs back to inputs. Forward mixes values; backward mixes gradients.you felt it: backprop §1 — blame flowing right to left →
curvature has directions $L = \tfrac{1}{2}(x^2 + \kappa y^2)$: A bowl can be 40× steeper one way than another (those steepnesses are the eigenvalues you'll meet later). Conditioning is why optimizers exist.you felt it: gradient descent §2 — the ravine →

Probability

raid: MML ch. 6 · the glossary, for now

a probability as output $\sigma(z) = \frac{1}{1 + e^{-z}} \in (0, 1)$: The net doesn't answer "class 1" — it answers "0.83 sure." Squashing a raw score into (0,1) is what lets a loss price confidence.you felt it: neural net — the tint of every grid cell →
negative log-likelihood $L = -\log p_{\text{truth}}$: Score a prediction by the probability it gave reality, then take −log so confident wrongness explodes. Cross-entropy is this, averaged.you felt it: core ml §1 — the price of confidence →
expectation = average $L = \tfrac{1}{n} \sum_i L_i$: Every loss you watched was a mean over examples — an estimate of the true average over all possible data. Train/val is two estimates of it.you felt it: core ml §2 — homework vs exam →
entropy of a coin flip $-\log \tfrac{1}{2} = \ln 2 \approx 0.69$: The loss of pure ignorance on a binary task. Every untrained net starts here; a loss stuck here means nothing was learned.you felt it: core ml — the dashed coin-flip line →
randomness needs seeds $W \sim \mathcal{N}(0, \tfrac{1}{n_{\text{in}}})$: Init and data are draws from distributions — so a single run is a single sample, and a "result" from one seed might be luck. (This becomes "≥3 seeds" in research method.)you felt it: neural net — the re-roll ⚄ button →

§ 2 · The protocol

How to raid a reference

1
Hit a real wall
You're mid-build and something won't click — a shape mismatch, a gradient that makes no sense, a paper line you can't parse. That confusion is the prerequisite. Without it, math doesn't stick.
2
Name the missing concept
Use this ledger (or the glossary) to turn "I don't get it" into a noun — "I don't get what the transpose is doing here." A nameable gap is a small gap.
3
Read exactly one section
Open Mathematics for Machine Learning to that one section, or watch the matching 3Blue1Brown chapter. Twenty minutes, with your build open in the next tab. Not the chapter before it. Not "from the beginning."
4
Return and re-derive
Go back to the wall and explain it to your log in your own words. If you can't yet, you found a second concept — loop once more. The build is the exam; the book is just the hallway.

That's the whole discipline. The phases ahead will force the rest of the list out of you — attention needs dot products, scaling laws need logarithms, evals need statistics — each at the moment it's needed, never before.

phase 1 — back to the build →Mathematics for Machine Learning — the raid target ↗3Blue1Brown — essence of linear algebra ↗

Math, just-in-time

Math you already did

Calculus

Linear algebra

Probability

How to raid a reference

Hit a real wall

Name the missing concept

Read exactly one section

Return and re-derive