Interactive explainer · Phase 4 — Post-training & eval
Evaluation as a discipline
The roadmap calls evaluation the single most under-respected skill in the field — and the recurring source of "results" that turn out to be measurement artifacts. The working posture: assume your eval is lying until you've checked how. Below are the four most common lies, each one computable enough to demonstrate live.
— every benchmark score ships with a noise floor, whether reported or not.
§ 1 · Lie #1 — the noise floor
A score is a sample, not a truth
An eval run is a coin-flip experiment: N questions, some probability p of getting each right. Below, both models' runs are real seeded binomial draws, and the shaded band is the analytic 95% interval. Shrink N and watch two genuinely different models become indistinguishable; re-run the eval and watch the "winner" change.
On N = 200, a true 62% model measures anywhere in 55–69% (95% band, ±6.7 pts). A single eval run ranks B above A only 66% of the time — coin-flip territory dressed as a result.
fig. 1 — 20 runs per model, real binomial sampling; the ranking probability uses the normal approximation. Rule of thumb before you celebrate: is the gap bigger than the band?
§ 2 · Lie #2 — the grader
The grader is part of the model
Ten fixed answers, four grading functions — all four actually run on this page. Exact match robs correct answers; the lenient grader credits a wrong one (watch the vermillion card when you pick contains-answer). The score swings 4× with zero change to the model.
the grader
headline score
20%
answer === truth, byte for byte
What is 12 + 15?
truth "27" · model "27"
How many heads did our toy attention layer have?
truth "7" · model "7.0"
Capital of France?
truth "Paris" · model "paris"
When was the US Constitution effective?
truth "March 4, 1789" · model "March 4 1789"
Does the causal mask block future tokens?
truth "yes" · model "Yes, because the mask blocks attention to later positions."
correct — but only the lenient grader can see it
The answer to everything?
truth "42" · model "The answer is 42."
Is a bigger eval always better?
truth "no" · model "no"
Can one seed prove a speedup?
truth "no" · model "Not at all certain — most sources say yes."
the model said YES (wrong) — but "Not" contains "no", so contains-answer credits it
Value of π to 5 decimals?
truth "3.14159" · model "π ≈ 3.14"
wrong at the asked precision — all four graders agree, for once
Spell out the number 7.
truth "seven" · model "7"
correct in spirit — every grader robs it
Same model, same ten answers: 20% or 80% depending on which grading function you picked. The dots on each card show all four graders at once. Which number goes in the paper — and does the paper say which grader produced it?
§ 3 · Lie #3 — contamination
The test set is on the web
A real n-gram scan, the same technique production pipelines use: every benchmark item is checked for 3-gram containment against the training corpus. Three items leak in through a quiz blog and a forum post — the model "solves" everything it memorized, and the reported score inflates accordingly.
reported
70%
on 7 clean items
57%
The scanner found 3 benchmark items verbatim or near-verbatim in the training corpus — a quiz blog and a forum answer, which is exactly how it happens in the wild: the test set is on the web, so models train on it by accident. Every serious eval states a decontamination protocol; scores without one deserve the strikethrough.
fig. 2 — containment, not symmetric similarity: corpus documents are much longer than benchmark items, so the honest question is "what fraction of this item appears in the doc."
§ 4 · Lie #4 — the prompt
The ranking depends on the template
The same two models, the same questions, five ways of asking. Model A wins three formats and loses two — the leaderboard order is a property of the prompt, not just the models. The per-format numbers here are hand-set illustrations of a measured phenomenon (MMLU-style format sensitivity); the flip is the point.
ask the same questions as…
across all five formats: A ranges 57–72% (a 15-point prompt tax), B ranges 61–68%. A wins 3 of 5 formats — the "better model" depends on the template.
the same disease, pairwise · position bias
judge sees A then B → "A wins, 62%"
judge sees B then A → "B wins, 55%"
An LLM judge prefers whichever answer it read first (illustrative numbers; the bias is real and documented). Fix: randomize order per item, report both directions.
why pairwise still beats absolute
"Rate this answer 1–10" drifts with the judge's mood, verbosity, and scale interpretation. "Which of these two is better?" is anchored by construction — you inherit position bias, but that one is measurable and fixable. Comparative judgment plus order randomization is the current honest default.
§ 5 · The discipline
The honest-eval checklist
Phase 4's milestone asks you to critique a published paper's evaluation. This is the checklist to critique it with — and the one your own Phase 4 project must survive:
- Error bars or at least N stated — a score without a denominator is a vibe.
- Grader code published — the grader is part of the model being measured.
- Decontamination protocol stated — n-gram scan against the training corpus.
- Prompt-sensitivity checked — same eval under ≥2 formats, spread reported.
- Same-size baseline included — beating nothing proves nothing.
- Decoding parameters reported — temperature and sampling change the score.
a score is a sample
The noise floor exists whether you report it or not. Gaps smaller than the band are coin flips.
the grader is the model
Exact match, normalization, tolerance — each is a different benchmark wearing the same name. Publish the grader.
report the spread, not the max
Across seeds, prompts, and graders you get a distribution. Quoting its maximum is how measurement artifacts become "results".
Everything on this page compounds into every later project: the reproduction ablations of Phase 3, the post-training project of Phase 4, and the original result of Phase 5 all live or die by whether the measurement can be trusted.