Research method

You don't "read" a paper start to finish. Use a multi-pass approach:

1. 1

   Pass 1 — skim (5 min). Title, abstract, figures, and the results table. Figures carry most of the signal. Ask: what problem, what's the claimed result, does it look real?

2. 2

   Pass 2 — method (15–30 min). Read the method section and the experimental setup. What exactly did they do? What's the baseline? What's the key comparison?

3. 3

   Pass 3 — deep (only for papers that matter). Work through the details, the math, the ablations. Try to find the weakness — what would you need to check to believe this?

Always write a note — even 3 sentences: what they claim, how they measured it, what the one weakness is. A paper you didn't take a note on, you didn't read. And read with the right question: not "is this true?" but "how was this measured, and how could it be wrong?" That question, applied relentlessly, is most of research taste. Aim for 2–3 papers/week, every week, forever.

• — Reproduce before you extend. You can't trust your extension if you can't reproduce the baseline.
• — Containerize your environment (Docker/uv lockfiles) — "works on my machine" is the enemy of reproducibility, including your own future machine.
• — When your reproduction doesn't match the paper, that gap is the most educational thing in the whole process. Chase it down; don't paper over it.
• — A reproduction that's fully scripted, seeded, and documented is the template for every artifact you'll ship. In 2026 a clean reproduction is itself a publishable, valued contribution.

The technical heart of the craft. A result is only worth as much as its design.

• —One independent variable. Vary the method or the data or the eval — never several at once, or you can't attribute the effect.
• —The right baseline. Almost always a same-size / same-budget baseline, not a giant model. "My method beats a model 100× bigger" usually means you picked the wrong baseline.
• —Seeds. ≥3 seeds for any comparison you'll report. A single-run delta is often just noise; report variance, not a point.
• —Ablations. To claim component X matters, show the result with and without X, everything else fixed.
• —Controls for confounds. Match params and FLOPs; control for implementation quality; keep calibration and eval sets disjoint.
• —Pre-register the question. Decide what would confirm or refute your hypothesis before you run it — the antidote to moving the goalposts to wherever the data landed.

Assume your metric is lying until proven otherwise. Check for contamination, prompt sensitivity, and the gap between the metric and the behavior you actually care about. Prefer comparative/pairwise judgment over absolute scores for taste-based work. Never let a system grade its own output.

A dated log (LOG.md or daily notes) is the highest-ROI habit in this whole roadmap. Each entry: what I tried, what happened, what I learned, what's next.

• — It's how you avoid re-running failed experiments and re-deriving conclusions you already reached.
• — It's the raw material of every writeup — a paper/blogpost is mostly a cleaned-up log.
• — It externalizes your thinking so you can see your own reasoning errors.
• — After compaction-of-memory (yours, biological), the log is what survives.

start it in phase 0. never skip it.

Unwritten research barely exists. Writing is not a final step — it's a thinking tool that exposes the holes in your understanding. If you can't write it clearly, you don't understand it.

• —The artifact writeup is the deliverable. Not the code alone: what question, what you did (reproducibly), what you found, what the limitations are. Stating limitations honestly builds far more credibility than overclaiming.
• —Teach to learn, then animate it. After you implement a concept, build a 20–40s animated explainer of it ("what backprop computes," "why FlashAttention saves memory"). Moving the pieces on screen forces the understanding, and the clip doubles as your proof-of-understanding artifact, a learn-in-public post, and storytelling reps. Tools: SVG/CSS for simple concepts, Manim for real math visualization. One rule: animate after you've built it, never instead of building it.
• —Clarity over polish. Reproducible numbers and honest framing beat elegant prose. The bar is open weights + a reproducible eval + honest baselines, not rhetoric.

• —Seek harsh feedback early. Show half-finished work to people who'll tell you it's wrong. Praise is useless; the person who finds your confound is doing you a favor.
• —Adversarial self-review. Before believing your own result, try to refute it. What's the most likely reason it's wrong?
• —Taste is pattern-matching, built by volume. Hundreds of papers read, dozens reproduced, a few bad experiments that burned you. You can't shortcut it; you can only accelerate the loop — faster, and in public.