How to Read Papers

One of the key skills for any academic researcher is reading other peoples' papers. The trouble is that our peers write papers faster than we can read them, and to fully understand a paper takes a long time, because each paper summarizes months to years of effort. So you cannot fully read all the relevant literature, even if you did nothing but reading. This means you need to have a way to prioritize papers based on their relevance to your interests. Here's how I do it.

1. Scanning. (time per paper: seconds to minutes) When a conference lists its newly accepted papers, or a Google search produces a list of links, you have to decide which of the dozens of possible papers you might want to look at. At this point, you are going through the papers, looking at their titles and abstracts, and categorizing the papers into "definitely not" and "maybe". Here, you aren't trying to assess relevance, you are trying to identify the papers which are definitely irrelevant to you.
2. Skimming. (time per paper: minutes to hours) When you skim a paper, you have judged (based on title and abstract) that it may have some relevance to your interests. So now, you can fully read the introduction, the examples, the conclusions, the related work, and anything with more English than equations or code. Basically you are trying to figure out what was actually done, so that you can conclude whether it actually is something you find interesting or relevant.
3. Reading. (time per paper: hours to days) If you've decided a paper is probably worth reading, then it's something that is definitely relevant to your interests. Now it's time to figure out what they actually did. I tend to work in the more mathematical regions of semantics, so this is the stage at which I try to make sure I understand all the definitions and theorems in the paper. The goal is to fully understand every definition and equation which occurs in the paper, and to understand the dependency graph of theorems which leads to the final results. I'm not trying to understand the proofs of the theorems, but I am trying to understand exactly what was done, and what techniques were used in the doing.
4. Reproducing. (time per paper: days to weeks) If something is relevant to your interests, and is using a method that you want to use yourself, then it's time to sit with the paper and try to fully assimilate it. This is the point at which you sit down and try to prove all the theorems yourself, so that you have a full understanding of everything in the paper. Your goal is to reach the point where you could sit down and write a similar paper yourself. Ideally, the paper has an accompanying technical report with all the proofs, which makes this task a whole lot easier when you get stuck, but often they don't, and so you still have to sit down and prove things.

My time is probably spent about equally on these four levels. This means I have understood perhaps three to six papers a year well enough to reproduce them, have read to understanding several dozen papers, skimmed several hundred papers, and scanned probably thousands of abstracts each year.