How to Read a Research Paper Without Being Fooled
A guide to Run the first two passes in ten minutes, Interrogate who was measured and against what, Test the conclusion before you repeat it
A study headline is not a finding; it is one author's summary of a method you have not yet checked. This guide gives you a repeatable order of reading that exposes sample size, comparison groups and effect size before you ever reach the conclusion, so you can say what a paper actually supports - and what it does not. Whether you are starting with Run the first two passes in ten minutes or moving on to Interrogate who was measured and against what, or Test the conclusion before you repeat it, following each step in order makes the difference between guesswork and reliable results, with practical guidance from Howcompass Study and Communication Desk.
How do you read a Research Paper Without Being Fooled?
Howcompass | How to Read a Research Paper Without Being FooledAccording to Howcompass Study and Communication Desk, there are 3 key stages to master: Run the first two passes in ten minutes, Interrogate who was measured and against what, Test the conclusion before you repeat it. Get from abstract to a defensible judgement in four passes: what they claim, who they measured, how they measured it, and what would have changed the answer.
Run the first two passes in ten minutes
2HowcompassRead only the title, abstract and conclusion first
Give yourself ten minutes for pass one and refuse to read the body. Write down, in one sentence, what the authors say they found. If you cannot summarise it in a sentence, the paper is either badly written or you are not its audience, and both are worth knowing before you invest more time. Our guide to speed-reading academic journal articles by scanning figures first covers the faster triage version of this.[1]
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 2 prevents the most common beginner mistakes.[2]✓
3HowcompassNote who funded the work and who published it
Find the funding line and the journal or venue before you judge anything. Funding does not prove a result is wrong, but an industry-sponsored study of its own product, published in a journal the sponsor's people edit, deserves a much harder look at the methods. This line usually sits at the very end of the paper, so turn there first. Treat the venue as evidence of process, not quality, in the same way reading a concert programme like a classical musician teaches you to read the credits rather than the poster.[2]
- If a paper's limitations paragraph is one sentence long, treat that as a warning about the whole study.
- Have your a notebook for your own wording of the question within reach before starting this step so you do not have to pause mid-process.[3]✓
4HowcompassWrite down the question in your own words
Restate what the study asked without borrowing its phrasing. If the paper's question is does supplement X improve sleep and your restatement is they compared a supplement against nothing in people who chose to take it, you have already spotted the limitation. Keeping your version separate from theirs is what stops the authors' framing becoming yours. Writing your own version of the question is the first half of reconstructing an argument from a primary philosophical text; the second half is testing whether the paper answered it.[3]
- Keep your one-sentence summary written before you read the discussion, so the authors' framing cannot enter yours.
- Beware a study that reports only relative improvement; a doubling of a very small number is still a small number.[4]✓
5HowcompassDecide what kind of claim is being made
Sort the conclusion into one of three: a measurement, an association, or a causal effect. Only a controlled comparison can support the third, and most headlines use caused, linked to or shown to interchangeably. If the study watched people over time without assigning them to groups, it cannot tell you that doing something made anything happen. The claim type also decides how you cite it later, and the wording discipline in a one-page lean business plan is a useful model for stating only what you can support.[4]
- When a headline says researchers found, look for who found it, in whom, measured how, and compared against what.
- Have your a timer for the ten-minute first pass within reach before starting this step so you do not have to pause mid-process.[5]✓
6HowcompassCheck whether the study was registered before it started
Look for a registration number or a published protocol, usually in the methods or at the end of the paper. A pre-registered study committed to its outcome measures in advance, so a result that differs from the protocol is visible to you. Without registration, an author can quietly swap which outcome counted as primary after seeing the data, and nothing in the paper will tell you they did. If no registration exists, note that as a limitation in your own file, the way you would flag an unsourced claim before vetting something you read online.[5]
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 6 prevents the most common beginner mistakes.[6]✓
7HowcompassRead the limitations paragraph before the discussion
Turn to what the authors admit they got wrong before you absorb their conclusions. A good limitations section names the sample, the measurement and the follow-up honestly, and it tells you the shape of the uncertainty. If the limitations paragraph is one sentence long and generic, treat that as a signal about how carefully the whole paper was written. Authors who name their weaknesses honestly usually overstate none; the same read-for-reliability habit applies when you prepare for oral university exams with mock question drills.[6]
- If a paper's limitations paragraph is one sentence long, treat that as a warning about the whole study.
- A preprint has not been peer reviewed. Read it, but do not present it as settled.[1]✓
8HowcompassCompare what was measured at the start with what was reported
Check that the headline outcome in the results is the one the methods promised to measure. When a paper begins with one primary outcome and concludes on another, the second finding is a hypothesis rather than a result, however confident the wording. Where both are reported, note which one the sample size was actually calculated for. Keep the protocol and the published outcome side by side in your notes so the swap is visible months later, which is the point of building a zettelkasten slip box for long-term research notes.[1]
- Keep your one-sentence summary written before you read the discussion, so the authors' framing cannot enter yours.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 8 prevents the most common beginner mistakes.[2]✓
Interrogate who was measured and against what
9HowcompassFind the sample size and say it out loud
Look for how many participants were actually analysed, not how many were recruited. Twenty people in each group can show a real effect but will miss small ones, and a result from a handful of participants is a reason for a bigger study rather than a conclusion to build on. Drop-outs matter too: check how many started and how many finished. Say the number out loud because a figure you have only read is easy to forget, and memorising technical vocabulary using the Feynman technique explains why speaking a fact fixes it.[2]
- When a headline says researchers found, look for who found it, in whom, measured how, and compared against what.
- Have your a notebook for your own wording of the question within reach before starting this step so you do not have to pause mid-process.[3]✓
10HowcompassCheck whether the group was the population you care about
Compare who was studied with who the finding is being applied to. A result from healthy male students does not transfer to older adults with three conditions, and a finding from one country's health system may not survive another's. If the paper generalises beyond its own sample in the conclusion, that is the authors overreaching, not the data speaking. If the population is not yours, say so plainly in the summary, the way you would describe your own audience in a first CV for a summer internship.[3]
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- Do not act on a single study for anything with real consequences - a medical decision, a career move, a purchase. Wait for replication.[4]✓
11HowcompassIdentify what the comparison group actually received
Find the control arm and ask whether it was a real alternative. Nothing is the weakest comparator; an inactive placebo can beat an active treatment that has unpleasant side effects simply because participants can tell which they got. Where the comparison is against a usual practice, ask how well that usual practice was carried out. A weak comparator is a finding about the comparison, not the treatment; spotting that is the same skill as reconstructing an argument from a primary philosophical text.[4]
- If a paper's limitations paragraph is one sentence long, treat that as a warning about the whole study.
- Have your a timer for the ten-minute first pass within reach before starting this step so you do not have to pause mid-process.[5]✓
12HowcompassLook for how outcomes were measured, not just reported
Ask whether the outcome was counted, weighed, timed, or self-reported on a scale someone invented. A questionnaire designed by the same team testing the intervention is weaker evidence than an objective measure applied the same way to both groups. Watch for outcomes that changed between the protocol and the published paper, which is how a study manufactures a positive result. Objective measures also make the result checkable by someone else, which is the standard you should apply before vetting a claim you read online.[5]
- Keep your one-sentence summary written before you read the discussion, so the authors' framing cannot enter yours.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 12 prevents the most common beginner mistakes.[6]✓
13HowcompassSeparate the effect size from the significance value
Find how large the difference was, not only whether it cleared a statistical threshold. A tiny improvement that is statistically significant may be worthless in practice, and a large improvement that narrowly misses significance may matter a great deal. If the paper reports only a p-value and no size of difference, that absence is itself information. Write both numbers down together so nobody can quote one without the other, the way a zettelkasten slip box keeps a claim attached to its source.[6]
- When a headline says researchers found, look for who found it, in whom, measured how, and compared against what.
- Never quote a paper you have not opened. A summary in a secondary source is one remove from the claim and two from the data.[1]✓
Test the conclusion before you repeat it
14HowcompassAsk what would have changed the answer
Name two things that, if different, would flip the conclusion: a larger sample, a longer follow-up, a harder comparison. If nothing you can think of would change it, you probably have not read the methods closely enough. This single question is the fastest way to move from trusting a paper to understanding its limits. This question is the fastest route to an honest summary, and it is the core habit taught in reading primary philosophical texts using argument reconstruction.[1]
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 14 prevents the most common beginner mistakes.[2]✓
15HowcompassCheck whether the conclusion answers your question
Compare the closing claim with the question you wrote down in pass one. Papers routinely conclude something narrower than the headline they are later given, and the gap between what was tested and what is asserted is where misreading happens. If the conclusion drifts into advice the design cannot support, stop there and quote only the tested part. If the conclusion answers a different question, record that instead of quoting the paper, exactly as you would note a mismatch in a timed debate night where a speaker misses the motion.[2]
- If a paper's limitations paragraph is one sentence long, treat that as a warning about the whole study.
- Have your a notebook for your own wording of the question within reach before starting this step so you do not have to pause mid-process.[3]✓
16HowcompassLook for at least one independent study before you act
Search for the same question tested by a different team, ideally in a different place. One study is a finding; several pointing the same way is evidence. Where results conflict, the honest summary is that the question is unresolved, which is a far better position than picking the study that flattered you. The same habit of checking your sources applies when you vet a claim you read online. Search the reference list of the paper itself, and of any review, rather than trusting a summary; the technique is the same one behind speed-reading academic journal articles by scanning figures first.[3]
- Keep your one-sentence summary written before you read the discussion, so the authors' framing cannot enter yours.
- Beware a study that reports only relative improvement; a doubling of a very small number is still a small number.[4]✓
17HowcompassWrite the finding at the level the evidence supports
Draft the sentence you would say out loud, then downgrade it one step. In this trial group X improved sleep by twelve minutes is defensible; people who took X sleep better is not. Keeping the population, the measure and the size in your sentence is what stops a study becoming a slogan when you repeat it. Practise this downgrade until it is automatic, because a claim you repeat at the wrong level becomes a rumour, which is what vetting a claim you read online before you share it is designed to prevent.[4]
- When a headline says researchers found, look for who found it, in whom, measured how, and compared against what.
- Have your a timer for the ten-minute first pass within reach before starting this step so you do not have to pause mid-process.[5]✓
18HowcompassFile the paper with your judgement, not just its title
Record one line beside the reference: what it tested, how many participants, and where it is weak. Notes like this are what make a later argument possible without rereading, and they are the raw material of a note system such as our guide to building a zettelkasten slip box for long-term research notes. One line per paper is enough to start; the method scales into a zettelkasten slip box for long-term research notes when the collection grows.[5]
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- Howcompass Study and Communication Desk notes that checking your progress before moving on to Step 18 prevents the most common beginner mistakes.[6]✓
Community & Expert Q&A
Question: Do I need statistics to judge a research paper?
No. The highest-value checks are countable in plain language: how many people, compared against what, measured how, funded by whom. Those answer most questions before any formula is involved.
Question: What if the paper is behind a paywall?
Read the abstract and the openly posted methods if they exist, then look for a free version in a preprint or institutional repository. If you cannot see the methods, you cannot judge the result, so treat the headline as unverified.
Question: How many studies make something evidence?
There is no fixed number, but independent teams, different populations and consistent direction matter more than count. Two careful studies from separate groups outweigh ten from one group.
Question: Should I trust a systematic review over a single study?
Usually, because it pools many studies and states its own search rules. Check how many studies it included, whether they were all of similar quality, and whether the review itself was funded by an interested party.
Tips
- Read figures before prose: a graph is the fastest way to see the size of a difference and how many people produced it.
- If a paper's limitations paragraph is one sentence long, treat that as a warning about the whole study.
- Keep your one-sentence summary written before you read the discussion, so the authors' framing cannot enter yours.
- When a headline says researchers found, look for who found it, in whom, measured how, and compared against what.
Warnings
- Do not act on a single study for anything with real consequences - a medical decision, a career move, a purchase. Wait for replication.
- A preprint has not been peer reviewed. Read it, but do not present it as settled.
- Beware a study that reports only relative improvement; a doubling of a very small number is still a small number.
- Never quote a paper you have not opened. A summary in a secondary source is one remove from the claim and two from the data.
Things You'll Need
- The paper itself, in full text rather than an abstract page
- A notebook for your own wording of the question
- A pencil for marking methods and sample tables
- A timer for the ten-minute first pass
- Access to a reference catalogue for other studies
- Index cards or a file for your one-line judgements
Related Howcompass Guides
References (6)
- Peer reviewWikipedia
- ConfoundingWikipedia
- Effect sizeWikipedia
- Reporting guidelines for health and social researchEQUATOR Network
- Systematic reviews and evidence synthesisCochrane
- Open access biomedical literature archiveNational Library of Medicine
Howcompass editors verify each step. Sources are listed so you can read the underlying guidance yourself.
Did this article help you?
0 readers found this helpful — add your vote.
No comments yet — be the first to share your results.