Browse the site

Start with a goal 40 things people take supplements for — sleep, stress, joints, focus.
Home Get the app
  • Methodology
  • robust evidence
  • 4 min · 3 sources
  • Published Apr 13, 2026

Publication Bias

Studies with positive results are more likely to get published.

It can make evidence look stronger and more reliable than the full research record really is.

Also known as file drawer problem · small-study effects · missing studies bias · bias due to missing results

Publication Bias in brief

  • Positive findings are more likely to be published, indexed, and noticed than null findings.1
  • Systematic reviews and meta-analyses are most vulnerable when unpublished studies stay invisible.
  • Publication bias differs from reporting bias, which hides some outcomes inside published papers.

What Publication Bias means

The standing ovation problem

Imagine judging a music festival after hearing only the songs that got encores. The flops happened too, they just never reached the stage. That is the trap behind publication bias. In real research, studies with dramatic, statistically significant, or tidy results are often more likely to be submitted, accepted, and cited than studies finding little or nothing.

The surprise is that publication bias is not mainly a flaw inside one study. It is a distortion of the lineup. A single trial can be perfectly well run, but if similar trials with dull or non-significant results stay buried in a file drawer, the published literature starts to clap for an effect that may be smaller, shakier, or sometimes absent.

Why meta-analyses are especially vulnerable

This is why publication bias in meta-analysis gets so much attention. A meta-analysis is supposed to combine the whole body of evidence. But if the available body is missing ribs, the final skeleton is crooked. PRISMA 2020 explicitly treats missing studies or missing results as a risk of bias in the synthesis itself, not just a footnote about inconvenience.

A classic clue is the funnel plot. In a healthy evidence base, big precise studies cluster near the true effect, while smaller studies scatter more widely, making an upside-down funnel. If one side of that funnel looks oddly hollow, often where small studies with disappointing results would be, reviewers worry about publication bias or other small-study effects. But this is where people overreach: an uneven funnel plot does not prove publication bias by itself. Real differences between studies, random scatter, and measurement choices can also bend the shape.

Publication bias is not the same as reporting bias

A helpful distinction: publication bias means whole studies are less likely to appear because of their results. Reporting bias is broader. A study may get published, but only the favorable outcome gets highlighted while an unfavorable outcome stays out of the paper. So the first problem is missing songs from the concert; the second is a published album with the worst tracks quietly removed.

One decision that improves your reading today

If you read a systematic review, whether it is about antidepressants, publication bias psychology findings, or a supplement ingredient like ashwagandha, do not stop at the pooled effect size. Scroll to the part on publication bias, funnel plots, or bias due to missing results. If the review has only a handful of small studies, an asymmetrical funnel, or no serious search for unpublished evidence, read the conclusion as more fragile than it sounds. That one move will protect you from treating a loud literature as the same thing as a complete literature.

Deep dive

How it works

In meta-analysis, publication bias often interacts with precision. Small studies have wider statistical scatter, so if journals preferentially publish the small studies that happen to land on the 'exciting' side of the result, the literature becomes enriched for exaggerated effects. That is why funnel-plot methods look for asymmetry across study size or standard error rather than inspecting any one paper in isolation.

When you'll see this

The term in the wild

  1. You open a systematic review on ashwagandha for stress and see that most included trials are small, positive, and industry-linked, with a note about possible funnel plot asymmetry.

    That does not mean the supplement does nothing. It means the published record may be tilting toward the studies most likely to show benefit, so the pooled effect may look cleaner than the full hidden evidence would.

    This is the difference between 'promising' and 'settled', a useful guardrail before you overspend or overclaim.

  2. A psychology meta-analysis reports a strong average effect, but the authors also say unpublished dissertations and conference abstracts were hard to locate.

    That is a publication bias warning sign. Psychology has long discussed the file drawer problem because null findings often remain harder to discover than journal articles.

    The headline number may reflect what was easiest to publish, not the true average effect across all attempts.

  3. In a PubMed search, you find ten upbeat trial papers on a therapy and almost no null results, even though the topic has been studied for years.

    PubMed is excellent, but it mostly shows what reached publication and indexing. Publication bias can therefore survive even when your database search feels thorough.

    A careful reviewer will also look for trial registries, dissertations, preprints, and other grey literature.

Myths vs reality

What people get wrong

  • Myth

    Publication bias means the published studies are fraudulent or low quality.

    Why people believe it People hear the word 'bias' and assume it describes a flawed experiment rather than a distorted collection of experiments.

    Reality

    No. Many published studies are competently done. The bias comes from who made it onto the shelf, not automatically from bad craft inside each paper.

  • Myth

    A lopsided funnel plot proves publication bias.

    Why people believe it Textbooks and review papers often teach funnel plots as the standard visual check, and the image is so intuitive that readers mistake a clue for a verdict.

    Reality

    A funnel plot is a smoke pattern, not a fingerprint. Missing studies can create it, but so can real differences between studies, chance, or the way effects were measured.

  • Myth

    If a study is published, reporting bias is no longer a concern.

    Why people believe it PRISMA 2020 had to explicitly separate 'missing studies/results' from other bias domains because readers and authors often collapse them into one vague problem.

    Reality

    A paper can reach print and still hide disappointing outcomes. Publication bias hides whole studies; selective non-reporting can hide parts of studies that did get published.

Putting Publication Bias to work

Specific failure mode to avoid: do not treat 'there are 12 published studies' as proof of a mature evidence base. Twelve tiny, positive studies with no registry checks can give you a more distorted picture than four larger preregistered trials.

Frequently asked

Common questions

Can you give an example of publication bias in research?

Suppose twenty teams test the same ingredient, but only the five studies with clear positive results reach journals. A reviewer then summarizes those five and gets an inflated impression because the null fifteen are effectively invisible.

Why is publication bias a problem?

Because it can make effects look larger, more consistent, and more trustworthy than they really are. That can skew treatment decisions, policy, and product claims, especially when people rely on meta-analyses.

How does publication bias differ from selection bias?

Selection bias usually refers to how participants or studies are selected into a study or analysis in ways that distort results. Publication bias is narrower: studies are selected into the published literature based partly on their findings.

How is publication bias different from reporting bias?

Publication bias hides entire studies from view. Reporting bias means a study is visible, but some outcomes, time points, or analyses are left out or downplayed.

Can reviewers fix publication bias completely?

Not completely. They can reduce it by searching trial registries, dissertations, conference abstracts, preprints, and other grey literature, but they still cannot recover studies that were never registered or never shared.

Sources

Sources

  1. 1. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021)
  2. 2. Reproducibility and Replicability in Science (2019)
  3. 3. The Perils of Misinterpreting and Misusing 'Publication Bias' in Meta-analyses: An Education Review on Funnel Plot-Based Methods (2024)

Related

Where this term shows up

Evidence guides and other glossary entries that touch this concept.