Statistical vs. Clinical Significance

The 0.05 line can make a tiny effect look important

A research abstract may say a result was statistically significant because the p value was below 0.05. That line became famous because many journals, researchers, and readers treat it as a pass or fail mark. But the p value does not tell you whether the effect was useful, impressive, or worth paying for. It answers a narrower question: if there were truly no effect, how surprising would this result be? The American Statistical Association has warned that p values are often stretched far beyond what they can prove.

Here is the surprise: a change can be statistically significant and still be too small to notice. If a sleep supplement improves sleep time by 4 minutes in a large trial, the study may have enough people to show that the 4 minute difference is unlikely to be random noise. That does not mean a tired person would feel better the next day.

Two different questions, not two levels of proof

Statistical significance is a signal detection question. It asks whether the study found a pattern that probably was not produced by random chance alone. The usual shorthand is a p value, often written as p < 0.05. This means the result crossed a commonly used cutoff, not that the finding has a 95 percent chance of being true.

Clinical significance is a human impact question. It asks whether the size of the change matters in real life. That may mean less pain, fewer symptoms, better movement, improved sleep, lower risk, or a change large enough that a clinician or consumer would make a different decision. Many fields use a minimum clinically important difference, often shortened to MCID. That means the smallest change patients would usually notice or value.

The bridge between the two is effect size, which means how large the change actually was. A p value tells you how confidently a study detected something. Effect size tells you how much changed. Confidence without size can mislead.

Read the size before the celebration

The strongest one decision is this: when you see “statistically significant,” find the actual difference before believing the claim. For supplements, that may be milligrams per deciliter of a blood marker, minutes of sleep, points on a symptom scale, or number of colds per season. If the paper or product page only shows p < 0.05 and hides the size of the effect, treat the claim as incomplete.

This does not mean small effects are always useless. A small blood pressure reduction across millions of people can matter for public health. A tiny improvement in one person’s mood score may not. Clinical significance depends on context: baseline risk, cost, side effects, burden, and what outcome the person actually cares about. Good medical reporting separates “we detected a difference” from “this difference changes lives.” CONSORT trial reporting guidance pushes researchers to report both estimated effects and uncertainty, not just whether a result crossed a p value line.

The takeaway is simple: statistical significance can open the door to belief. Clinical significance decides whether the result deserves action.