Scroll through any supplement marketing page and you'll see phrases like "clinically studied" and "backed by research." Both can be technically true while the actual evidence is weak enough to be useless.
The gap between "there's a study" and "there's *good* evidence this works" is enormous. Supplements are one of the industries where understanding study design pays off most — because the financial incentive to publish favorable results is huge.
This lesson teaches you to read supplement studies like a scientist, not a customer. Five minutes of scrutiny can save you thousands of dollars on compounds that don't do what their marketing claims.
Learning Objectives
- •Rank different types of studies from most to least reliable
- •Distinguish effect size from statistical significance
- •Spot common red flags in supplement research (industry funding, surrogate endpoints, tiny sample sizes)
- •Decide whether a claimed benefit is likely real, likely hyped, or somewhere in between
The Evidence Hierarchy
Not all evidence is equal. From strongest to weakest:
- Systematic review / meta-analysis — pools data from many studies, usually the highest bar. Look for ones registered on PROSPERO or published in Cochrane.
- Randomized controlled trial (RCT) — people are randomly assigned to the supplement or placebo. Double-blinded RCTs in humans are the workhorse of good evidence.
- Cohort / observational study — tracks people over time but doesn't assign treatments. Can show correlation, not causation. Prone to confounding (e.g., "vitamin D users live longer" may just mean "healthier people take vitamin D").
- Case-control study — compares people with an outcome to people without. Useful for rare outcomes, weaker for supplement efficacy.
- Animal / in vitro — a compound works in a petri dish or a mouse. About 90% of these results do not replicate in humans.
- Anecdote / testimonial — the weakest possible signal. Influencers' "I tried X and my sleep improved" is not evidence.
The supplement industry has mastered the art of citing animal and in vitro studies for compounds that have never been tested in humans.
Effect Size > P-Value
A study can be "statistically significant" (p < 0.05) and still describe a change too small to matter. Conversely, a medium-sized study can show a dramatic improvement that misses statistical significance — and that's often the more interesting finding.
Effect size is how *big* the change is. A supplement that reduces blood pressure by 0.5 mmHg (with p < 0.001) is statistically real but clinically irrelevant. One that reduces it by 8 mmHg is clinically meaningful.
When reading a study, always ask:
- How large is the effect?
- Is it clinically meaningful (not just mathematically detectable)?
- Would a person actually notice it?
Marketing will often quote "statistically significant" without telling you the effect was tiny.
Resveratrol: A Case Study in Hype
Resveratrol extended lifespan in obese mice in a famous 2006 paper. It was marketed as the 'anti-aging molecule' and built into hundreds of products. Over the next decade, human studies repeatedly failed to show meaningful benefit, and the compound's poor oral bioavailability emerged as a core problem. The lesson: a dramatic animal result is a hypothesis, not a conclusion. Wait for human RCTs.
'Clinically studied' on a label means the product works.
'Clinically studied' can mean anything from 'a strong human RCT' to 'a 12-person uncontrolled open-label trial funded by the manufacturer.' It's a legally vague phrase. Check the actual study: who funded it, how many people were in it, was there a placebo group, was it blinded?
Industry-Funded Studies Need Extra Scrutiny
Manufacturer-funded supplement trials show positive results roughly 3-4x more often than independently funded trials of the same compound. This isn't necessarily fraud — it's a combination of publication bias, outcome selection, and study design. Independent replication is the gold standard. If every positive study is funded by the seller, treat it as a hypothesis until someone else confirms it.
Quick Check
A supplement ad cites a 'double-blind placebo-controlled clinical trial showing statistically significant improvements in cognition (p = 0.04, n = 22).' Which is the biggest concern?
Summary
- →Evidence hierarchy: meta-analysis > RCT > cohort > case-control > animal > anecdote
- →Animal and in vitro results are hypotheses — ~90% do not replicate in humans
- →Statistical significance tells you the effect is probably real; effect size tells you whether you'll notice it
- →Industry-funded studies show positive results far more often than independent ones — demand replication
- →'Clinically studied' on a label is a marketing phrase, not a quality guarantee
Next lesson: the major longevity pathways — mTOR, AMPK, sirtuins, autophagy — and the supplement classes that try to modulate each one.