When the sample lies: evidence fallacies

Definition

These three fallacies corrupt statistical and inductive reasoning before the inference even begins. Cherry-picking keeps only the favorable facts; a biased sample asks the wrong slice of the group; survivorship bias studies only what made it through a filter. Each produces evidence that is true, piece by piece — and a picture that is false.

Memory hook

“Before trusting a sample, ask what never made it in.”

What it sounds like

• Just look at these five glowing reviews. (cherry-picking)
• Everyone I asked at the harbor agrees. (biased sample)
• They built things to last in the old days — look at these old houses. (survivorship bias)