What is the Simpson's paradox?

The basic idea is that we assume just because we are comparing percentages we are comparing equal measures, but because the sample sizes are split differently, we aren't.

Look at it this way. You and I are going to the pub this Tuesday and Wednesday and we are going to play a game where we throw darts and try and hit the bulls eye.

On Tuesday you only throw the dart once, but you hit it. You now have 100% for that night. I throw the dart 99 times and hit the bulls eye 98 times. That would give me right around 99% accuracy. Looking just at those percentages without knowing how many times we both tried, it looks like you did better.

Now we come back Wednesday, this time though we switch, I throw the dart only once and I miss, leaving me with 0% accuracy on the night. You then throw 99 times, and hit the bulls eye 10 times, which gives you right around 10% accuracy on Wednesday. Again you seem to have won.

The trick is you really haven't. The data was just split weird, making it misleading. Really, over the course of two days, I hit the bulls eye 98 times out of 100, and you got only 11 out of 100.

+++

Simpson's paradox is best demonstrated graphically. Consider this scatter plot.

Overall the groups that received more treatment end up doing worse than the groups that received less treatment. But within each group more treatment gives better outcomes.

One possible cause is that group membership is correlated with both the amount of treatment and the outcome. For example, treatment could be chemotherapy and the groups could be based on how the cancer was detected (which affects how quickly you notice it). The treatment is helping, it's just that late-detections require more treatment and still don't do as well.

+++

You see this stuff so often in nutrition studies that it's ridiculous.

Example: People who consume red meat have lower life expectancy.

But then control for smoking, stress, and if the person has healthy lifestyle choices and you get something completely opposite.

Of course people who don't care about their health are not going to care about eating healthy, so they'll eat more of whatever. This includes red meat.

Another: Do runners enjoy a longer lifespan because of running or are they just more likely to be mindful of their health?

Or the worst is the titles you see on women's magazines: "Eat these foods to lose weight". Makes sense, eat calories to lose weight. I saw one saying you should eat X foods to increase apoptosis of fat cells. Autophagy/apoptosis occurs more frequently when you HAVEN'T eaten... Do those foods actually increase apoptosis, or are they simply fewer in calories making it more likely for apoptosis of fat cells to occur? Autophagy is also increased by exercise, so is it the food or is it health-minded people exercising more?