Simpson’s Paradox
What hospital to go to: A or B? Well, hospital A’s patients survive 90% of the time and hospital B’s survive 10% of time.
Clearly hospital A is the right choice. Well, not necessarily. Hospital B’s patients could be a much sicker group to begin with. Should you buy stock in company A or company B? Well, A has higher margin and faster growth, So clearly company A. Well, not necessarily. Company A also has too much debt and its growth is all from acquisitions. Company B’s growth is organic and its balance sheet is debt free. Similarly, is there gender discrimination in at college x. Well, say 60% of women who apply are admitted while 90% of men are. Clearly there is discrimination, right? Not necessarily. Perhaps the women are applying to more competitive departments. It is extraordinarily easy to come to the wrong conclusion based on incomplete data. Demagogues love twisting your emotions with data that sound compelling when critical, granular data is omitted.
THE MOST FAMOUS EXAMPLE (from Wikipedia)
UC Berkeley gender bias
One of the best-known examples of Simpson’s paradox is a study of gender bias among graduate school admissions to University of California, Berkeley. The admission figures for the fall of 1973 showed that men applying were more likely than women to be admitted, and the difference was so large that it was unlikely to be due to chance.[14][15]
| Applicants | Admitted | |
|---|---|---|
| Men | 8442 | 44% |
| Women | 4321 | 35% |
But when examining the individual departments, it appeared that six out of 85 departments were significantly biased against men, whereas only four were significantly biased against women. In fact, the pooled and corrected data showed a “small but statistically significant bias in favor of women.”[15] The data from the six largest departments is listed below.
| Department | Men | Women | ||
|---|---|---|---|---|
| Applicants | Admitted | Applicants | Admitted | |
| A | 825 | 62% | 108 | 82% |
| B | 560 | 63% | 25 | 68% |
| C | 325 | 37% | 593 | 34% |
| D | 417 | 33% | 375 | 35% |
| E | 191 | 28% | 393 | 24% |
| F | 373 | 6% | 341 | 7% |
The research paper by Bickel et al.[15] concluded that women tended to apply to competitive departments with low rates of admission even among qualified applicants (such as in the English Department), whereas men tended to apply to less-competitive departments with high rates of admission among the qualified applicants (such as in engineering and chemistry).