Wednesday, March 8, 2017

Refutation of Summers' Hypothesis for the CS Gender Gap

Summers' Hypothesis is a widely-cited hypothesis purporting to explain gender spreads in academic/occupational fields, especially STEM fields. The idea is that gender spreads are driven by difference in variance of individual intelligence. Specifically, intelligence variance is higher among males, meaning that more males have either very high or very low intelligence, even though average intelligence is roughly the same across genders.

(The hypothesis is named for Harvard president and US Treasury secretary Larry Summers, who became a liberal pariah shortly after floating the hypothesis in public.)

The key word here is variance. I’ve seen lots of “refutations” of Summers' hypothesis which take a bunch of IQ data, and show that the average is the same (or at least very close) between the two groups. But that’s not actually Summers' hypothesis: the hypothesis states that the variance is different, and that difference explains the gender gap. I’ve never seen any popular media present a correct refutation of the hypothesis, so that’s what we’re going to do here.

We’ll focus on the gender gap in computer science. We’ll compute what gender gap we’d expect based on Summers' hypothesis, then compare that to the real gender gap.


Down to business. To start off, we need three key numbers: IQ variances for males and females, and average IQ for computer scientists.


The average IQ for computer scientists is fairly straightforward: SAT scores do a good job of measuring IQ, and there’s data out there on SAT scores by major. In fact, people have even crunched the numbers already! We’ll use the IQ-by-major estimates here; this source lists an average IQ of 124 for computer and information science majors.


IQ variance for males and females is trickier: it’s been the subject of considerable debate thanks to Summers' hypothesis, so of course people of various political stripes have published heavily-biased “studies” and arguments trying to prove their views. I’ll pull from this study. I like this study for several reasons:

  • It uses sibling pairs, so lots of potential confounders are controlled for 
  • The sample size is large (~1200 sibling pairs) 
  • It draws from the US National Longitudinal Survey for Youth, so it’s fairly representative of the US population 
  • The authors are careful to address g-factor specifically 
In short, the study is really carefully done from a technical standpoint.
Anyway, that study found a male intelligence standard deviation about 1.11-1.16 times the female standard deviation, depending on the exact measure used. Also noteworthy: the males had significantly higher variance on all but two subtests. (Difference in mean intelligence was tiny, as expected.)

The next bit involves some math. I’ll omit the calculations, and illustrate what’s going on with a picture:



The picture shows two normal curves. (Intelligence isn’t normally distributed, but it’s a good enough approximation for our purposes.) The taller curve (blue) represents females - I’ve set its standard deviation to 15, which is the usual standard deviation for IQ. The flatter curve (green) represents males - its standard deviation is 1.16 * 15, reflecting the study above.

Right at the mean IQ of 100, the blue curve is noticeably higher - among a sample of people with IQ exactly 100, there should be more females than males (the exact calculation predicts about 16% more). The curves intersect somewhere between 115 and 120, and between 80 and 85. Around these IQ levels, the females and males are about even.


We saw that “computer and information science” majors have an average IQ around 124. At that level, we’d expect about 20% more males than females. Put differently, based only on IQ variance differences, we’d expect about 45.5% of computer and information science majors to be female.


Now, anyone in CS knows that “information science” is a very different field, and those information science folks… well, their reputation isn’t as strong. I suspect that may be dragging down the IQ estimate. So to double-check, I looked here and found an estimated average IQ of 128.5 for computer scientists. At that level, we’d expect about 42% females. Another important factor is that we’re setting female IQ standard deviation to 15 - if we instead set male IQ standard deviation to 15, then we get an estimate of 38% female. This is just a side effect of lazy back-of-the-envelope math; a more careful calculation would be somewhere between the 38% and 42% numbers.


Anyway, Summers' hypothesis seems to predict roughly 38%-45% females in CS, depending on calculation details. What fraction of computer scientists are actually female? According to payscale, computer science is 85% male, 15% female.


So, Summers' hypothesis? Not even close. Differences in IQ variance are nowhere near large enough to account for the gender gap in CS. Other STEM fields are left as an exercise to the reader.

3 comments:

  1. Hi just started reading your blog from the Slatestarcodex. You got some great content!
    One point tho, despite of what you might have heard Larry Summer's lecture was not that radical at all: he thought gender gaps (this is a loaded phrase incidentally) were due to a mixture of discrimination, differences in the variance, and differences in interests.
    As a indication of how long the latter can be, on 'people versus things' men and women differ a whopping 1 SD[1].

    [1] https://www.ncbi.nlm.nih.gov/pubmed/19883140

    ReplyDelete
    Replies
    1. I didn't want to devote space it it in the post, but there's significant divergence between stuff Larry Summers actually said, and the various IQ-variance-gender-gap claims which have his name attached these days. Personally, I think people give the guy more crap than he deserves, but the naming has stuck.

      Thanks for bringing that up, it's an important point which I'm glad to have here.

      Delete
  2. You didn't disprove summers' hypothesis, you just showed that it doesn't explain the entire gap. There are also sex differences in preferences (Smart women are less likely to be interested in CS than smart men).

    The means also aren't exactly equal. You only see equal means when the tests are pre-puberty or mostly verbal. On post-puberty non-verbal tests, like adult raven's progressive matrices scores, men do about half a standard deviation better than women.

    ReplyDelete