Why the gender gap on physics assessments?

Researchers are lost on the question of why women consistently score lower than men on assessments of conceptual understanding of physics. Previous research claimed to have found the "smoking gun" that would account for the differences, but new researchers claim that their new synthesis of past research has shown that there is no pattern to be found.

"These tests have been very important in the history of physics education reform," said S. B. McKagan, who co-authored the new analysis. Past studies have shown that students in classrooms using interactive techniques get significantly higher scores on these tests than do students in more traditional lecture settings; "these results have inspired a lot of people to change the way that they teach," said McKagan. But several studies had also reported that women's scores on these tests are typically lower than men's. Lead author Madsen said, "We set out to determine whether there is a gender gap on these concept inventories, and if so, what causes it."

But what problem are these researchers arguing over anyway? Are they thinking that all men and women on average should understand physics equally? Another question that this raises is: do they think that these tests accurately measure understanding of physics? Well in order to keep this essay short, I'll ignore the question of whether or not the tests accurately measure what it's creators claim they measure. So that still leaves the question: why should men and women understand physics equally?

To illustrate that this is the wrong question, consider two individuals, one that loves physics and doesn't particularly like art and another that isn't fond of physics but loves art. Should these two people be expected to learn physics equally? Should they be expected to learn how to draw equally?

Interest drives learning. Everybody knows this yet somehow these researchers ignore it. When somebody is interested in a subject, he spends a lot of time thinking about it, enjoyably. But without interest, the person wouldn't think much on that subject. And trying to do so in spite of lack of interest is not enjoyable at all. It's painstaking. And that's precisely why lack of interest is a barrier to learning.

So why are these researchers thinking that lumping all women together and all men together is the correct way to figure out what's going on here? Do they think that interest in physics by women on average should be equal to interest in physics by men on average? Does that even make sense? I think these researchers are completely ignoring the concept that interest drives learning, and that differences in interest causes differences in learning. So then the question is: why are there differences in interest between men and women?

To be clear, even if we restrict the study only to people who claim to be interested in physics, that doesn't mean that they all are interested equally. Interest is not a 0 or 1 phenomenon. It's not "I am interested" or "I'm not interested." It comes in degrees. And more importantly, interest is not associated with fields of study, and instead it's associated with specific ideas. So a person might be interested in how mechanical things work, but not interested in how light works, both of which are physics. This means that self-proclomations of interest in physics are not useful indicators of interest in specific ideas within the field of physics.

Further, a person's interests compete with his other interests. So a person might be interested in seemingly everything, but because he's interested in so many things, he finds it tough to focus on any one thing for very long. But as soon as this person satisfies some of his other interests, he might come full circle back to older interests that he hasn't focussed on in many years.

Back to the question of differences in interest between men and women, I'd like to consider a more primary question: why make this arbitrary division by gender? Why not divide by race? Should we expect that all races on average should have equal interest in physics? What about dividing by culture? I suspect that these researchers would think that dividing by race or culture wouldn't make sense because there are huge differences in background knowledge among the groups. But then that raises the question: why should we think that men and women in US schools, whom theoretically receive the same education opportunities on average, should have the same background knowledge? Well, men and women don't share the same background knowledge. Boys and girls are raised differently by their parents, and society treats them differently, so girls grow up with different background knowledge than boys. And it's these differences in background knowledge that result in differences in interest, which then results in differences in learning.

Just consider the two hypothetical individuals from before. One loves physics and the other loves art. The question is: Why do they love different things? Is it that there are differences in genes between them that cause differences in interest? Or are the interests learned?

Even if genes are a factor then isn't it possible for the X chromosome and the Y chromosome to contain some genes that affect interest in things? And if this is the case, then what would the researchers be looking for exactly? If it's possible that men's greater interest in physics is due to a gene on the Y chromosome, which women do not have, then whatever those researchers are looking for could easily be drowned out by this gender-specific gene difference. So even if some data "emerged" from the analysis, there is no way to know whether that identified variable is the cause or if the cause is actually a gender-specific gene. So why do the researchers think that the answer they are looking for would emerge from the analysis?

If it's a genetics issue, then the researchers are looking in the wrong places. And if it's not a genetics issue, well then we should be talking about the cultural differences between men and women as a factor in why they learn physics differently. And again, analogous to the genetic question, if it's a cultural issue, then researchers are looking in the wrong places again. Women and men are part of different subcultures.

So what's going on here? Why are these researchers looking in the wrong places? What are they doing wrong? Well this has already been answered decades ago by Karl Popper, a philosopher of science. He explained that many scientists aren't doing science. Doing science means creating a testable theory, and then testing that theory. What these researchers are doing instead is sifting through data looking for theories to "emerge", and never actually doing any tests that could possibly rule out a theory. It's a problem of scientific methodology.

Popper taught us that not all science is being done right. We need to be selective in figuring out what is good science and what just looks like science. To address this dilemma, he created his Line of Demarcation to separate science from non-science. In summary: a theory is scientific if and only if it can (in principle) be ruled out by experiment. So that means that if a theory cannot be ruled out by experiment, then it's not scientific -- instead it is scientism, stuff that looks like science but isn't because there is no way to rule out (by experiment) the theories being hypothesized.

So the way to test whether or not a theory is scientific is to ask yourself, “what would it take to make this theory false?” If the answer is nothing, then it isn’t science.

These researchers are assuming that there is a scientifically-measurable difference between men and women that should account for the dissimilar testing results. They are sifting through data hoping for the theory (the "smoking gun") to jump out at them. But this is backwards. Where is the part about creating an experiment that could rule out the theory? It's not there. They aren't even thinking about it. This is not science. It is scientism.

For more on the Line of Demarcation, see _Conjectures and Refutations_ (Chapter 11: The Demarcation Between Science and Metaphysics), by Karl Popper, or see the more recent and easier to read _The Beginning of Infinity_ (Chapter 1: The Reach of Explanations), by David Deutsch.

-------------------------------------------------------------------------------

Citation:

"The gender gap on concept inventories in physics: what is consistent, what is inconsistent, and what factors influence the gap?" A. Madsen, S. B. McKagan and E. C. Sayre, Physical Review Special Topics – Physics Education Research.