Friday, February 19, 2016

How A Star Is Born. A Study On Gender in How Biology Undergrads Rate Their Classmates' Knowledge

You can read the whole study here*.  It's an investigation into one type of possible gender discrimination:  Do our colleagues (either at work or at school) take gender into account when evaluating our performance?

The answer, from this study, is intriguing.  The Washington Post summarizes it like this:

Anthropologist Dan Grunspan was studying the habits of undergraduates when he noticed a persistent trend: Male students assumed their male classmates knew more about course material than female students — even if the young women earned better grades.
“The pattern just screamed at me,” he said.
So, Grunspan and his colleagues at the University of Washington and elsewhere decided to quantify the degree of this gender bias in the classroom.
After surveying roughly 1,700 students across three biology courses, they found young men consistently gave each other more credit than they awarded to their just-as-savvy female classmates.
Men over-ranked their peers by three-quarters of a GPA point, according to the study, published this month in the journal PLOS ONE. In other words, if Johnny and Susie both had A's, they’d receive equal applause from female students — but Susie would register as a B student in the eyes of her male peers, and Johnny would look like a rock star.

The researchers used data from three different classes of a fairly introductory biology course with large numbers of students.  In each class, students were asked to pick those of their fellow students (from class rosters) who seemed to show the greatest command of the material that was being taught.  These picks were repeated several times during each lecture series, and constitute the data the rest of the study analyzes.

As one might expect, not every student in those large classes got picked equally often as showing great command of the material.  Certain individuals got many more votes.  The researchers call them "the celebrities."

How does one become such a celebrity?

The objective basis in this context is actually having an extraordinary grasp of the material being taught.  But that is insufficient (and sometimes not even necessary) on its own.

Others also need to learn about that extraordinary grasp, either by observing the class participation of those celebrities, by observing them in the attached lab sessions or by learning about that grasp through friendships and other informal channels.  Perhaps some students tell others that they scored As in their exams?  Perhaps some students are much more vocal during the classes?

The crucial question here is how an impression of someone's excellence is created.  Is it based on just objective facts or is it also affected by various cultural prejudices?  Do students tell each other the grades they are earning?  Is the impression of excellence based solely on class participation, or does it matter who your friends are?  Does it matter if the potential celebrity is female or male?  Does it matter if the student doing the evaluating is female or male?

The study uses two fairly objective measures of someone's actual competence.  They are the student's exam grades and an evaluation of the students' class participation by the professors who taught the three courses.**  Note that the latter variable also measures one of the likely channels through which other students form their opinions.  Someone expressing smart opinions in the class will be assumed to have good command of the material, even if that person's examination grades are private information.

After controlling for those variables, the study finds a residual gender effect:

Results reveal that males are more likely than females to be named by peers as being knowledgeable about the course content. This effect increases as the term progresses, and persists even after controlling for class performance and outspokenness. The bias in nominations is specifically due to males over-nominating their male peers relative to their performance. The over-nomination of male peers is commensurate with an overestimation of male grades by 0.57 points on a 4 point grade scale, indicating a strong male bias among males when assessing their classmates. Females, in contrast, nominated equitably based on student performance rather than gender, suggesting they lacked gender biases in filling out these surveys. These trends persist across eleven surveys taken in three different iterations of the same Biology course. In every class, the most renowned students are always male. This favoring of males by peers could influence student self-confidence, and thus persistence in this STEM discipline.

In ordinary language, guys give other guys extra celebrity points for just guyness.  Gals, on the other hand, don't seem to be*** affected by the gender of the person they are evaluating.

How do we explain this?

The researchers explain their findings as implicit biases:

The finding that a gender bias impacts the perception of millennial students may at first seem surprising, but is supported by work on implicit biases. Implicit biases are unconscious associations that people hold related to certain groups. Across many cultures, STEM is associated with males and not females [26]. Interestingly, male STEM majors in the US hold the strongest associations between maleness and science, while female STEM majors show some of the weakest implicit biases between gender and science [27]. These differences in the gender-STEM stereotypes held may explain why male undergraduate STEM majors nominate more males, but females do not demonstrate this bias. It also helps explain why male faculty demonstrate biases in hiring and mentoring, but many female STEM faculty do not [28].

Bolds are mine.

That explanation makes sense on one level, but it introduces a deeper question:  Why would young men be more likely to hold these implicit biases than young women?   After all,  most male and female students in US universities share roughly the same cultural background and should have the same average implicit biases.****

I pondered this question for some time.  All I came up were these two theorettes (my name for baby theories), assuming that the results of the study hold after scrutiny by those more methodologically aware:

First, all this may have something to do with gender differences in circles of friendship.  Suppose that the way the "celebrities" are determined is in a wider circle of friends.  Suppose, moreover, that men tend to have mostly male friends and women both male and female friends.  If both of those suppositions were true, then the results of this study could follow, because more men than women simply wouldn't know high-performing women in the classes.

Second, it may have the same roots as women's inaudibility in meetings or women's invisibility as potential experts.  I have read about these phenomena and even experienced them myself. 

The usual scenario goes like this:  A woman says something in a meeting at work.  Her comment drops like a stone into still water.  Then, later in the same meeting, a man makes the same comment and it is eagerly discussed or debated. 

This is an actual pattern, but I don't know if it would apply in situations where women are not a small minority of those present.  Neither do I know how that odd invisibility/inaudibility is theoretically accounted for.

I'd love to see this study replicated and also carried out in other academic disciplines.  What the researchers write about may not be a STEM-problem but a more general one.



* I have read the study pretty carefully, but should note that the method it uses is not one I am capable of criticizing in any detail.  So caveat emptor.

** Two of the courses had only male professors, the third had one female professor and two male professors.  That most of the "experts" in these courses were men could have had a subconscious male-favoring effect on the student evaluations.  But that effect should have worked equally for both male and female students.

The class participation measure is about outspokenness, not about outspokenness-combined-with-smartness.  This could matter, though I can't figure out how.  Also, the professors' evaluations of students' class participation levels could themselves be biased.  For these reasons the actual grades are probably a better measure of excellence.

***  The exception is the last survey in the course which had three professors, one of them female. Its results show a small and non-significant bias for women to nominate other women.   But the larger bias of men towards nominating men still persists.  As the researchers note, the solution to potential bias probably isn't to introduce opposite bias, however.

****  Indeed, most other theories I can think of slam into that same hurdle.  Take statistical discrimination.  It's a theory which applies to, say, evaluating job applicants when knowledge of their true potential is only partially known. The evaluators may then use more general evidence (either true evidence or just prejudice) about the group the applicant belongs to.  If that group, on average, does better than other groups in the job under consideration, the particular applicant may be given extra points for that group membership.  If that group does worse, on average, then the particular applicant will have points deducted from his or her final assessment.

Statistical discrimination hurts top performers who belong to a group which has lower average scores, because it pulls down their overall assessment.  As the average female grades in the courses the current study analyzed are slightly lower than the average male grades, a statistical discriminator might give top performing men extra points.  But that assumes the students knew the overall average grades in the courses.  I doubt that was the case, and even if those grades were known, the theory doesn't explain why men would adjust their assessment of men's performance upwards but women would not.