Tuesday, June 30, 2009

Nasty Post VI: FIRST READ THE STUDY. THEN POPULARIZE ITS FINDINGS.






Do you ever get so angry at something that your fingers shake too much for you to type? Wonder why that thought occurred to me while reading about new research concerning possible discrimination against female playwrights? Well, aren't you lucky, because you are going to find out.

The research I mentioned consists of three different empirical studies, carried out by Emily Glasberg Sands (an undergraduate student at Princeton University), each of them testing various economic theories of discrimination or its lack against actual data on playwrights. The studies are pretty neat, the empirical analysis careful and the revealed knowledge of the theoretical field fine. Ms. Sands' overall findings deserve a careful post, and I might write one when I'm not quite so angry. But for this post I'm only going to discuss one of the three studies, the second one, the one which every anti-feminist is writing about. Well, sort of writing about, because I see no evidence that the writers actually read the studies, even though the file is so very conveniently linked to this New York Times article on them.

Let's begin with that NYT article, because it is such a very juicy example of what roused my anger. First, here's a summary of the oh-so-scandalous second study:

For the second study, Ms. Sands sent identical scripts to artistic directors and literary managers around the country. The only difference was that half named a man as the writer (for example, Michael Walker), while half named a woman (i.e., Mary Walker). It turned out that Mary's scripts received significantly worse ratings in terms of quality, economic prospects and audience response than Michael's. The biggest surprise? "These results are driven exclusively by the responses of female artistic directors and literary managers," Ms. Sands said.

Amid the gasps from the audience, an incredulous voice called out, "Say that again?"

Ms. Sands put it another way: "Men rate men and women playwrights exactly the same."

Ms. Sands was reluctant to explain the responses in terms of discrimination, suggesting instead that artistic directors who are women perhaps possess a greater awareness of the barriers female playwrights face.

Get it? Me neither, actually. There's a disconnect in that quote and I would have thought that anyone writing about this would have either asked Ms. Sands more clarifying questions or actually read the study.

To summarize that quote: The same plays, all written by real women, were given false female or male names and sent out randomly for evaluation. Because the actual scripts were the same, except for the female or male name attached to them, any consistent differences in the way they were evaluated would be a reaction to the playwright's gender and nothing else.

Given this, the fact that female respondents ranked the quality of the plays with female author names lower is indeed very thought-provoking. But why doesn't Ms. Sands seem to get provoked here? Why does she give superficially paradoxical answers?

The author of the NYT piece isn't excited about that at all! Nope. Instead, she begins her piece like this:

When more than 160 playwrights and producers, most of them female, filed into a Midtown Manhattan theater Monday night, they expected to hear some concrete evidence that women who are authors have a tougher time getting their work staged than men.

And they did. But they also heard that women who are artistic directors and literary managers are the ones to blame.

Get it? It's women who are discriminating against women here.

Other writers caught the same fever. Here's Bloomberg's version:

Female playwrights, long aware that they're produced less frequently than their male counterparts, may now have someone to blame: female artistic directors.

Even the knuckledraggers who seem to breed in the comments threads of Salon joined in:

Are you referring to mynameisdan? If so, go reread his post, because he's exactly right. The evidence from this study showed that women, not men, were discriminating against women, which is the opposite of what the researcher expected to find, so she "charitably" theorized--apparently on the basis of no evidence at all--that this must be caused by male discrimination somewhere else. Why? What possible reason is there for this theory other than that the researcher has a bias that only men, and never women, can be responsible for sexism?

Hate men much, GeorgiaProg?

I need to go and bang my head against the garage door for a few centuries now.

OK. That helped a bit. Let's see what the second study actually says about all this.

First, Sands created a long list of questions for the respondents. These were intended to capture the respondents' views about the quality of the play, the likely reception it would get from the audience and from the actors and the number of prizes it might take. The questions about the likely reception of a play intend to measure customer discrimination (by the audience, in this case) and co-worker discrimination (by the actors). They are not measures of what the respondent himself or herself thinks about the play. THEY ARE NOT MEASURES OF WHAT THE RESPONDENT HIMSELF OR HERSELF THINKS ABOUT THE PLAY.

What they measure instead is the respondent's beliefs about the sexism of the customers or the theater workers. That female respondents would have more negative views on these matters DOES NOT MEAN THAT THEY THEMSELVES WOULD DISCRIMINATE AGAINST FEMALE PLAYWRIGHTS. All it means is that they believe the audience and the workers in the theater are less accepting of female playwrights than of male playwrights. Male respondents don't believe this, but then they don't have the same life experiences as the female respondents.

Second, only three of the questions in the study were explicitly about how good the respondent thought the script was. The first of these asked the respondent to rate the play along its exceptionality, the second asked the respondent to rate the play based on how likable the characters in it are and the third asked the respondent to assess how likely it is that the play would receive a prize. Note that even here the third question would be affected by what one thinks about the theater world in general. If the respondent believes that women are discriminated in the awarding of such prizes it would be perfectly possible to rate a study both exceptional and its chances of winning a prize as poor.

Here's the bit from the original study on how female and male respondents rated the quality of the play (p. 77):

On aggregate, male respondents assign nearly identical ratings to a script irrespective of the gender of the pen-name. Female respondents, however, assign markedly lower ratings to a script when that script bears a female pen-name. The lower ratings assigned by female respondents to purportedly female-written scripts may be attributable to heightened awareness among female respondents of the barriers faced by female playwrights.

Female respondents believe a script written by women will be perceived by the theater community to be of lower overall quality... However, female respondents do not report personally believing that a script with a female pen-name is of lower quality. Specifically, female respondents assign lower ratings for Likable and Prize when a script has a female pen-name; these questions ask generally if the characters are likable and how likely it is that the playwright will win a prize. In the more personal rating measuring the extent to which the respondent herself deems the play to be an example of artistic exceptionalism, in contrast, female respondents assign a given script the same rating irrespective of the playwright's gender.

All bolding above by me.

Are you ready to join me in the head banging yet? If not, let me give you a similar example: Suppose you come and interview me about authors, say, and ask me to tell which ones I think are really good and what their future prospects might be. And I give you some female and some male names of writers I like and regard as equally good and then tell you that the female ones have not as great prospects as the male ones. Then you go out and write that THIS PROVES I'M THE REASON FOR THEIR NOT-SO-GREAT PROSPECTS! That's pretty much how this whole thing works: To notice discrimination is to be guilty of it.*
-----
*Sands' third study in the package provides some evidence on the continuing discrimination against women's plays in the marketplace.