Friday, April 07, 2006

Bad Popularizations of Gender Research



NOTE: Jason in the comments to this post noted that I missed getting the real comparison study to the men's study. In my defense I can point out that the article on the men's study linked to the study I used, not to the actual comparison study. The link said: Change your life with Jane Austen: the books that inspired women.

This means that the rest of this post is mostly rubbish, and I apologize for that.

Though comparing the proper studies doesn't make me change my opinions very much as the only thing that is changed by this is an alignment of the research questions and rough sample sizes in the two studies. Now both of the samples are small, perhaps too small if the search was for wider trends. The other reservations I have remain: the samples are not randomly drawn, the women's sample appears to consist largely of various types of famous women from assorted fields whereas the men's sample seems to drawn largely from publishing (still apples and oranges), and it is not at all clear what percentage of the respondents said what.



This could become a continuing series. There is so much bad popularization of research that I sometimes weep into my keyboard. Like after reading about the recent research into what men and women like to read. The Guardian has a long article on the novels that changed men's lives:

The novel that means most to men is about indifference, alienation and lack of emotional responses. That which means most to women is about deeply held feelings, a struggle to overcome circumstances and passion, research by the University of London has found.

Professor Lisa Jardine and Annie Watkins of Queen Mary College interviewed 500 men, many of whom had some professional connection with literature, about the novels that had changed their lives. The most frequently named book was Albert Camus's The Outsider, followed by JD Salinger's Catcher in the Rye and Kurt Vonnegut's Slaughterhouse Five. The project, called Men's Milestone Fiction, commissioned by the Orange prize for fiction and the Guardian, followed on from similar research into women's favourite novels undertaken by the same team last year.

The results are strikingly different, with almost no overlap between men's and women's taste. On the whole, men preferred books by dead white men: only one book by a woman, Harper Lee, appears in the list of the top 20 novels with which men most identify.

Women, by contrast, most frequently cited works by Charlotte and Emily Brontë, Margaret Atwood, George Eliot and Jane Austen. They also named a "much richer and more diverse" set of novels than men, according to Prof Jardine. There was a much broader mix between contemporary and classic works and between male and female authors.

Okay, I guess. So I looked for the comparable study on women's choices of novels that changed their lives. The only link I found was to a study which didn't ask about life-changing choices at all, but about something slightly different:

It is a truth universally acknowledged that the book women feel has most transformed their lives is the one that has assured them for the past two centuries that, yes, they will marry the wealthy, handsome man next door and live happily ever after.

Pride and Prejudice, Jane Austen's salty-tongued commentary on the plight of women in the 19th century, perhaps best known today for providing Colin Firth with the opportunity to pose in a wet shirt in front of many grateful viewers, has won the Women's Watershed Fiction poll, it was announced yesterday on Radio 4's Woman's Hour.

Despite being specifically about women's lives 200 years ago, the relevance of Austen's classic has not diminished, according to the 14,000 voters who took part in the poll, 93% of whom were women. It is, according to the poll, the novel that "has spoken to you on a personal level; it may have changed the way you look at yourself, or simply made you happy to be a woman".

Note that neither of these studies is based on proper randomly drawn samples of men and women. Note that the women's study is based on 14,000 people who felt strongly enough to send their answers in, and the men's study is based on 500 people, many of whom have a professional connection to books. To compare the two studies is to compare one gigantic and eager apple to a very small and rather professional orange. Makes no sense at all.

Then add the suspicion I have that the two studies asked different questions. The study about men's reading habits asked about personal change, not about books that were meaningful for them as men, whereas the women's study asked specifically about books that mattered to the readers as women. Now, these are very different research questions, and the answers are not comparable.

Except that compared they are, all over the media. And not only are they compared, the conclusions that are arrived at are extreme:

The results are strikingly different, with almost no overlap between men's and women's taste. On the whole, men preferred books by dead white men: only one book by a woman, Harper Lee, appears in the list of the top 20 novels with which men most identify.

Women, by contrast, most frequently cited works by Charlotte and Emily Brontë, Margaret Atwood, George Eliot and Jane Austen. They also named a "much richer and more diverse" set of novels than men, according to Prof Jardine. There was a much broader mix between contemporary and classic works and between male and female authors.

...

"We were completely taken aback by the results," said Prof Jardine, who admitted that they revealed a pattern verging on a gender cliche, with women citing emotional, more domestic works, and men novels about social dislocation and solitary struggle.

She was also surprised she said, "by the firmness with which many men said that fiction didn't speak to them". The historian David Starkey said, for instance: "I fear fiction, of any sort, has never worked on me like that ... Is that perhaps interesting in itself?"

...

"On the whole, men between the ages of 20 and 50 do not read fiction. This should have some impact on the book trade. There was a moment when car manufacturers realised that it was women who bought the family car, and the whole industry changed. We need fiction publishers - many of whom are women - to go through the same kind of recognition," Prof Jardine said.

Do men and women really read totally different books? Who in their right mind would regard Jane Austen's books as being about passion? After all, Austen was the most cited author in the study about women's reading experiences. And is it really true that hardly any men between the ages of 20 and 50 read fiction? I very much doubt that. In fact, I doubt this whole enterprise. It's yet another attempt to bring back the women-are-from-Venus-and-men-from-Mars mythology.

So let me summarize: Two studies are done with different sized groups of respondents, neither selected properly. The two studies have different questions for the subject to answer, and result in two different lists of books. Conclusion: men and women are different breeds of people! Perhaps. But it's much more likely that these are not well-done studies and that we can't draw many conclusions from them.

As an aside, I have read all the books on the men's list and also all the books on the women's list. None of those changed my life much. What did change my life was Agatha Christie's The Murder on the Orient Express. I was around eight years old when I read it and it blew my brain. To think that they were all guilty!