Wednesday, February 25, 2015

White Privilege in Australian Buses. A Look At An Audit Study.

Ian Ayres' opinion column in the New York Times is about an audit study carried out in Australia.

I love audit studies, because they are a way to control for all the alternative explanations to pure discrimination in consumer and labor markets.  Here's why:

An audit study uses trained individuals to play the role of, say, a car buyer or a job seeker.  The trained individuals are all given the same rules about how to behave, what to ask for and how, how to negotiate and, when relevant, they are also provided resumes etc. of equal value.  The goal is to have these individuals differ in only the characteristic the study is interested in, such as race or gender or both.

If it turns out that the tester's chances of getting a job interview or a good price on a second-hand car indeed vary by race and/or gender, we have ruled out that something else caused the apparent correlation.  Well, we have ruled it out if the audit study was well designed.

The flaw in audit studies is that they cannot continue for years and years, which means that they cannot tell us much about how people are rewarded in their jobs, whether they are promoted purely on the basis of merit, say.  But they are pretty good for measuring potential gender and/or race discrimination against job seekers or car buyers or renters of apartments.

The audit study Ayres writes about was carried out in Brisbane, Australia.  It consisted of having a group of young men and women of various racial and ethnic backgrounds (the testers) take buses.  They were told to get on a bus (actually, on many, many buses during the study), then tell the driver that they didn't have enough money for the ticket.  The focus of the study was in finding out how often the bus drivers let the testers ride without paying, what determined that frequency and how it differed by the race or ethnic origin of the tester.  The researchers coined the bus driver's decision to let someone ride free "a favor," and the study also tried to find out which theoretical model of discrimination would best explain the favors granted.

Ayres links to a working paper of the study.  I found that weird, given that the working paper came out in 2013.  The study should have been published somewhere by now.  But there's an explanation for that state of the affairs.  Ayres writes in Forbes:

Professors Mujcic and Frijters deserve our thanks for authoring a study that is not only illuminating about what white privilege means.  But their employer, the University of Queensland, has not thrown them a parade.  After the City of Brisbane complained that the study encouraged fare evasion, the University initiated a complaint process against Professor Frijters and has ordered the authors to suppress this important paper. 

Anyway, the working paper can be read here.   The researchers played with several versions of the audit study.  In the main study the testers were dressed the way young people dress in Australia, most of the time (t-shirts, jeans or shorts).  In two alternative versions some testers dressed in business attire or in an Australian military uniform*.  As Ayres writes in the NYT, the results are striking**:

As they describe in two working papers, Redzo Mujcic and Paul Frijters, economists at the University of Queensland, trained and assigned 29 young adult testers (from both genders and different ethnic groups) to board public buses in Brisbane and insert an empty fare card into the bus scanner. After the scanner made a loud sound informing the driver that the card did not have enough value, the testers said, “I do not have any money, but I need to get to” a station about 1.2 miles away. (The station varied according to where the testers boarded.) 
With more than 1,500 observations, the study uncovered substantial, statistically significant race discrimination. Bus drivers were twice as willing to let white testers ride free as black testers (72 percent versus 36 percent of the time). Bus drivers showed some relative favoritism toward testers who shared their own race, but even black drivers still favored white testers over black testers (allowing free rides 83 percent versus 68 percent of the time). 
The study also found that racial disparities persisted when the testers wore business attire or dressed in army uniforms. For example, testers wearing army uniforms were allowed to ride free 97 percent of the time if they were white, but only 77 percent of the time if they were black.
The authors considered several economic theories of discrimination which could explain the observed findings, such as the possibility that certain groups are regarded as less trustworthy if those groups, on average, have high crime rates, the possibility that people prefer to grant favors to individuals who belong to the same racial or ethnic group, and so on.  They argue that the results support the hypothesis that people are more likely to grant favors to individuals who appear to belong to the "in-group.".  Or in the words of the study itself:

The story that best fits the observations from our natural field experiment is that existing groups discriminate against members of racial/ethnic groups deemed less likely than them to be in the in-group, where clothing is taken as a visual cue of the degree to which particular individuals belong to this group. Discrimination of out-group members is then rationalized on the basis of following the rules and distrusting the motivations when not granting a favor, while believing in the honesty and general deservingness of the recipient when granting a favor. This is consistent with recent field studies examining both the extent and nature of racial discrimination in the United States and other countries (e.g., Shayo and Zussman 2011; Gneezy et al. 2012; Zussman 2012).

Is there anything about the study that deserves criticism?***  I was a little concerned about the number of testers.  Twenty-nine is not necessarily too low a total number of testers, but when you realize that six of them were white, twelve Asian, six Indian and five Black you start worrying about the possible impact of just one or two individual testers on the results.  The 1500+ observations do not mitigate this problem because that large number was created by repeated contacts carried out by the same 29 individuals.

I also found the role of gender potentially underplayed in the findings.  Granted, I scanned the statistical tables pretty quickly, but I failed to see a race-and-sex interaction term.  That term would have been a statistical way to get at the idea of intersectionality, though its absence might be explained by the same small-numbers problem.  The number of women in the racial or ethnic groups varied from the high of six in the Asian group to two in the Indian and Black groups.

The study did find a statistically significant difference between the overall numbers of favors granted to men and to women.  Men were slightly more likely to receive them than women (67% compared to 59%).  This fits well with the general in-group theory (women are less likely to be members of the in-group, when racial or ethnic characteristics are held constant or ignored).  But it also suggests that the damsel-in-distress is not a scenario the bus drivers honored in the study.

My apologies for potentially boring you.  But it's good to know what studies say and do not say, and this study actually said quite a bit more than the gist Ian Ayres transmitted. 

*   To test the possibility that apparent income level of the tester or patriotism would affect the results.
**  The average percentage of favors granted in the basic study was 63%

The racial categories in the study also included Indians and Asians (mostly Chinese).  The Asian group was treated  similarly to the white group in the basic study (without business attire or military uniforms).  The treatment of the Indian testers fell between the white/Asian testers and the Black testers.

In the business attire and military uniform versions the percentage of favors granted to white testers approached 100%.  The percentages rose for other racial and ethnic groups, too, but the Black testers still received the fewest favors (67% in the business attire version and 77% in the military uniform version).   Indians received favors 83%  and Asians 69% of the time when dressed in a business suit and carrying a briefcase.  The equivalent percentages for the military uniform version were 93% and 90%.

Economic theory calls this effect of the clothing of an individual signaling.  One is signaling either higher income (and perhaps thereby the idea that one doesn't usually take buses without paying) or patriotic sacrifice (which individual bus drivers might reward).  This signaling works but does not completely eradicate the observed racial or ethnic differences.

***These thoughts are based on a quick reading of the study.