The New York Times article this week about the study of racial bias in NBA officiating by Joseph Price and Justin Wolfers generated quite a bit of commentary. What is amazing is how little people understand, or are willing to understand, about statistics. Here's what the authors claim in the abstract of the study:
We find that--even conditioning on player and referee fixed effects (and specific game fixed effects)--that more personal fouls are awarded against players when they are officiated by an opposite-race officiating crew than when officiated by an own-race refereeing crew.
Much of the reaction among sportswriters has been to take the authors to task for calling the refs racist. (See Mike Wise in his column in Thursday's Washington Post and Kevin Hench at FoxSports.) Having taken a look at the study myself, I am surprised that those who make a living based on the sport would be so dismissive of the result. The main result of the paper is that the foul rate (fouls called per 48 minutes played) increases for black players when the racial composition of the three-person crew of referees goes from black to white. (See Table 4 and the discussion on page 8.) Any honest sportswriter should hold the NBA accountable for the result--why are the outcomes for fouls different across different racial configurations of refs and players?
It is very difficult to posit an explanation for these results that would attribute them to something other than race. First, no one disputes the NBA's claim that it does not assign referees to games based on their race or the racial composition of the two teams. (See page 4 and Table 1 of the study for discussion and evidence.) With (conditionally) random assignment, and the fact that the explanatory variables are fixed characteristics of people (i.e., race), we have the conditions for a clinical trial here, where "controlling" for possibly confounding factors is not likely to be important. Second, the authors do in fact control for a number of "fixed effects," exploiting the fact that their dataset is a panel consisting of a limited number of individuals observed in numerous interactions. This includes characteristics of the player and the refs that don't change over time. As the authors note, the most comprehensive results "are identified only off the differential propensity of teammates to earn extra fouls when the refereeing crew is of the opposite race."
Having said that, I think the authors soft-pedal one possible explanation of the results that would exonerate the refs. The following passage appears on pages 12-13:
The fourth point speaks to a relatively subtle interpretation issue: while we document a correlation between a player’s foul rate and the race of the referee, this may reflect the players responding to the race of the referees, rather than the referees policing opposite-race players more aggressively. Strategic responses by players would lead to an attenuation bias: expecting to receive more fouls for a given style of play, the players may play less aggressively, minimizing the impact of referee discrimination on realized fouls. This suggests that our results understate the amount of discrimination. Alternatively, if players exhibit oppositional responses, they may play more aggressively when policed by the opposite race. Importantly, such oppositional responses suggest that our findings are driven by changes in player behavior, rather than referee behavior. Yet if this were driving our results, one might expect to see effects not just on the number of fouls earned, but on the likelihood of fouling out, as well as other indicators of aggression, including blocks and steals. Instead, we find that blocks and steals actually decline under opposite-race referees.
I'm not persuaded by this reasoning. The player response needn't take the form of aggression--it merely needs to be a general decline in player performance in the presence of opposite-race referees. What if, for example, players find it more difficult to concentrate on their tasks when the refs are of opposite race? Elsewhere in the paper, the authors write, "Player-performance appears to deteriorate at every margin when officiated by a larger fraction of opposite-race referees." So why assume that it's the refs not the players? And why make a statement, "Basically, it suggests that if you spray-painted one of your starters white, you’d win a few more games," even under the possible coaxing of a reporter?
The interpretation of the results that it's the players, not the refs, may also reconcile the results of NBA's internal studies that claim that, on a call-by-call basis, there is no evidence of racial bias. (The NBA has not released the results of these studies, much less the data.) If the players are changing their game based on the racial composition of the refereeing crew, then it is possible that every call or non-call is legitimate, and both studies can be accurate.