By Jonathan Mummolo, Stanford University
The following is part of a series of posts written by MPSA award recipients highlighting outstanding research presented at previous MPSA annual conferences and in the American Journal of Political Science.
The question of whether and to what degree police officers respond to orders from their commanders is fundamental to understanding the prospects of effective police reform. But for decades, the policing literature has offered no consensus. Indeed, social scientists since the publication of James Q. Wilson’s landmark study, Varieties of Police Behavior (1968), have tended to paint patrol officers as autonomous bureaucrats who are relatively impervious to rules and supervision because of the large amount of discretion they are granted and because the nature of their work allows them to operate largely out of view from their superiors.
There are several reasons this question deserves renewed empirical testing. For one, in light of a spate of recent high-profile incidents of police misconduct, there are widespread calls for police reform today with little agreement on which policy changes will produce the best results. Much research on police misconduct also tends to focus on individual-level officer traits such as implicit bias and personality which, while important determinants of police behavior, suggest few policy-based remedies because prior work shows these traits may be immutable. In addition, many sources of high-resolution data on police behavior have only recently come online.
In my study, I leverage a sudden change in the New York Police Department’s procedure for implementing “Stop, Question and Frisk” (SQF) in order to gauge the responsiveness of officers to orders from their commanders. SQF is a controversial police practice that has historically targeted disadvantaged communities of color, and has been widely criticized for being over-zealously applied in major cities across the U.S. Combining quantitative data covering millions of police stops in New York City with qualitative evidence from original interviews and court transcripts, I use an interrupted time series analysis to estimate the causal effect of a procedural reform within the NYPD on the nature of police-citizen interactions. Specifically, I measure the impact of an order mandating that officers provide their commanders with narrative descriptions of the reasons they stopped criminal suspects on the “hit rate,” the proportion of stops conducted by officers which produced evidence of the suspected crime that motivated those stops. This metric has often been used to approximate the rate at which officers are stopping people actually engaged in criminal activity, rather than needlessly detaining innocent citizens.
The results of this analysis are stark. The day the reform was put into place, the hit rate— which had been relatively stable for years leading up to this date— effectively doubled by some estimates. Further analysis shows that this increase in the hit rate was driven by a sudden and sustained decrease in the number of stops being conducted, which occurred even as the number of stops producing evidence of a crime remained relatively constant. Further, contrary to claims by critics of SQF reforms that crime surged in New York City following this change, an analysis of homicide and robbery data shows no detectable change following the intervention. Faced with the prospect of increased scrutiny from their superiors, officers suddenly and dramatically refrained from detaining thousands of innocent New York residents with no discernible impact on public safety.
There are of course some necessary caveats. Though analyzing the immediate discontinuity in the hit rate at the moment of the intervention provides valuable causal leverage, it also confines inferences about this intervention’s effectiveness to the short term. The high hit rate observed post-intervention persists, and appears to grow, through the end of 2015. But we cannot attribute this persistence to the new directive with much confidence, as intervening events could be responsible. This study also examines data from a single city, and the efficacy of similar reforms should be tested and validated in other settings. Future work that selectively implements similar interventions experimentally across multiple departments could test the robustness and persistence of these effects.
The intervention was also followed by a sharp reduction in the number of stops producing a weapon. While we cannot necessarily attribute this change to the reform—since, again, there was no immediate change in this outcome the day of the reform, and intervening events could have easily been responsible for future changes—we also cannot rule out the possibility that this reduction was due to a lagged treatment effect. If the treatment did cause this decline, that would represent an important public welfare tradeoff. However, it is also worth noting that the primary purpose of removing weapons from the street according to proponents of SQF is to reduce violent crime. As the results show, the intervention did not lead to any detectable increase in homicides or robberies, a result that is consistent with earlier work finding no robust evidence that increases in SQF activity reduced crime rates in New York.
Despite the impact of this reform, the difficulty of improving the quality of police-citizen interactions should also not be understated. Officers still enjoy immense power and discretion as well as substantial barriers to prosecution in the event of wrongdoing. The effect observed here is limited to a single aspect of police work, and it is possible that the performance of other tasks which do not generate reports—or ones performed in environments where the press and populous are less able to scrutinize police behavior—would be much more difficult to improve. And even if similar interventions lead to widespread improvements in policing nationwide (a best case scenario), it may still take years, if not decades, to rebuild the atrophied levels of trust between residents of over-policed communities and law enforcement personnel.
But as solutions to the problems facing law enforcement continue to be sought, these findings should underscore for reformers the strong influence of institutional factors on police behavior. The trope of the “rogue cop” in discussions surrounding police misconduct has led to an individuation of social justice problems that, to a large extent, have institutional support. To be clear, this paper does not dispute that individual-level factors such as racial bias and personality affect police-citizen interactions, but rather that such results, at present, suggest few policy-based remedies. Even if some prejudice reduction strategies are effective, police organizations have often failed to demonstrate this by scientifically evaluating them during implementation. Indeed, the failure to adequately assess the merit of these initiatives may indicate a willful ignorance, and illustrate the resistance of institutions to more sweeping structural remedies. Announcing prejudice reduction initiatives while failing to properly evaluate them may allow political leaders to appear concerned about injustice while distracting attention from the fact that the institutions they control play a substantial role in shaping police behavior.
About the Authors: Jonathan Mummolo is a PhD Candidate in the Department of Political Science at Stanford University. Mummolo’s research “Can New Procedures Improve the Quality of Policing? The Case of ‘Stop, Question and Frisk’ in New York City” was recently named as the Best Paper Presented by a Graduate Student presented at the 2016 MPSA conference.