Friday, April 1, 2022

By Mason Roberts and Jeremy Reynolds

On March 2, Police Chief Maris Herold shared a summary of crime statistics dating from 2013 to 2022. The presentation displayed increased crime rates that grabbed the attention of citizens and pundits alike.

However, the presented graphs need to be interpreted with caution. Over-interpreting these trends may lead our community to unjustified decisions; for example, increasing budgets without proper evidence.

Without a doubt, every crime is a violation which can lead to unrepairable emotional damage in addition to the initial physical and/or monetary damage. As someone who has experienced crime (Mason was robbed at gunpoint in 2006), we sympathize with every victim. But making emotional decisions about resources for our community will not lead to the best outcomes. If our community is going to make evidenced-based decisions on how to best spend our limited resources, the evidence needs to be thorough and complete.

Reporting statistics, and in particular crime statistics, responsibly is notoriously difficult. The data are messy: Definitions of crime categories, incentives and reporting technology change over time. Proving causation versus correlation is difficult in any analysis. Providing the proper context to develop a meaningful, accurate and easily actionable message is difficult.

Even if we get the analysis and message right, the data, analysis and message do not provide the recommendation of how to act in response to discovered trends. For instance, tying performance metrics of policing to statistics has led to increased policing of marginalized communities and falsified arrests. We need to be thoughtful in our analysis and what we choose to do with the results.

Our police chief presented raw data points with a regression line. The presenters did not provide appropriate context and did not account for external variables such as the onset of the COVID-19 pandemic. There were no “error bars” — no information was provided as to the statistical significance or size of the identified trends.

All measurements are noisy, and statistical significance provides a way of measuring whether the identified trends could have been observed by chance. The size of the effect is important, because visual representations (i.e. graphs) can misrepresent the strength of an effect by growing or shrinking the vertical-axis. In other words, an average increase of 10 violent crimes per year could appear to be as visually steep as an average increase of one violent crime per year if one adjusts the vertical-axis.

It should be noted that the presented data is more than many police departments provide, and we appreciate the efforts that are currently being made. Unfortunately, it is simply not where we need to be.

Dr. Daniel Reinhard, BPD’s new data analyst, did note that there will be more in-depth analysis in the next release, including data from comparable cities. We should also consider external variables, including but not limited to economic crashes, changes that would impact reporting and common statistical phenomenon (i.e. Simpson’s Paradox). For instance, if you take into account COVID by segmenting the data into pre- and post- onset, you can see limited changes in crime rates over the past two years (the post-COVID onset time period).

We should avoid hubris and gravitate towards humility.

During the presentation, there were insinuations that the public could not understand in-depth data analysis. Data analysis, done effectively, presents information in a way that allows stakeholders to make informed decisions. Anything less is a failure on the analysts’ part. The public needs to be given the opportunity to think critically about this data. So far, the conversation has been limited to questions from city council based on the BPD’s analysis.

Without inclusion from the community in the analysis, we risk embedding bias. Without inclusion of the community in the message, we are left with only political voices. Political spin robs our city of effective policing and critical funds for other programs that could have a direct impact on crime. It is time to focus on evidence-based budget decisions.

Police may not have a meaningful impact on crime rates.

Studies have shown you would need to hire more than 10 additional police to avoid one homicide. This is because police, in large part, respond to crime, not prevent it.

The data presented by the BPD does not inform us on the effectiveness of our police force. If we want to measure police impact, we might want to focus on other metrics (response times, conviction rates) or other relationships (the direct relationship between police force size and crime rates). We might find that the resources would be better spent on retention rather than adding personnel to the force. Otherwise we could focus on addressing the root cause of crime by funding housing, education and social services, which may reduce crime more cost effectively than increased policing while improving the working conditions for police.

This is important work.

The most basic purpose of a government is to provide recourse to its citizens through the rule of law and to use the resources of its people wisely. In order for the city of Boulder to do this properly, its stakeholders — council, city staff and voters — need the best analysis possible. To achieve this, we need transparency and true partnership between the police department and the people it serves.

All data should be made available, all analysis should be made public and a robust discussion surrounding the final outcome needs to be inclusive. Certain limitations will likely arise and, when they do, we will work to remove those limitations. Democracy works best when we have full participation, inclusion and information.

Mason Roberts is an actuary who specializes in the financial evaluation of healthcare intervention programs. He served as treasurer of councilwoman Lauren Folkerts’ election campaign

Jeremy Reynolds is a data scientist and the husband of council member Nicole Speer.

Opinion Boulder city council city of Boulder crime data opinion