Chicago's heat list changed who got arrested, not who got shot

Everyone Chicago arrested got a number

A person arrested in Chicago for a non-violent misdemeanor, such as speeding, could be given a police risk score. A person shot in the street, if they had never been arrested, got none. The city's Office of Inspector General offered that exact pair to illustrate how the Chicago Police Department decided who landed on its Strategic Subject List, the roster officers called the heat list.¹ The list was a risk score: a single number meant to predict how likely a person was to be a "party to violence," the department's term for the victim or shooter in a future gun crime.¹

Miles Wernick, an engineering professor at the Illinois Institute of Technology, built the model, applying machine learning to Chicago crime data.² The work ran on federal money: $3.8 million in grants to build the models, including a $2,999,984 National Institute of Justice award for the demonstration.¹³ Every person arrested at least once in the four years before the model ran got a score, from 0 for the lowest risk to 500 for the highest. It did not matter whether they had any history of violence.¹⁴

The department first put the score to work on March 26, 2013. It placed the 426 people it judged highest-risk on a pilot list, to see whether flagging them could hold gun violence down.⁵

What happened to the 426

The only independent study of the pilot came from three RAND Corporation researchers: Jessica Saunders, Priscillia Hunt, and John Hollywood, who published it in 2016.⁵ To measure the list's effect they used propensity-score matching. They paired each listed person with a similar person who was not listed, alike on the factors that drive risk, and compared what happened to the two groups over the following year.⁵

The listed group was no more and no less likely to be shot or murdered than their matched counterparts. They were 2.88 times more likely to be arrested for a shooting. In absolute terms that was 9 listed people arrested for a shooting in the next year, against 5 in the matched group.⁵

The list was supposed to do more than mark people. In one district, commanders visited some of the listed at home to offer social services, a program the department called Custom Notifications. That was the only formal attempt to prevent violence rather than record risk, and because a single district ran it, it reached under 5% of the people on the list.⁵

RAND was careful about what the null result proved. The researchers could not tell whether the list failed because the idea was wrong or because the prevention program was never built out, a difference they wrote as theory failure versus implementation failure.⁵ They found no sign the list made violence worse, allowed that it may have identified more genuine offenders, and noted the algorithm improved in later versions.⁵

Who ended up on the list

The pilot list of 426 was the small version. The department kept scoring everyone it arrested. The Chicago Sun-Times pried the data loose in 2017 through a public-records lawsuit, after the department refused to release even a copy with names removed.⁶ The released list held 398,684 people. More than two-thirds of them, 287,404, scored above 250, the level the department said drew heightened scrutiny.⁷

Many of those people had no record of violence at all. 127,513 had never been arrested and had never been shot, and roughly 90,000 of them were still rated high-risk.⁷ The strongest single predictor of a score was not any crime. It was age: one analysis found age alone explained about 89% of the variation in scores.⁷

At the very top the pattern was sharper. 153 people carried the maximum score of 500, and another 3,568 scored between 400 and 499. Of the 153 highest-scored, 20 had never been arrested for either guns or violence.⁶ The released data also carried each person's race, and 85% of the people with the highest scores were African-American men.⁶⁴ The legal scholars Rashida Richardson, Jason Schultz, and Kate Crawford, reading the same data, reported that 56% of Black men under thirty in Chicago carried a score.⁸

Then the city audited its own tool. In January 2020 the Office of Inspector General reported that the scores were unreliable, that the officers using them had not been properly trained, and that no controls governed who inside or outside the department could see them.¹ It flagged how the scores were used against people: the interventions they drove "may have attached negative consequences to arrests which did not result in convictions."¹ The department decommissioned the program on November 1, 2019.¹

Why the list found the already-policed

The list did not measure crime. It measured arrests, and the two are not the same. An arrest is a record of what the police did, not a verified record of what a person did. When a model is trained to predict who will be a party to violence, but the history it learns from is arrests, it learns the pattern of policing. This is the proxy-label problem this desk met in a health algorithm that predicted cost and was read as need. A model learns the stand-in you give it, not the goal you had in mind, and it carries every difference between the two.

In the same period those arrest records were generated, the U.S. Department of Justice investigated the Chicago Police Department and found a pattern of unlawful force that fell disproportionately on Black and Latino residents.⁹ A model built on such records, Richardson and her co-authors argued, "cannot escape the legacies of the unlawful or biased policing practices that they are built on."⁸ Because most inputs were arrests, not convictions, a person who had committed no crime could still be ranked, and the ranking sent officers back to the neighborhoods they already patrolled hardest.⁸ A score built from arrests sent police out to make the arrests that fed the score.

A risk score is also only as good as what follows it, and for the people on this list almost nothing followed but police contact. The inspector general found a department directive urging the highest possible charges against listed people who were later arrested.¹ A high score rarely brought help. It brought heavier policing.

The last problem is arithmetic, and it would sink the list even with clean data. Gun violence, statistically, is rare. In the year after the pilot, the listed group's homicide rate was about 0.7%.⁵ That is far above the rest of the city, and it still means that for every 1,000 people the list marked as highest-risk, about 993 were not killed. When the thing you are predicting almost never happens, even an accurate model produces far more false alarms than hits, the base-rate problem that made a store's face scanner flag mostly the innocent. RAND found the pilot model identified fewer than 1% of the city's actual homicide victims, 3 out of 405.⁵

Where this kind of score lives now

Chicago retired the heat list. The design behind it did not. A score that predicts who will commit a crime, built from records of who the police have already stopped, outlived this one program. What has changed is how that design is treated in law. Since February 2, 2025, the European Union's AI Act has prohibited it. Article 5 bans using an AI system to "assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality traits and characteristics."¹⁰ The regulation leaves one opening: an AI tool may support a human judgment that already rests on objective, verifiable facts tied directly to a criminal activity.¹⁰ A number assigned to a person for having been arrested is the thing the rule names.

The weakness the EU named in its statute is the one Chicago's data exposed. A score is only ever as good as the label it learns and the base rate it works against. Predict a proxy for crime and you predict the policing that produced it. Predict a rare event and most of your flags are wrong. Any system that ranks individuals for police attention out of their contact history inherits both faults, whatever it is called and wherever it runs.