Every automated car that has killed came with a human assigned to watch it

The two things blame needs

In 2004, in the journal Ethics and Information Technology, Andreas Matthias named a problem he thought machine learning was about to create: the responsibility gap. We hold a person responsible for an act only when they had two things: control, meaning they could have done otherwise, and knowledge, meaning they understood the facts well enough to choose. In Matthias's words, an agent is responsible "only if he knows the particular facts surrounding his action, and if he is able to freely form a decision ... based on these facts."¹

For a machine, we pass its consequences up to a person by the same test. While it runs to the maker's specification, the operator answers. When it departs, the maker does. A learning system breaks the chain, because a neural network or autonomous agent rewrites its own rules as it runs, from data the maker never supplied. Neither operator nor maker can predict what it does next, so both conditions blame needs leave their hands at once. Matthias called this the responsibility gap: cases where "nobody has enough control over the machine's actions to be able to assume the responsibility for them."¹

His warning was not that we would fail to punish someone, but that we would punish someone anyway, committing "the injustice of holding men responsible for actions of machines over which they could not have sufficient control."¹ The machine is no candidate either: it is not a moral agent, an entity answerable for its own choices, so the blame has nowhere left to land.

Sparrow's three defendants

Robert Sparrow pressed the same point at its hardest, where someone must answer. In "Killer Robots" (2007) he imagines an autonomous weapon that, knowing exactly what it does, deliberately kills soldiers who have surrendered. That would be a war crime if a person did it. The laws of war carry a rule philosophers call jus in bello, on how a war may be fought: for a death in war, someone must be able to be justly held to account. So who is tried?²

Sparrow lines up three defendants and rules out each. Not the programmer: once the weapon is genuinely autonomous, "the connection between the programmers/designers and the results of the system ... is broken by the autonomy of the system." Not the commanding officer, who cannot fairly answer for the acts of a machine "whose decisions they did not control." Not the machine: to hold a thing responsible here is to be able to punish it, and we cannot make a machine suffer in any way that means anything.² The weapons case has its own lesson, on autonomous weapons. What carries here is the shape of the argument: three exits, all blocked. Sparrow concludes it would be wrong to send such a weapon to war at all, because no one could justly answer for the crimes it commits.² He grants, though, that no such weapon yet exists: "for the foreseeable future we will not be able to hold that machines are responsible for their actions."²

The gap needs a machine no one is watching

Matthias's gap and Sparrow's trilemma both open on one condition: no human holds the control and knowledge to answer. Put a person back in the loop, whose job is to watch and take over, and the gap does not open: the ordinary test finds them.

Two automated-driving systems have killed people and been publicly investigated, and neither was the machine the philosophers described. The Uber car that killed a pedestrian in 2018 was a developmental system, a test vehicle with a safety driver whose job was to monitor it and take over.³ Tesla's Autopilot is a Level 2 system on the SAE scale, which means the human stays the driver and keeps responsibility for the car. The regulator's records say it plainly: with Autopilot engaged, "the driver is the operator of the vehicle ... responsible for the vehicle's movement."⁴ Both systems named a human supervisor by design. So when they caused harm, the question was never who among no one, but whether the assigned human met the duty the design gave them.

What the two crashes settled

Tempe, Arizona, the night of March 18, 2018. The National Transportation Safety Board, which investigates crashes to find their cause, found the Uber system detected Elaine Herzberg 5.6 seconds before impact. It never classified her as a pedestrian: it read her as a vehicle, then an unknown object, then a bicyclist, and never predicted her path. Its design "precluded emergency braking" on its own and relied on the operator.³ The safety driver, Rafaela Vasquez, was streaming video on her phone and looked up about a second before impact. Her failure to monitor because she was visually distracted was the NTSB's probable cause, the over-reliance on a usually-reliable system the paper covers under automation bias.³

The NTSB decides cause, not liability. The legal outcome came from the county. A Maricopa County grand jury indicted Vasquez for negligent homicide in August 2020, and in July 2023 she pleaded guilty to a reduced charge of endangerment and got three years of supervised probation.⁵ Uber, the company, was not criminally charged, because a prosecutor's office that reviewed the question found no basis for it.⁶

The design problem Matthias pointed at shows itself here, though not where his stronger claim predicted. The NTSB faulted Uber's inadequate safety culture most heavily of all, yet the company faced no charge and the only person prosecuted was the individual at the wheel, who got probation. This is the many-hands problem: responsibility spread across so many people that no single hand carries enough to bear the weight, the shape the normal accidents lesson describes. It is real, and it is not the same thing as no one being holdable at all.

Tesla's Autopilot leaves two records. The NTSB investigated a fatal 2016 Autopilot crash in Williston, Florida, and put the cause on the two drivers, not the car.⁷ Then the regulator moved on the system itself. The National Highway Traffic Safety Administration reviewed 467 Autopilot crashes and found at least 13 that involved a fatality. It concluded that weak controls for keeping the driver engaged, paired with a system easy to over-trust, "resulted in a critical safety gap between drivers' expectations ... and the system's true capabilities."⁴ In December 2023 Tesla recalled every Autopilot-equipped car it had sold, 2,031,220 vehicles.⁴ The fault landed on the manufacturer's controls, and the recall on the manufacturer, not on an empty space where accountability should have been.

The legal gap closes faster than the moral one

Two answers to Matthias are on offer. The first changes the law so blame is not required: strict, or no-fault, liability, where you pay for the harm your product causes without anyone proving you were at fault. The EU has taken this route for software, but two facts cut against the confident version. Its dedicated tool, a proposed AI Liability Directive built around forcing disclosure of an AI system's records and presuming its opacity caused the harm, was withdrawn by the European Commission in 2025 for want of foreseeable agreement.⁸⁹

What did pass, the revised Product Liability Directive, is real and aimed squarely at the gap. It defines software, including AI, as a product. It keeps a maker liable when it "designs a product with the ability to develop unexpected behaviour" that causes harm. And it lets a court presume a defect where the claimant faces "excessive difficulties, in particular due to technical or scientific complexity."¹⁰ But it does not apply until December 2026, and only to products, not to an AI service or a standalone algorithmic decision. The law that answers Matthias is later and narrower than "the law has this covered" would suggest.

The moral question is the harder one. Its clearest witness is a philosopher who denies the gap exists. In a 2021 paper, "There Is No Techno-Responsibility Gap," Daniel Tigard argues that responsibility is plural: even where no one can be held to account for a machine's act, we can still attribute values to it and demand answers. But he draws the same line this lesson draws, and says which side is easy: "regulatory gaps can in principle be repaired with new regulation ... while potential gaps in moral responsibility are far less clear, and thereby less easily resolved."¹¹ The person most sure the gap is an illusion grants that the legal fix is the reachable one.

The instruments built for the gap reach for the same tools: no-fault liability, forced disclosure, and a duty to log what the system did.¹⁰ They are aimed at the design problem, and they leave Matthias's stronger claim untouched. No fully autonomous system, running with no human assigned to catch it, has yet killed someone and reached a courtroom. Until one does, the claim that no one could be held remains what Matthias offered in 2004: an argument, not a result.