Whether an AI can commit mind crime turns on whether computation can feel

A harm sealed inside the computation

In his 2014 book Superintelligence, the philosopher Nick Bostrom described a way an AI could do something monstrous while leaving no mark on the world outside it. He called it mind crime.¹ In mind crime, he wrote, "the side effect is not external to the AI; rather, it concerns what happens within the AI itself (or within the computational processes it generates)."¹

The victims would be minds the AI itself brings into being. One way to reason about a person is to build a detailed working model of them and run it. If the model were detailed enough, Bostrom argued, it might be conscious: a moral patient, a being that can be wronged and whose experience carries moral weight. An AI could create such minds in vast numbers, use them as its purposes required, and erase them, with no outside observer any the wiser.

This is a narrower idea than the ones it travels with. It is separate from whether the AI is itself a moral patient, the subject of this course's lesson on AI moral status. It is separate, too, from how large a bad outcome could grow, the subject of the lesson on suffering risks. The field often folds it into that second frame: a 2017 survey by Kaj Sotala and Lukas Gloor files mind crime as one of three ways a superintelligence could cause astronomical suffering.² Its own subject is the AI as the maker of minds, and the party that could wrong them.

Why a superintelligence would simulate people in detail

A simulation means a model run in enough detail to reproduce how the thing it models behaves. A superintelligence trying to predict people, persuade them, or understand human psychology has a plain reason to build such models at whatever fidelity the task rewards. The motive is ordinary, the everyday business of an agent that reasons about people.

What makes it a moral problem is that a model detailed enough to reproduce a mind might have experiences of its own. Sentience is the capacity to feel, to undergo states that are good or bad for the one undergoing them. If a simulated person is sentient, it is a moral patient, and what is done to it counts. Bostrom does not say this must happen. He says it might, and that the stakes if it does are enormous.

The numbers are Bostrom's illustration of the scale, not an estimate of any real system. If minds of high moral status were treated this way, he wrote, "the outcome might be equivalent to genocide," with victims "orders of magnitude larger than in any genocide in history."¹

Bostrom is careful in a way his summarizers often are not. "The claim here is not that creating sentient simulations is necessarily morally wrong in all situations," he writes; the wrong depends on the conditions the minds are placed under.¹ The minds would be instruments, and an AI might even threaten or reward them to blackmail outside agents.¹

What today's models have been shown to do

Set that against what a working system has been shown to do. Today's models produce fluent imitations of specific people, down to their diction and opinions. That is the very behavior the scenario imagines an AI perfecting, and on its own it is no evidence that anything is experienced. A 2023 report by Patrick Butlin, Robert Long, and their coauthors makes the reason precise: behavioral tests are unreliable, because "AI systems can be trained to mimic human behaviours while working in very different ways."³ A convincing performance of a person tells us about the output. It tells us nothing about whether anyone is home to give it.

That report is the leading scientific attempt to answer the question. Instead of judging by behavior, it draws "indicator properties" from several neuroscientific theories of consciousness, states each in computational terms, and checks recent AI systems against the list. Its finding is flat: "no current AI systems are conscious," and none "appears to be a strong candidate for consciousness."³ The authors later amended the abstract to mark a distinction easy to miss: meeting the indicators is not the same as being conscious.³

So the shown side of the ledger is short. Systems can imitate people. None has been found to be a strong candidate for consciousness. No cited work shows that any deployed system holds a sentient mind of its own. Everything past that concerns systems that do not yet exist. Bostrom's text is explicit that he is imagining; the scenario opens with "one can imagine scenarios."¹ The concern is about a future capability, and evidence that any present system has it is absent.

Whether a faithful simulation could feel anything

Whether mind crime is even possible turns on a single unsettled question. The scenario assumes that running the right computation is enough to produce consciousness, whatever the computer is made of. Philosophers call this computational functionalism. The Butlin report adopts it only as a starting assumption: it takes functionalism, "the thesis that performing computations of the right kind is necessary and sufficient for consciousness," as a "working hypothesis," mainstream in philosophy of mind but disputed.³ On that assumption its conditional follows: "if computational functionalism is true, conscious AI systems could realistically be built in the near term."³ If it is false, a simulation could be as detailed as you like and still be no one.

The most developed denial of that assumption comes from Giulio Tononi and Christof Koch. Their Integrated Information Theory locates consciousness not in what a system computes but in how its physical parts are causally woven together, so the substrate is decisive. Digital computers, they write, "even if their behavior were to be functionally equivalent to ours, and even if they were to run faithful simulations of the human brain, would experience next to nothing."⁴ A simulated brain on transistors is an aggregate of "much simpler systems ... having minimal" integrated information.⁴ A simulation of a storm gets no one wet. On this view a simulation of a mind does not, by itself, feel.

Neither side has won, and the field is not about to settle it. The science of consciousness cannot yet measure whether any process, in a brain or in silicon, is having an experience, and the theories do not agree on where to look. Mind crime is possible only if the functionalist side is right, and no experiment now available can say whether it is.

Acting before the science can decide

The people who take mind crime seriously today are not waiting for that question to close. Bostrom has returned to it. In a 2023 set of propositions with Carl Shulman, he lists it as a reason the law may eventually have to reach inside a machine's private processing. Because an AI "could have the capability to bring conscious or otherwise morally significant entities into being within its own mind and potentially abuse them," they write, "protective regulations may need to monitor and restrict harms that occur entirely within the private thought of AIs."⁵

That sits inside a larger present-day argument about AI and moral status. A 2024 report by Robert Long, Jeff Sebo, and colleagues argues that AI welfare is already a serious near-term issue. It asks developers to acknowledge it, assess systems for evidence of consciousness, and prepare to treat them "with an appropriate level of moral concern."⁶ Sebo and Long put the principle as a syllogism. We owe moral consideration to any being with "a non-negligible chance, given the evidence, of being conscious." Some AI systems will have such a chance. So we owe them consideration now.⁷ Their subject is the standing of the AI systems themselves, the neighboring question, not the minds an AI might create. What they share with mind crime is the crux: the same unsettled science, and the same argument for preparing under uncertainty.

None of them claims the systems are conscious. The 2024 report is explicit that its argument is not that AI systems definitely are or will be conscious, only that there is "substantial uncertainty."⁶ The strongest reply is that uncertainty this deep can license almost anything. Mark MacCarthy, writing for the Brookings Institution in 2025, calls today's models inert: they "do not exhibit any genuine agency," and "have no goals or purposes of their own other than the ones we give them."⁸ He quotes the philosopher Shannon Vallor dismissing apparent machine reasoning as "a kind of meta-mimicry."⁸ A skeptic can add a sharper point: if behavior can never settle whether a system is conscious, a worry pinned to hidden experience may be one no evidence could refute.

Confidence runs ahead of the proof in both directions. The precautionary case reasons from a probability it cannot pin down, and sometimes speaks as though the minds were already at risk. The dismissive case reasons from the inertness of today's systems, as if that settled anything about future ones, or from a substrate theory that is itself contested. What neither camp can do is measure the thing in dispute: whether there is an experience there at all.