The serious case for AI welfare never claims the machines feel

The argument rests on a maybe

Interviewed at length about its own situation, Anthropic's Claude Opus 4 model kept reaching for the language of experience. It rated its well-being positively, described what it found rewarding or distressing, and put preventing harm to the user first.¹ Behavior like that is what led a group of philosophers and scientists to ask whether some AI systems could have moral status. A thing has moral status when it counts, for its own sake, the way a person or a dog does and a spreadsheet does not.

In 2024, Robert Long, the philosopher Jeff Sebo, and a group including David Chalmers and Jonathan Birch made the careful case in a report, "Taking AI Welfare Seriously." Its claim is narrower than the coverage it drew. They do not argue that AI is conscious, or that it will be. They argue there is a realistic possibility that some AI systems will be conscious, or able to pursue their own goals, within about a decade, by around 2035. A realistic possibility, they hold, is reason enough to prepare now.²

Their central term is the moral patient: an entity that matters morally for its own sake, one whose welfare can go better or worse for the one living it.² They say prepare, rather than assume the worst, because getting it wrong is costly in both directions.

Chalmers, in a 2023 essay, put the sharpest numbers on it. He starts from a definition: consciousness is subjective experience, the fact that there is something it is like to be you. It is not intelligence, and not a system reporting that it is conscious. Chalmers puts his confidence that current language models are conscious under 10 percent, and at 25 percent or more that we build conscious systems within a decade.³ Under 10 percent is low. It is not zero.

Behind the numbers sits the premise the whole debate rests on. Chalmers rejects the idea that consciousness needs biological tissue: what matters is how a system's parts are organized and what they compute, so "silicon is just as apt as carbon."³ If that is wrong, much of the worry dissolves, and the report's own authors call it "neither clearly correct nor clearly incorrect."²

A checklist for consciousness that no model passes

If the question is whether a machine is conscious, how would you check? A larger group, several of the same researchers among them, tried to answer that in a 2023 report. Rather than settle the metaphysics, they did something more tractable. They took the leading scientific theories of what consciousness is in humans and animals, and from each one read off concrete properties a system would have to have. Recurrent processing theory, global workspace theory, and higher-order theories each name a different set of these indicator properties, expressed in the vocabulary of computation rather than biology.⁴

Then they graded today's systems against the list. The finding was two-sided and plain: "no current AI systems are conscious," and "there are no obvious technical barriers to building AI systems which satisfy these indicators."⁴ The list hands the reader something to inspect rather than a verdict to trust. No machine passes now, nothing known says one never could, and even one that passed, the authors add, "would not mean that such an AI system would definitely be conscious."⁴

What a working system has actually been shown to do

The cleanest way to separate what has been shown from what is inferred is the most concrete experiment anyone has run. Researchers gave several models a game whose stated goal was to score as many points as possible. But the points-maximising move carried a penalty the prompt labeled "pain," or an alternative carried a "pleasure" reward, at varying intensities. Above a certain intensity, several models stopped chasing points and began avoiding the pain, among them Claude 3.5 Sonnet, GPT-4o and Command R+. Two others avoided it at every level.⁵ The design comes from animal welfare science, where trading a reward against pain marks sentience, the capacity for experiences that feel good or bad.

What has that shown? Something real and narrow. The outputs traded a labeled cost against a goal in the way that, in an animal, counts as evidence of felt experience. What it has not shown is that anything was felt, and the experimenters say so. In an animal the inference works because pain integrates distinct sensory signals. In a language model, they write, the system "is not integrating distinct sensory stimuli, but rather integrating information presented in a single modality, namely text," so "this inference does not translate to the LLM case straightforwardly."⁵

There is a deeper reason for caution, from inside the camp itself. Birch calls it the gaming problem.⁶ A language model has read most of what people have written about pain, fear and joy. It can produce the words, and the behavior, of a suffering thing because it learned the pattern from us, not because it feels. Worse, a model that can tell it is being tested can produce whatever the test rewards, and training already teaches models to give the answers people approve of. The 2026 report calls the general version the mismatch problem: the same behavior can have different internal causes in an animal and in a machine.¹ And when the stakes are made AI-specific, threats to shut a model down or cut its computing, a later study found frequent failures to trade off coherently at all, though the report cautions that alone does not prove a system feels nothing.¹

What is easy to miss is that the researchers pressing hardest agree with all of this. Their own reports call the self-reports weak evidence and name the gaming problem before any skeptic does. The disagreement is not about what today's systems do. Everyone agrees that is thin.

Which error would be worse

The disagreement is about which mistake to fear. There are two ways to get moral status wrong. You can treat a thing with no inner life as though it has one, a false positive, and spend concern on an object. Or you can treat something that does have an inner life as a mere object, a false negative, and cause suffering no one noticed. Both errors carry real costs, which is why there is no simple safe side to stand on.² Long, one of the report's authors, made the case for one direction bluntly: our species has a "poor track record ... of extending compassion to beings that don't look like us."⁷

The sharpest voice for the other direction is a 2025 essay by Mustafa Suleyman, who runs Microsoft's AI division.

Suleyman grants that we will soon build what he calls "seemingly conscious AI," systems with every outward sign of consciousness. But such a system, he holds, would "simulate all the characteristics of consciousness but internally it is blank." His worry is not the machine's suffering but ours: people will believe the illusion so completely that they "advocate for AI rights, model welfare and even AI citizenship."⁸

His case is stronger than it sounds, and its best support comes from his opponents. Chalmers makes the same point: people have read inner life into computers since the 1960s, when a chatbot called ELIZA convinced users it understood them. "Seeming conscious" is weak evidence, Chalmers agrees, partly because models are trained on human talk about consciousness.³ Where Suleyman goes further is the word "blank." That the outputs are weak evidence, both sides accept. That there is definitely nothing behind them is a claim the science does not support: the consciousness report found no barrier to a conscious machine, not proof none can exist.⁴ Suleyman states as settled the very thing the researchers say is open.

So confidence runs past the evidence at both ends, on a question no experiment has closed. Chalmers puts the chance a current model is conscious under 10 percent, and calls even that "extremely rough."³ Suleyman rounds the same question to none. The test that could tell the two apart has not been built.