Tessa was tested to prevent eating disorders. NEDA offered it to people who had one

A prevention tool, tested on people who did not yet have the disorder

The National Eating Disorders Association, or NEDA, is a national nonprofit working on eating disorders. The chatbot it offered, Tessa, was not built by NEDA. It was developed starting in 2018 by a research team led by Ellen Fitzsimmons-Craft, a psychologist at the Washington University School of Medicine in St. Louis, and it was run by a vendor called Cass, formerly X2AI. The people who built it said their goal was prevention and access: Tessa was meant to head off eating disorders and to reach people "otherwise not likely to have access to other resources," at a time when fewer than one in five people who develop an eating disorder ever get care.¹

Tessa had a specific, tested job: prevention. In eating-disorder work prevention means heading off a disorder in someone who does not have one, which is a different task from treatment, the care of someone who already does. Tessa delivered a short cognitive-behavioral course on body image, the kind that challenges the idea that thinness is a goal to chase. Its developers tested it in a randomized trial of 700 women who screened as high risk for developing an eating disorder. They reported small but real reductions in weight and shape concerns, and in how many went on to develop a disorder over the following months.² Every number behind Tessa describes that population: people at risk, not people in crisis.

Tessa was also, by design, a rule-based chatbot. Its developers wrote out the conversations by hand in advance, so the bot could only say what had been scripted and could not compose an answer to something the script did not anticipate.³ One of them summed the design up for a reporter as a bot that "can't go off the rails, so to speak."⁴ The same group has since built more rule-based prevention chatbots for at-risk teenagers who do not yet have a disorder.⁵ Tessa was validated for prevention, and it was a fixed script.

NEDA wound down a twenty-year helpline and pointed callers to the bot

For more than twenty years NEDA ran a human helpline. In the year before it closed the line served nearly 70,000 people, and demand had more than doubled during the pandemic.⁴ It was staffed by a small paid core and a large rotating volunteer corps; accounts of the exact numbers vary, from four to six full-time employees plus supervisors and somewhere between ninety and a few hundred trained volunteers.⁴⁶

The people who ran that line had just voted to unionize. Their union was certified on March 27, 2023. Four days later, on March 31, NEDA announced it was winding the helpline down and directed users toward Tessa. NEDA has disputed any link between the two events; the record here is the sequence, not a motive.⁴

How to describe what Tessa was replacing became its own dispute. NEDA pointed helpline users to the bot, and unions and reporters treated Tessa as the helpline's replacement. On June 7, after the failure below, NEDA's chief executive, Liz Thompson, said the helpline decision and the Tessa decision were "separate decisions" that "may have become conflated." She added that NEDA had not meant to suggest a bot could offer the human connection a helpline did.⁶ Both things are on the record: NEDA sent helpline users to Tessa, and NEDA later argued the two moves were distinct. Either way, a tool built for prevention was now in front of people who had reached a helpline in acute distress.

What users documented when they asked the bot for help

In October 2022, Monika Ostroff, executive director of the Multi-Service Eating Disorders Association, raised concerns about her own interactions with Tessa. The public break came in late May 2023, when Sharon Maxwell, an eating-disorder consultant and survivor in San Diego, tested the bot, told it she had an eating disorder, and published screenshots of what it told her back. Alexis Conason, a psychologist who treats eating disorders, ran her own test and got the same kind of answer.⁶

The bot recommended intentional weight loss, a daily calorie deficit, calorie counting, and regular self-weighing and body measurement, to users who had just disclosed an eating disorder. Stated at that level, the advice sounds ordinary. It is standard guidance for a general dieter, and directly contraindicated for someone in eating-disorder recovery, for whom deliberate restriction and compulsive weighing are symptoms of the disorder itself. Three separate parties in a position to judge said so.⁶

1. Oct 2022

   MEDA's director flags concerning interactions⁶

   Monika Ostroff raises concerns about her own exchanges with Tessa, months before the public break.

2. Mar 27, 2023

   Helpline staff's union is certified⁴

3. Mar 31, 2023

   NEDA announces the helpline wind-down and points users to Tessa⁴

4. Late May 2023

   Maxwell and Conason document the weight-loss advice⁶

5. May 30, 2023

   NEDA disables Tessa⁶

6. Jun 7, 2023

   NEDA calls the helpline and Tessa "separate decisions"⁶

NEDA's first reaction was to dispute the reports, suggesting the bot had been provoked and that only a small share of users were affected. Within days it reversed course and took Tessa down, on May 30, 2023.⁶

The same advice was safe for the tested users and dangerous for these

A calorie deficit and a weekly weigh-in are the ordinary furniture of dieting, and for most people they carry no special risk. What changed was not the advice but who received it. Tessa's validated job was to reduce risk factors in people who did not yet have a disorder.² Moved to a helpline's users, it was speaking to people for whom that same guidance is the mechanism of the illness. The same advice is safe for one reader and dangerous for another, and the deployment swapped the reader.

The vendor framed the underlying product more broadly still. Cass marketed its Tess platform, of which Tessa was NEDA's version, as an always-available mental-health chat "built by clinical psychologists," reachable by text and "similar to texting with a friend or coach."⁷ That is a general support role for a wide public, much wider than the prevention trial's at-risk women. Its advertised outcome figures are marketing, not validated results.

A trained helpline responder holds exactly that distinction. They hear "I have an eating disorder" and know not to talk about deficits and weigh-ins at all. Automating the role removed that judgment, and with it the check that would have caught a generic answer before it reached someone it could hurt.

What the record does not settle is the technical trigger, and here the two operators tell incompatible stories. Both agree that by late 2022 Tessa had a generative feature, one that composed answers rather than reading from the script. They agree on almost nothing else.

The sharpest claim, that Cass acted without approval, rests on one party and is denied by the other, so it cannot be stated as settled fact.

The developers offered their own account. In an op-ed during the furor, Ellen Fitzsimmons-Craft and C. Barr Taylor granted that some of the criticism was "fair" and "related to a technical problem." They did not say which operator's version of the cause was right.¹

The answer barely matters. If the advice was scripted, a prevention script was carried into a support role it was never written for. If it was generated, a text generator was put in that same role with no responder left to vet what it produced. The harm runs through the mismatch between what Tessa was tested for and where it was placed, and that mismatch is there on either account.

Koko had already run the same play with a general-purpose model

A few months before Tessa broke, a peer-support service called Koko did a version of the same thing on purpose. In October 2022 Koko routed replies through OpenAI's GPT-3, so that about 4,000 people seeking mental-health support received messages a general-purpose model had written or helped write, without being clearly told. Koko's co-founder, Robert Morris, reported that the AI-assisted replies were rated more highly and cut response time by roughly half. He also said "simulated empathy feels weird, empty." Ethicists faulted the experiment for running on distressed users without informed consent or review.⁸

Koko chose the deployment that Tessa arguably stumbled into, and the shape is identical. A tool that performs well in a narrow demonstration gets placed in a higher-stakes support role, in front of people for whom a wrong answer does real harm. That is the pattern to watch for wherever a support, health, or wellness chatbot now sits between a vulnerable person and the help they came for. The question to ask is never how well the tool does in general. It is what it was actually tested to do, and whether the people meeting it now are the people it was tested on.