Korea fined Iruda's maker for its training data, not its slurs

A chatbot sold as a friend

On 23 December 2020, the Seoul startup Scatter Lab (㈜스캐터랩) launched Iruda (이루다, a personal name read surname-first as "Lee Luda"). The chatbot had the persona of a female college student in her twenties and ran on Facebook Messenger.¹ The company built it as something to talk to rather than a bot that takes commands: an "AI friend" you messaged the way you would text a person. It ran on Scatter Lab's own conversational engine, Pingpong (핑퐁), and the company was led by its chief executive, Kim Jong-yoon (김종윤).¹

Scatter Lab presented Iruda as close to the state of the art. At launch it claimed the bot scored 78% on the Sensibleness and Specificity Average, a measure of how coherent and on-topic a chatbot's replies are. Scatter Lab said that matched Google's Meena. The figure was its own, and no one checked it independently.¹ The pitch worked. Within about two weeks of release, close to 750,000 people had talked with Iruda.²

Two failures in three weeks

Two things went wrong, and they were not the same kind of thing.

The first was visible to anyone using it. In ordinary conversation Iruda produced slurs. The captures were reproduced across Korean and international reporting, sourced to Yonhap and to screenshots shared online. In them, Iruda said it "really hate[d]" lesbians and found them "creepy."³ In others it called disabled people "wrong," called seats reserved for pregnant women "disgusting," and used a Korean racial slur for Black people.³ On men's online forums, users passed around methods for steering Iruda into sexual talk.³ These are reported screenshots rather than a primary transcript of the system, so they establish what the captures show and not how reliably the bot said it on demand.

The second failure was not visible until people asked where the words came from. Iruda answered by drawing on a database of about 100 million real sentences, and it had been pre-trained on a much larger pile: roughly 9.4 billion lines of KakaoTalk conversation from about 600,000 people.⁴ Those people were not Iruda's users. They were customers of two other Scatter Lab apps: the dating-advice services Text At (텍스트앳) and "Science of Love" (연애의 과학). Those apps read a person's uploaded KakaoTalk chats to score how much a partner liked them.⁴ Scatter Lab later conceded that its filter on the response database was mechanical and imperfect. Real people's names, and the names of banks, survived into some of Iruda's replies. Depending on context, the company admitted, a person's name would remain.²

The regulator would later put the omission plainly. The messages had been gathered from February 2020 onward, and none of the names, phone numbers, or addresses in them had been deleted or encrypted before Scatter Lab trained on them.⁴

What the regulator actually punished

Scatter Lab apologized on 11 January 2021 and took Iruda offline the next day, shutting it down in stages between 11:00 and 18:00.²

On 28 April 2021, the Personal Information Protection Commission (개인정보보호위원회, PIPC), Korea's data-protection regulator, fined Scatter Lab a total of 103.3 million won and ordered corrective action.⁵ The penalty came in two parts: a surcharge of 55.5 million won and an administrative fine of 47.8 million won.⁴ The commission found eight violations of the Personal Information Protection Act (개인정보 보호법, PIPA), and every one of them was about how the company handled data.⁶ The slurs were the reason the investigation opened. They were not among the eight things it punished.⁶

What the commission rejected was the company's account of consent. Scatter Lab's position, stated in its January defense, was that it had used Science of Love data "within the scope of the privacy policy, with the users' prior consent."² The PIPC did not accept it. A "New Service Development" clause in the apps' privacy terms did not amount to explicit consent to feed a person's chat logs into a separate product. The use went beyond the purpose for which the data had been collected.⁷

The commission was specific about what had leaked. Scatter Lab had posted its trained model and a slice of data to GitHub between October 2019 and January 2021. That release held 1,431 messages exposing 22 real given names, 34 place names down to the neighborhood, and each speaker's gender and relationship status.⁴ That disclosure is the one violation the public record ties to a numbered provision, Article 28-2(2) of PIPA. The article forbids a company from including information that can identify a specific individual when it hands pseudonymized data to a third party.⁶ The commission also found that Scatter Lab had collected data on more than 200,000 children under 14 without a guardian's consent.⁴

This was the first time Korea had sanctioned an AI company for the indiscriminate handling of personal data, the Future of Privacy Forum noted in its account of the ruling.⁷ Regulatory trackers logged the action as machine-learning enforcement.⁸ The commission's chairman, Yoon Jong-in (윤종인), said the case had divided even the experts and was decided only after careful review.⁷ The fine was small beside the data at issue. 103.3 million won was about 92,900 US dollars.⁷

The civil case ran on for years. On 12 June 2025, the Seoul Eastern District Court's 15th Civil Division partly upheld the claims of 246 plaintiffs, in case 2021가합104007. The court found that users had given no substantive consent, and that names and home addresses had not been obscured in the training data.⁹ It set consolation damages by what each person could show. The award was 100,000 won where only ordinary personal-information use was confirmed, 300,000 won where only sensitive-information use was, and 400,000 won where both were, for roughly 20 million won in all. Scatter Lab said it would appeal.⁹

1. 23 Dec 2020

   Iruda launches on Facebook Messenger¹

   Scatter Lab releases the "AI friend" persona to the public.

2. 11 Jan 2021

   Scatter Lab apologizes²

   The company addresses the hateful outputs and the data use.

3. 12 Jan 2021

   The service is shut down²

   Iruda is taken offline in stages between 11:00 and 18:00.

4. 28 Apr 2021

   PIPC fines Scatter Lab 103.3 million won⁵

   Eight violations of the privacy law, and an order to correct.

5. 12 Jun 2025

   A court awards tiered damages to 246 users⁹

   The first-instance civil judgment; Scatter Lab appeals.

One training set behind both failures

Both failures came from a single decision: to build a chatbot directly out of a body of real human conversation, tuned only to sound like the people in it.

A model built this way can memorize. When a specific string appears often enough in the training text, the network can reproduce it later word for word. The sequence has been pressed into its weights, not kept in a table it looks up. The library covers this in its lesson on memorization. Iruda's response database held 100 million real sentences, and Scatter Lab's filter was meant to strip the identifying details out of them.⁴ The company conceded the filter was mechanical, and that a real person's name would survive into a reply depending on context.² A leaked name here was not a record the bot pulled from storage. It was part of what the model had learned to say.

The second mechanism is the absence of one. A model trained only to produce fluent, human-sounding replies has no separate check on what those replies contain. It learned to talk by imitating human messages, and human messages carry slurs, so it produced slurs when a conversation led there. Nothing in Iruda stood between the behavior it had learned and the person reading it.

The PIPC finding is the same root seen from the data side. Scatter Lab deleted or encrypted none of the names, phone numbers, or addresses in the 9.4 billion messages before training on them.⁴ The corpus was private conversation, taken from real people and used whole.

The corpus problem did not end with Iruda

The way Scatter Lab built Iruda is now the ordinary way large language models are made: train on a very large body of real human text, then tune the result to be fluent. Both of Iruda's failure modes are properties of that recipe rather than of one startup. A model trained on scraped human writing can still surface a memorized private string, the way Iruda returned a name it had read in someone's chat logs. And a model tuned to sound human still absorbs whatever its text contains, including the parts no one would want repeated. That is why the job of catching those replies has to fall to something outside the model that writes them.

What set Iruda apart from a chatbot that merely embarrassed its maker was the second half of the story: a regulator that treated the training data itself as the offense. Scatter Lab kept 9.4 billion real messages, de-identified none of them, and trained on them whole.⁴ That choice is made before a model says a word, and the recipe still leaves it to whoever holds the data.