Facebook translated ‘good morning’ as ‘attack them,’ and police made the arrest

A good-morning post and an arrest

In mid-October 2017, a construction worker posted a photograph of himself on Facebook. He was standing beside a bulldozer at the site where he worked, in Beitar Illit, a settlement in the West Bank. The caption, in colloquial Arabic, was a good-morning greeting.¹

Facebook’s software translated the caption automatically. In Hebrew it rendered the greeting as “attack them.” In English it rendered the same words as “hurt them.”¹ Officers saw a man beside a bulldozer, a vehicle that had been used in past ramming attacks, above a caption their screens told them read “attack them.”²

They arrested him at the site on suspicion of incitement.³ The account that broke the story, in the Israeli newspaper Haaretz, reported that no Arabic-speaking officer read the original post before the arrest.¹ The Times of Israel reported that the officers “relied solely on Facebook’s automatic translation.”³ Every version of that detail traces back to the Haaretz report. No police statement has confirmed or denied it. Haaretz did not name the worker, who declined to speak to the paper.¹

He was held for several hours. Once someone read the Arabic, the error was plain, and he was released.¹ Facebook apologized. An engineering manager in its language-technologies group gave the company’s account of what had gone wrong.

How one word became ‘attack them’

Machine translation does not read a sentence, understand it, and restate it. It predicts the most probable text in the target language, learned from millions of example translations. Handed an input it has seen little of, it still returns its best guess, and the guess can be confidently wrong. Models also see far less of most languages than of English, and their tokenizers split those languages into more pieces, so a model guesses worst exactly where its training text was thinnest.

Arabic makes that thinness worse. The written, formal language, Modern Standard Arabic, differs sharply in grammar and vocabulary from the spoken dialects people actually use, and the dialects have no standard spelling, so writers improvise.⁴ A system trained mostly on formal text is a poor fit for a casual dialect post. In one study, a system trained on 1.5 million words of dialect scored 6.3 to 7.0 BLEU points higher than one trained on a hundred times more formal text, where BLEU is a standard 0-to-100 score for how close a machine translation sits to a human one.⁴

The posted word was يصبحهم. Linguists who examined it read it as a colloquial morning blessing, short for a phrase meaning roughly “may God grant them a good morning,” rather than the textbook greeting صباح الخير that a class teaches first.⁵ Facebook’s system reportedly reached for a rare sense of the root, “to attack in the morning,” that carries no weight in everyday speech, and the string also resembles a form meaning “to hurt.”⁵ The Times of Israel reported that a single letter separates the colloquial greeting from the reading the software chose.³

None of this is exotic. A system that guesses the likeliest words, fed a short, casual, dialectal string it had seen little of, returned an unlikely and violent reading. That is the kind of error such a system makes, and it will make others like it.

The decisive failure was the missing check

The translation error alone harmed no one. It became an arrest when a person treated the machine’s output as a fact. Researchers have a name for the tendency behind that. Automation bias, as the engineer Mary Cummings defines it, is the human inclination to stop searching for contradictory information once a computer has offered an answer that looks right.⁶

It is measurable. In one study Cummings cites, 39 of 40 people monitoring a simulated flight followed an incorrect automated instruction even though the evidence to catch it was in front of them and checking was possible.⁶ These were not careless people behaving strangely. This is what ordinarily careful people do when a machine sounds certain.

The check the arrest lacked was cheap: one officer who reads Arabic, or one call to someone who does. Any Arabic speaker, Facebook itself later noted, could see that the output did not match the post.² It is tempting to file the whole episode under bad machine translation. The model behaved exactly as such models do. The arrest required a second thing: a person who read “attack them” on a screen and acted before reading the words underneath it.

The same pattern runs in asylum screening and moderation

The same arrangement is in use now: a machine translation, a high-stakes decision, and no one in the loop who reads the language. The United States asylum system routes applicants through an app, CBP One, that translates official instructions by machine. Translators at Respond Crisis Translation documented errors that block access. In Haitian Creole the app renders “Customs,” the border agency, as “koutim,” a word for social customs and traditions, so the instruction no longer points at the agency it names.⁷

Content moderation is the other place it runs, and Meta hit the wall again in its own product. In October 2023, Instagram’s automatic translation turned Arabic user bios that paired “Palestinian,” a flag, and “alhamdulillah” — praise be to God — into “Palestinian terrorists are fighting for their freedom.”⁸ Meta apologized, saying it had fixed a problem that briefly caused inappropriate Arabic translations in some of its products.⁹

In each system the machine’s output reaches someone who will act on it — an officer, a screening decision, a removed account — faster than it reaches anyone who can read the original.