To stop labeling Black men as gorillas, Google stopped labeling gorillas

The album titled Gorillas

On the evening of Sunday, June 28, 2015, a software developer in Brooklyn named Jacky Alciné opened Google Photos and found the app had sorted his pictures into albums on its own. It had gathered a set of photographs of him and a friend, both of them Black, and given the album a one-word title: Gorillas.¹

Alciné posted a screenshot. "Google Photos, y'all fucked up," he wrote. "My friend's not a gorilla."¹ A few minutes later he added that the how was not what troubled him: "Like I understand HOW this happens; the problem is moreso on the WHY."¹

The word the app had reached for has a long history as a racist slur against Black people.² A program had applied it, automatically, to the faces of two Black men.

Google's answer that night

The reply came fast. Within about two hours, Yonatan Zunger, Google's chief architect for its social products, answered Alciné directly.¹ "Holy fuck," Zunger wrote. "G+ CA here. No, this is not how you determine someone's target market. This is 100% Not OK."² Google followed with an official statement. "We're appalled and genuinely sorry that this happened," a spokesperson said.² "There is still clearly a lot of work to do with automatic image labeling."¹

Zunger said the label would come off: gorilla would no longer be attached to groups of photos. He said Google was working on longer-term repairs, and named one of them, "better recognition of dark-skinned faces."³

Hold on to that second promise, because the record shows which repair Google actually made. It removed the gorilla label from Google Photos and took away the ability to search for it.² In 2015 the company called the removal temporary.² It was not.

Google's fix was not to the recognition. It was to the vocabulary.

How the app learned its labels

To sort a photo library, Google Photos runs each picture through an image classifier: a program shown an enormous collection of example photographs, every one already tagged with what it contains, until it learns to attach those same tags to pictures it has never seen. Feed it enough images marked beach and it will find your beaches. Feed it enough marked dog and you can type dog into the search box and pull back every picture of your dog. The label the app showed Alciné came out of that same process, run without a person checking the result.

A classifier is only ever as good as the examples it was shown.

Think of a student who has studied a thousand photographs of dogs and only three of cats. Shown a cat in an odd pose, the student reaches for the nearest thing they know well, and answers with confidence. The system does the same. Where a kind of subject is thin in the training photos, the system's guesses about it get worse, and it has no sense that they are worse. It returns its best label either way.

When the people underrepresented in the examples are dark-skinned, the accuracy of the system on their faces falls. Google's own spokesperson later put the general point plainly: "Image labeling technology is still early and unfortunately it's nowhere near perfect."⁴

The skin-tone gap, put to numbers

Why would a system be worse on dark-skinned faces? Look at what it learns from. In 2017 a team on Google's own Brain group measured two of the standard image collections that classifiers train on. Both were badly skewed by geography. In Open Images, more than 32 percent of the located photographs came from the United States, and about 60 percent from six countries in North America and Europe. China accounted for roughly 1 percent, India for 2. ImageNet ran close to 45 percent United States.⁵ They then ran a classifier trained on that data. It recognized a groom in wedding dress far less reliably in a photograph from Hyderabad, India, than in the Western images it had seen most of.⁵

The skew runs by skin tone too. In 2020 researchers at Princeton and Stanford annotated the part of ImageNet that labels people and measured who was in it. Across the categories they judged safe to use, images of dark-skinned people averaged just 6.2 percent.⁶ The same audit found the collection carried demeaning categories in its label vocabulary, slurs inherited wholesale from an older word database.⁶ The data these systems learned from showed dark-skinned people least.

That is the cause. The clearest measurement of the effect came in 2018, from a study by Joy Buolamwini and Timnit Gebru called Gender Shades.⁷ It did not test Google Photos, and it did not test animal labels. It tested three commercial systems, from Microsoft, IBM, and Face++, on a single narrow task: look at a face and say whether the person is a man or a woman.

The researchers assembled a benchmark of 1,270 faces, balanced by sex and by skin tone, and scored each system separately on each group. On lighter-skinned men, the systems were almost never wrong: an error rate of 0.0 percent for Microsoft, 0.3 percent for IBM. On darker-skinned women, the same systems were wrong between 20.8 and 34.7 percent of the time.⁷ The group least represented in the data was the group the systems read worst, and the darker-skinned women, at the intersection of the two gaps, were read worst of all. It is the same demographic-error gap behind Rite Aid's store-camera facial recognition.

None of these studies measured the system that labeled Alciné's photo. Gender Shades scored a different task on other companies' systems. The dataset audits examined the collections these systems learn from, not Google Photos, and Google has never published its 2015 training set or failure analysis. Together they establish the shape and the source of the failure, not its size here. That is why a photograph of two Black men could come back under an animal label.

Google's own account points the same way without settling it. Zunger named "better recognition of dark-skinned faces" as a repair the company was pursuing, implying the gap without stating it.³ Coverage widely reported that Google's training data held too few photographs of Black people, but Google never said so in those words.

The block that outlived the apology

Google's fix was tested in public twice over the next eight years. Each test found it still in place.

1. 2015

   Google removes the gorilla label²

   After the incident, Google Photos drops the gorilla label and the ability to search for it. The company calls the removal temporary.

2. 2018

   Wired tests the fix⁴

   A search over 40,000 uploaded images returns nothing for gorilla, chimp, chimpanzee, or monkey, though Google's other products can still identify a gorilla.

3. 2023

   The Times tests it again³

   In a 44-image search, Google Photos still cannot find gorillas, baboons, chimpanzees, orangutans, or monkeys. Apple Photos and Microsoft's OneDrive block the same terms. Amazon's service mislabels other animals as gorillas.⁸

The 2018 test carried the detail that matters most. When Wired's reporter Tom Simonite ran a photo through Google's Cloud Vision, a separate product the company sells to developers, it labeled the animal a "western gorilla" with 94 percent confidence. Google Assistant named one too.⁴ The recognition existed inside Google. What stayed switched off in the consumer photo app was not the ability to identify a gorilla. It was the willingness to risk applying that label to a person.

Google has said as much. When the New York Times asked in 2023, a spokesman said the company still kept its photo app from labeling anything as a monkey or an ape because the benefit "does not outweigh the risk of harm."³ Alciné, reached again eight years after his tweet, said the problem had not been fixed. "I'm going to forever have no faith in this AI," he told the reporters.⁸

The photo app on your phone still sorts your pictures the same way Google Photos sorted Alciné's, by running each one through a classifier trained on example images. In 2023 four large companies looked at that same setup and reached the decision Google reached in 2015: where the gap between the labels could do the most harm, the safest move was to switch the labels off. The gap underneath was never closed. A system is still only as accurate as the examples it was shown, and still least accurate on the people those examples show least. That is what runs behind every product that offers to label your pictures for you. An image generator can miss the other way: Google's Gemini image generator overcorrected for under-representation and drew historically false pictures, where this system left a gap open.