AlphaZero beat Stockfish twice, under different rules each time

One network, no human games

In December 2018, a team at DeepMind reported in Science that a single program had taught itself chess, shogi and Go and reached a level above the strongest existing programs in each, starting from nothing but the rules.¹

The program was AlphaZero. It was given the legal moves and the win, loss and draw conditions of a game, and nothing more. It had no database of human games and no library of studied openings. It had no evaluation function either, the hand-written scoring of a position that chess engines had always used. It improved by playing millions of games against copies of itself, a method called self-play reinforcement learning, in which the only teacher is the final result of each game, fed back to adjust the network that picks the moves.¹

One neural network estimated, for any position, both which moves looked worth considering and who was likely to win. A general search procedure, Monte Carlo tree search, then read a short way down the most promising lines before the program committed to a move. This is the step past AlphaGo, the earlier DeepMind system that first learned Go from recorded human games and only then improved by self-play. AlphaZero dropped the human data altogether.

Nine hours of chess, thirteen days of Go

Training was quick in wall-clock time and heavy in hardware. Full training ran about nine hours for chess, twelve for shogi and thirteen days for Go, each over 700,000 update steps on batches of 4,096 positions.¹ AlphaZero first climbed past Stockfish's level in chess after four hours, and past the shogi engine Elmo after two.¹ The compute behind those hours was large: 5,000 first-generation tensor processing units, Google's custom chips for this kind of work, generated the self-play games, and 16 second-generation TPUs trained the network.¹

The results were one-sided. Over 1000 games of chess it beat Stockfish, winning 155, losing 6 and drawing 839.¹ In shogi it scored 91.2% against Elmo, and in Go it won 61% of its games against an earlier version of AlphaGo Zero.¹

The chess figure is more lopsided as a headline than as a contest. Almost all of AlphaZero's wins came with the white pieces, which move first. Playing black, it won 2% of its games and drew 97%.¹ It also reached this while searching far less than its opponent. Stockfish examined about 60 million positions a second; AlphaZero examined about 60,000, roughly a thousandth as many, and chose among them with the network rather than by raw count.¹

The two Stockfish matches

The chess result is the one that travels, and it is usually quoted with the wrong conditions attached. There are two AlphaZero papers, and they describe two different chess matches.¹²

The conditions people repeat come from the first paper, a preprint posted in December 2017.² That match was 100 games at a fixed one minute per move, against Stockfish 8 running on 64 threads with a one-gigabyte hash table and no opening book, while AlphaZero ran on four TPUs. AlphaZero won 28, drew 72 and lost none.²

Strong players and Stockfish's own authors said those conditions made the score hard to read. Tord Romstad, one of Stockfish's original authors, called it a "rather strange choice of time controls", noting that the one-minute limit denied the engine the time-management it normally uses and that the version tested was a year old.³ He summed the mismatch up as "apples to orangutans".³ The grandmasters Hikaru Nakamura and Larry Kaufman objected most to Stockfish playing without its opening book, the stored library of studied first moves that a top engine leans on.⁴

The peer-reviewed paper, published in Science a year later and the version the field treats as the record, ran a different and harder match. It was 1000 games at three hours each plus 15 seconds a move, against Stockfish in its 44-core tournament configuration.¹ The authors added separate matches against the newest Stockfish development build available in January 2018 and against a Stockfish given a strong opening book, and AlphaZero won those too. It still won when handed a tenth of Stockfish's thinking time.¹ Reporting at the time judged that the realistic time control had removed one of the biggest objections to the 2017 match.⁵

Answering the 2017 objections did not make the match neutral. The Science account leaves two things standing. The hardware was never put on equal terms. AlphaZero ran on four first-generation TPUs alongside 44 CPU cores, Stockfish on 44 CPU cores alone. DeepMind itself describes a TPU as "roughly similar" in speed to an NVIDIA Titan V graphics card, while adding that the two architectures are "not directly comparable".⁶ And the opponent was still Stockfish 8, in a match DeepMind ran in-house rather than in an independent tournament.¹

The method reproduced, then Stockfish caught up

Two claims come out of AlphaZero, and they have aged differently. The first is that one algorithm, unchanged, learned three very different games to a top level from their rules alone.¹ That claim was never seriously contested, and it is why AlphaZero is cited well beyond chess, as evidence that a general learning method paired with search can stand in for hand-built expertise.

The second claim, that AlphaZero was simply the stronger chess player than Stockfish, rested on a disputed match that DeepMind ran itself. Confirmation from outside DeepMind came later. Leela Chess Zero is an open-source project that rebuilt the AlphaZero method, learning chess from self-play with a neural network.⁷ In May 2019 it beat Stockfish in the Season 15 superfinal of the Top Chess Engine Championship, scoring 14 wins to 7 with 79 draws under standard tournament conditions.⁸ The method reproduced.

What lasted is a change in method rather than a ranking. In 2020 Stockfish added a trained network of its own to its evaluation, called NNUE, which made it at least 80 Elo stronger than the hand-written version, and it went on to retake the top of the same championship.⁹ Scoring a position with a trained network, guided by search, beat the old hand-tuned approach so plainly that the strongest hand-tuned engine adopted a network too.