AlphaStar reached Grandmaster while capped at 22 actions every five seconds

The paper behind the headline

AlphaStar is a StarCraft II player built at DeepMind and described in a 2019 Nature paper by Oriol Vinyals, Igor Babuschkin, and dozens of colleagues.¹ The paper's central result is one sentence: evaluated in the full game against human players, AlphaStar was rated Grandmaster, the ladder's top tier, for all three of the game's races, and above 99.8% of ranked players.¹ Most people remember that sentence in a shorter form, “an AI reached Grandmaster at StarCraft,” and a looser one, “an AI beat the pros.”

The rank came from anonymous games on Battle.net, Blizzard's official online ladder. DeepMind entered the agent under the same conditions as any human: real opponents who did not know they faced a bot, standard maps, and matchmaking that chose who it played.² Its strongest versions were rated 6,275 MMR at Protoss, 6,048 at Terran, and 5,835 at Zerg, where MMR is Battle.net's single skill number for a player. Measured against a pool of about 90,000 active players on the European server, that put the average in the top 0.15%.³

StarCraft drew AI research because it breaks the assumptions that made chess and Go tractable. It runs in real time rather than in turns. It is also a game of imperfect information: each side sees only the part of the map its own units can see. Both players commit to moves while half-blind to the other.¹ AlphaStar learns by reinforcement learning, in which an agent tries actions and is rewarded for the ones that lead to winning, the same family of method behind DeepMind's earlier work on Atari. What made the paper famous was the claim that this recipe could reach the top of a human ladder in a game this messy.

From human replays to a league of rivals

AlphaStar was not built from self-play alone. Its training ran in stages, and human games seeded every one of them.¹

1. Copy strong human players

   The agent first learned by supervised imitation, adjusting itself to reproduce the moves in a public set of 971,000 anonymized human replays from players in the top 22% of the ladder. That step alone produced an agent rated about 3,699 MMR on average, already above 84% of human players.¹

2. Study the best games more closely

   A second pass fine-tuned the agent on 16,000 replays from players above 6,200 MMR, which raised its win rate against StarCraft's built-in elite bot from 87% to 96% in Protoss mirror matches.¹

3. Play a league of opponents

   Reinforcement learning then improved the agents by having them play each other inside what DeepMind calls a league: a population of agents kept training together so that no single strategy can quietly dominate. Three main agents, one per race, are the ones actually deployed. Alongside them run three main exploiters and six league exploiters, which exist only to hunt weaknesses in the others and force them out. Human data stays in the loop even here: each agent is pulled back toward its imitation policy when it drifts, and rewarded for following build orders drawn from human games.¹

The full league ran on 32 third-generation tensor processing units per agent for 44 days, creating roughly 900 distinct agents along the way.¹ This is the sense in which AlphaStar “taught itself”: the reinforcement stage is self-driven, but it begins and stays anchored in human play, which is what sets it apart from a pure self-play system like AlphaZero.

The limits on how it sees and acts

A StarCraft player's skill is partly mechanical: how much of the map you keep track of, and how fast you can issue commands. To keep the result about strategy rather than raw speed, DeepMind constrained both of these, and the exact constraints are the most misremembered part of the whole story.¹

The Nature version saw the game through a camera it had to move, the way a person scrolls the screen, rather than reading the entire map at once. Its rate of action was capped at 22 non-duplicate actions per five-second window.¹ Actions per minute, or APM, is the standard count of how many commands a player issues in a minute, and StarCraft professionals routinely sustain several hundred.⁴ The agent also acted on a delay, choosing its next observation on average about 370 milliseconds ahead and responding within roughly 110 milliseconds.¹

DeepMind then tested whether these limits were doing real work, in ablations that remove a single feature to see what it was worth. Restricting the agent to the camera lowered its performance against a version that read the whole map. Pushing its action rate either below or above the chosen cap also made it play worse.¹ The constraints cost the agent strength. It reached Grandmaster carrying them.

Two agents, one memory

The “it only clicked faster” criticism was not invented. It was aimed at an earlier, looser version of AlphaStar, and the confusion in popular memory comes from treating that version and the ladder one as the same agent.¹

In December 2018, DeepMind recorded a series against two professionals, Dario “TLO” Wünsch and Grzegorz “MaNa” Komincz, and showed it in January 2019.⁴ That agent was, in the paper's words, less restricted: it had no limited camera and read the map fully zoomed out, and it could act more often.¹ It won all ten recorded games, 5-0 against each player.⁴ Two caveats sit inside that “beat the pros” result. TLO is a professional Zerg player who played Protoss, not his own race, in the series.¹ And when DeepMind later put a camera-restricted prototype into a live game, MaNa beat it, making the public record 10-1.⁵

An independent analysis by Rick Korzekwa found that AlphaStar's average action rate in one December game was in fact lower than MaNa's. But the agent banked its budget and spent it in bursts: 14 stretches above 400 APM and 6 above 500 APM, each sustained for five seconds in combat and executed with no wasted clicks.⁶ Together with seeing the whole map at once, he judged that speed advantage large enough to have swung the games.⁶ Every part of that critique describes the December agent. None of it describes the version that later climbed the ladder.¹

Grading the “superhuman” claim

Grandmaster on the ladder and beating the world's best in a controlled match are different claims, and the paper is explicit about the gap between them. Its ladder conditions, it says, “do not directly measure its susceptibility to exploitation under repeated play.”¹ A ranked ladder rewards an agent for being strong against a broad field of anonymous opponents. It does not test whether a top professional, given a run of games to study the agent, could find a weakness and repeat it. The paper does not claim that AlphaStar beat the best players in a set series under its ladder constraints.¹

One of the paper's own authors is a professional player. TLO is quoted in it saying that, with the limits in place, the agent “doesn't feel superhuman—certainly not on a level that a human couldn't theoretically achieve.”¹ That is the team stating, inside the document, that the constrained version's edge was human-shaped.