AlphaStar (software)


AlphaStar is a computer program by DeepMind that plays the video game StarCraft II. It was unveiled to the public by name in January 2019. In a significant milestone for artificial intelligence, AlphaStar attained Grandmaster status in August 2019.

Background

Games created for humans are considered to have external validity as benchmarks of progress in artificial intelligence. IBM's chess-playing Deep Blue and DeepMind's AlphaGo were considered major milestones; some argue that StarCraft would also be a major milestone, due to StarCraft's "real-time play, partial observability, no single dominant strategy, complex rules that make it hard to build a fast forward model, and a particularly large and varied action space." Though difficult, StarCraft may still be tractable with current technology because "its rules are known and the world is discrete with only a few types of objects".
StarCraft II is a popular fast-paced online real-time strategy game by Blizzard Entertainment.

History

DeepMind Technologies was founded in the UK in 2010. As early as 2011, founder Demis Hassabis called StarCraft "the next step up" after games like Go. DeepMind became a Google subsidiary in 2014, after demonstrating self-learning bots with superhuman ability at a variety of Atari 2600 games. In February 2015, computer scientist Zachary Mason predicted Deepmind's research "leads to StarCraft in five or ten years". In March 2016, following AlphaGo's victory over a world champion Go player, Hassabis publicly mulled building an AI for StarCraft, citing it as a strategic game with incomplete information where much of the "board" is invisible. A formal collaboration was announced at BlizzCon in November 2016, alongside a plan to release an open development environment for bots in Q1 of 2017.
By 2017, DeepMind was experimenting with feeding StarCraft data into its software. In August 2017, DeepMind and Blizzard released development tools to assist in bot development, as well as data from 65,000 past games. At the time, computer scientist and StarCraft tournament manager David Churchill guessed it would take five years for a bot to beat a human, but made the caveat that AlphaGo had beaten expectations. In Wired, tech journalist Tom Simonite stated, "No one expects the robot to win anytime soon. But when it does, it will be a far greater achievement than DeepMind's conquest of Go."
On 19 December 2018, DeepMind's bot defeated "a top professional player", Grzegorz "MaNa" Komincz, 5-0. DeepMind announced the bot, named "AlphaStar", on 24 January 2019. A journalist at Ars Technica and others argued that AlphaStar still had unfair advantages: "AlphaStar has the ability to make its clicks with surgical precision using an API, whereas human players are constrained by the mechanical limits of computer mice". AlphaStar also had a global view rather than being limited by the in-game camera. Furthermore, while there was an API cap on the number of actions over a five-second window, AlphaStar was free to allocate its action quota unevenly across the window in order to launch superhuman bursts of activity at critical moments. DeepMind quickly retrained AlphaStar under more realistic constraints, and then lost a rematch with Komincz. Starting in July 2019, the new, constrained version of AlphaStar anonymously competed against players who "opted in" on the public 1v1 European multiplayer ladder. By the end of August 2019, AlphaStar had attained "grandmaster level", ranking among the top 0.2 percent of human players.

Algorithms

Unlike AlphaZero, AlphaStar initially learns to imitate the moves of the best players in its database of human vs. human games; this step is necessary to solve what DeepMind's Dave Silver calls "the exploration problem": discovering new strategies would otherwise be like finding a "needle in a haystack". Agents then play each other and deploy reinforcement learning. These main agents also learn by playing against suboptimal "exploiter agents" whose purpose is to expose weaknesses in the main agents.

Reactions

After his 5-0 defeat in December 2018, Komincz stated "I wasn't expecting the AI to be that good".
Stuart Russell assessed that AlphaStar's 2018 victory required "a fair amount of problem-specific effort" and that general-purpose methods were "not quite ready for StarCraft".
An article in Wired UK judged AlphaStar's new constraints, adopted for the July 2019 matches, to be "fair" this time around. StarCraft pro Raza "RazerBlader" Sekha stated AlphaStar was "impressive" but had its quirks, succumbing in one game to an unorthodox army composition made up of only air units. The UK's top player, Joshua "RiSky" Hayward, expressed some disappointment, saying AlphaStar "often didn't make the most efficient, strategic decisions". Pro Diego "Kelazhur" Schwimer called AlphaStar's play "unimaginably unusual; it really makes you question how much of StarCraft's diverse possibilities pro players have really explored". AlphaStar's opponents often did not realize they were playing a bot.
Ian Sample of The Guardian called AlphaStar a "landmark achievement" for the field of artificial intelligence. Churchill stated that he had previously seen bots that master one or two elements of StarCraft, but that AlphaStar was the first that can handle the game in its entirety. Gary Marcus expressed his continuing skepticism about deep learning, stating: "So far the field has struggled to take techniques like this out of the laboratory and game environments and into the real world, and I don't immediately see this result as progress in that direction". AI researcher Jon Dodge was surprised by AlphaStar, stating he did not expect such a "superhuman" performance for "another couple of years"; in contrast, Churchill states "StarCraft is nowhere near being 'solved', and AlphaStar is not yet even close to playing at a world champion level".

Legacy

DeepMind argues that insights from AlphaStar might benefit robots, self-driving cars and virtual assistants, which need to operate with "imperfectly observed information". Silver has indicated his lab "may rest at this point", rather than try to substantially improve AlphaStar. Silver himself argues that "AlphaStar has become the first AI system to reach the top tier of human performance in any professionally played e-sport on the full unrestricted game under professionally approved conditions... Ever since computers cracked Go, chess and poker, the game of StarCraft has emerged, essentially by consensus from the community, as the next grand challenge for AI."
Computer scientist Noel Sharkey argues, disapprovingly, that "military analysts will certainly be eyeing the successful AlphaStar real-time strategies as a clear example of the advantages of AI for battlefield planning". In contrast, Silver argues: "To say that this has any kind of military use is saying no more than to say an AI for chess could be used to lead to military applications".