While I believe SC2 and Dota would fail today with sufficient effort, the models didn’t quite perform superhuman, and as far as I am aware no community bots do either.
Ah yup, for Starcraft that’s what I get for relying on hazy 2019 memory.
https://deepmind.google/discover/blog/alphastar-grandmaster-level-in-starcraft-ii-using-multi-agent-reinforcement-learning/ includes “I’ve found AlphaStar’s gameplay incredibly impressive – the system is very skilled at assessing its strategic position, and knows exactly when to engage or disengage with its opponent. And while AlphaStar has excellent and precise control, it doesn’t feel superhuman – certainly not on a level that a human couldn’t theoretically achieve. Overall, it feels very fair – like it is playing a ‘real’ game of StarCraft.”
Dota2, though, I do feel was fairly superhuman—it beat the world champion team (I think? I don’t know Dota2 that well so it’s plausible I’m misunderstanding the 2019 showmatch it won against OG) and won 99.7% of its matches in a public demo, which is totally insane from a human perspective. Seems analogous-ish to DeepBlue first beating Kasparov, though Carlsen could maybe beat DeepBlue now. I guess it just goes to show that, well, “superhuman” isn’t well specified.
EDIT: ah, I missed the not-all-heroes limitation. That’s big. At the time I remember following the “they got invulnerable couriers” handicap and being excited when they lifted it, but yeah, restricting the number of heroes is definitely too big for it to count as superhuman.
Also, thanks! Edited the post to correct the error.
While I believe SC2 and Dota would fail today with sufficient effort, the models didn’t quite perform superhuman, and as far as I am aware no community bots do either.
Ah yup, for Starcraft that’s what I get for relying on hazy 2019 memory.
https://deepmind.google/discover/blog/alphastar-grandmaster-level-in-starcraft-ii-using-multi-agent-reinforcement-learning/ includes “I’ve found AlphaStar’s gameplay incredibly impressive – the system is very skilled at assessing its strategic position, and knows exactly when to engage or disengage with its opponent. And while AlphaStar has excellent and precise control, it doesn’t feel superhuman – certainly not on a level that a human couldn’t theoretically achieve. Overall, it feels very fair – like it is playing a ‘real’ game of StarCraft.”
Dota2, though, I do feel was fairly superhuman—it beat the world champion team (I think? I don’t know Dota2 that well so it’s plausible I’m misunderstanding the 2019 showmatch it won against OG) and won 99.7% of its matches in a public demo, which is totally insane from a human perspective. Seems analogous-ish to DeepBlue first beating Kasparov, though Carlsen could maybe beat DeepBlue now. I guess it just goes to show that, well, “superhuman” isn’t well specified.
EDIT: ah, I missed the not-all-heroes limitation. That’s big. At the time I remember following the “they got invulnerable couriers” handicap and being excited when they lifted it, but yeah, restricting the number of heroes is definitely too big for it to count as superhuman.
Also, thanks! Edited the post to correct the error.