• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Dota 2: an AI beats the world’s top professionals at 1v1 matches

Wok

Wok

Member
I posted the other day about the collaboration between Google DeepMind and Blizzard to develop an AI for Starcraft 2, little did I know that OpenAI was working on an AI for Valve's game Dota 2.

It turns out that there already exists (yes, today) « a bot which beats the world’s top professionals at 1v1 matches of Dota 2 under standard tournament rules », including Dendi, SumaiL and Arteezy, in best-of-three matches.

The event happened during the International, Valve's annual competition at Dota 2, and the VOD is here:

https://openai.com/the-international/

Dota 1v1 is a complex game with hidden information. Agents must learn to plan, attack, trick, and deceive their opponents.
The correlation between player skill and actions-per-minute is not strong, and in fact, our AI’s actions-per-minute are comparable to that of an average human player.

The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situations involving real humans.

Success in Dota requires players to develop intuitions about their opponents and plan accordingly.
In the above video you can see that our bot has learned — entirely via self-play — to predict where other players will move, to improvise in response to unfamiliar situations, and how to influence the other player’s allied units to help it succeed.
Click to expand...

Full blog post: https://blog.openai.com/dota-2/

Edit: These are the constraints: Shadow Fiend 1 vs 1 and:

yMTqi2n.png
Click to expand...
 
Ploid 3.0

Ploid 3.0

Member
Makes me feel slightly less bad about getting curb stomped by the AI the first time I tried DOTA2, while fighting camera controls and learning stuff. I should give it another try.

But I didn't play against OpenAI
 
H

Haly

One day I realized that sadness is just another word for not enough coffee.
DrKirby, of NeoGAF fame, beat the OpenAI at the venue.

What now, Skynet?
 
Ploid 3.0

Ploid 3.0

Member
Wow that was awesome. I already hate how accurate bots can be in other MOBA games, like third person ones. Can't compete with Google's I'm sure.
 
C

ChrisD

Member
Haly said:
DrKirby, of NeoGAF fame, beat the OpenAI at the venue.

What now, Skynet?
Click to expand...
There's a really mean joke here about it not being able to beat bad players.

but I want to make clear that I don't mean any ill-will against DrKirby, it just popped in my head as soon as I read your post lol
 
P

PrinnyBomb

Member
Did DrKirby creep cut as SF? Heard the AI has only played against other SF's, and so the arcana was only offered to those that won as him
 
I

irrotational

Member
It's amazing that they can do these things now... But I'd love them to connect up the servers to a robot head and arm, and get it to move and click around the screen. Would really make it feel like skynet is getting close.
 
S

Soule

Member
What's blowing me away is how it learned independently to do stuff like creep block. When I found out that want preprogrammed but learnt I was like holy shit this is the end for humanity we've been superceded
 
E

Evolution of Metal

Member
That hero is so boring, none of the spells are interesting just damage, I wonder what would AI do with a stealth or some kind of specialist hero. If someone managed to exploit this AI by lane manipulations, then it is a one trick pony as of now. It is a good start, but all I see is a mechanical trading and farming prowess.

Cassio scripts were pretty crazy in LoL as far as doing combos, dodging skills and providing a superior awareness.
 
Wok

Wok

Member
Soule said:
What's blowing me away is how it learned independently to do stuff like creep block. When I found out that want preprogrammed but learnt I was like holy shit this is the end for humanity we've been superceded
Click to expand...

Also, this creep block is insane:

SDe9fxp.gif


Evolution of Metal said:
Cassio scripts were pretty crazy in LoL as far as doing combos, dodging skills and providing a superior awareness.
Click to expand...

Keep in mind this AI is self-trained with no programmed strategy or mechanics.
 
H

Hylian7

Member
Haly said:
DrKirby, of NeoGAF fame, beat the OpenAI at the venue.

What now, Skynet?
Click to expand...

He didn't beat it, he killed it once. You had to kill it twice, destroy the tower, or be up in kills by 10 minutes to win. So he didn't get the Arcana. He was close though.

I tried it and failed horribly. I was not used to that mouse and the settings, I had a panic moment trying to hit Alt+E to use my Salve only to find out it was on X! I fixed some of the settings, and it was better for the second attempt. I still got my ass handed to me.

To quantify how it works, think of how a good player tries to find weaknesses in his opponents play and exploit those, and take that to an extreme level. That's what this thing does. On my second attempt, he got a perfect creep block and got his creeps really close to his tower. Mine was clearly not good enough. He skipped the creep wave to trade hits with me (we were both level 1 with Necromastery) and he aggroed one creep that was hitting him. He kept trading anyway, and I had to back further (he got a couple of hits first), and I got high ground and kept trading hits while the creep was hitting him. He knew as soon as it would get to the point where he wouldn't be ahead of me in HP, he would back. He knew it would go south for him because of the HP obviously, but the fact that I was on high ground, and he could just get farm under his tower and I was out of XP range, so he just got level 2 and killed me. It animation cancels perfectly and is super aggressive with Razes. I don't think it missed a Raze once.
 
R

Randdalf

Member
I hope they can expand this to 5v5 successfully. It would be amazing if they could ship it with the game with a slider for quality (i.e. pick between bots with different experience levels)
 
poodaddy

poodaddy

Member
A buddy of mine who was working security at the event today mentioned that he got to see an AI beat pro players. I don't know or understand anything about DOTA 2 so I didn't pay it any mind, but this must be what he was talking about. Seems cooler than he made it sound lol.
 
H

Hylian7

Member
Randdalf said:
I hope they can expand this to 5v5 successfully. It would be amazing if they could ship it with the game with a slider for quality (i.e. pick between bots with different experience levels)
Click to expand...

They said they plan to and will have it at TI8.

poodaddy said:
A buddy of mine who was working security at the event today mentioned that he got to see an AI beat pro players. I don't know or understand anything about DOTA 2 so I didn't pay it any mind, but this must be what he was talking about. Seems cooler than he made it sound lol.
Click to expand...

Yep, this was it. It's always interesting seeing the Key Arena staff/security see Dota stuff and are impressed by it.
 
T

traveler

Not Wario
More impressive than chess, less impressive than Go. Seeing a computer stomp a pro in a real time game isn't quite the same as turn based- its advantages aren't just its ability to learn but also frame perfect canceling, guaranteed landing on moves since it can tell exact distances and compute travel speed in real time, and perfect last hitting ability- advantages that prove especially valuable in a Shadow Fiend 1v1 of all heroes. In Go and Chess, the player isn't physically hindered by their, well, human-ess. They're equals with the computer since you each get one deliberate input per turn. Still neat, of course, and the applications beyond dota should prove way more valuable.

Now, set up an ai in a robot separate from the computer hosting the game that can beat a human by actually playing the game with keyboard and mouse and I'll be truly impressed.
 
Wok

Wok

Member
poodaddy said:
A buddy of mine who was working security at the event today mentioned that he got to see an AI beat pro players. I don't know or understand anything about DOTA 2 so I didn't pay it any mind, but this must be what he was talking about. Seems cooler than he made it sound lol.
Click to expand...

Is he any of these two big boys?

phWSHu7.jpg
 
poodaddy

poodaddy

Member
Hylian7 said:
They said they plan to and will have it at TI8.



Yep, this was it. It's always interesting seeing the Key Arena staff/security see Dota stuff and are impressed by it.
Click to expand...
Yeah, he and I work for the same event security staffing company and I could have worked the event that day but unfortunately I had some personal issues I had to handle all week. It sounds like it was a fun gig and he said it was easy money, so I wish I could have worked it.

Wok said:
Is he any of these two big boys?

phWSHu7.jpg
Click to expand...
Na he ain't a big guy at all, he's a short curly haired white guy with black hair and a red beard. Apparently he was ushering so may have seen him in that capacity at some point.
 
N

NewGame

Banned
I don't really get it, they only compared creep score? ofc a bot would win, its aim bot last hitting

If this works 5v5 i will assume we have already made true, real and personable AI
 
H

Hylian7

Member
Piers said:
Learning/getting better against that AI will still be useless compared to actual players.
Click to expand...
Not really. If you can learn to position yourself, last hit, etc against something that can do it perfectly, those are fundamental skills you can use against real players.

Honestly when I played against it, one of the biggest takeaways from it was it's positioning, which was pretty much perfect. Granted I did try to beat it in a straightforward manner, which was a disastah, and you can kind of confuse it if you do something unorthodox like Drkirby did, with a creep skip.

Assuming this eventually does get released to the public, I think everyone can learn a lot from it. However I don't know if this requires and special hardware to run. It didn't seem like it though, as far as I could tell all the PCs in the LAN area were the same, for both regular matches and vs the bot. The bot just had a separate version of the Dota exe you had to use.
 
P

pronk420

Member
I don't think this is that impressive, when its a computer playing a game where timing matters its always going to have faster reaction times than a person. I guess its more impressive than learning something like COD since its picked up strategies as well but it also has an inherent advantage.

It can also have perfect 'vision' in that it can measure the distance in screen space etc exactly, can pay attention to the whole screen at once rather than focussing their eyes in one location etc. Plus if they are feeding it actual game data rather than making it interpret the video, it has a much easier job to interpret the data and react to things happening.
 
P

Piers

Member
Hylian7 said:
Not really. If you can learn to position yourself, last hit, etc against something that can do it perfectly, those are fundamental skills you can use against real players.

Honestly when I played against it, one of the biggest takeaways from it was it's positioning, which was pretty much perfect. Granted I did try to beat it in a straightforward manner, which was a disastah, and you can kind of confuse it if you do something unorthodox like Drkirby did, with a creep skip.
Click to expand...

You're right in that it definitely helps with fundamentals which is good for starting off. Long term, though, it doesn't do much for the game sense which is more important. Last hitting and positioning won't get a player that far if they're unable to react and think and their feet against ever changing tactics from real players — bots, given their nature, always act on scripted behaviour. It becomes exploitation (like you did with confusing it) more than actual strategy.
 
H

Hylian7

Member
Piers said:
You're right in that it definitely helps with fundamentals which is good for starting off. Long term, though, it doesn't do much for the game sense which is more important. Last hitting and positioning won't get a player that far if they're unable to react and think and their feet against ever changing tactics from real players — bots, given their nature, always act on scripted behaviour. It becomes exploitation (like you did with confusing it) more than actual strategy.
Click to expand...

That's the thing: it learns and will get to that point. Right now it's a god at 1v1 mid, but if people keep creep skipping it, it would eventually learn how to deal with that, just like how they said it even fed itself to towers in other lanes at first.

They said it would be able to do 5v5 in a year from now. This is quite different than the default bots in the game. The default bots don't learn, have little to no game sense, and part of the difficulty increase on them is the get items faster (not by actually farming efficiently, the game just gives them to them faster). They have little to no game sense, and I have seen them do some very head scratching things.

This bot can learn that game sense.

aspaceman55 said:
The presumption is that the bot uses a DNN. In which case, any consumer GPU should be able to run the thing trivially while playing the game. There's two components to a DNN's operation: training and inference. The former is much more computationally expensive, but is still typically done with consumer-grade 900-series and 1000-series NVIDIA cards. Once you're doing inference, pretty much any consumer grade card can handle the math quite easily.

I imagine this bot just incorporates some more GPU code while you're playing against it. Since DOTA isn't a graphics intensive game by any means, I imagine there's nothing stopping this from being released from a technical perspective.
Click to expand...
Cool, I look forward to hopefully playing against it at home some day.
 
E

emag

Member
Piers said:
You're right in that it definitely helps with fundamentals which is good for starting off. Long term, though, it doesn't do much for the game sense which is more important. Last hitting and positioning won't get a player that far if they're unable to react and think and their feet against ever changing tactics from real players — bots, given their nature, always act on scripted behaviour. It becomes exploitation (like you did with confusing it) more than actual strategy.
Click to expand...

Once the bot is available for long-term training in a public environment, it will adapt to those tactics. The bot isn't exhibiting pre-scripted behavior. It's a neural network that watched/played DotA and learned everything from scratch, without any predefined rules (beyond winning is good, losing is bad).

Next year, it'll dominate 5 v 5, mark my words. Hopefully we'll see more neural network based AI for a wider variety of games in the future -- they do take time and effort to train (vis a vis conventional scripted "AI"), but the results are fantastic.
 
F

Flayer

Member
The most interesting part of this is that the bot taught itself. Self learning means the possibilities for self improvement are endless and this bot will only get better. The really scary moment will be when one bot is able to train another and so forth, until they create an invincible perfect team of AI trained AIs.
 
Wok

Wok

Member
More details about the training of the Dota 2 AI:
https://blog.openai.com/more-on-dota-2/

  • March 1st: had our first classical reinforcement learning results in a simple Dota environment, where a Drow Ranger learns to kite a hardcoded Earthshaker.
  • May 8th: 1.5k MMR tester says he's been getting better faster than the bot.
  • Early June: beat 1.5k MMR tester
  • June 30th: winning most games against 3k MMR tester
  • July 8th: barely get first win against 7.5k MMR semi-pro tester.
  • August 7th: beat Blitz (6.2k former pro) 3-0, Pajkatt (8.5k pro) 2-1, and CC&C (8.9k pro) 3-0. All agreed that Sumail would figure out how to beat it.
  • August 9th: beat Arteezy (10k pro, top player) 10-0. He says Sumail could figure out this bot.
  • August 10th: beat Sumail (8.3k pro, top 1v1 player) 6-0, who says it's unbeatable. Plays the Aug 9th bot, where he goes 2-1.
  • August 11th: beat Dendi (7.3k pro, former world champion, old-school crowd favorite) 2-0. Bot has 60% win rate versus August 10th bot.
Click to expand...

Video of Sumail losing 1vs1: https://youtu.be/IbaHI38Ewws
 
D

Drkirby

Corporate Apologist
I did something similar to the Creep Pulling technique discussed, but my version was a bit unique in that it resulted in the Bot dying instead of killing the tower.
 
C

Cels

Member
Actions: Actions accessible by the bot API, chosen at a frequency comparable to humans, including moving to a location, attacking a unit, or using an item.
Click to expand...

they should limit the bot's reaction times too not just APM.
 
H

hurzelein

Member
Wok

Wok

Member
hurzelein said:
Didn' read the whole thread so not sure if already posted but it turned out that the bot used the game API and wasn't completely self trained, e.g. it was specifically taught that creep blocking is a thing.
Click to expand...

It is completely self-trained, but in two parts: one bot learns creep blocking, one bot learns how to win after that.

Moreover, the bot uses the game API, but does not abuse it:

The bot operated off the following interfaces:

  • Observations: Bot API features, which are designed to be the same set of features that humans can see, related to heroes, creeps, courier, and the terrain near the hero. The game is partially observable.
  • Actions: Actions accessible by the bot API, chosen at a frequency comparable to humans, including moving to a location, attacking a unit, or using an item.
  • Feedback: The bot received incentives for winning and basic metrics like health and last hits.
Click to expand...

On top of that, pros did not have the same constraints as other people who faced the bot during the event, so Pajkatt beat the bot using Faerie Fire and Magic Stick charges. However, OpenAI says that the bot has learnt how to use these items now.
 
You must log in or register to reply here.

Similar threads

AI and Games: How Machine Learning is Transforming the Video Games Industry
Opinion Analysis Trailer
8
1K
hyperbertha replied
Glorbo isn't Coming to WoW, but the Impact of AI is Already Here
News
8
818
ZoukGalaxy replied
Sony Patents: System for Smart Parental Controls, Help PlayStation Players Get Good, AI Assistant
Game Dev
1
508
01011001 replied
"Mass Effect: Parallax" -- Can an AI write a complex, branching video game story?
Game Dev
31
2K
Denton replied
PD GT7 A.I goes self aware: Sophy at the wheel
2 3 4
179
9K
IbizaPocholo replied
Top Bottom