Video

Transcript

Dota is one of the most popular and challenging competitive video games ever.

Playing Dota means you have to coordinate and focus as a team of five.

Last year, we built a bot that defeated the world’s best players at the 1v1 mini game.

But this year, we want to beat the best pro teams at the full game.

To do this, we’ve built a new AI system, which we’ve spent the last few months training.

It uses a reinforcement learning at a much larger scale,

to train our bots to play together as a team.

We’re now starting to play against amateur teams to test our skill,

and to our surprise so far we’ve won our first

games against every team we’ve tested.

[PLAYER:] Careful, he’s gonna kill you too! Someone else TP bottom!

EVERYONE TP! EVERYONE TP!

Get this guy! I can’t even.

[NARRATOR:] But to figure out how good our bots really are,

we asked a Dota expert to take a look.

My name is William Lee, better known in the Dota community as Blitz.

Blitz here with Christy, just gonna give a

brief explanation of the game for people that don’t know about the game.

The rules are very simple: it’s “kill the enemy team, take the buildings.”

Every hero has a variety of unique spells;

they’ll be playing a mirror mode where both teams have the exact same heroes.

[NARRATOR:] Here, the human team is

dealing significant damage to the base as 5, and Crystal Maiden comes in to

defend the barracks. She jumps in with Blink Dagger, and shields herself with

Black King Bar, while channeling Freezing Field to deal massive amounts of damage

while being uninterruptible. Due to her combo, she ends up taking up four human

players in a 2v5 fight.

And “GG” is called — “Good Game”. The humans no longer think they can win.

They would be absolutely correct.

So, their dev team got absolutely crushed.

I think it was the fastest casting gig of my life.

Then it went into game number two. The humans had time to like think about the game and stuff like that…

they got crushed even harder.

And the bots did exactly what I hope for:

They own this area of the map. You take away two thirds of the map,

they didn’t even touch these two bottom towers.

And they would be 100% correct in this.

This is like one of the highest level plays that you can make:

This side of the map is incredibly hard for the bots to control,

and so they’re just playing this top side and this mid side,

because they understand that these are the two most important parts to control the game.

The ability to intuitively do this is insane.

Doing it in one game, I can maybe chalk it up to just dumb luck, doing it

two games in a row, flipping the sides — means that it’s more than just coincidence.

It took me (and I’m fairly reasonably good at the game) eight years

to learn the strategies that the bot was intuitively doing.

To train our bots, we use reinforcement learning with self-play.

We run the game on over 100,000 CPUs

and our bots learn from every game they play.

Because Dota is so complex to learn even for a single player,

we created a hyper-parameter which we call “team spirit”.

The five bots start out completely selfish,

but tuning this knob tells them to care about their teammates,

so that they can learn to play together as one unit.

After seeing the bots win against the test team,

Blitz wanted to challenge it himself. He has a higher rating than the test team,

and we paired him up with the best players in our audience.

[CASTER 1:] And so we can see, Blitz is about to die here.

[CASTER 2:] Blitz is definitely dead, yes.

[CASTER 1:] Now, the humans are in trouble. Two of their team members are dead with 20

seconds remaining, and the bots are about to take their first lane of rax.

Another big fight happening here with the bots… actually winning.

The game, at first, started to go to the humans.

But eventually, our bot managed to beat this stronger team as well.

We’re still far away from beating pro teams, but I think everyone here was surprised to see this.

I think the teamfight aspect of the bot was excellent.

It didn’t mess up.

When it came to coordination it was some of the best pure teamfighting,

because it felt like I was getting hammered every single time that I made a mistake

and I feel like normal humans don’t do that.

The Dota world championships will take place in August.

While the best players in the world are getting ready to compete,

we’re also working on the next version of our bot.

To see how far we’ve come, we’ll host a live match in July, where we’ll play a team of top players.

Overall, what we’re excited about is that the training method we use is very general.

We’re focused on learning Dota, but we’re hoping that this will give us more and

more insight about how AI can solve complex problems of any kind.

That’s my favorite part when I hear the bot makes advancements: I don’t get scared,

I get excited. Because I think this is another thing for me to have to

challenge myself to beat. This is something that not a lot of people will be able to do.

Let’s have at it.