Why AI Chess Bots Are Virtually Unbeatable (ft. GothamChess)

I'm about to play Stockfish 16,

the strongest chess computer in the world.

And we are going to see how long I can survive.

What?

What the [beep] is that? Yeah, I'm fine. I'm not fine.

Queen takes Bishop, Checkmate.

That was horrible.

I never want to do that again. I got Checkmated in 34 moves.

The most important thing is I had a lot of fun. No I didn't.

My name is Levy Rozman.

I wanna find out a little bit more about

how Stockfish thinks so I can use its tricks

to beat other humans.

Today we'll talk to Gary Linscott, software engineer

who has worked in computer chess for over 20 years.

So Gary, you worked on Stockfish.

What exactly makes it so good?

Stockfish really takes advantage

of super powerful computers.

It searches tens of millions of physicians per second.

So if you give it 30 seconds,

it's gonna be searching upwards of a billion positions

and that's more positions than a human

will see in their entire lifetime.

I could live a hundred lifetimes.

I would not be able to win this game.

Not only that, the evaluation function it uses,

it's a way of assessing the board.

By itself is probably stronger than almost all the

chess players in the world.

Your chess Elo is basically a number

that quantifies your skill level

and the highest ever Elo achieved

by a human is by Magnus Carlsen.

My Elo right now is 2,322.

What exactly is Stockfish's Elo right now?

It's probably greater than 3,500.

I never had a chance.

To understand how Stockfish actually thinks,

we should start at the very beginning of the game.

Most chess openings are subjective for humans.

A human will choose an opening

because they get more fun out of it.

You might like something because it gives you a big center

or very active pieces.

That's how most players choose their openings.

In my game against Stockfish, I played Pawn to D4.

Let's see which opening Stockfish plays

and it responded Knight to F6.

Gary, so the way humans learn chess openings, we read books,

we create chess files and we try to memorize them

'cause we can't access them during tournament games.

That would be like cheating on a school test.

An opening book in chess is a gigantic chess database.

Does Stockfish have a chess opening opinion as early

as the second or even first move?

The way that it thinks about the opening is

really different than a human.

It is not using chess theory.

It very rarely assesses an opening position.

What it's assessing is the resulting structures

that might evolve from that opening,

but way, way down the game tree, it's already assessing

the end game right from the getgo.

So on its second move, Stockfish played D5,

which is a very popular move.

And here I made a decision to capture this Knight.

Humans might see doubled pawns and feel unpleasant.

They might see a weakness in their position.

Does Stockfish have these little buzzwords like weakness

or inactive piece?

What actually goes into that?

The evaluation function is trained off of a huge library

of chess games that have been played.

And so it kind of absorbs that knowledge.

It's just all going into a giant neural net.

And then you can train your evaluation function

to learn what's good and what's bad.

And then that network runs when Stockfish is searching the

the position and then uses it to navigate the opening.

So to succeed in our quest

of understanding Stockfish a little bit better,

let's dive into middlegames.

Most middlegames begin around move 10 to 15.

It's the stage right

after the opening when you've gotten all your pieces out.

You can think of the middlegame in chess as a

gigantic ocean, this massive tree of possibility.

So in my middle game against Stockfish,

Pawn to C4 is what I played.

Stockfish played a move

that I don't think any human being

would play here against me.

The absurd pawn to G5, which made me audibly gasp.

Ho, ho, ho, ho. What is that? Oh my God.

Is that actually the best move in the position?

That move violates most human chess principles

that we have been brought up with?

You shouldn't push ponds in front of your king.

You should not weaken an area of the board

where you can't claim any advantage.

How does Stockfish break these age old principles?

In this case, it might just be that the move is too

far past kind of the human frontier to be a reasonable one.

It's gonna assess every possible

move and it's gonna rank them.

It all comes down to a single number,

which is what's the likelihood

that I'm gonna win from this position?

Stockfish doesn't have emotions

and opinions, it will just play the best move

that it thinks exists.

Yes, Stockfish definitely does not

emotionally consider it.

It is assessing with like grand master level quality,

but it then searches 50 moves, 60 moves into the future

to evaluate what is the best possible move.

I'm curious, like as an incredibly strong human,

how many moves do you search into the future?

I didn't realize this was the roast of Levy Rozman.

Sometimes I can go 10 moves, if it's an end game position

and I can kind of identify the forcing moves for both sides.

Sometimes I'm stuck in a middlegame

and I'm already indecisive.

It's like being at a restaurant.

You have three good options.

They're all very difficult to evaluate.

And I'll flip a coin,

but that doesn't work against Stockfish.

I counted, in this position, black has 41 legal moves.

How does Stockfish know which of those to scrub

and ultimately decide on one,

if three or four look really, really good,

It builds a game tree

and that game tree is gonna have all the legal moves.

It's gonna rank all of those moves.

It takes one step forward at a time.

It'll search two moves ahead,

then three moves ahead, then four moves ahead.

[Levy] Three or four moves speak to me

in about five seconds

and that's probably why I'm a good speed chess player.

But deciding on the best move,

sometimes I need to spend 5 minutes, 10 minutes

uncovering the truth about one of those moves.

Chess Computers actually do a similar sort of process.

Stockfish actually only looks at about two moves per

position and that's what's called the alpha beta search

technique, which is what Stockfish

and most other engines are built on top of.

Alpha Beta allows the engine to eliminate many,

many moves from a position because it knows that they are

worse than the best move that it's found so far.

And that allows alpha beta

to prune a huge amount of the search trees.

For example, if it was white's move,

one of the legal moves is Knight to E4 which is really,

really stupid because the Pawn would take the Knight

and white would be at a catastrophically worse position.

Some computers will just be able to discard this right away

and humans will too.

Alpha beta by itself can take those 35 moves that you have

to look at on average down to about 15 or so,

which is a huge reduction.

The last stage of chess is the endgame.

Is there any difference in the way

that Stockfish approaches the endgame

versus the middlegame or the opening?

Yes. Once the number of pieces goes

below a certain amount, seven,

then Stockfish can actually solve the game perfectly.

Chess is solved if there are seven pieces

remaining on a chessboard or less.

Exactly. Yeah.

There's what are called endgame table bases.

And by solved we mean literally every combination

of moves possible.

If the Bishop goes here and the Knight goes here,

or if the Bishop goes to that square originally

and the Knight goes there or there or there

or any of these combinations,

all of that is solved.

Literally Stockfish just has

to look up the position in its database

and it knows exactly what the answer is.

Even for seven pieces, it's only about 10 to 20 terabytes

of data, which is a lot.

But you know, manageable.

There is no more evaluating.

A best move is the best move

and a position is either winning,

a draw, or a losing, and that's it.

It's completely non-negotiable.

Exactly. Now the chess computers are capable

of playing without the table bases as well,

and they will still play incredibly strong chess there.

But they could make a mistake.

Gary, I lost to Stockfish.

I got checkmated in 34 moves,

so we didn't make it into an end game.

All right.

But that's just as well.

Stockfish always knows how

to close out a game in the least amount of moves,

even when the world's greatest players don't.

I have a game here, Magnus Carlsen versus Fabiano Caruana

from the 2018 World Chess Championship.

Magnus was in an end game where he was down a piece,

but neither one of them found the right technique.

Fabiano has a Bishop and a Knight,

and Magnus only has a Bishop.

Fabiano could not convert despite his extra material

because it looked like Magnus had a defensive fortress.

The game ended in a draw.

While the players were playing

and the entire world was watching,

Stockfish was screaming at them from the digital cyberspace

going, you idiots forced checkmate in 35 moves.

That is ridiculous.

Like how could it have possibly known

that if both sides made a move

for 35 moves, black would win.

What Stockfish can do is look forward into the table

bases and every path to the table bases.

The best that you can possibly do is that mate in 35.

If Stockfish was playing black here instead of Fabiano,

it would've played Bishop to H4,

then bring its Knight here

and then trap its own Knight on the edge of the board.

So the Knight now cannot escape anywhere

because white's bishop would take it.

No human being would play chess like this.

You cannot trap your Knight on the edge of the board,

but apparently white actually runs out of moves first

and slowly but surely black would've won that end game.

Stockfish saw that from a distance.

No human on the planet saw that.

So Stockfish is a very, very specialized AI.

How does it compare to some other AIs,

autonomous cars or something else?

Stockfish is only good at one thing

and it's super good at that one thing,

which is playing chess.

Chess engines have now borrowed the state

of the art research and artificial intelligence,

which is deep learning.

However, they're still fundamentally limited

to the domain of chess.

Chess has, I think, been opened up, in a way, by AI.

People can leverage the chess engines

to improve their own skills,

and that's kind of like the ideal use case

for AI technologies, is that they help humans,

you know, do things better.

I agree with Gary.

I think AI has been a net benefit for chess.

I think it's helping people improve at a rate

like we've never experienced.

We do obviously have to tackle the issue

of potential cheating,

but I'm gonna stay positive and optimistic.

Anyway, now that I know a little bit more about Stockfish,

I'm gonna go have my rematch.