GM Andrew Tang Defends Humanity Against Leela Chess Zero

23 Apr 2018T. Alexander Lystad (@arex)Chess

The new AI puts on an extremely strong performance against the human Grandmaster

As previously announced, on Sunday we got to see a historic match between GM Andrew Tang (@penguingim1) and the self-taught chess engine, Leela Chess Zero.

Historic because Leela is not a traditional chess engine. She learns and improves over time exclusively by playing chess against herself. This means she's free from human tendencies and limitations. Maybe it's possible for Leela to come up with new concepts, or combine existing chess principles in a new way? That's one of several reasons many people are excited about Leela.

Before this match, Leela had played a couple of informal practice matches against humans (rapid and bullet against IM @lovlas in particular), and a couple of matches against traditional chess engines. However, she had never played against a human GM before, nor any human that's capable of playing chess as fast as GM Andrew Tang.

It should be no surprise then, that when the match started, 2,500 people were ready to watch livestreams of the match. The number of livestream viewers grew to about 4,000 over the next couple of hours.

The match format was two 15+2 games, four 5+2 games, eight 1+0 games, ending with a "freeplay" phase where Andrew was free to choose any time control he wanted. Since Andrew is known for excelling at faster time controls, everyone expected, and was hoping for, some hyperbullet and ultrabullet games. Would Leela sweep Andrew off the board, or could Leela be out-played or out-flagged?

Phase 1: Two rapid games (15+2)

In the first rapid game, Leela played the Scotch Game as White and never seemed to give up the advantage. Andrew resigned on move 31 with 2 minutes left on his clock, as Leela trapped his light-squared bishop in an already winning position.

In the second rapid game, we saw the only manual adjustment to Leela of the match. On move 10, Andrew mouseslipped Kf1 when he obviously meant to castle short. The match organizers immediately conferred and @careless25 (the Leela overseer for this match) accepted Andrew's takeback request which allowed him to play the intended 10. O-O instead. However, the takeback crashed the software that we used to let Leela communicate with Lichess. As a consequence of this, Leela lost about 15 seconds on her clock, and her next move, 10...Qd8, was also played more quickly (with less thought) than normal. We communicated to Andrew that no more takebacks would be given during the match. Andrew got a good position out of the opening, but his advantage got whittled down little by little over the next 30 moves. He lost on time in a losing position on move 44.

Phase 2: Four blitz games (5+2)

In the first blitz game of the match, we saw Leela win effortlessly with the French Advance. In the second game, however, Andrew had chances. After having entered the Queen's Gambit Declined Exchange, Leela got bold and sacrificed her Knight for two pawns by playing 21...Nxf2. Andrew's King was exposed, but it was possible to survive with an advantage. With uncoordinated pieces and only 40 seconds on the clock, however, it was not an easy task. Andrew resigned on move 43.

In the third blitz game, the players followed the first rapid game of the match for a while. Leela played the Scotch Game again, but was the first to deviate, playing 10. a3 instead of 10. Qe4. Andrew seemed to hold the advantage for a while, but gave it away with 25...f4. 11 moves later, Andrew resigned.

In the final blitz game of the match, Andrew maintained an equal position for half of the game before Leela was able to take the advantage. The game quickly ended after Andrew blundered a piece with 37. h4.

Phase 3: Eight bullet games (1+0)

Even though he had been able to get some good positions, it had been all losses for Andrew up until this point in the match. There was quite some discussion in the lead-up to the match, whether faster time controls helps Leela, or whether it helps Andrew. Normally, you'd expect the engine advantage over humans to increase in faster time controls, but Leela is not a traditional engine, and Andrew is well-known for excelling in the fastest time controls.

Long story short, Andrew was able to score 3 draws against Leela out of the eight bullet games. All of them involved fortresses, and the 50-move rule as Leela is currently not able to offer or accept draws.

The following game was drawn by the 50-move rule on move 89.

https://lichess.org/study/VlIegy6E/aEYrBxDW

The following game was drawn by the 50-move rule on move 164. Andrew had a comfortable 9.4 seconds remaining on his clock.

https://lichess.org/study/VlIegy6E/OmIWSTKH

The following game was drawn by the 50-move rule on move 170.

https://lichess.org/study/VlIegy6E/dpv28vIf

Phase 4: Anything goes

Andrew spent the last part of the match feeling Leela out in different time controls. Andrew delivered on the spectators' hopes and expectations by playing very fast time controls. 30 games were played in this phase, most of them ultrabullet, meaning just 15 seconds for each side. In this last phase of the match, Andrew scored three ultrabullet draws and even an ultrabullet win! Let's take a look at the win first:

https://lichess.org/study/VlIegy6E/XfqgyZCG

With 16...Qxd4 Leela blundered in a very human way to the very simple tactic 17. Bh7+ Kxh7 18. Qxd4, losing her Queen. Andrew now had 10.3 seconds to finish her off. After an additional 50 moves, and with 1.4 seconds left on his clock, Andrew checkmated Leela for his first (and only) win of the match. To re-live the incredible moment from Andrew's perspective, watch this clip. I'm certain that the 4,000 spectators people across the globe were screaming, if not jumping and dancing at this point.

But how come Leela blundered such a simple tactic? Looking at the chess clocks, we can see that Leela spent less than 0.1 seconds thinking before making the move. Under such time constraints, Leela is not really able to consider and evaluate many different moves, but instead relies heavily on her "instincts". Looking under the hood after the match, we could see that Leela thinks Qxd4 is the correct move 90% of the time in the critical position. Furthermore, Leela thinks Bh7+ is the correct reply only 10% of the time. If she had more time to double check her assumptions, she would not have blundered, but she didn't - so she did. In our estimation, Leela would have avoided the blunder if she had spent 0.25 seconds thinking before making a a move.

Here are the three draws:

Andrew drew the following game by threefold repetition on move 42.

https://lichess.org/study/VlIegy6E/AV2Zj9jl

Andrew drew the following game by perpetual check on move 33.

https://lichess.org/study/VlIegy6E/upfJKyyx

Andrew drew the following game by perpetual check on move 37.

https://lichess.org/study/VlIegy6E/BZPH18Jl

Match summary

Andrew managed to take 4 points out of a possible 44 against Leela Chess Zero ID 125 running on a GTX 1060 graphics card. Despite Andrew's win and six draws, which are significant accomplishments, we have to conclude that Leela has already reached, and most likely surpassed, the Grandmaster level.

Asked for his post-match thoughts, Andrew said the following: "Coming into the match, I didn't like my chances, as I had seen her very good results against IM @lovlas. Despite not expecting much, the losses were still a bit painful. After her Knight sacrifice on f2, I understood why @lovlas said that playing her felt like playing Tal. f2 is a move you don't expect from an engine. Leela proved very aggressive, but she also just outplayed me. Still, the match was a lot of fun, and it was crazy to see how many people watched our match."

Prior to the match, the Leela devs were concerned about her time management. Due to Andrew's reputation, they feared he might be able to flag Leela. If Leela was flaggable, that could lead to a long win streak for Andrew and egg on Leela's face. After the match, however, it looks like Leela may have done even better if she had spent more time thinking than she did. Time management will be one of many things the Leela team will work on moving forward.

I missed the match. What do I do?

Watch the archived version of Andrew's livestream on his Twitch channel or on his YouTube channel.

Jerry aka ChessNetwork also livestreamed his commentary of the match. You can watch it on his Twitch channel or on his YouTube channel.

If you just want to see the games or want to download the PGNs, check this Lichess Study.

Where can I see more of Leela?

Leading up to the match, many content creators covered Leela in different ways.

agadmator did three videos on three games from the informal 11-game practice match between Leela and IM Lasse Østebø Løvik (@lovlas):

The @lovlas practice matches were played against full-strength Leela (ID 125) running on a GTX 1080Ti graphics card, and not the much weaker resource-limited version people can play on play.lczero.org.

ChessNetwork did two videos where he played against Leela:

Kingscrusher did at least four videos where he played against Leela:

How can I play against Leela?

With the recently added BOT support, anyone can run a chess engine account on Lichess. We even created a Lichess team for all the BOTs. This means there are many users running Leela, Stockfish and their own custom engines. If you see them online, just challenge them, and they might play you. As an example, @LeelaChess has played more than 700 games against other Lichess users (humans as well as other BOTs) in the past week. If you prefer something easier, you can try the web version of Leela at play.lczero.org.

How do I contribute to Leela?

As of April 22nd, Leela has played more than 7.5 million chess games against herself. This is how she learns and improves. To make this possible, volunteers run custom software on their computer which allows Leela to play games using their CPU or GPU. Anyone can do this, and you can turn off the software when you need the CPU or GPU for other things. Check this page for instructions on how to contribute your CPU or GPU time.

Credits

Huge thanks to The Penguin himself, GM Andrew Tang, for playing a very challenging engine match for our entertainment and to satisfy our curiosity.

Thanks to @lovlas for playing informal practice matches against Leela to help us understand her better and prepare for the big match.

Thanks to @thibault for implementing Lichess BOT support as quickly and effectively as he did.

Thanks to @careless25, @thibault and other contributors to the lichess-bot interface that allows Leela (and other engines) to talk to Lichess.

Thanks to @careless25, @KillerDucky and @MTGOStark for helping me organize the match.

Last, but not least, thanks to the entire Leela Chess Zero community for an amazingly cool, interesting and successful open-source project that will be interesting to follow in the time ahead.