The 19^th Top Chess Engine Championship, TCEC19

1.The Qualification League: 12 engines, 1 DRR, 132 games @ 30′+5″/m

TCECwelcomed three new competitors this time: A0lite, Roc and SlowChess. A0lite, in contrast to LC0’s thousands of lines of engine code, has an engine based on just 95 lines of Python code. As such, it initiates an interesting experiment in rebuilding the core engine for NN MCTS/UCT competitors. ChessBrainVB’s seasonal holiday led to Counter being readmitted to League 3.

The second author here provided 4-ply openings. After the first round robin, SlowChess was in command on +7 with Monolith and Roc on +3. In game 18, Combusken–Asymptote, 53.f6?? handed Black a potential QP-rrp mate in 157 moves! Has a deeper position appeared on the board in TCEC or elsewhere? The opportunity was spurned with 54...Ra3?? where 54...R7a6 (or Ra4+) was needed. Black shuffled blindly on the a-file: the draw adjudication came swiftly.

In g65, Tucano–chess22k, White won the pawn race, the initiative and the game, a surprising outlier-win. The TCEC ‘win adjudication’ pre-empted the subsequent 27-move mate. In g103/18.1, A0lite–Roc, White was three pawns up with opposite-colour bishops at position 75b: the game had clearly landed gently under parachute in the shared Chapter 5 territory of both Müller & Konoval (2018) and Dvoretsky (2020). Karsten Müller’s (2020) analysis: “118.Ke4 missed the win which needed 118.Be7. After this it was drawn. White brings the bishop to the queenside, the king to the kingside and plays f5 to win.” The position is manageable by FinalGen (Romero, 2012) which confirms that White had a win available.

In the end, SlowChess was almost out of sight, unbeaten and scoring at least 75% against all other engines, see Table 2. It is particularly exciting to see an engine performing in this way, particularly a new entrant to TCEC and a revived experiment in computer chess. Roc finished clear second on +8 so we enjoyed the rare sight of debutante engines taking both the promotion places. Wool (2020) usefully picked out games 12, 14, 23, 35-6, 38, 47, 59, 65 (see above), 75, 78 (a chess22k blunder), 80, 85, 88, 93, 103 (also above) and 125.

Table 2

The TCEC19 Qualification League cross-table

2.League 3: 10 engines, 1 DRR, 2RRs, 90 games @ 30′+5″/m

We had 8-ply openings for this league from Nelson’s ‘Book B’. After the first round robin, the still unbeaten dark horse debutant SlowChess led with +3 but Igel and Minic were close behind on +2. No other engine was positive. Igel’s win over Minic in game 20 was to prove crucial as head-to-head results are the second tiebreaker. Marvin and Roc advanced the schedule with a draw on move 12 in game 41. In Season 10 Stage 1, Texel-Gull similarly drew on move 9.

The second round robin did not change matters at the top. SlowChess on +8 and Igel on +5 remained unbeaten. And so we had a second promotion for SlowChess, #8 on CCRL’s (2020) 40/15 rating list and maybe going all the way to the Premier Division. Forty unbeaten games is at least an encouraging indicator. Minic was also on +5 but, as mentioned, had lost to Igel which placed it third. The other promotee, Roc, was well clear of relegation but Counter’s reappearance in League 3 terminated. There were very few wins against what became the form book, the cross-table of Table 3. These ‘wins in the South West’ were both achieved by the aptly-named Topple against fellow mid-fielders, see game 15 against Pirarucu and game 73 against Gogobello.

Wool (2020) highlighted games 7, 15, 20, 22, 27, 4, 46, 48, 85 and 88, some of which we mention above. These notes on how wins were achieved are an excellent introduction to some of the decisive games.

Table 3

The TCEC19 League 3 cross-table

3.League 2: 10 engines, 1 DRR, 2 RRs, 90 games @ 30′+5″/m

This TCEC season continued with the same tempo, two round-robins and the same ‘SSS’ small sample size luck factor. The openings were arrayed by Eduardo Sauceda aka ‘Cookie Monster’ with background guidance from the second author here. They were variable-length openings, providing an extra degree of freedom. The two promotees provided the main interest from the start. SlowChess was yet to concede. Igel took advantage of the TCEC update rule, not to fix a bug or introduce some minor tweaks but to take on a completely new engine, one including the exciting technology du jour, NNUE. There are no penalties for doing so as in Formula One: even so, this was a brave move.

An indication that this might also be a very good move was game 12 when Igel with Black beat SlowChess in an 86-move thriller which was adjudicated in a QRNqrb endgame which could have shown us another ten interesting moves until the passed pawn won out.

Igelon +10 and SlowChess on +8 promoted cum laude, clear of Rubichess on +4: these were the only three engines in positive territory, see Table 4. SlowChess was clearly good but Igel had leapt past it in one NNUE-sprung bound like a blade-runner. In fact, Rubichess also promoted to fill the chair vacated by Booot as the latter had not fixed a crash problem as required.

Nemorinoand Wasp (which crashed again) relegated to League 3. Wasp exited in style, pulling off the outlier win of the event with Black in the last round against Pedone, see game 86. Wool (2020) highlighted games 1, 8, 12, 28, 46, 51, 73 and 85-6 with clear notes, the three promotees very much to the fore.

Table 4

The TCEC19 League 2 cross-table

4.League 1: 10 engines, 1 DRR, 2 RRs, 90 games @ 45′+5″/m

For this slower-tempo league, Nelson this time combined his previous League 1 and Premier Division opening books in an effort to reduce the likely draw-percentage. As in League 2, SlowChess and Igel were of particular interest. Figure 2 provides the logos for the TCEC19 engines which had not yet played.

Rubichess, standing in for ‘in repair’ Booot, performed well in fact. Xiphos had not been updated since November 2019 and so provided a benchmark as to the improving standard in this League.

Fig. 2.

The logos for the engines originally in League 1 and the Premier Division.

After the first round-robin, the standings were Fire +4, ScorpioNN/Xiphos +2, SlowChess/Igel +1. Rubichess notably beat ScorpioNN in game 6/2.1, a game covered by Wool (2020) who also touched on games 7, 24, 32 and 41.

Table 5

The TCEC19 League 1 cross-table

Ultimately, as in Table 5, Fire and ScorpioNN promoted to the Premier Division where the real TV Money is – but not before Fire lost to rofChade in g55/11.5 and ScorpioNN lost to SlowChess in g69/14.4. Wool (2020) covered these games and games 50, 62 and 75. In fact, no engine went unbeaten. Arasan and Fritz were demoted to Division 2 but the other eight engines were tightly grouped in the scoring. There should be another close battle at this stage in TCEC Season 20 and hopefully, we will see the return of Booot.

5.The Premier Division: 8 engines, 4 DRRs, 8 RRs, 224 games @ 90′+5″/m

At this point as in TCEC18, the TCEC19 championship stepped into a different mode – a doubled time-budget and three times the round robins. Cato supplied 16-ply openings from his Premier Division book.

In fact, there was an administrative slip in proceedings, as surprising as it was rare these days: the operations side of TCEC had been looking more solid lately in Aloril’s care. Nelson’s opening book for TCEC19 League 1, the new combination of previous League 1 and Premier Division, was taken to be also the TCEC19 Premier Division opening book. This was discovered after some 19 games in the first round-robin and clearly, this first round-robin and its return games could not be discarded. The remedy adopted was to complete the first double round-robin, switch to the correct opening book after this, and add a fourth double round-robin to the event.11

As per the vibe in the chat room, Stockfish did in fact come to the start line in a new mode with the NNUE technology. We were wondering how this would change its playing style, so successful in the past. Only the Stockfish team knew whether it would be better in attack or defence, whether it would be more strategic and less tactical.

The answers slowly revealed themselves. Despite having four Whites in the first round, Stockfish failed to post a win, in itself a testament to the playing strength across the field. AllieStein and Komodo got the upper hand but their chances were snuffed out by move 25. Spoiler alert: the draw-rate overall was a touch high, even ignoring the erroneously included ‘Division 1’ openings. The TCEC20 opening book will have added spice! Back to round 1: Leela jumped to +2, the only positive score. Stoofvlees beat Komodo in game 4 but lost to Ethereal in game 6.

Had Stockfish lost some teeth? Not in fact. In the second round of ‘return games’, Stockfish racked up four wins to jump into the lead. AllieStein and Stoofvlees took one step into positive territory while Leela was in turn constrained to seven draws. Stockfish’s win against AllieStein particularly drew the attention of GM Matthew Sadler (2020), featuring both ‘excellent positional play’ and ‘a mind-boggling tactical sequence’. His beautifully written report illuminates the quality play of this season’s Premier Division and Superfinal. We are of course extremely grateful once again for the richly informed perspective of this leading professional. Candidate Master ‘Kingscrusher’ (2020a, 2020b, 2020c), aka Tryfon Gavriel, and ‘GMTheChessPuzzler’ (2020) provided very welcome video-commentary on many decisive games.22 Wool (2020) gave notes on many decisive games. Now Stockfish led with +4 followed by Leela with +2, Stoofvlees/AllieStein on +1.

The third round proved to be Leela’s best with three wins. Stockfish also advanced with two more wins while AllieStein and Stoofvlees stepped back to +0 again and looked unlikely to contend the two Superfinal spots. It was already clear that the other four engines would be in a desperate fight to the end to avoid relegation, see Fig. 3. We came to the half-way point with a quieter round 4 which did however include a win by Komodo over Stoofvlees in game 22.4/88. It was as if the dragon was awakening from its slumbers but of course, it was not.

In round 5, Stockfish stepped up the pace again with two wins while AllieStein lost more ground to Leela. ScorpioNN scored a notable win against the inconsistent Stoofvlees in game 35.1/137 and was to repeat this feat in round 7. Round 6 was another bad one for the one-time mercurial but now struggling Ethereal: it lost not only to Leela but also to the neighbouring Fire. It was fortunate for Komodo that there was another double round-robin to come as it was seventh at this point.

The last two rounds were again good for Stockfish with +3, despite the fact that it was surprisingly beaten in game 47.3/191 by Komodo. The latter also lifted clear of relegation with wins against ScorpioNN and Fire, see Fig. 3. Ethereal got even with Fire, which also lost to ScorpioNN, a result which confirmed its relegation. So, en fin as in Table 6, we had the first TCEC Superfinal between two ‘neural network’ engines, Stockfish with added NNUE, and Leela, playing better but maybe not improved enough.

Table 6

The TCEC19 Premier League cross-table

Fig. 3.

The TCEC19 Premier League: net wins, round by round.

6.The Superfinal

Once again, Stockfish and Leela Chess Zero faced off in the TCEC Superfinal, the fifth time since Leela led the NN-engine charge up the rankings. As can be seen from Fig. 4, it was to be a match of two halves, tied after game 51 but with Stockfish edging ahead in game 52 and easing away (but not ‘warming up’) after a pause in the ‘60s to a +9 win. This result was almost exactly predicted in the pre-match poll: the chat room are a knowledgeable bunch!

Jeroen Noomen (2020) explained his approach to the openings. The tireless Wool (2020) provided notes on many of the games, going more deeply into all the decisive ones as listed in Table 7. ‘Kingscrusher’ (2020a–c) covered games 5 and 6. GM Matthew Sadler (2020), again delighted by the selection of openings, provided his own very personal reactions to 19 games33 of the match in a companion piece complementing this report. We are happy and thankful to be able to refer the reader to these various accounts, all of which add to our understanding of this match. Matthew majored particularly on games 18, 60, 78 and 92 for which we provide the ‘teaser’ evaluation curves in Fig. 5. Game 78 took his ‘game of the Sufi’ prize so if you only have time to study one game in depth, this is the one: ‘awesome play on both side.’

In the spirit of ‘what happened next?’ we have selected five positions from these games, see Fig. 6. Clues, and sometimes the answers can be found in Matthew’s narrative but of course, the reader should ponder them first without, or with, engine assistance.

Fig. 4.

The TCEC19 Superfinal: Stockfish’s net wins.

Table 7

The TCEC19 Superfinal: score detail

The underdog Leela was given White first in each mini-match, an innovation to be applauded. Each time Leela won, we wondered if Stockfish would bounce back. This happened on eight of nine occasions. Jeroen Noomen emphasises that ‘1-0/1-0’ openings are not busted but, taken across all TCEC matches, they certainly define their own category of ‘brave choices for Black’. It seems to be a combination of difficult and unnatural to find openings, which give Black the advantage and there were no Black wins, let alone 2-0 scores. Stockfish won ten mini-matches to Leela’s one, which speaks to at least its defensive capability. While the Superfinal may seem to have been ‘one way’, only five Elo points were exchanged between the engines after all was done.

Fig. 5.

Evaluations for Sufi games 18, 60, 78 and 92: ‘red’ = kibitzer Crystal (CPW, 2020).

Fig. 6.

(a) Game 18, position 26w; (b) g60 p150w; g78 (c) p27w and (d) p63w; g92 p22w.

7.In conclusion

So Stockfish became the TCEC Grand Champion for the ninth time: many congratulations to its developers and connections on this result, particularly for the NNUE development and decision.

For those who mourned the passing of ‘Shannon Stockfish’, TCEC generously provided three ‘Superfinal Bonus’ games. These were between Stockfish 202007311012, the most recent non-NN version, and Komodo, StockfishNN and Leela Chess Zero, all slightly different (a pity) from their Premier Division and Superfinal versions. The results with ‘StClassic’ first, were 69-31 against Komodo (no surprise), 39-61 against StockfishNN and 41-59 against Leela. These last two results indicate that the Stockfish move to NNUE was not only a no-brainer but that it was also ‘just in time’. Leela could have beaten StClassic in the Superfinal even more heavily than StockfishNN beat Leela.

Clearly then, Stockfish and Leela Chess Zero have moved up the Elo scale again, distancing the rest of the field and particularly the Shannon-AB engines in the Premier Division. Further innovations will open or close that gap: we have only seen ‘version 1’ of the integration of ‘Shannon’ and ‘NN’ ideas in Stockfish. NNUE technology is available to all and presumably allowed by TCEC in all engines. Will NNUE also find its way into Chessbase’s seasonal offerings? At the time of writing, Christmas is coming even if the Covid cloud still hangs over us.

As the engines move inexorably onward and upward, the gap widens ever more between their and we onlookers’ level of understanding. In this context, GM Matthew Sadler’s (2019, 2020) commentaries are welcome, particularly as they are so accessible and readable in their own right. Long may they continue. Is there also an opportunity here for engines and/or computer graphics to help our understanding here? DeepMind (Moxie Pictures, 2017) clearly had some sophisticated graphics to help them understand AlphaGo’s confidence levels about various aspects of the positions.

The ICGA’s prize for an annotation engine was never awarded but should now perhaps be relaunched. NN-engines are now both superhuman and fallible in some non-game domains so we need to know more of when they succeed and when they fail. Isaac Asimov’s (1942) First Law of Robotics is “A robot may not injure a human being or, through inaction, allow a human being to come to harm.” Written at a time when neural networks were not even on the horizon, it perhaps needs to be rewritten. If we do not use a robot that is better than a human, we violate the law. If we use a robot and it makes a rare mistake, we also violate the law. Discuss!

Once again, we congratulate all participants whether engine authors, trainers or fans, TCEC subscribers and supporters, authors of opening books or chat room entertainers. ‘noobpwnftw’ gets our particular thanks as the principle sponsor of TCEC. He has made major contributions to the progress of computer chess through the provision of the TCEC platform and in the endgame field (de Man et al., 2018).

Haworth and Hernandez (2020b) provides not only the games of the TCEC19 event but also the games of the three Sufi bonus matches. We feel that these are valuable context and of historical value. All decisive games have been played out by the first author’s Fritz17 at search-depth 24 ply for the benefit of beginners. Tables 8 and 9 below provide the usual cross-event statistics.

Table 8

Generic statistics for each phase of TCEC19: results, terminations and average game-length

Table 9

The shortest and longest 1-0, drawn and 0-1 games in each phase of TCEC19