The future of computer chess tournaments
In recent years the strength of chess programs have increased significantly, for a large part due to the rise of neural-network based evaluation. In 2018 Leela Chess Zero participated in the ICGA tournaments in Stockholm. Although it did not score well there (4 draws out of 7 games) the potential of the neural net was already clear. The next year in Macao Leela was not able to participate, but two other programs (and former world champions) did evolve into using neural networks: Shredder and Jonny. Jonny reached 2nd place in the software championship, which showed that the neural networks could also perform on regular computers without major GPU calculation power. The next tournament in Vienna 2022 was the first one after the NNUE (Efficiently Updatable Neural Networks) revolution made all top computer chess engines switch to using neural networks. Only one participant there (The Baron) did not make this step and was punished for this by losing all games in the open hardware tournament.
In the tournaments up to the 2022 event there have been editions where one or more programs tied for first place. In 2018 this happened in both the WCCC and WCSC tournaments. In both instances the tie was broken by a 2-game playoff match. In 2019 no tie-breaking matches were required. In 2022 again a tie-breaking match was required for both tournaments. Before the tie-breaking matches were played there was a discussion amongst the participants whether this would lead to a fair result. The rules stated that when in four 2-game matches all ended in a tie, an Armageddon game was to be played. The participants foresaw already that when it would come to this the player lucky enough to draw the lots to play black would have an almost decisive advantage. As no better (feasible) alternative could be decided upon the playoff matches were played as stipulated in the rules. And both tournaments were ultimately decided by an Armageddon game, both of which were won by black. This would be something to improve upon for the next tournament.
Fig. 1.
The playing hall in 2022, Vienna (photo: Jan Krabbenbos).
Experiences in other tournaments. Several suggestions were made to solve the Armageddon problem. One could establish an auction system where the white advantage (in time) is gradually increased until one player is willing to take the white colors. However, this only works as long as one player is willing to take the gamble and make a bid for the white colors.
The TCEC (Top Chess Engine Competition) is a long running competition with the goal of having chess engines battle it out at long time control on a high-spec computer. As the main goal was to have the engines compete without the use of tools like opening books or learning features a different method needed to be found to avoid repeated games. This was found in the selection of a set of starting positions. As the chess engines evolved it also became clear that it was not enough to just provide 100 random positions, but if meaningful decisive games are required the selection of the opening positions required a careful process (see article ‘Not 100 draws, no double kills’ of Matthew Sadler in the New In Chess Yearbook 141).
Changes in the 2023 tournament. The goal of the WCCC tournament has historically not been a software competition, but instead a competition to find select the best chess playing entity, whether it is software, hardware or databases (e.g. opening book) based decision making. The opening-book has traditionally been one of the major discriminators in the WCCC (and WCSC) tournaments, so replacing this with pre-selected opening positions for the whole tournament was not easy to be accepted by all participants. Instead two measures were agreed upon during the player’s meeting:
Doubling the number of games to be played by reducing the time controls. More games will increase the chances of decisive games between the participants, and as the current hardware is huge compared to a decade ago, the reduction of thinking time was not perceived as a major degradation in the playing strength.
When the tournament requires tiebreaking the playoff matches would be played with pre-selected starting positions, to increase the chances of a decision. The Armageddon game remained the final stage of the playoff match. It was also decided that opening books should be disabled during the playoffs to avoid ‘booking-up’ and repeating a win from this position in the previous round.
The position may have only few games with it in my database to avoid that an opening book would include this position and play many moves from its opening book instead
To maximize the test for the program the number of moves leading to the opening position should be minimal.
After a 2-game test on this opening position (Stockfish 15 and 15.1 with their standard (and different) neural nets were used) should give as a maximum one decisive game and non-drawish evaluation scores during the majority of the game.
Arasan by Jon Dart (USA) Arasan is an open source engine that has been around since the early 90s. Arasan was adopted to use neural networks in 2021.
Jonny by Johannes Zwanzger (Germany) Jonny has been participating in the WCCC since 2003 when the tournament was organized in Graz. It usually runs using a big university cluster providing thousands of cores. Jonny has been a world champion in 2015 before, e.g., Komodo, Hiarcs and Ginkgo, but is also the runner-up in the WCSC of 2019 in Macao.
Fig. 2.
Playing area in Valencia (photo: Richard Pijl).
Fritz by Frank Schneider (Germany) Fritz (formerly known as Ginkgo) is participating in the WCCC since 2015. Frank participated with his previous program Gromitchess in the WCCC 1999. Ginkgo is the WCSC champion of 2022.
Stoofvlees by Gian-Carlo Pascutto (Belgium) Gian-Carlo is also known for his previous chess program Deep Sjeng and Go program Leela which served as the starting point for the development of Leela Chess Zero.
The lack of a graphics cards that is on par with the computing power of the provided CPU put Stoofvlees at a disadvantage. By losing one game to Fritz the operator of Stoofvlees decided to go all-in in the second encounter which did not yield the wanted result. All other games were drawn, but due to the two wins of Fritz against Stoofvlees no playoffs were needed (See Table 1 for an overview of all results). Fritz is the world software champion of 2023
Table 1
Results of the WCSC 2023
World computer chess championship. In the WCCC (or open-hardware tournament) there has been a variety of hardware selected which indicates where the strongpoints of the different participants lie
This tournament was decided in the games against the participant with the weakest hardware, but that advantage was not big enough to give a decisive advantage for all other participants (See Table 2 for an overview of all results). In fact, a careful selected side-line was needed to avoid getting into an equal position once the massive big book of Arasan gives control over to the engine.
| Participant | Hardware |
| Arasan | AMD Threadripper 32 cores |
| Jonny | Cluster of 100 x AMD EPYC (32 cores each) |
| Fritz | AMD EPYC 2x96 cores (overclocked), 1.5 TB RAM |
| Stoofvlees | AMD Threadripper 1x64 cores, 4x nVidia RTX 4090 |
Even though many games ended in a draw it does not mean that there was exciting play in many of the games. One example is from the game Fritz-Stoofvlees (see the diagram in Fig. 3).
Table 2
Results of the WCCC 202
Stoofvlees now uncorked the surprise 20..Rxf2, resulting in a very narrow road to a draw. A small inaccuracy on either side would have resulted in a decisive game.
Fig. 3.
Fritz – Stoofvlees: position after 20.Nh2.
The game was analyzed by Albert Silver in a youtube video: https://www.youtube.com/watch?v=qqVU6aSF8I0
World computer speed chess championship. The Speed chess tournament is a special tournament in the sense that the operator has a major influence on the results. Where in the other tournaments problems with hardware/connection can be solved with stopped clocks, in this tournament that is not the case. Additionally, operator errors (e.g. executing the wrong move on the board) will be punished with a loss.
Operator errors and/or technical problems were the main cause of the decisive games in this tournament. But somehow this evened out over all participants and after 12 rounds we had the standings as in Table 3.
All participants had exactly the same amount of points after 12 rounds, which meant that the foreseen tiebreaking rule (using SB points to select the two topmost participants) had no effect.
Table 3
Results of the Speedchess tournament after 12 rounds
After some discussion it was decided to add an additional double-round robin cycle of 6 rounds to the tournament. As the additional games could be considered a playoff stage, it was also decided to use play-off rules and use pre-selected starting positions instead of the opening books. In total 3 different positions were used, one for each round where two participants played each other with the same starting position with both black and white colors.
Where in the two cycles of the regular tournament most decisive games were due to operator errors and/or technical problems, the playoff cycle had no such results. Instead four out of 12 games were decided on the chessboard (see Table 4 for an overview of the results). And even in games that ended in a draw the scores reported by the engines were longer unbalanced then when using the opening book.
Table 4
Results of the Speedchess tournament after the playoff rounds
Summary. When excluding games where hardware plays a major role (e.g., Stoofvlees in the software championship due to the basic graphics card, Arasan in the WCCC) all games are likely to end in a draw. This is even more clear when looking at the results of Jonny. Jonny’s book only has main lines, and all of its games were a draw, even when at a hardware advantage.
When using pre-selected starting positions the games can still result in a draw, but the drawing rate is much less than with playing with an opening book. On top of that the games will be more interesting, also from the computer point of view.
