Institute of Information Science, Academia Sinica

Game-playing programs based on the Alpha Zero architecture have achieved superhuman level in games such as chess and Go. How close are these programs to perfection? We analyse two top open source programs, Leela Chess Zero and KataGo, on endgame positions for which a comparison to perfect play is possible. We study the performance of these engines both with and without search, and at two different stages of their training. We identify both strong and weak points of these state of the art approaches, and show some surprising examples. We discuss the gap between general purpose and specialised systems, and possible implications for building reliable AI systems.

Martin Müller is a professor in the Department of Computing Science at the University of Alberta, DeepMind Chair in Artificial Intelligence, Canada CIFAR AI Chair at Amii (the Alberta Machine Intelligence Institute), and an Amii Fellow. His main research interests are in search and planning, especially for two-player games such as Go.