Adapting the AlphaZero Algorithm to Pawn Dama: Implementation, Training, and Performance Evaluation

Abstract

This research uses deep reinforcement learning techniques, notably the AlphaZero algorithm, to construct an artificial intelligence system that can play Pawn Dama at a level that surpasses human players. Pawn dama, a simplified variant of Dama, is a perfect platform to explore AI's ability to think strategically and make decisions. The primary goal is to develop an AI that can use self-play to develop sophisticated strategies and comprehend the game's dynamics and regulations. The project incorporates MCTS to improve decision-making during games and uses a Convolutional Neural Network (CNN) to enhance the AI's learning capabilities. Creating an intuitive graphical user interface, putting the reinforcement learning algorithm into practice, and testing the system against real players are steps in the development process. The accomplishment of this project will contribute to the field of strategic game AI research by providing insights that may be applied to other domains and spurring further advancements in AI-driven game strategies.

Keywords

• Deep Reinforcement Learning
• Deep Learning
• AlphaZero Algorithm
• Pawn Dama
• Monte Carlo Tree Search (MCTS)
• Convolutional Neural Network (CNN)

AlphaZero Algoritmasının Piyon Daması’na Uyarlanması: Uygulama, Eğitim ve Performans Değerlendirmesi

Abstract

This research uses deep reinforcement learning techniques, notably the AlphaZero algorithm, to construct an artificial intelligence system that can play Pawn Dama at a level that surpasses human players. Pawn dama, a simplified variant of Dama, is a perfect platform to explore AI's ability to think strategically and make decisions. The primary goal is to develop an AI that can use self-play to develop sophisticated strategies and comprehend the game's dynamics and regulations. The project incorporates MCTS to improve decision-making during games and uses a Convolutional Neural Network (CNN) to enhance the AI's learning capabilities. Creating an intuitive graphical user interface, putting the reinforcement learning algorithm into practice, and testing the system against real players are steps in the development process. The accomplishment of this project will contribute to the field of strategic game AI research by providing insights that may be applied to other domains and spurring further advancements in AI-driven game strategies.

Keywords

• Deep Reinforcement Learning
• Deep Learning
• AlphaZero Algorithm
• Pawn Dama
• Monte Carlo Tree Search (MCTS)
• Convolutional Neural Network (CNN)

Thanks

araştırmalarımız sırasında bize bir çatı sağlayan marmara ünv. bilg müh. bölümüne teşekkür ederiz

References

1. Shannon, C.E. (1950). Programming a Computer for Playing Chess. Philosophical Magazine, 41(314), 256-275.
2. Knuth, D.E., & Moore, R.W. (1975). An Analysis of Alpha-Beta Pruning. Artificial Intelligence, 6(4), 293-326.
3. Newborn, M. (1997). Kasparov versus Deep Blue: Computer Chess Comes of Age. Springer.
4. Schaeffer, J. (1997). One Jump Ahead: Challenging Human Supremacy in Checkers. Springer-Verlag.
5. Tesauro, G. (1995). Temporal Difference Learning and TD-Gammon. Communications of the ACM, 38(3), 58-68.
6. Samuel, A.L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3(3), 210-229.
7. Coulom, R. (2006). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In Proceedings of the 5th International Conference on Computers and Games (pp. 72-83).
8. Kocsis, L., & Szepesvári, C. (2006). Bandit Based Monte-Carlo Planning. Machine Learning, 282, 282-293.

9. Gelly, S., & Silver, D. (2007). Combining Online and Offline Knowledge in UCT. In Proceedings of the 24th International Conference on Machine Learning (273-280).
10. Ciancarini, P., & Favini, G.P. (2010). Monte Carlo tree search in Kriegspiel. Artificial Intelligence, 174, 670-684.
11. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012). A Survey of Monte Carlo Tree Search Methods. Artificial Intelligence Review, 34(1), 1-49.
12. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., & Lanctot, M. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484-489.
13. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., & Hassabis, D. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676), 354-359.
14. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., & Hassabis, D. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144.
15. Vinyals, O., Babuschkin, I., Czarnecki, W.M., Mathieu, M., Dudzik, A., Chung, J., Choi, D., Powell, R., Ewalds, T., Georgiev, P., & Silver, D. (2019). Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782), 350-354.
16. Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., & Silver, D. (2020). Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. Nature, 588(7839), 604-609.
17. Dong, H., Ding, Z., & Zhang, S. (2020). Fundamentals, Research and Applications. In Deep Reinforcement Learning (pp. 391-414).
18. Thakoor, S., Nair, S., & Jhunjhunwala, M. (2016). Learning to play othello without human knowledge. Stanford University.
19. Hassabis, D. (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. ArXiv. Retrieved from https://arxiv.org/abs/1712.01815
20. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
21. Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning.
22. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1), 1929-1958.
23. Nair, V., & Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the International Conference on Machine Learning.
24. Tomašev, N., Paquet, U., Hassabis, D., & Kramnik, V. (2020). Assessing game balance with AlphaZero: Exploring alternative rule sets in chess. arXiv preprint arXiv:2009.04374. https://arxiv.org/abs/2009.04374
25. Ye, W., Liu, S., Kurutach, T., Abbeel, P., & Gao, Y. (2021). Mastering Atari games with limited data. Advances in Neural Information Processing Systems, 34, 14917–14929.
26. Schmid, M., Moravčík, M., Burch, N., Kadlec, R., Davidson, J., Waugh, K., Bard, N., Timbers, F., Lanctot, M., Holland, G. Z., Davoodi, E., Christianson, A., & Bowling, M. (2023). Student of Games: A unified learning algorithm for both perfect and imperfect information games. Science Advances, 9(45), eadg3256. https://doi.org/10.1126/sciadv.adg3256
27. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.