Sensör Tabanlı DDQN Modeline Ödül Fonksiyonu Belirleme

Öz

Bu çalışmada DDQN Modeli ile Pekiştirmeli Öğrenme içerisinde 100 engeli/nesneyi geçmek üzere eğitilen ajanın uygun ödül fonksiyonunun belirlenmesi amaçlanmaktadır. Ajanı eğitmek için çevre alt problemlere bölümüştür. Alt problemler için çeşitli kurallar ve farklı ödül fonksiyonları tanımlanmıştır. Eğitim için gNet adında geliştirilmiş mini derin öğrenme kütüphanesi kullanılmıştır.

Anahtar Kelimeler

• yapay zeka
• derin öğrenme
• DDQN
• gNet
• ödül fonksiyonu
• sensor tabanlı

Setting Reward Function of Sensor Based DDQN Model

Abstract

In this study, it is aimed to determine the appropriate reward function of the agent which trained to pass 100 obstacles/objects in Reinforcement Learning (RL) with Double Deep Q Network (DDQN) model. To train the agent, environment is split into sub problems. Several rules and different reward functions defined for the sub problems. A developed mini deep learning library which is called gNet is used for the training.

Keywords

• artificial intelligence
• deep learning
• DDQN
• gNet
• reward function
• sensor based

References

1. R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning, MIT Press, 1998.
2. E. Ratner, D. Hadfield-Menell and A. D. Dragan, “Simplifying Reward Design through Divide-and-Conquer,” CoRR, vol. abs/1806.02501, 2018, [Online] http://arxiv.org/abs/1806.02501.
3. Z. Hu, K. Wan, X. Gao, and Y. Zhai, “A Dynamic Adjusting Reward Function Method for Deep Reinforcement Learning with Adjustable Parameters,” Mathematical Problems in Engineering, vol. 2019, pp. 1-10, DOI: 10.1155/2019/7619483.
4. C. J. C. H. Watkins and P. Dayan, “Q-Learning,” Machine Learning, vol. 8, 1992, pp. 279-292.
5. R. E. Bellmann and S. E. Dreyfus, Applied Dynamic Programming, Princeton, NJ, USA: Princeton University Press, 1962.
6. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra and M. Riedmiller, "Playing Atari with Deep Reinforcement Learning," CoRR, vol. abs/1312.5602, 2013, [Online] https://arxiv.org/abs/1312.5602
7. L. Lin, “Reinforcement Learning for Robots Using Neural Networks,” Ph.D. dissertation, School of Computer Science, Carnegie Mellon Univ., Pittsburgh, PA, USA, 1993.
8. H. van Hasselt, A. Guez, D. Silver, "Deep Reinforcement Learning with Double Q-Learning," in Proc. of the AAAI Conference on Artificial Intelligence, vol. 30, No.1, 2016, [Online] https://arxiv.org/abs/1509.06461

9. gNet, Avalaible: https://github.com/MGokcayK/gNet.
10. D. P. Kingma, J. Ba, (2014, 12), Adam: A Method for Stochastic Optimization in International Conference on Learning Representations, [Online] https://arxiv.org/abs/1412.6980.