Balkan Journal of Electrical and Computer Engineering

2147-284X 2147-284X

Balkan Yayın

10.17694/bajece.1114868

Computer Software Software Architecture

Bilgisayar Yazılımı Yazılım Mimarisi

An Approach for DC Motor Speed Control with Off-Policy Reinforcement Learning Method

https://orcid.org/0000-0001-9815-7724

Tüfenkçi

Sevilay

MALATYA TURGUT ÖZAL ÜNİVERSİTESİ

https://orcid.org/0000-0003-2651-5005

Kavuran

Gürkan

MALATYA TURGUT ÖZAL ÜNİVERSİTESİ

https://orcid.org/0000-0002-6106-2374

Yeroğlu

Celaleddin

İNÖNÜ ÜNİVERSİTESİ

06 04 2023

11 2 184 189 05 10 2022 05 30 2023

2013

Balkan Journal of Electrical and Computer Engineering

In the literature, interest in automatic control systems that do not require human intervention and perform at the desired level increases day by day. In this study, a Twin Delay Deep Deterministic Policy Gradient (TD3), a reinforcement learning algorithm, automatically controls a DC motor system. A reinforcement learning method is an approach that learns what should be done to reach the goal and observes the results that come out with the interaction of both itself and the environment. The proposed method aims to adjust the voltage value applied to the input of the DC motor in order to reach output with single input and single output structure to the desired speed.

Deep reinforcement learning DC motor PI controller Twin-delayed deep deterministic policy gradient

R.S. Sutton, "Reinforcement Learning: Past, Present and Future", Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), Vol. 1585, 1998, 195–197.

L.P. Kaelbling, M.L. Littman, A.W. Moore, "Reinforcement Learning: A Survey", J. Artif. Intell. Res., Vol. 4, 1996, pp. 237–285.

R.S. Sutton, A.G. Barto, "Reinforcement Learning: An Introduction", 1998.

J. Xue, Q. Gao, W. Ju, "Reinforcement learning for engine idle speed control", 2010 Int. Conf. Meas. Technol. Mechatronics Autom. ICMTMA 2010, Vol. 2, 2010, pp. 1008–1011.

E. Uchibe, M. Asada, K. Hosoda, "Behavior coordination for a mobile robot using modular reinforcement learning", IEEE Int. Conf. Intell. Robot. Syst., Vol. 3, 1996, pp. 1329–1336.

Z. Linan, Y. Peng, C. Lingling, Z. Xueping, T. Yantao, "Obstacle avoidance of multi mobile robots based on behavior decomposition reinforcement learning", 2007 IEEE Int. Conf. Robot. Biomimetics, ROBIO, 2007, pp. 1018–1023.

N.J. Van Eck, M. Van Wezel, "Application of reinforcement learning to the game of Othello", Comput. Oper. Res., Vol. 35, 2008, pp. 1999–2017.

C.J.C.H. Watkins, "Learning from delayed rewards", 1989.

C.J.C.H. Watkins, P. Dayan, "Q-learning", Mach. Learn. 1992, Vol. 83, 8, 1992, pp. 279–292,

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller, " Playing Atari with Deep Reinforcement Learning", 2013.

D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, "Deterministic Policy Gradient Algorithms".

T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, " Continuous control with deep reinforcement learning", 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc., 2015.

S. Fujimoto, H. Hoof, D. Meger, "Addressing Function Approximation Error in Actor-Critic Methods", http://proceedings.mlr.press/v80/fujimoto18a.html, 2018.

F. Harashima, S. Kondo, "Design Method For Digital Speed Control System Of Motor Drives", PESC Rec. - IEEE Annu. Power Electron. Spec. Conf., 1982, pp. 289–297.

D. Germanton, M. Lehr, "Variable speed DC motor controller apparatus particularly adapted for control of portable-power tools", 1989.

Y. Hoshino, "A proposal of Reinforcement Learning System to Use Knowledge effectively", 2003, pp. 1582–1585.

S.J. Russell, P. Norvig, "Artificial Intelligence A Modern Approach", 2003.

R.S. Sutton, D. Mcallester, S. Singh, Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation", Adv. NEURAL Inf. Process. Syst. 12, Vol. 12, 2000, pp. 1057--1063.

H. van Hasselt, A. Guez, D. Silver, "Deep Reinforcement Learning with Double Q-Learning", Proc. AAAI Conf. Artif. Intell. 30, 2016.

W.B. Knox, P. Stone, "Reinforcement learning from human reward: Discounting in episodic tasks", Proc. - IEEE Int. Work. Robot Hum. Interact. Commun., 2012, pp. 878–885.

University of Michigan: Control Tutorials for MATLAB and Simulink - Motor Speed: System Modeling, https://ctms.engin.umich.edu/CTMS/index.php?example=MotorSpeed&section=SystemModeling.