Research Article
BibTex RIS Cite

An Approach for DC Motor Speed Control with Off-Policy Reinforcement Learning Method

Year 2023, Volume: 11 Issue: 2, 184 - 189, 04.06.2023
https://doi.org/10.17694/bajece.1114868

Abstract

In the literature, interest in automatic control systems that do not require human intervention and perform at the desired level increases day by day. In this study, a Twin Delay Deep Deterministic Policy Gradient (TD3), a reinforcement learning algorithm, automatically controls a DC motor system. A reinforcement learning method is an approach that learns what should be done to reach the goal and observes the results that come out with the interaction of both itself and the environment. The proposed method aims to adjust the voltage value applied to the input of the DC motor in order to reach output with single input and single output structure to the desired speed.

References

  • R.S. Sutton, "Reinforcement Learning: Past, Present and Future", Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), Vol. 1585, 1998, 195–197.
  • L.P. Kaelbling, M.L. Littman, A.W. Moore, "Reinforcement Learning: A Survey", J. Artif. Intell. Res., Vol. 4, 1996, pp. 237–285.
  • R.S. Sutton, A.G. Barto, "Reinforcement Learning: An Introduction", 1998.
  • J. Xue, Q. Gao, W. Ju, "Reinforcement learning for engine idle speed control", 2010 Int. Conf. Meas. Technol. Mechatronics Autom. ICMTMA 2010, Vol. 2, 2010, pp. 1008–1011.
  • E. Uchibe, M. Asada, K. Hosoda, "Behavior coordination for a mobile robot using modular reinforcement learning", IEEE Int. Conf. Intell. Robot. Syst., Vol. 3, 1996, pp. 1329–1336.
  • Z. Linan, Y. Peng, C. Lingling, Z. Xueping, T. Yantao, "Obstacle avoidance of multi mobile robots based on behavior decomposition reinforcement learning", 2007 IEEE Int. Conf. Robot. Biomimetics, ROBIO, 2007, pp. 1018–1023.
  • N.J. Van Eck, M. Van Wezel, "Application of reinforcement learning to the game of Othello", Comput. Oper. Res., Vol. 35, 2008, pp. 1999–2017.
  • C.J.C.H. Watkins, "Learning from delayed rewards", 1989.
  • C.J.C.H. Watkins, P. Dayan, "Q-learning", Mach. Learn. 1992, Vol. 83, 8, 1992, pp. 279–292,
  • V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller, " Playing Atari with Deep Reinforcement Learning", 2013.
  • D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, "Deterministic Policy Gradient Algorithms".
  • T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, " Continuous control with deep reinforcement learning", 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc., 2015.
  • S. Fujimoto, H. Hoof, D. Meger, "Addressing Function Approximation Error in Actor-Critic Methods", http://proceedings.mlr.press/v80/fujimoto18a.html, 2018.
  • F. Harashima, S. Kondo, "Design Method For Digital Speed Control System Of Motor Drives", PESC Rec. - IEEE Annu. Power Electron. Spec. Conf., 1982, pp. 289–297.
  • D. Germanton, M. Lehr, "Variable speed DC motor controller apparatus particularly adapted for control of portable-power tools", 1989.
  • Y. Hoshino, "A proposal of Reinforcement Learning System to Use Knowledge effectively", 2003, pp. 1582–1585.
  • S.J. Russell, P. Norvig, "Artificial Intelligence A Modern Approach", 2003.
  • R.S. Sutton, D. Mcallester, S. Singh, Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation", Adv. NEURAL Inf. Process. Syst. 12, Vol. 12, 2000, pp. 1057--1063.
  • H. van Hasselt, A. Guez, D. Silver, "Deep Reinforcement Learning with Double Q-Learning", Proc. AAAI Conf. Artif. Intell. 30, 2016.
  • W.B. Knox, P. Stone, "Reinforcement learning from human reward: Discounting in episodic tasks", Proc. - IEEE Int. Work. Robot Hum. Interact. Commun., 2012, pp. 878–885.
  • University of Michigan: Control Tutorials for MATLAB and Simulink - Motor Speed: System Modeling, https://ctms.engin.umich.edu/CTMS/index.php?example=MotorSpeed&section=SystemModeling.
Year 2023, Volume: 11 Issue: 2, 184 - 189, 04.06.2023
https://doi.org/10.17694/bajece.1114868

Abstract

References

  • R.S. Sutton, "Reinforcement Learning: Past, Present and Future", Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), Vol. 1585, 1998, 195–197.
  • L.P. Kaelbling, M.L. Littman, A.W. Moore, "Reinforcement Learning: A Survey", J. Artif. Intell. Res., Vol. 4, 1996, pp. 237–285.
  • R.S. Sutton, A.G. Barto, "Reinforcement Learning: An Introduction", 1998.
  • J. Xue, Q. Gao, W. Ju, "Reinforcement learning for engine idle speed control", 2010 Int. Conf. Meas. Technol. Mechatronics Autom. ICMTMA 2010, Vol. 2, 2010, pp. 1008–1011.
  • E. Uchibe, M. Asada, K. Hosoda, "Behavior coordination for a mobile robot using modular reinforcement learning", IEEE Int. Conf. Intell. Robot. Syst., Vol. 3, 1996, pp. 1329–1336.
  • Z. Linan, Y. Peng, C. Lingling, Z. Xueping, T. Yantao, "Obstacle avoidance of multi mobile robots based on behavior decomposition reinforcement learning", 2007 IEEE Int. Conf. Robot. Biomimetics, ROBIO, 2007, pp. 1018–1023.
  • N.J. Van Eck, M. Van Wezel, "Application of reinforcement learning to the game of Othello", Comput. Oper. Res., Vol. 35, 2008, pp. 1999–2017.
  • C.J.C.H. Watkins, "Learning from delayed rewards", 1989.
  • C.J.C.H. Watkins, P. Dayan, "Q-learning", Mach. Learn. 1992, Vol. 83, 8, 1992, pp. 279–292,
  • V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller, " Playing Atari with Deep Reinforcement Learning", 2013.
  • D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, "Deterministic Policy Gradient Algorithms".
  • T.P. Lillicrap, J.J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, " Continuous control with deep reinforcement learning", 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc., 2015.
  • S. Fujimoto, H. Hoof, D. Meger, "Addressing Function Approximation Error in Actor-Critic Methods", http://proceedings.mlr.press/v80/fujimoto18a.html, 2018.
  • F. Harashima, S. Kondo, "Design Method For Digital Speed Control System Of Motor Drives", PESC Rec. - IEEE Annu. Power Electron. Spec. Conf., 1982, pp. 289–297.
  • D. Germanton, M. Lehr, "Variable speed DC motor controller apparatus particularly adapted for control of portable-power tools", 1989.
  • Y. Hoshino, "A proposal of Reinforcement Learning System to Use Knowledge effectively", 2003, pp. 1582–1585.
  • S.J. Russell, P. Norvig, "Artificial Intelligence A Modern Approach", 2003.
  • R.S. Sutton, D. Mcallester, S. Singh, Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation", Adv. NEURAL Inf. Process. Syst. 12, Vol. 12, 2000, pp. 1057--1063.
  • H. van Hasselt, A. Guez, D. Silver, "Deep Reinforcement Learning with Double Q-Learning", Proc. AAAI Conf. Artif. Intell. 30, 2016.
  • W.B. Knox, P. Stone, "Reinforcement learning from human reward: Discounting in episodic tasks", Proc. - IEEE Int. Work. Robot Hum. Interact. Commun., 2012, pp. 878–885.
  • University of Michigan: Control Tutorials for MATLAB and Simulink - Motor Speed: System Modeling, https://ctms.engin.umich.edu/CTMS/index.php?example=MotorSpeed&section=SystemModeling.
There are 21 citations in total.

Details

Primary Language English
Subjects Computer Software, Software Architecture
Journal Section Araştırma Articlessi
Authors

Sevilay Tüfenkçi 0000-0001-9815-7724

Gürkan Kavuran 0000-0003-2651-5005

Celaleddin Yeroğlu 0000-0002-6106-2374

Early Pub Date May 30, 2023
Publication Date June 4, 2023
Published in Issue Year 2023 Volume: 11 Issue: 2

Cite

APA Tüfenkçi, S., Kavuran, G., & Yeroğlu, C. (2023). An Approach for DC Motor Speed Control with Off-Policy Reinforcement Learning Method. Balkan Journal of Electrical and Computer Engineering, 11(2), 184-189. https://doi.org/10.17694/bajece.1114868

All articles published by BAJECE are licensed under the Creative Commons Attribution 4.0 International License. This permits anyone to copy, redistribute, remix, transmit and adapt the work provided the original work and source is appropriately cited.Creative Commons Lisansı