Research Article
BibTex RIS Cite

Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning

Year 2024, Volume: 13 Issue: 3, 30 - 46, 31.12.2024

Murat Erhan Çimen

Abstract

In recent years, the global decline in fossil fuel reserves and the alarming rise in greenhouse gas emissions have significantly heightened the need for renewable energy sources. This urgent shift towards sustainability has made the development and optimization of efficient energy systems a top priority for countries and communities worldwide. This study focuses on the modeling and control of a DC-DC boost converter, a critical component widely utilized in renewable energy applications such as solar panels, battery systems, and fuel cells. The research explores three control strategies: the conventional Proportional-Integral (PI) controller, state feedback controller with integral action, and Q-learning-based controller, which employs reinforcement learning principles. Comparative experiments were conducted in the Matlab/Simulink environment to evaluate the performance of each controller. Results demonstrate that the Q-learning controller outperformed the traditional methods in terms of performance metrics, including Integral Squared Error (ISE), Integral Absolute Error (IAE), and settling time, showcasing its potential for enhancing the efficiency and stability of renewable energy systems.

Keywords

Boost Converter Q learning PI Control State Feedback

References

  • Abdalla, A. O. M., Ibrahim, A. A. Z., Fadul, S. M. E. 2022. Modeling single diode PV using particle swarm optimization (PSO) techniques. Gaziosmanpaşa Bilimsel Araştırma Dergisi, 11(1), 44–56.
  • Abdalla, A. O. M., Önbilgin, G. 2024. An Approach to Optimized the Output Power of Photovoltaic System Using Artificial Neural Networks. Gaziosmanpaşa Bilimsel Araştırma Dergisi, 13(1), 18–30.
  • Ahmed, S. H., Ahmad, I. 2024. Optimal wireless power transfer to hybrid energy storage system for electric vehicles: A comparative analysis of machine learning-based model-free controllers. Journal of Energy Storage, 75, 109534.
  • Akyurek, H. A., Bucak, İ. Ö. 2012. Zamansal-Fark, Uyarlanır Dinamik Programlama ve SARSA Etmenlerinin Tipik Arazi Aracı Problemi için Öğrenme Performansları. Akıllı Sistemlerde Yenilikler ve Uygulamaları Sempozyumu.
  • Alfred, D., Czarkowski, D., Teng, J. 2024. Reinforcement Learning-Based Control of a Power Electronic Converter. Mathematics, 12(5), 671.
  • Alkrunz, M., Yazıcı, İ. 2016. Design of discrete time controllers for the DC-DC boost converter. Sakarya University Journal of Science, 20(1), 75–82.
  • Angiuli, A., Fouque, J. P., Laurière, M. 2022. Unified reinforcement Q-learning for mean field game and control problems. Mathematics of Control, Signals, and Systems, 34(2), 217–271.
  • Borase, R. P., Maghade, D. K., Sondkar, S. Y., & Pawar, S. N. 2021. A Review of PID Control, Tuning Methods and Applications. International Journal of Dynamics and Control, 9, 818–827.
  • Bououden, S., Hazil, O., Filali, S., Chadli, M. 2014. Modelling and model predictive control of a DC-DC Boost converter. In 2014 15th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA), 643–648.
  • Çimen, M., Garip, Z., Boz, A. 2021. Chaotic flower pollination algorithm based optimal PID controller design for a buck converter. Analog Integrated Circuits and Signal Processing.
  • El Fadil, H., Giri, F. 2007. Backstepping based control of PWM DC-DC boost power converters. In 2007 IEEE International Symposium on Industrial Electronics, 395–400.
  • Farajdadian, S., Hajizadeh, A., & Soltani, M. (2024). Recent developments of multiport DC/DC converter topologies, control strategies, and applications: A comparative review and analysis. Energy Reports, 11, 1019–1052.
  • Garip, Z., Çimen, M. E., Boz, A. F. 2021. Meta-sezgisel algoritmalar kullanarak güneş pili modellerinin parametre çıkarımında karşılaştırmalı performans analizi. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 2, 1133–1144. https://doi.org/10.17341/gazimmfd.586269
  • Gheisarnejad, M., Khooban, M. H. 2023. Quantum Deep Learning for Fast Switching of Full-Bridge Power Converters. Designs, 7(3), 60. Güldemir, H. 2005. Sliding mode control of DC-DC boost converter. Journal of Applied Sciences, 5(3), 588–592.
  • Güngör, O., Yüksek, H. İ. 2020. Modeling of Boost and Cuk Converters and Comparison of Their Performance in MPPT. Sigma Journal of Engineering and Natural Sciences, 11(1), 83–101.
  • Harmon, M. E., Harmon, S. S. 1996. Reinforcement learning: A tutorial. WL/AAFC, WPAFB Ohio, 45433, 237–285.
  • Kang, H., Jung, S., Kim, H., Jeoung, J., Hong, T. (2024). Reinforcement learning-based optimal scheduling model of battery energy storage system at the building level. Renewable and Sustainable Energy Reviews, 190, 114054.
  • Ibrahim, O., Yahaya, N. Z., Saad, N. 2016. Comparative studies of PID controller tuning methods on a DC-DC boost converter. In 2016 6th International Conference on Intelligent and Advanced Systems, 1–5.
  • Meng, Q., Hussain, S., Luo, F., Wang, Z., Jin, X. (2024). An online reinforcement learning-based energy management strategy for microgrids with centralized control. IEEE Transactions on Industry Applications.
  • Muktiadji, R. F., Ramli, M. A., & Milyani, A. H. 2024. Twin-Delayed Deep Deterministic Policy Gradient Algorithm to Control a Boost Converter in a DC Microgrid. Electronics, 13(2), 433.
  • Nishanthi, B., Kanakaraj, J. 2024. Enactment of deep reinforcement learning control for power management and enhancement of voltage regulation in a DC micro-grid system. Electric Power Components and Systems, 52(4), 555–565.
  • Li, Y., Wu, J., Pan, Y. (2024). Deep reinforcement learning for online scheduling of photovoltaic systems with battery energy storage systems. Intelligent and Converged Networks, 5(1), 28-41.
  • Liu, Q., Guo, Y., Xu, T. (2024). Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networks. arXiv preprint arXiv:2408.06776.
  • Palpandian, P., Govindaraj, V., Dinesh, S., Megalan, K., Sivaprasanth, K., Vigneshwaran, G. 2024. Intelligent Voltage Regulation Using Machine Learning-Enhanced Boost Converter. In 2024 International Conference on Science Technology Engineering and Management (ICSTEM), 1–7.
  • Panggabean, J., Sutisna, N., Syafalni, I., & Adiono, T. 2023. Comparison of MPPT based on Deep Reinforcement Learning by DQN, DDPG and TD3. In 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 261–266.
  • Rajamallaiah, A., Karri, S. P. K., & Sankar, Y. R. (2024). Deep Reinforcement Learning Based Control Strategy for Voltage Regulation of DC-DC Buck Converter Feeding CPLs in DC Microgrid. IEEE Access.
  • Saha, U., Shahria, S., & Rashid, A. B. (n.d.). Proximal Policy Optimization-Based Reinforcement Learning Approach for DC-DC Boost Converter Control: A Comparative Evaluation Against Traditional Control Techniques. 2023. ArXiv Preprint ArXiv:2310.02945.
  • Sezen, A., Keskin, K. 2021. Hybrid Control of DC-DC Buck Boost Converter. Demiryolu Mühendisliği, 14, 99–109. Smart, W. D., Kaelbling, L. P. 2000. Practical Reinforcement Learning in Continuous Spaces. ICML.
  • Su, T., Wu, T., Zhao, J., Scaglione, A., & Xie, L. (n.d.). A Review of Safe Reinforcement Learning Methods for Modern Power Systems. ArXiv Preprint ArXiv:2407.00304.
  • Sun, Z., Lu, T. (2024). Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism. IET Generation, Transmission & Distribution, 18(1), 39-49.
  • Uçmaz, B. N., Yakut, Y. B. (2024). PEM yakıt pillerinde PI, PSO ve FOPI kontrollü DC/DC dönüştürücülerine ilişkin performanslarının karşılaştırılması. Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Dergisi, 15(1), 23–29.
  • Wang, H., Emmerich, M., Plaat, A. (2018). Monte Carlo Q-learning for General Game Playing. ArXiv Preprint ArXiv:1802.05944.
  • Xu, J. H., Zhang, B. X., Yan, H. Z., Ding, Q., Zhu, K. Q., Yang, Y. R., & Wang, X. D. (2024). Sliding mode–Extended state observer control strategy to improve energy transfer of PEMFC connected DC-DC boost converter system. Sustainable Energy Technologies and Assessments, 63, 103645.
  • You, W., Yang, G., Chu, J., & Ju, C. (2023). Deep reinforcement learning-based proportional–integral control for dual-active-bridge converter. Neural Computing and Applications, 35(24), 17953–17966.

Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning

Year 2024, Volume: 13 Issue: 3, 30 - 46, 31.12.2024

Murat Erhan Çimen

Abstract

In recent years, the global decline in fossil fuel reserves and the alarming rise in greenhouse gas emissions have significantly heightened the need for renewable energy sources. This urgent shift towards sustainability has made the development and optimization of efficient energy systems a top priority for countries and communities worldwide. This study focuses on the modeling and control of a DC-DC boost converter, a critical component widely utilized in renewable energy applications such as solar panels, battery systems, and fuel cells. The research explores three control strategies: the conventional Proportional-Integral (PI) controller, state feedback controller with integral action, and Q-learning-based controller, which employs reinforcement learning principles. Comparative experiments were conducted in the Matlab/Simulink environment to evaluate the performance of each controller. Results demonstrate that the Q-learning controller outperformed the traditional methods in terms of performance metrics, including Integral Squared Error (ISE), Integral Absolute Error (IAE), and settling time, showcasing its potential for enhancing the efficiency and stability of renewable energy systems.

Keywords

Boost Converter Q learning PI Control State Feedback

References

  • Abdalla, A. O. M., Ibrahim, A. A. Z., Fadul, S. M. E. 2022. Modeling single diode PV using particle swarm optimization (PSO) techniques. Gaziosmanpaşa Bilimsel Araştırma Dergisi, 11(1), 44–56.
  • Abdalla, A. O. M., Önbilgin, G. 2024. An Approach to Optimized the Output Power of Photovoltaic System Using Artificial Neural Networks. Gaziosmanpaşa Bilimsel Araştırma Dergisi, 13(1), 18–30.
  • Ahmed, S. H., Ahmad, I. 2024. Optimal wireless power transfer to hybrid energy storage system for electric vehicles: A comparative analysis of machine learning-based model-free controllers. Journal of Energy Storage, 75, 109534.
  • Akyurek, H. A., Bucak, İ. Ö. 2012. Zamansal-Fark, Uyarlanır Dinamik Programlama ve SARSA Etmenlerinin Tipik Arazi Aracı Problemi için Öğrenme Performansları. Akıllı Sistemlerde Yenilikler ve Uygulamaları Sempozyumu.
  • Alfred, D., Czarkowski, D., Teng, J. 2024. Reinforcement Learning-Based Control of a Power Electronic Converter. Mathematics, 12(5), 671.
  • Alkrunz, M., Yazıcı, İ. 2016. Design of discrete time controllers for the DC-DC boost converter. Sakarya University Journal of Science, 20(1), 75–82.
  • Angiuli, A., Fouque, J. P., Laurière, M. 2022. Unified reinforcement Q-learning for mean field game and control problems. Mathematics of Control, Signals, and Systems, 34(2), 217–271.
  • Borase, R. P., Maghade, D. K., Sondkar, S. Y., & Pawar, S. N. 2021. A Review of PID Control, Tuning Methods and Applications. International Journal of Dynamics and Control, 9, 818–827.
  • Bououden, S., Hazil, O., Filali, S., Chadli, M. 2014. Modelling and model predictive control of a DC-DC Boost converter. In 2014 15th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA), 643–648.
  • Çimen, M., Garip, Z., Boz, A. 2021. Chaotic flower pollination algorithm based optimal PID controller design for a buck converter. Analog Integrated Circuits and Signal Processing.
  • El Fadil, H., Giri, F. 2007. Backstepping based control of PWM DC-DC boost power converters. In 2007 IEEE International Symposium on Industrial Electronics, 395–400.
  • Farajdadian, S., Hajizadeh, A., & Soltani, M. (2024). Recent developments of multiport DC/DC converter topologies, control strategies, and applications: A comparative review and analysis. Energy Reports, 11, 1019–1052.
  • Garip, Z., Çimen, M. E., Boz, A. F. 2021. Meta-sezgisel algoritmalar kullanarak güneş pili modellerinin parametre çıkarımında karşılaştırmalı performans analizi. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 2, 1133–1144. https://doi.org/10.17341/gazimmfd.586269
  • Gheisarnejad, M., Khooban, M. H. 2023. Quantum Deep Learning for Fast Switching of Full-Bridge Power Converters. Designs, 7(3), 60. Güldemir, H. 2005. Sliding mode control of DC-DC boost converter. Journal of Applied Sciences, 5(3), 588–592.
  • Güngör, O., Yüksek, H. İ. 2020. Modeling of Boost and Cuk Converters and Comparison of Their Performance in MPPT. Sigma Journal of Engineering and Natural Sciences, 11(1), 83–101.
  • Harmon, M. E., Harmon, S. S. 1996. Reinforcement learning: A tutorial. WL/AAFC, WPAFB Ohio, 45433, 237–285.
  • Kang, H., Jung, S., Kim, H., Jeoung, J., Hong, T. (2024). Reinforcement learning-based optimal scheduling model of battery energy storage system at the building level. Renewable and Sustainable Energy Reviews, 190, 114054.
  • Ibrahim, O., Yahaya, N. Z., Saad, N. 2016. Comparative studies of PID controller tuning methods on a DC-DC boost converter. In 2016 6th International Conference on Intelligent and Advanced Systems, 1–5.
  • Meng, Q., Hussain, S., Luo, F., Wang, Z., Jin, X. (2024). An online reinforcement learning-based energy management strategy for microgrids with centralized control. IEEE Transactions on Industry Applications.
  • Muktiadji, R. F., Ramli, M. A., & Milyani, A. H. 2024. Twin-Delayed Deep Deterministic Policy Gradient Algorithm to Control a Boost Converter in a DC Microgrid. Electronics, 13(2), 433.
  • Nishanthi, B., Kanakaraj, J. 2024. Enactment of deep reinforcement learning control for power management and enhancement of voltage regulation in a DC micro-grid system. Electric Power Components and Systems, 52(4), 555–565.
  • Li, Y., Wu, J., Pan, Y. (2024). Deep reinforcement learning for online scheduling of photovoltaic systems with battery energy storage systems. Intelligent and Converged Networks, 5(1), 28-41.
  • Liu, Q., Guo, Y., Xu, T. (2024). Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networks. arXiv preprint arXiv:2408.06776.
  • Palpandian, P., Govindaraj, V., Dinesh, S., Megalan, K., Sivaprasanth, K., Vigneshwaran, G. 2024. Intelligent Voltage Regulation Using Machine Learning-Enhanced Boost Converter. In 2024 International Conference on Science Technology Engineering and Management (ICSTEM), 1–7.
  • Panggabean, J., Sutisna, N., Syafalni, I., & Adiono, T. 2023. Comparison of MPPT based on Deep Reinforcement Learning by DQN, DDPG and TD3. In 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 261–266.
  • Rajamallaiah, A., Karri, S. P. K., & Sankar, Y. R. (2024). Deep Reinforcement Learning Based Control Strategy for Voltage Regulation of DC-DC Buck Converter Feeding CPLs in DC Microgrid. IEEE Access.
  • Saha, U., Shahria, S., & Rashid, A. B. (n.d.). Proximal Policy Optimization-Based Reinforcement Learning Approach for DC-DC Boost Converter Control: A Comparative Evaluation Against Traditional Control Techniques. 2023. ArXiv Preprint ArXiv:2310.02945.
  • Sezen, A., Keskin, K. 2021. Hybrid Control of DC-DC Buck Boost Converter. Demiryolu Mühendisliği, 14, 99–109. Smart, W. D., Kaelbling, L. P. 2000. Practical Reinforcement Learning in Continuous Spaces. ICML.
  • Su, T., Wu, T., Zhao, J., Scaglione, A., & Xie, L. (n.d.). A Review of Safe Reinforcement Learning Methods for Modern Power Systems. ArXiv Preprint ArXiv:2407.00304.
  • Sun, Z., Lu, T. (2024). Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism. IET Generation, Transmission & Distribution, 18(1), 39-49.
  • Uçmaz, B. N., Yakut, Y. B. (2024). PEM yakıt pillerinde PI, PSO ve FOPI kontrollü DC/DC dönüştürücülerine ilişkin performanslarının karşılaştırılması. Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Dergisi, 15(1), 23–29.
  • Wang, H., Emmerich, M., Plaat, A. (2018). Monte Carlo Q-learning for General Game Playing. ArXiv Preprint ArXiv:1802.05944.
  • Xu, J. H., Zhang, B. X., Yan, H. Z., Ding, Q., Zhu, K. Q., Yang, Y. R., & Wang, X. D. (2024). Sliding mode–Extended state observer control strategy to improve energy transfer of PEMFC connected DC-DC boost converter system. Sustainable Energy Technologies and Assessments, 63, 103645.
  • You, W., Yang, G., Chu, J., & Ju, C. (2023). Deep reinforcement learning-based proportional–integral control for dual-active-bridge converter. Neural Computing and Applications, 35(24), 17953–17966.
There are 34 citations in total.

Details

Primary Language English
Subjects Reinforcement Learning, Electrical Circuits and Systems, Power Electronics, Control Theoryand Applications
Journal Section Araştırma Makaleleri
Authors

Murat Erhan Çimen 0000-0002-1793-485X

Publication Date December 31, 2024
Submission Date September 11, 2024
Acceptance Date November 20, 2024
Published in Issue Year 2024 Volume: 13 Issue: 3

Cite

APA Çimen, M. E. (2024). Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning. Gaziosmanpaşa Bilimsel Araştırma Dergisi, 13(3), 30-46.
AMA Çimen ME. Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning. GBAD. December 2024;13(3):30-46.
Chicago Çimen, Murat Erhan. “Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning”. Gaziosmanpaşa Bilimsel Araştırma Dergisi 13, no. 3 (December 2024): 30-46.
EndNote Çimen ME (December 1, 2024) Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning. Gaziosmanpaşa Bilimsel Araştırma Dergisi 13 3 30–46.
IEEE M. E. Çimen, “Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning”, GBAD, vol. 13, no. 3, pp. 30–46, 2024.
ISNAD Çimen, Murat Erhan. “Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning”. Gaziosmanpaşa Bilimsel Araştırma Dergisi 13/3 (December 2024), 30-46.
JAMA Çimen ME. Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning. GBAD. 2024;13:30–46.
MLA Çimen, Murat Erhan. “Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning”. Gaziosmanpaşa Bilimsel Araştırma Dergisi, vol. 13, no. 3, 2024, pp. 30-46.
Vancouver Çimen ME. Controller Design For Dc-Dc Boost Converter Using PI, State Feedback and Q Learning. GBAD. 2024;13(3):30-46.
 Download Cover Image