Design of Cardiac Pacemaker Controller Based on Reinforcement Learning

Kağan Orbay; Mehmet Sagbas; Murat Demir

Design of Cardiac Pacemaker Controller Based on Reinforcement Learning

Abstract

This study investigates the derivation of PID controller parameters, commonly used for pacemaker control, using both genetic algorithm (GA) and reinforcement learning (RL) methods. We compare the PID parameters obtained by RL with those obtained by GA, a well-known and often preferred method in the literature. The aim of the study is to analyze the performance of the control parameters obtained by both methods and to determine which approach is more effective in pacemaker applications. In particular, comparisons on important control criteria such as rise time, settling time and overshoot of the system will reveal the advantages and disadvantages of these methods.

Keywords

Project Number

BBAP.2024.011

References

[1] Ponikowski, P., Anker, S. D., AlHabib, K. F., Cowie, M. R., Force, T. L., Hu, S., et al. (2014). Heart failure: preventing disease and death worldwide. ESC Heart Failure, 1(1), 4–25.
[2] Bajpai, S., Alam, S., Ali, M.A. (2017). Intelligent Heart Rate Controller using Fractional Order PID Controller Tuned by Genetic Algorithm for Pacemaker. International Journal of Engineering Research & Technology. 6(05), 715-720.
[3] Arunachalam, S. P., Kapa, S., Mulpuru, S.K., Friedman P.A., Tolkacheva, E.G. (2017). Intelligent Fractional-Order PID (FOPID) Heart Rate Controller for Cardiac Pacemaker, 2016 IEEE Healthcare Innovation Point-Of-Care Technologies Conference (HI-POCT), Cancun, Meksika, 2016, s. 105-108
[4] Bikki, P., Dhiraj, Y., & Kumar, R. N. (2023). Implementation of a Dual-Chamber Pacemaker for Low-Power Applications. https://doi.org/10.1109/icecct56650.2023.10179677
[5] Sutton R.S. and Barto, A.G. Reinforcement Learning: An Introduction. The MIT Press, London, 2018.
[6] Holland, J.H., Adaptation in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press, 1975.
[7] Biswas, S.C., Das, A., Guha, P. (2006). Mathematical Model of Cardiovascular System by Transfer function Method, Calcutta Medical Journal, 4, 15-17.
[8] Noble, D., A Modification of the Hodgkin-Huxley Equations Applicable to Purkinje Fibre Action and Pace-maker Potentials, 1962 Journal of Physiology , 160, 317-352. PubMed ID: 14480151

[9] Beeler, G.W. and Reuter, H. (1977) Reconstruction of the action potential of ventricular myocardial fibres. The Journal of physiology, 268, 177-210.
[10] Ziegler, J. G., & Nichols, N. B. (1942). Optimum Settings for Automatic Controllers. Trans. ASME, 64(11), 759–768.
[11] Cohen, G. H., & Coon, G. A. (1953). Theoretical Consideration of Retarded Control. Trans. ASME, 75, 827–834.
[12] Momani, S., Batiha I.M., El-Khazali, R. (2019). Design of PIλDδ-Heart Rate Controllers for Cardiac Pacemaker, 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Ajman, United Arab Emirates, pp. 1-5. https://doi.org/10.1109/ISSPIT47144.2019.9001785
[13] Govind K.R.A., Sekhar, R.A.: Design of a novel PID controller for cardiac pacemaker. 2014 International Conference on Advances in Green Energy (ICAGE), Thiruvananthapuram, India, pp. 82-87 (2014). https://doi.org/10.1109/ICAGE.2014.7050147
[14] Alfarhan, K.A., Mashor, M.Y., Mohd Saad, A.R., Omar, M.I. (2018). Wireless heart abnormality monitoring kit based on raspberry pi. Journal of Biomimetics, Biomaterials and Biomedical Engineering. 35, 96–108. https://doi.org/10.4028/www.scientific.net/JBBBE.35.96
[15] Srivastava, R., Kumar, B. (2022). Design of Anfis based pacemaker controller having improved transient response and its FPGA implementation. Biomedical Signal Processing and Control. https://doi.org/10.1016/j.bspc.2021.103186
[16] Lima, G.S., Savi, M.A., Bessa, W.M. (2023). Intelligent control of cardiac rhythms using artificial neural networks. Nonlinear Dynamics. 111(12), 11543–11557. https://doi.org/10.1007/s11071-023-08447-1
[17] Chen, E.Z., Wang, P., Chen, X., Chen, T., Sun, S. (2022). Pyramid Convolutional RNN for MRI Image Reconstruction. in IEEE Transactions on Medical Imaging. 41(8), 2033-2047. https://doi.org/10.1109/TMI.2022.3153849
[18] Nako, J., Psychalinos, C., & Elwakil, A. S. (2023). Minimum Active Component Count Design of a PIλDμ Controller and Its Application in a Cardiac Pacemaker System. J. Low Power Electron. Appl. https://doi.org/10.3390/jlpea13010013 (7)
[19] Yürdem, B., Aksu, M. F., & Sağbaş, M. (2024). Design of Fractional/Integer Order PID Controller Using Single DVCC and Its Cardiac Pacemaker Application. Circuits Syst Signal Process. https://doi.org/10.1007/s00034-024-02810-2 (10)
[20] Beg, A. H., & Islam, M. Z. (2016, June). Advantages and limitations of genetic algorithms for clustering records. In 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA) (pp. 2478-2483). IEEE.
[21] Chen, X., Qu, G., Tang, Y., Low, S., & Li, N. (2022). Reinforcement learning for selective key applications in power systems: Recent advances and future challenges. IEEE Transactions on Smart Grid, 13(4), 2935-2958.
[22] Zhang, T., & Mo, H. (2021). Reinforcement learning for robot research: A comprehensive review and open issues. International Journal of Advanced Robotic Systems, 18(3), 17298814211007305.
[23] Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26-38.
[24] Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6), 750-797.
[25] Köse, B., Işıklı, İ., Sagbas, M. (2024). Estimation of Weibull Probability Distribution Parameters with Optimization Algorithms and Foça Wind Data Application. Gazi University Journal of Science, 37(3), 1236-1254. https://doi.org/10.35378/gujs.1311992
[26] Armoogum S., Li, X., (2019). Big Data Analytics and Deep Learning in Bioinformatics with Hadoop. Deep Learning and Parallel Computing Environment for Bioengineering Systems, Elsevier. pp. 17–36. https://doi.org/10.1016/B978-0-12-816718-2.00009-9

Details

Primary Language

English

Subjects

Reinforcement Learning

Journal Section

Research Article

Authors

Kağan Orbay
0009-0002-2463-8618
Türkiye

Mehmet Sagbas ^*
0000-0001-5776-3947
Türkiye

Murat Demir
0000-0003-4407-9673
Türkiye

Publication Date

May 1, 2025

Submission Date

December 23, 2024

Acceptance Date

April 9, 2025

Published in Issue

Year 2025 Volume: 5 Number: 1

IZ

https://izlik.org/JA46HU55HW

Cite

RIS / Bibtex

APA

Orbay, K., Sagbas, M., & Demir, M. (2025). Design of Cardiac Pacemaker Controller Based on Reinforcement Learning. Artificial Intelligence Theory and Applications, 5(1), 29-41. https://izlik.org/JA46HU55HW

AMA

1.Orbay K, Sagbas M, Demir M. Design of Cardiac Pacemaker Controller Based on Reinforcement Learning. AITA. 2025;5(1):29-41. https://izlik.org/JA46HU55HW

Chicago

Orbay, Kağan, Mehmet Sagbas, and Murat Demir. 2025. “Design of Cardiac Pacemaker Controller Based on Reinforcement Learning”. Artificial Intelligence Theory and Applications 5 (1): 29-41. https://izlik.org/JA46HU55HW.

EndNote

Orbay K, Sagbas M, Demir M (May 1, 2025) Design of Cardiac Pacemaker Controller Based on Reinforcement Learning. Artificial Intelligence Theory and Applications 5 1 29–41.

IEEE

[1]K. Orbay, M. Sagbas, and M. Demir, “Design of Cardiac Pacemaker Controller Based on Reinforcement Learning”, AITA, vol. 5, no. 1, pp. 29–41, May 2025, [Online]. Available: https://izlik.org/JA46HU55HW

ISNAD

Orbay, Kağan - Sagbas, Mehmet - Demir, Murat. “Design of Cardiac Pacemaker Controller Based on Reinforcement Learning”. Artificial Intelligence Theory and Applications 5/1 (May 1, 2025): 29-41. https://izlik.org/JA46HU55HW.

JAMA

1.Orbay K, Sagbas M, Demir M. Design of Cardiac Pacemaker Controller Based on Reinforcement Learning. AITA. 2025;5:29–41.

MLA

Orbay, Kağan, et al. “Design of Cardiac Pacemaker Controller Based on Reinforcement Learning”. Artificial Intelligence Theory and Applications, vol. 5, no. 1, May 2025, pp. 29-41, https://izlik.org/JA46HU55HW.

Vancouver

1.Kağan Orbay, Mehmet Sagbas, Murat Demir. Design of Cardiac Pacemaker Controller Based on Reinforcement Learning. AITA [Internet]. 2025 May 1;5(1):29-41. Available from: https://izlik.org/JA46HU55HW