Design of Cardiac Pacemaker Controller Based on Reinforcement Learning

Year 2025, Volume: 5 Issue: 1, 29 - 41, 01.05.2025

Kağan Orbay , Mehmet Sagbas , Murat Demir

Abstract

This study investigates the derivation of PID controller parameters, commonly used for pacemaker control, using both genetic algorithm (GA) and reinforcement learning (RL) methods. We compare the PID parameters obtained by RL with those obtained by GA, a well-known and often preferred method in the literature. The aim of the study is to analyze the performance of the control parameters obtained by both methods and to determine which approach is more effective in pacemaker applications. In particular, comparisons on important control criteria such as rise time, settling time and overshoot of the system will reveal the advantages and disadvantages of these methods.

Keywords

heart rhythm regulation, pacemaker control system, PID controller optimization, reinforcement learning

Project Number

BBAP.2024.011

References

• [1] Ponikowski, P., Anker, S. D., AlHabib, K. F., Cowie, M. R., Force, T. L., Hu, S., et al. (2014). Heart failure: preventing disease and death worldwide. ESC Heart Failure, 1(1), 4–25.
• [2] Bajpai, S., Alam, S., Ali, M.A. (2017). Intelligent Heart Rate Controller using Fractional Order PID Controller Tuned by Genetic Algorithm for Pacemaker. International Journal of Engineering Research & Technology. 6(05), 715-720.
• [3] Arunachalam, S. P., Kapa, S., Mulpuru, S.K., Friedman P.A., Tolkacheva, E.G. (2017). Intelligent Fractional-Order PID (FOPID) Heart Rate Controller for Cardiac Pacemaker, 2016 IEEE Healthcare Innovation Point-Of-Care Technologies Conference (HI-POCT), Cancun, Meksika, 2016, s. 105-108
• [4] Bikki, P., Dhiraj, Y., & Kumar, R. N. (2023). Implementation of a Dual-Chamber Pacemaker for Low-Power Applications. https://doi.org/10.1109/icecct56650.2023.10179677
• [5] Sutton R.S. and Barto, A.G. Reinforcement Learning: An Introduction. The MIT Press, London, 2018.
• [6] Holland, J.H., Adaptation in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press, 1975.
• [7] Biswas, S.C., Das, A., Guha, P. (2006). Mathematical Model of Cardiovascular System by Transfer function Method, Calcutta Medical Journal, 4, 15-17.
• [8] Noble, D., A Modification of the Hodgkin-Huxley Equations Applicable to Purkinje Fibre Action and Pace-maker Potentials, 1962 Journal of Physiology , 160, 317-352. PubMed ID: 14480151
• [9] Beeler, G.W. and Reuter, H. (1977) Reconstruction of the action potential of ventricular myocardial fibres. The Journal of physiology, 268, 177-210.
• [10] Ziegler, J. G., & Nichols, N. B. (1942). Optimum Settings for Automatic Controllers. Trans. ASME, 64(11), 759–768.
• [11] Cohen, G. H., & Coon, G. A. (1953). Theoretical Consideration of Retarded Control. Trans. ASME, 75, 827–834.
• [12] Momani, S., Batiha I.M., El-Khazali, R. (2019). Design of PIλDδ-Heart Rate Controllers for Cardiac Pacemaker, 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Ajman, United Arab Emirates, pp. 1-5. https://doi.org/10.1109/ISSPIT47144.2019.9001785
• [13] Govind K.R.A., Sekhar, R.A.: Design of a novel PID controller for cardiac pacemaker. 2014 International Conference on Advances in Green Energy (ICAGE), Thiruvananthapuram, India, pp. 82-87 (2014). https://doi.org/10.1109/ICAGE.2014.7050147
• [14] Alfarhan, K.A., Mashor, M.Y., Mohd Saad, A.R., Omar, M.I. (2018). Wireless heart abnormality monitoring kit based on raspberry pi. Journal of Biomimetics, Biomaterials and Biomedical Engineering. 35, 96–108. https://doi.org/10.4028/www.scientific.net/JBBBE.35.96
• [15] Srivastava, R., Kumar, B. (2022). Design of Anfis based pacemaker controller having improved transient response and its FPGA implementation. Biomedical Signal Processing and Control. https://doi.org/10.1016/j.bspc.2021.103186
• [16] Lima, G.S., Savi, M.A., Bessa, W.M. (2023). Intelligent control of cardiac rhythms using artificial neural networks. Nonlinear Dynamics. 111(12), 11543–11557. https://doi.org/10.1007/s11071-023-08447-1
• [17] Chen, E.Z., Wang, P., Chen, X., Chen, T., Sun, S. (2022). Pyramid Convolutional RNN for MRI Image Reconstruction. in IEEE Transactions on Medical Imaging. 41(8), 2033-2047. https://doi.org/10.1109/TMI.2022.3153849
• [18] Nako, J., Psychalinos, C., & Elwakil, A. S. (2023). Minimum Active Component Count Design of a PIλDμ Controller and Its Application in a Cardiac Pacemaker System. J. Low Power Electron. Appl. https://doi.org/10.3390/jlpea13010013 (7)
• [19] Yürdem, B., Aksu, M. F., & Sağbaş, M. (2024). Design of Fractional/Integer Order PID Controller Using Single DVCC and Its Cardiac Pacemaker Application. Circuits Syst Signal Process. https://doi.org/10.1007/s00034-024-02810-2 (10)
• [20] Beg, A. H., & Islam, M. Z. (2016, June). Advantages and limitations of genetic algorithms for clustering records. In 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA) (pp. 2478-2483). IEEE.
• [21] Chen, X., Qu, G., Tang, Y., Low, S., & Li, N. (2022). Reinforcement learning for selective key applications in power systems: Recent advances and future challenges. IEEE Transactions on Smart Grid, 13(4), 2935-2958.
• [22] Zhang, T., & Mo, H. (2021). Reinforcement learning for robot research: A comprehensive review and open issues. International Journal of Advanced Robotic Systems, 18(3), 17298814211007305.
• [23] Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26-38.
• [24] Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6), 750-797.
• [25] Köse, B., Işıklı, İ., Sagbas, M. (2024). Estimation of Weibull Probability Distribution Parameters with Optimization Algorithms and Foça Wind Data Application. Gazi University Journal of Science, 37(3), 1236-1254. https://doi.org/10.35378/gujs.1311992
• [26] Armoogum S., Li, X., (2019). Big Data Analytics and Deep Learning in Bioinformatics with Hadoop. Deep Learning and Parallel Computing Environment for Bioengineering Systems, Elsevier. pp. 17–36. https://doi.org/10.1016/B978-0-12-816718-2.00009-9