Reinforcement learning for energy optimization in IoT based landslide early warning systems

Year 2024, Issue: 059, 32 - 57, 31.12.2024

Seyfullah Arslan , Safa Dörterler , Fırat Aydemir

Abstract

This study introduces a novel energy management model based on Deep Reinforcement Learning for IoT-based landslide early warning systems, aiming to achieve energy neutrality and enhance system resilience, efficiency, and sustainability. Unlike traditional energy optimization methods, the proposed model employs a Deep Q-Network (DQN) to dynamically optimize the duty cycle of sensor nodes by leveraging real-time energy availability. By adaptively balancing energy harvesting and consumption, sensor nodes can maintain continuous operation even under highly variable environmental conditions, maximizing their performance during high-energy periods while preserving battery life when energy is limited. Extensive simulations using real-world solar radiation data demonstrate the model's superior capability in extending system longevity and operational stability compared to existing approaches. Addressing critical energy management challenges in landslide monitoring systems, this work enhances system reliability, scalability, and adaptability, offering a robust foundation for broader IoT applications deployed in energy-limited and dynamic environments. The proposed method represents a significant improvement over conventional techniques, as it autonomously optimizes energy resources to ensure the continuous and sustainable operation of IoT ecosystems

Keywords

Reinforcement learning, internet of things, energy management optimization, landslide early warning systems

References

• [1] R. Steen, E. Roud, T. M. Torp, and T.-A. Hansen, “The impact of interorganizational collaboration on the viability of disaster response operations: The Gjerdrum landslide in Norway,” Saf Sci, vol. 173, p. 106459, 2024.
• [2] D. Petley, “Global patterns of loss of life from landslides,” Geology, vol. 40, no. 10, pp. 927–930, 2012.
• [3] M. T. Chaudhary and A. Piracha, “Natural disasters—origins, impacts, management,” Encyclopedia, vol. 1, no. 4, pp. 1101–1131, 2021.
• [4] F. Guzzetti et al., “Geographical landslide early warning systems,” Earth Sci Rev, vol. 200, p. 102973, 2020.
• [5] M. A. Jamshed, K. Ali, Q. H. Abbasi, M. A. Imran, and M. Ur-Rehman, “Challenges, applications, and future of wireless sensors in Internet of Things: A review,” IEEE Sens J, vol. 22, no. 6, pp. 5482–5494, 2022.
• [6] N. Casagli, E. Intrieri, V. Tofani, G. Gigli, and F. Raspini, “Landslide detection, monitoring and prediction with remote-sensing techniques,” Nat Rev Earth Environ, vol. 4, no. 1, pp. 51–64, 2023.
• [7] M. Esposito, L. Palma, A. Belli, L. Sabbatini, and P. Pierleoni, “Recent advances in internet of things solutions for early warning systems: A review,” Sensors, vol. 22, no. 6, p. 2124, 2022.
• [8] J. Singh, R. Kaur, and D. Singh, “Energy harvesting in wireless sensor networks: A taxonomic survey,” Int J Energy Res, vol. 45, no. 1, pp. 118–140, 2021.
• [9] A. J. Williams, M. F. Torquato, I. M. Cameron, A. A. Fahmy, and J. Sienz, “Survey of energy harvesting technologies for wireless sensor networks,” IEEE Access, vol. 9, pp. 77493–77510, 2021.
• [10] T. Sanislav, G. D. Mois, S. Zeadally, and S. C. Folea, “Energy harvesting techniques for internet of things (IoT),” IEEE access, vol. 9, pp. 39530–39549, 2021.
• [11] S. Shresthamali, M. Kondo, and H. Nakamura, “Multi-objective reinforcement learning for energy harvesting wireless sensor nodes,” in 2021 IEEE 14th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), IEEE, 2021, pp. 98–105.
• [12] A. Karras, C. Karras, I. Karydis, M. Avlonitis, and S. Sioutas, “An Adaptive, Energy-Efficient DRL-Based and MCMC-Based Caching Strategy for IoT Systems,” in International Symposium on Algorithmic Aspects of Cloud Computing, Springer, 2023, pp. 66–85.
• [13] E. O. Arwa and K. A. Folly, “Reinforcement learning techniques for optimal power control in grid-connected microgrids: A comprehensive review,” Ieee Access, vol. 8, pp. 208992–209007, 2020.
• [14] L. M. Highland and P. Bobrowsky, The landslide handbook-A guide to understanding landslides, no. 1325. US Geological Survey, 2008.
• [15] A. Tekerek and M. Dörterler, “The adaptation of gray wolf optimizer to data clustering,” Politeknik Dergisi, p. 1, 2020.
• [16] S. Dörterler, “Kanser Hastalığı Teşhisinde Ölüm Oyunu Optimizasyon Algoritmasının Etkisi,” Mühendislik Alanında Uluslararası Araştırmalar VIII, p. 15, 2023.
• [17] İ. Akgül and V. Kaya, “Learning performance of optimization algorithms in convolutional neural networks: An application,” in INSAC Scientific Researches in Natural and Engineering Sciences, 1st ed., vol. 1, Duvar, 2022, ch. 11, pp. 215–236.
• [18] N. Yağmur, H. Temurtaş, and İ. Dağ, “Anemi hastalığının yapay sinir ağları yöntemleri kullanılarak sınıflandırılması,” Journal of Scientific Reports-B, no. 008, pp. 20–34, 2023.
• [19] N. N. Arslan, E. Şahin, and M. Akçay, “Deep learning-based isolated sign language recognition: a novel approach to tackling communication barriers for individuals with hearing impairments,” Journal of Scientific Reports-A, no. 055, pp. 50–59, 2023.
• [20] V. Kaya, I. Akgül, and Ö. Tanır, “A novel hybrid model based on machine and deep learning techniques for the classification of microalgae.,” Phyton-International Journal of Experimental Botany, vol. 92, no. 9, pp. 2519-2534, 2023.
• [21] G. Arslan, F. Aydemir, and S. Arslan, “Enhanced license plate recognition using deep learning and block-based approach,” Journal of Scientific Reports-A, no. 058, pp. 57–82, 2023.
• [22] F. Aydemir and S. Arslan, “A System Design With Deep Learning and IoT to Ensure Education Continuity for Post-COVID,” IEEE Transactions on Consumer Electronics, 2023.
• [23] S. Dörterler, H. Dumlu, D. Özdemir, and H. Temurtaş, “Hybridization of Meta-heuristic Algorithms with K-Means for Clustering Analysis: Case of Medical Datasets,” Gazi Mühendislik Bilimleri Dergisi, pp. 1–23.
• [24] E. Şahin, D. Özdemir, and H. Temurtaş, “Multi-objective optimization of ViT architecture for efficient brain tumor classification,” Biomed Signal Process Control, vol. 91, p. 105938, 2024.
• [25] F. Aydemir and S. Arslan, “Covid-19 pandemi sürecinde çocukların el yıkama alışkanlığının nesnelerin interneti tabanlı sistem ile izlenmesi,” Mühendislik Bilimleri ve Araştırmaları Dergisi, vol. 3, no. 2, pp. 161–168, 2021.
• [26] M. C. Özbalcı, H. Şahin, and T. T. Bilgin, “Makine Öğrenmesi Yöntemleri ile GTZAN Veri Kümesine Ait Müzik Türlerinin Sınıflandırılması,” Mühendislik Bilimleri ve Araştırmaları Dergisi, vol. 6, no. 1, pp. 77–87.
• [27] E. Şahin and M. F. Talu, “WY-NET: A NEW APPROACH to IMAGE SYNTHESIS with GENERATIVE ADVERSARIAL NETWORKS,” Journal of Scientific Reports-A, no. 050, pp. 270–290, 2022.
• [28] E. Şahin and M. F. Talu, “Automatic Mustache Pattern Production on Denim Fabric with Generative Adversarial Networks,” Computer Science, vol. 7, no. 1, pp. 1–9, 2022.
• [29] N. Yagmur, İ. Dag, and H. Temurtas, “Classification of anemia using Harris hawks optimization method and multivariate adaptive regression spline,” Neural Comput Appl, pp. 1–20, 2024.
• [30] N. Yagmur, I. Dag, and H. Temurtas, “A new computer‐aided diagnostic method for classifying anaemia disease: Hybrid use of Tree Bagger and metaheuristics,” Expert Syst, p. e13528, 2023.
• [31] İ. Akgül, V. Kaya, E. Karavaş, S. Aydın, and A. Baran, “A Novel Artificial Intelligence-Based Hybrid System to Improve Breast Cancer DetectionUsing DCE-MRI,” Bulletin Of The Polish Academy Of Sciences. Technical Sciences, vol. 72, no. 3, 2024.
• [32] H. Ali, U. U. Tariq, M. Hussain, L. Lu, J. Panneerselvam, and X. Zhai, “ARSH-FATI: A novel metaheuristic for cluster head selection in wireless sensor networks,” IEEE Syst J, vol. 15, no. 2, pp. 2386–2397, 2020.
• [33] H. Ali et al., “A survey on system level energy optimisation for MPSoCs in IoT and consumer electronics,” Comput Sci Rev, vol. 41, p. 100416, 2021.
• [34] H. Thirugnanam, M. V. Ramesh, and V. P. Rangan, “Enhancing the reliability of landslide early warning systems by machine learning,” Landslides, vol. 17, pp. 2231–2246, 2020.
• [35] T. Hemalatha, M. V. Ramesh, and V. P. Rangan, “Effective and accelerated forewarning of landslides using wireless sensor networks and machine learning,” IEEE Sens J, vol. 19, no. 21, pp. 9964–9975, 2019.
• [36] E. Collini, L. A. I. Palesi, P. Nesi, G. Pantaleo, N. Nocentini, and A. Rosi, “Predicting and understanding landslide events with explainable AI,” IEEE Access, vol. 10, pp. 31175–31189, 2022.
• [37] C. W. W. Ng, B. Yang, Z. Q. Liu, J. S. H. Kwan, and L. Chen, “Spatiotemporal modelling of rainfall-induced landslides using machine learning,” Landslides, vol. 18, pp. 2499–2514, 2021.
• [38] R. Franceschini, A. Rosi, F. Catani, and N. Casagli, “Exploring a landslide inventory created by automated web data mining: the case of Italy,” Landslides, vol. 19, no. 4, pp. 841–853, 2022.
• [39] X. Qian, “Regional Geological Disasters Emergency Management System Monitored by Big Data Platform,” Processes, vol. 10, no. 12, p. 2741, 2022.
• [40] X. Chen, “Early Warning of Regional Landslide Disaster and Development of Rural Ecological Industrialization Based on IoT Sensor,” Sci Program, vol. 2022, 2022.
• [41] C. V. L. Pennington et al., “A near-real-time global landslide incident reporting tool demonstrator using social media and artificial intelligence,” International Journal of Disaster Risk Reduction, vol. 77, p. 103089, 2022.
• [42] G. Herrera et al., “Landslide databases in the Geological Surveys of Europe,” Landslides, vol. 15, pp. 359–379, 2018.
• [43] K. Sassa and P. Canuti, Landslides-disaster risk reduction. Springer Science & Business Media, 2008.
• [44] B. Rosser, S. Dellow, S. Haubrock, and P. Glassey, “New Zealand’s national landslide database,” Landslides, vol. 14, pp. 1949–1959, 2017.
• [45] C. S. Juang, T. A. Stanley, and D. B. Kirschbaum, “Using citizen science to expand the global map of landslides: Introducing the Cooperative Open Online Landslide Repository (COOLR),” PLoS One, vol. 14, no. 7, p. e0218657, 2019.
• [46] M. J. Froude and D. N. Petley, “Global fatal landslide occurrence from 2004 to 2016,” Natural Hazards and Earth System Sciences, vol. 18, no. 8, pp. 2161–2181, 2018.
• [47] Q. Han, K. Sassa, and M. Mikoš, “International programme on landslides (ipl): a programme of the icl for landslide disaster risk reduction,” Understanding and Reducing Landslide Disaster Risk: Volume 1 Sendai Landslide Partnerships and Kyoto Landslide Commitment 5th, pp. 187–203, 2021.
• [48] A. C. Teodoro and L. Duarte, “The synergy of remote sensing and geographical information systems in the management of natural disasters,” in Nanotechnology-Based Smart Remote Sensing Networks for Disaster Prevention, Elsevier, 2022, pp. 217–230.
• [49] Z. Yang, J. Li, J. Hyyppä, J. Gong, J. Liu, and B. Yang, “A comprehensive and up-to-date web-based interactive 3D emergency response and visualization system using Cesium Digital Earth: taking landslide disaster as an example,” Big Earth Data, pp. 1–23, 2023.
• [50] M. N. I. Sarker, Y. Peng, C. Yiran, and R. C. Shouse, “Disaster resilience through big data: Way to environmental sustainability,” International Journal of Disaster Risk Reduction, vol. 51, p. 101769, 2020.
• [51] K. Goniewicz et al., “Geographic information system technology: review of the challenges for its establishment as a major asset for disaster and emergency management in Poland,” Disaster Med Public Health Prep, vol. 15, no. 5, pp. 573–578, 2021.
• [52] C. Nefros, G. Kitsara, and C. Loupasakis, “Geographical Information Systems and Remote Sensing Techniques to Reduce the Impact of Natural Disasters in Smart Cities,” IFAC-PapersOnLine, vol. 55, no. 11, pp. 72–77, 2022.
• [53] P. V Gorsevski, M. K. Brown, K. Panter, C. M. Onasch, A. Simic, and J. Snyder, “Landslide detection and susceptibility mapping using LiDAR and an artificial neural network approach: a case study in the Cuyahoga Valley National Park, Ohio,” Landslides, vol. 13, pp. 467–484, 2016.
• [54] N. M. Abdullah, N. Sulaiman, U. Nazir, M. Ismail, S. K. K. A. Latib, and N. P. N. Mahmud, “Geographical Information System (GIS) and Remote Sensing (RS) Applications in Disaster Risk Reduction (DRR) in Malaysia,” International Journal of Integrated Engineering, vol. 14, no. 5, pp. 25–37, 2022.
• [55] I. Alcántara-Ayala and R. J. Garnica-Peña, “Landslide Warning Systems in Upper Middle-Income Countries: Current Insights and New Perspectives,” in Progress in Landslide Research and Technology, Volume 1 Issue 2, 2022, Springer, 2023, pp. 159–168.
• [56] R. Can, S. Kocaman, and C. Gokceoglu, “A convolutional neural network architecture for auto-detection of landslide photographs to assess citizen science and volunteered geographic information data quality,” ISPRS Int J Geoinf, vol. 8, no. 7, p. 300, 2019.
• [57] G. T. Harilal, D. Madhu, M. V. Ramesh, and D. Pullarkatt, “Towards establishing rainfall thresholds for a real-time landslide early warning system in Sikkim, India,” Landslides, vol. 16, no. 12, pp. 2395–2408, 2019.
• [58] X. Fan et al., “Prediction of a multi-hazard chain by an integrated numerical simulation approach: the Baige landslide, Jinsha River, China,” Landslides, vol. 17, pp. 147–164, 2020.
• [59] J.-Y. Park, S.-R. Lee, D.-H. Lee, Y.-T. Kim, and J.-S. Lee, “A regional-scale landslide early warning methodology applying statistical and physically based approaches in sequence,” Eng Geol, vol. 260, p. 105193, 2019.
• [60] T. Salvatici et al., “Application of a physically based model to forecast shallow landslides at a regional scale,” Natural Hazards and Earth System Sciences, vol. 18, no. 7, pp. 1919–1935, 2018.
• [61] V.-H. Nhu et al., “Landslide susceptibility mapping using machine learning algorithms and remote sensing data in a tropical environment,” Int J Environ Res Public Health, vol. 17, no. 14, p. 4933, 2020.
• [62] B. Hariharan and R. Guntha, “Crowdsourced landslide tracking-lessons from field experiences of landslide tracker mobile app,” in EGU General Assembly Conference Abstracts, 2021, pp. EGU21-12711.
• [63] R. Guntha, S. N. Rao, and M. V Ramesh, “Deployment experiences with Amrita Kripa: a user-friendly feature rich crowdsourced humanitarian application,” Procedia Comput Sci, vol. 171, pp. 302–311, 2020.
• [64] C. He, N. Ju, Q. Xu, and J. Huang, “Automated data processing and integration of large multiple data sources in geohazards monitoring,” International Journal of Georesources and Environment-IJGE (formerly Int’l J of Geohazards and Environment), vol. 3, no. 1–2, pp. 9–21, 2017.
• [65] T. F. Fathani, D. Karnawati, and W. Wilopo, “An adaptive and sustained landslide monitoring and early warning system,” in Landslide Science for a Safer Geoenvironment: Volume 2: Methods of Landslide Studies, Springer, 2014, pp. 563–567.
• [66] T. Hemalatha, M. V. Ramesh, and V. P. Rangan, “Adaptive learning techniques for landslide forecasting and the validation in a real world deployment,” in Advancing Culture of Living with Landslides: Volume 5 Landslides in Different Environments, Springer, 2017, pp. 439–447.
• [67] L. Zhang and P. Lin, “Reinforcement learning based energy-neutral operation for hybrid EH powered TBAN,” Future Generation Computer Systems, vol. 140, pp. 311–320, 2023.
• [68] A. Murad, F. A. Kraemer, K. Bach, and G. Taylor, “Autonomous management of energy-harvesting iot nodes using deep reinforcement learning,” in 2019 IEEE 13th International Conference on Self-Adaptive and Self-Organizing Systems (SASO), IEEE, 2019, pp. 43–51.
• [69] A. Omidkar, A. Khalili, H. H. Nguyen, and H. Shafiei, “Reinforcement-learning-based resource allocation for energy-harvesting-aided D2D communications in IoT networks,” IEEE Internet Things J, vol. 9, no. 17, pp. 16521–16531, 2022.
• [70] M. Chu, X. Liao, H. Li, and S. Cui, “Power control in energy harvesting multiple access system with reinforcement learning,” IEEE Internet Things J, vol. 6, no. 5, pp. 9175–9186, 2019.
• [71] F. A. Aoudia, M. Gautier, and O. Berder, “RLMan: An energy manager based on reinforcement learning for energy harvesting wireless sensor networks,” IEEE Transactions on Green Communications and Networking, vol. 2, no. 2, pp. 408–417, 2018.
• [72] Y. Li, “Deep reinforcement learning: An overview,” arXiv preprint arXiv:1701.07274, 2017.
• [73] Y. Fei, Z. Yang, Y. Chen, and Z. Wang, “Exponential bellman equation and improved regret bounds for risk-sensitive reinforcement learning,” Adv Neural Inf Process Syst, vol. 34, pp. 20436–20446, 2021.
• [74] J. Clifton and E. Laber, “Q-learning: Theory and applications,” Annu Rev Stat Appl, vol. 7, no. 1, pp. 279–301, 2020.
• [75] F. Liu, L. Viano, and V. Cevher, “Understanding deep neural function approximation in reinforcement learning via $\epsilon $-greedy exploration,” Adv Neural Inf Process Syst, vol. 35, pp. 5093–5108, 2022.
• [76] T. T. Nguyen, N. D. Nguyen, and S. Nahavandi, “Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,” IEEE Trans Cybern, vol. 50, no. 9, pp. 3826–3839, 2020.
• [77] European Commission, “European Commission Photovoltaic Geographic Information System.” Accessed: Nov. 09, 2024. [Online]. Available: https://re.jrc.ec.europa.eu/pvg_tools/en/
• [78] V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
• [79] T. P. Lillicrap, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
• [80] F. Ait Aoudia, M. Gautier, and O. Berder, “RLMan: An Energy Manager Based on Reinforcement Learning for Energy Harvesting Wireless Sensor Networks,” IEEE Transactions on Green Communications and Networking, vol. 2, no. 2, pp. 408–417, Jun. 2018, doi: 10.1109/TGCN.2018.2801725.
• [81] A. Murad, F. A. Kraemer, K. Bach, and G. Taylor, “Autonomous management of energy-harvesting iot nodes using deep reinforcement learning,” in 2019 IEEE 13th International Conference on Self-Adaptive and Self-Organizing Systems (SASO), IEEE, 2019, pp. 43–51.
• [82] N. Charef, M. Abdelhafidh, A. Ben Mnaouer, K. Andersson, and S. Cherkaoui, “RL-Based Adaptive Duty Cycle Scheduling in WSN-Based IoT Nets,” in GLOBECOM 2023-2023 IEEE Global Communications Conference, IEEE, 2023, pp. 3777–3782.
• [83] A. F. E. Abadi, S. A. Asghari, M. B. Marvasti, G. Abaei, M. Nabavi, and Y. Savaria, “RLBEEP: Reinforcement-Learning-Based Energy Efficient Control and Routing Protocol for Wireless Sensor Networks,” IEEE Access, vol. 10, pp. 44123–44135, 2022, doi: 10.1109/ACCESS.2022.3167058.