ENHANCING GREEN COMPUTING THROUGH ENERGY-AWARE TRAINING: AN EARLY STOPPING PERSPECTIVE

Year 2024, Volume: 2 Issue: 2, 108 - 139, 17.01.2025

Abdulkadir Taşdelen

https://doi.org/10.71074/CTC.1594291

Abstract

This study delves into energy-efficient training strategies, emphasizing their alignment with green computing principles. In particular, it highlights the utility of early stopping mechanisms in optimizing the training process of deep learning models. Early stopping works by monitoring performance metrics, such as validation accuracy or loss, and halting the training process once these metrics stabilize or show no improvement over a predefined number of epochs. This approach eliminates redundant computations, leading to significant reductions in energy consumption and computational costs while preserving model accuracy. The research is centered on transfer learning models, specifically MobileNetV2, InceptionV3, ResNet50V2, and Xception, which are well-regarded for their versatility and performance in image classification tasks. By systematically varying patient values (3, 5, 7, 10, and 15), the study explores their impact on training duration, model accuracy, and computational efficiency. Each patience value determines how many epochs the training continues without improvement before stopping, allowing for a nuanced examination of its effects across different architectures. The findings reveal that early stopping not only streamlines the training process but also aligns well with the broader goals of sustainable artificial intelligence development. By effectively balancing computational efficiency with performance optimization, this strategy exemplifies how environmentally responsible practices can be integrated into AI workflows. This study contributes valuable insights into how adopting such techniques can mitigate the environmental impact of AI model training, highlighting their importance in the context of advancing green computing initiatives.

Keywords

Artificial Intelligence, Green Computing, Green AI, Early Stop Strategy

References

• K. I. Ibekwe, A. A. Umoh, Z. Q. S. Nwokediegwu, E. A. Etukudoh, V. I. Ilojianya, A. Adefemi, Energy efficiency in industrial sectors: A review of technologies and policy measures, Engineering Science & Technology Journal 5 (1) (2024) 169–184. doi:10.51594/estj.v5i1.742.
• A. Tasdelen, M. H. Habaebi, M. R. Islam, Exploring blockchain technologies: Insights into consensus mechanisms, mining pool dynamics, and energy consumption patterns, in: 2024 9th International Conference on Mechatronics Engi- neering (ICOM), IEEE, 2024, p. 95–100. doi:10.1109/icom61675.2024.10652588.
• V. Bolo´n-Canedo, L. Mora´n-Ferna´ndez, B. Cancela, A. Alonso-Betanzos, A review of green artificial intelligence: To- wards a more sustainable future, Neurocomputing 599 (2024) 128096. doi:10.1016/j.neucom.2024.128096.
• Z. Vale, L. Gomes, D. Ramos, P. Faria, Green computing: a realistic evaluation of energy consumption for building load forecasting computation, Journal of Smart Environments and Green Computing 2 (2) (2022) 34–45. doi:10.20517/ jsegc.2022.06.
• M. Z. Alom, T. M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, B. C. V. Esesn, A. A. S. Awwal, V. K. Asari, The history began from alexnet: A comprehensive survey on deep learning approaches (2018). arXiv:1803.01164. URL https://arxiv.org/abs/1803.01164
• Y. Zhou, X. Lin, X. Zhang, M. Wang, G. Jiang, H. Lu, Y. Wu, K. Zhang, Z. Yang, K. Wang, Y. Sui, F. Jia, Z. Tang, Y. Zhao, H. Zhang, T. Yang, W. Chen, Y. Mao, Y. Li, D. Bao, Y. Li, H. Liao, T. Liu, J. Liu, J. Guo, X. Zhao, Y. WEI, H. Qian, Q. Liu, X. Wang, W. Kin, Chan, C. Li, Y. Li, S. Yang, J. Yan, C. Mou, S. Han, W. Jin, G. Zhang, X. Zeng, On the opportunities of green computing: A survey (2023). arXiv:2311.00447. URL https://arxiv.org/abs/2311.00447
• D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, J. Dean, Carbon emissions and large neural network training (2021). arXiv:2104.10350. URL https://arxiv.org/abs/2104.10350
• X. Chen, Optimization strategies for reducing energy consumption in ai model training, ACS 6 (1) (Mar. 2023).
• L. Lannelongue, J. Grealey, M. Inouye, Green algorithms: Quantifying the carbon footprint of computation, Advanced Science 8 (12) (May 2021). doi:10.1002/advs.202100707.
• S. Georgiou, M. Kechagia, T. Sharma, F. Sarro, Y. Zou, Green ai: do deep learning frameworks have different costs?, in: Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, ACM, 2022, p. 1082–1094. doi:10.1145/3510003.3510221.
• B. M. Hussein, S. M. Shareef, An empirical study on the correlation between early stopping patience and epochs in deep learning, ITM Web of Conferences 64 (2024) 01003. doi:10.1051/itmconf/20246401003.
• Y. Xu, S. Martinez-Fernandez, M. Martinez, X. Franch, Energy efficiency of training neural network architectures: An empirical study (Feb. 2023). arXiv:2302.00967.
• H. Järvenpää, P. Lago, J. Bogner, G. Lewis, H. Muccini, I. Ozkaya, A synthesis of green architectural tactics for ml- enabled systems, in: Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Society, ICSE-SEIS’24, ACM, 2024, p. 130–141. doi:10.1145/3639475.3640111.
• L. Maroney, Rock paper scissors classification dataset, https://laurencemoroney.com/datasets.html, accessed: 2024-10-17.
• W. H. Kruskal, W. A. Wallis, Use of ranks in one-criterion variance analysis, Journal of the American Statistical Asso- ciation 47 (260) (1952) 583–621. doi:10.1080/01621459.1952.10483441.
• H. B. Mann, D. R. Whitney, On a test of whether one of two random variables is stochastically larger than the other, The Annals of Mathematical Statistics 18 (1) (1947) 50–60.
• W. Conover, R. Iman, Multiple-comparisons procedures. Informal report, 1979. doi:10.2172/6057803.
• S. S. Acmali, Y. Ortakci, H. Seker, Green ai-driven concept for the development of cost-effective and energy-efficient deep learning method: Application in the detection ofeimeriaparasites as a case study, Advanced Intelligent Systems 6 (7) (Jun. 2024). doi:10.1002/aisy.202300644.
• D. Reguero, S. Martinez-Fernandez, R. Verdecchia, Energy-efficient neural network training through runtime layer freezing, model quantization, and early stopping, Computer Standards & Interfaces 92 (2025) 103906. doi:10.1016/ j.csi.2024.103906.
• Y. Matsubara, M. Levorato, F. Restuccia, Split computing and early exiting for deep learning applications: Survey and research challenges, ACM Computing Surveys 55 (5) (2022) 1–30. doi:10.1145/3527155.
• S. Teerapittayanon, B. McDanel, H. Kung, Branchynet: Fast inference via early exiting from deep neural networks, in: 2016 23rd International Conference on Pattern Recognition (ICPR), IEEE, 2016, p. 2464–2469. doi:10.1109/icpr. 2016.7900006.
• S. Teerapittayanon, B. McDanel, H. Kung, Distributed deep neural networks over the cloud, the edge and end devices, in: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), IEEE, 2017, p. 328–339. doi:10.1109/icdcs.2017.226.
• Y. Wang, J. Shen, T.-K. Hu, P. Xu, T. Nguyen, R. Baraniuk, Z. Wang, Y. Lin, Dual dynamic inference: Enabling more efficient, adaptive, and controllable deep inference, IEEE Journal of Selected Topics in Signal Processing 14 (4) (2020) 623–633. doi:10.1109/jstsp.2020.2979669.
• H. Li, H. Zhang, X. Qi, R. Yang, G. Huang, Improved techniques for training adaptive deep networks (2019). URL https://arxiv.org/abs/1908.06294
• S. Laskaridis, S. I. Venieris, M. Almeida, I. Leontiadis, N. D. Lane, Spinn: synergistic progressive inference of neural networks over device and cloud, in: Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, MobiCom ’20, ACM, 2020, p. 1–15. doi:10.1145/3372224.3419194.
• Keras applications, https://keras.io/api/applications/, accessed: 2024-10-15.
• T. Fontanari, T. C. Fro´es, M. Recamonde-Mendoza, Cross-validation Strategies for Balanced and Imbalanced Datasets, Springer International Publishing, 2022, p. 626–640. doi:10.1007/978-3-031-21686-2_43.
• D. Berrar, Cross-Validation, Elsevier, 2019, p. 542–545. doi:10.1016/b978-0-12-809633-8.20349-x.
• X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: G. Gordon, D. Dunson, M. Dud´ık (Eds.), Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Vol. 15 of Proceedings of Machine Learning Research, PMLR, Fort Lauderdale, FL, USA, 2011, pp. 315–323. URL https://proceedings.mlr.press/v15/glorot11a.html