Federated Learning and Resource-Constrained Embedded Systems: A Comprehensive Survey

Year 2024, Volume: 1 Issue: 2, 40 - 55, 20.12.2024

Eda Bahar , Ozgun Pınarer

Abstract

Federated Learning (FL) has become a transformative approach in machine learning, allowing decentralized training of models across multiple devices while preserving data privacy. This paradigm addresses critical concerns related to data privacy, security, and communication overhead, making it particularly relevant for applications in domains such as healthcare, finance, and the Internet of Things (IoT). Resource-constrained FL extends this concept to environments where computational, communication, and energy resources are limited, such as edge networks and IoT devices. This extension focuses on optimizing various aspects of the learning process to enable effective model training even in resource-limited settings. The primary aim of this survey is to provide a comprehensive and structured overview of the current state of research in FL and resource-constrained FL. By examining 62 key publications, this survey synthesizes insights and developments across these domains, highlighting advancements, challenges, and gaps that exist. This survey aims to provide a holistic view of the advancements and ongoing challenges in FL and resource-constrained FL. It identifies research gaps and proposes future directions, such as improving communication efficiency, developing adaptive learning algorithms, and enhancing resource management strategies. This survey serves as a valuable resource for researchers, practitioners, and stakeholders in the field, offering practical insights and guiding future exploration and innovation in FL and its applications in resource-constrained environments.

Keywords

Decentralized Training, Edge Computing, Privacy-Preserving Algorithms, IoT Applications, Computational Efficiency

References

• AbdulRahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2020). A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE Internet of Things Journal, 8(7), 5476-5497.
• Abreha, H. G., Hayajneh, M., & Serhani, M. A. (2022). Federated learning in edge computing: a systematic survey. Sensors, 22(2), 450.
• Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020). Federated learning: A survey on enabling technologies, protocols, and applications. IEEE Access, 8, 140699-140725.
• Antunes, R. S., André da Costa, C., Küderle, A., Yari, I. A., & Eskofier, B. (2022). Federated learning for healthcare: Systematic review and architecture proposal. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4), 1-23.
• Banabilah, S., Aloqaily, M., Alsayed, E., Malik, N., & Jararweh, Y. (2022). Federated learning review: Fundamentals, enabling technologies, and future applications. Information processing & management, 59(6), 103061.
• Bharati, S., Mondal, M., Podder, P., & Prasath, V. B. (2022). Federated learning: Applications, challenges and future directions. International Journal of Hybrid Intelligent Systems, 18(1-2), 19-35.
• Bouacida, N., & Mohapatra, P. (2021). Vulnerabilities in federated learning. IEEE Access, 9, 63229-63249. Brecko, A., Kajati, E., Koziorek, J., & Zolotova, I. (2022). Federated learning for edge computing: A survey. Applied Sciences, 12(18), 9124.
• Chen, M., Poor, H. V., Saad, W., & Cui, S. (2020). Wireless communications for collaborative federated learning. IEEE Communications Magazine, 58(12), 48-54.
• Chen, H., Huang, S., Zhang, D., Xiao, M., Skoglund, M., & Poor, H. V. (2022). Federated learning over wireless IoT networks with optimized communication and resources. IEEE Internet of Things Journal, 9(17), 16592-16605.
• da Silva, L. G. F., Sadok, D. F., & Endo, P. T. (2023). Resource optimizing federated learning for use with IoT: A systematic review. Journal of Parallel and Distributed Computing, 175, 92-108.
• Du, M., Zheng, H., Gao, M., & Feng, X. (2023). Adaptive Decentralized Federated Learning in Resource-Constrained IoT Networks. IEEE Internet of Things Journal.
• Ficco, M., Guerriero, A., Milite, E., Palmieri, F., Pietrantuono, R., & Russo, S. (2024). Federated learning for IoT devices: Enhancing TinyML with on-board training. Information Fusion, 104, 102189.
• Gafni, T., Shlezinger, N., Cohen, K., Eldar, Y. C., & Poor, H. V. (2022). Federated learning: A signal processing perspective. IEEE Signal Processing Magazine, 39(3), 14-41.
• Gao, W., Zhao, Z., Min, G., Ni, Q., & Jiang, Y. (2021). Resource allocation for latency-aware federated learning in industrial internet of things. IEEE Transactions on Industrial Informatics, 17(12), 8505-8513.
• Ghimire, B., & Rawat, D. B. (2022). Recent advances on federated learning for cybersecurity and cybersecurity for federated learning for internet of things. IEEE Internet of Things Journal, 9(11), 8229-8249.
• Ghosh, A., Chung, J., Yin, D., & Ramchandran, K. (2020). An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33, 19586-19597.
• Gupta, A., Misra, S., Pathak, N., & Das, D. (2023). Fedcare: Federated learning for resource-constrained healthcare devices in iomt system. IEEE Transactions on Computational Social Systems, 10(4), 1587-1596.
• Hazra, A., Adhikari, M., Nandy, S., Doulani, K., & Menon, V. G. (2022). Federated-learning-aided next-generation edge networks for intelligent services. IEEE Network, 36(3), 56-64.
• Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2021a). A survey on federated learning for resource-constrained IoT devices. IEEE Internet of Things Journal, 9(1), 1-24.
• Imteaj, A., & Amini, M. H. (2021b). FedPARL: Client activity and resource-oriented lightweight federated learning model for resource-constrained heterogeneous IoT environment. Frontiers in Communications and Networks, 2, 657653.
• Imteaj, A., Khan, I., Khazaei, J., & Amini, M. H. (2021c). Fedresilience: A federated learning application to improve resilience of resource-constrained critical infrastructures. Electronics, 10(16), 1917.
• Imteaj, A., Mamun Ahmed, K., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022). Federated learning for resource-constrained iot devices: Panoramas and state of the art. Federated and Transfer Learning, 7-27.
• Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., ... & Zhao, S. (2021). Advances and open problems in federated learning. Foundations and trends® in machine learning, 14(1–2), 1-210.
• Khan, Latif U., et al. "Federated learning for internet of things: Recent advances, taxonomy, and open challenges." IEEE Communications Surveys & Tutorials 23.3 (2021): 1759-1799.
• Kushwaha, D., Redhu, S., Brinton, C. G., & Hegde, R. M. (2023). Optimal device selection in federated learning for resource-constrained edge networks. IEEE Internet of Things Journal, 10(12), 10845-10856.
• Li, L., Fan, Y., Tse, M., & Lin, K. Y. (2020a). A review of applications in federated learning. Computers & Industrial Engineering, 149, 106854.
• Li, Z., Sharma, V., & Mohanty, S. P. (2020b). Preserving data privacy via federated learning: Challenges and solutions. IEEE Consumer Electronics Magazine, 9(3), 8-16.
• Li, T., Hu, S., Beirami, A., & Smith, V. (2021a). Ditto: Fair and robust federated learning through personalization. In International conference on machine learning (pp. 6357-6368). PMLR.
• Li, Q., He, B., & Song, D. (2021b). Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10713-10722).
• Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., ... & He, B. (2021c). A survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 35(4), 3347-3366.
• Li, Z., Zhou, Y., Wu, D., Tang, T., & Wang, R. (2022). Fairness-aware federated learning with unreliable links in resource-constrained Internet of things. IEEE Internet of Things Journal, 9(18), 17359-17371.
• Liu, J., Huang, J., Zhou, Y., Li, X., Ji, S., Xiong, H., & Dou, D. (2022). From distributed machine learning to federated learning: A survey. Knowledge and Information Systems, 64(4), 885-917.
• Liu, Y., Kang, Y., Zou, T., Pu, Y., He, Y., Ye, X., ... & Yang, Q. (2024). Vertical federated learning: Concepts, advances, and challenges. IEEE Transactions on Knowledge and Data Engineering.
• Long, G., Tan, Y., Jiang, J., & Zhang, C. (2020). Federated learning for open banking. In Federated learning: privacy and incentive (pp. 240-254). Cham: Springer International Publishing.
• Lyu, L., Yu, H., & Yang, Q. (2020). Threats to federated learning: A survey. arXiv preprint arXiv:2003.02133. Ma, Z., Xu, Y., Xu, H., Meng, Z., Huang, L., & Xue, Y. (2021). Adaptive batch size for federated learning in resource-constrained edge computing. IEEE Transactions on Mobile Computing, 22(1), 37-53.
• Ma, X., Zhu, J., Lin, Z., Chen, S., & Qin, Y. (2022). A state-of-the-art survey on solving non-iid data in federated learning. Future Generation Computer Systems, 135, 244-258.
• Mammen, P. M. (2021). Federated learning: Opportunities and challenges. arXiv preprint arXiv:2101.05428. Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021). A survey on security and privacy of federated learning. Future Generation Computer Systems, 115, 619-640.
• Nguyen, H. T., Sehwag, V., Hosseinalipour, S., Brinton, C. G., Chiang, M., & Poor, H. V. (2020a). Fast-convergent federated learning. IEEE Journal on Selected Areas in Communications, 39(1), 201-218.
• Nguyen, V. D., Sharma, S. K., Vu, T. X., Chatzinotas, S., & Ottersten, B. (2020b). Efficient federated learning algorithm for resource allocation in wireless IoT networks. IEEE Internet of Things Journal, 8(5), 3394-3409.
• Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Poor, H. V. (2021). Federated learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3), 1622-1658.
• Nguyen, D. C., Pham, Q. V., Pathirana, P. N., Ding, M., Seneviratne, A., Lin, Z., ... & Hwang, W. J. (2022). Federated learning for smart healthcare: A survey. ACM Computing Surveys (Csur), 55(3), 1-37.
• Niknam, S., Dhillon, H. S., & Reed, J. H. (2020). Federated learning for wireless communications: Motivation, opportunities, and challenges. IEEE Communications Magazine, 58(6), 46-51.
• Pfeiffer, K., Rapp, M., Khalili, R., & Henkel, J. (2023). Federated learning for computationally constrained heterogeneous devices: A survey. ACM Computing Surveys, 55(14s), 1-27.
• Pfitzner, B., Steckhan, N., & Arnrich, B. (2021). Federated learning in a medical context: a systematic literature review. ACM Transactions on Internet Technology (TOIT), 21(2), 1-31.
• Posner, J., Tseng, L., Aloqaily, M., & Jararweh, Y. (2021). Federated learning in vehicular networks: Opportunities and solutions. IEEE Network, 35(2), 152-159.
• Saha, R., Misra, S., & Deb, P. K. (2020). FogFL: Fog-assisted federated learning for resource-constrained IoT devices. IEEE Internet of Things Journal, 8(10), 8456-8463.
• Salehi, M., & Hossain, E. (2021). Federated learning in unreliable and resource-constrained cellular wireless networks. IEEE Transactions on Communications, 69(8), 5136-5151.
• Salh, A., Ngah, R., Audah, L., Kim, K. S., Abdullah, Q., Al-Moliki, Y. M., ... & Talib, H. N. (2023). Energy-efficient federated learning with resource allocation for green IoT edge intelligence in B5G. IEEE Access, 11, 16353-16367.
• Siddique, A. A., Alasbali, N., Driss, M., Boulila, W., Alshehri, M. S., & Ahmad, J. (2024). Sustainable collaboration: Federated learning for environmentally conscious forest fire classification in green internet of things (IoT). Internet of Things, 25, 101013.
• Sun, W., Lei, S., Wang, L., Liu, Z., & Zhang, Y. (2020). Adaptive federated learning and digital twin for industrial internet of things. IEEE Transactions on Industrial Informatics, 17(8), 5605-5614.
• Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2022). Towards personalized federated learning. IEEE transactions on neural networks and learning systems, 34(12), 9587-9603.
• Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019). Adaptive federated learning in resource constrained edge computing systems. IEEE journal on selected areas in communications, 37(6), 1205-1221.
• Wang, H., Kaplan, Z., Niu, D., & Li, B. (2020, July). Optimizing federated learning on non-iid data with reinforcement learning. In IEEE INFOCOM 2020-IEEE conference on computer communications (pp. 1698-1707). IEEE.
• Wen, J., Zhang, Z., Lan, Y., Cui, Z., Cai, J., & Zhang, W. (2023). A survey on federated learning: challenges and applications. International Journal of Machine Learning and Cybernetics, 14(2), 513-535.
• Xia, Q., Ye, W., Tao, Z., Wu, J., & Li, Q. (2021). A survey of federated learning for edge computing: Research problems and solutions. High-Confidence Computing, 1(1), 100008.
• Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021). Federated learning for healthcare informatics. Journal of healthcare informatics research, 5, 1-19.
• Zhan, Y., Zhang, J., Hong, Z., Wu, L., Li, P., & Guo, S. (2021). A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing, 10(2), 1035-1044.
• Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021a). A survey on federated learning. Knowledge-Based Systems, 216, 106775.
• Zhang, H., Xie, Z., Zarei, R., Wu, T., & Chen, K. (2021b). Adaptive client selection in resource constrained federated learning systems: A deep reinforcement learning approach. IEEE Access, 9, 98423-98432.
• Zhang, T., Gao, L., He, C., Zhang, M., Krishnamachari, B., & Avestimehr, A. S. (2022). Federated learning for the internet of things: Applications, challenges, and opportunities. IEEE Internet of Things Magazine, 5(1), 24-29.
• Zhang, K., Song, X., Zhang, C., & Yu, S. (2022b). Challenges and future directions of secure federated learning: a survey. Frontiers of computer science, 16, 1-8.
• Zhu, G., Deng, Y., Chen, X., Zhang, H., Fang, Y., & Wong, T. F. (2024). ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices. IEEE Internet of Things Journal.
• Ziller, A., Trask, A., Lopardo, A., Szymkow, B., Wagner, B., Bluemke, E., ... & Kaissis, G. (2021). Pysyft: A library for easy federated learning. Federated Learning Systems: Towards Next-Generation AI, 111-139.