Research Article
BibTex RIS Cite

Federe Öğrenmede Birleştirme Algoritmalarının Model Performansına Etkisi

Year 2023, Volume: 14 Issue: 1, 65 - 73, 23.03.2023
https://doi.org/10.24012/dumf.1241947

Abstract

Hammaddesi büyük veri olan Yapay zeka (YZ) teknolojileri özellikle son yıllarda verinin gizliliği ve güvenliği gibi önemli gerekçelerle veriye ulaşmayı zorlaştıran sebeplerden ötürü bir takım zorluklarla karşılaşmaktadır. Öte yandan büyük verinin merkezi bir lokasyonda toplanmasının zorlukları ve yüksek kapasiteli depolama ve işlemci ihtiyaçları da YZ alanında karşılaşılan zorluklardır. Bu zorluklardan esinlenerek geliştirilen İşbirlikçi YZ konsepti olan Federe Öğrenme (FÖ), işbirliğine katılan katılımcıların, veri gizliliğini ihlal etmeden YZ model parametrelerinin kendi verileri ile işlenip model paremetrelerinin güncellenmesi ve güncellenen parametrelerin bir sunucuda belirli algoritmalar aracılığıyla birleştirilmesi ile iteratif olarak gerçekleştirilen bir konsepttir. FÖ konsepti, katılımcıların öznitelik ve örnek uzaylarının ortaklığına bağlı olarak Yatay FÖ, Dikey FÖ ve Federe Transfer Öğrenme şeklinde yaklaşımlar ile uygulanmaktdır. Bu çalışmada öznitelik uzaylarının ortak olduğu Yatay FÖ yaklaşımı için geliştirilen model parametrelerini birleştirme algoritmalarından FedAVG, FedAVGM ve FaultTolerantFedAVG’nin 5 katılımcı arasında özdeş olmayan bir şekilde dağıtılmış olan MNIST veri setinin ResNet-18 ve MobileNet V3 small sınıflandırıclarının performansına etkisi incelenmektedir.

References

  • [1] J. Park et al., “Communication-Efficient and Distributed Learning over Wireless Networks: Principles and Applications,” Proc. IEEE, vol. 109, no. 5, pp. 796–819, 2021, doi: 10.1109/JPROC.2021.3055679.
  • [2] “I (Legislative acts) REGULATIONS REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance).”
  • [3] H. Brendan McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” Proc. 20th Int. Conf. Artif. Intell. Stat. AISTATS 2017, vol. 54, 2017.
  • [4] P. Kairouz et al., “Advances and open problems in federated learning,” arXiv, pp. 1–105, 2019.
  • [5] X. Huang, Y. Ding, Z. L. Jiang, S. Qi, X. Wang, and Q. Liao, “DP-FL: a novel differentially private federated learning framework for the unbalanced data,” World Wide Web, vol. 23, no. 4, pp. 2529–2545, Jul. 2020, doi: 10.1007/s11280-020-00780-4.
  • [6] M. NERGİZ, “Collaborative Artifical Intelligence Concept: Federated Learning Review,” DÜMF Mühendislik Derg., Jun. 2022, doi: 10.24012/dumf.1130789.
  • [7] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, and M. Jirstrand, “A performance evaluation of federated learning algorithms,” in DIDL 2018 - Proceedings of the 2nd Workshop on Distributed Infrastructures for Deep Learning, Part of Middleware 2018, Dec. 2018, pp. 1–8, doi: 10.1145/3286490.3286559.
  • [8] S. Ek, F. Portet, P. Lalanda, and G. Vega, “A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison,” Oct. 2021, doi: 10.1109/PERCOM50583.2021.9439129.
  • [9] E. M. Campos et al., “Evaluating Federated Learning for intrusion detection in Internet of Things: Review and challenges,” Comput. Networks, vol. 203, p. 108661, Feb. 2022, doi: 10.1016/J.COMNET.2021.108661.
  • [10] “MNIST dataset,” [Online]. Available: https://github.com/myleott/mnist_png.
  • [11] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Dec. 2015, [Online]. Available: http://arxiv.org/abs/1512.03385.
  • [12] F. Ramzan et al., “A Deep Learning Approach for Automated Diagnosis and Multi-Class Classification of Alzheimer’s Disease Stages Using Resting-State fMRI and Residual Neural Networks,” J. Med. Syst., vol. 44, no. 2, Feb. 2020, doi: 10.1007/s10916-019-1475-2.
  • [13] A. Howard et al., “Searching for MobileNetV3.”
  • [14] T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,” Sep. 2019, [Online]. Available: http://arxiv.org/abs/1909.06335.
  • [15] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated Learning with Non-IID Data,” 2018, [Online]. Available: http://arxiv.org/abs/1806.00582.
  • [16] “PYTORCH,” [Online]. Available: https://pytorch.org/vision/stable/models.html.
Year 2023, Volume: 14 Issue: 1, 65 - 73, 23.03.2023
https://doi.org/10.24012/dumf.1241947

Abstract

References

  • [1] J. Park et al., “Communication-Efficient and Distributed Learning over Wireless Networks: Principles and Applications,” Proc. IEEE, vol. 109, no. 5, pp. 796–819, 2021, doi: 10.1109/JPROC.2021.3055679.
  • [2] “I (Legislative acts) REGULATIONS REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance).”
  • [3] H. Brendan McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” Proc. 20th Int. Conf. Artif. Intell. Stat. AISTATS 2017, vol. 54, 2017.
  • [4] P. Kairouz et al., “Advances and open problems in federated learning,” arXiv, pp. 1–105, 2019.
  • [5] X. Huang, Y. Ding, Z. L. Jiang, S. Qi, X. Wang, and Q. Liao, “DP-FL: a novel differentially private federated learning framework for the unbalanced data,” World Wide Web, vol. 23, no. 4, pp. 2529–2545, Jul. 2020, doi: 10.1007/s11280-020-00780-4.
  • [6] M. NERGİZ, “Collaborative Artifical Intelligence Concept: Federated Learning Review,” DÜMF Mühendislik Derg., Jun. 2022, doi: 10.24012/dumf.1130789.
  • [7] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, and M. Jirstrand, “A performance evaluation of federated learning algorithms,” in DIDL 2018 - Proceedings of the 2nd Workshop on Distributed Infrastructures for Deep Learning, Part of Middleware 2018, Dec. 2018, pp. 1–8, doi: 10.1145/3286490.3286559.
  • [8] S. Ek, F. Portet, P. Lalanda, and G. Vega, “A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison,” Oct. 2021, doi: 10.1109/PERCOM50583.2021.9439129.
  • [9] E. M. Campos et al., “Evaluating Federated Learning for intrusion detection in Internet of Things: Review and challenges,” Comput. Networks, vol. 203, p. 108661, Feb. 2022, doi: 10.1016/J.COMNET.2021.108661.
  • [10] “MNIST dataset,” [Online]. Available: https://github.com/myleott/mnist_png.
  • [11] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Dec. 2015, [Online]. Available: http://arxiv.org/abs/1512.03385.
  • [12] F. Ramzan et al., “A Deep Learning Approach for Automated Diagnosis and Multi-Class Classification of Alzheimer’s Disease Stages Using Resting-State fMRI and Residual Neural Networks,” J. Med. Syst., vol. 44, no. 2, Feb. 2020, doi: 10.1007/s10916-019-1475-2.
  • [13] A. Howard et al., “Searching for MobileNetV3.”
  • [14] T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,” Sep. 2019, [Online]. Available: http://arxiv.org/abs/1909.06335.
  • [15] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated Learning with Non-IID Data,” 2018, [Online]. Available: http://arxiv.org/abs/1806.00582.
  • [16] “PYTORCH,” [Online]. Available: https://pytorch.org/vision/stable/models.html.
There are 16 citations in total.

Details

Primary Language Turkish
Journal Section Articles
Authors

Mehmet Nergiz 0000-0002-0867-5518

Early Pub Date March 22, 2023
Publication Date March 23, 2023
Submission Date January 24, 2023
Published in Issue Year 2023 Volume: 14 Issue: 1

Cite

IEEE M. Nergiz, “Federe Öğrenmede Birleştirme Algoritmalarının Model Performansına Etkisi”, DUJE, vol. 14, no. 1, pp. 65–73, 2023, doi: 10.24012/dumf.1241947.
DUJE tarafından yayınlanan tüm makaleler, Creative Commons Atıf 4.0 Uluslararası Lisansı ile lisanslanmıştır. Bu, orijinal eser ve kaynağın uygun şekilde belirtilmesi koşuluyla, herkesin eseri kopyalamasına, yeniden dağıtmasına, yeniden düzenlemesine, iletmesine ve uyarlamasına izin verir. 24456