Ses ve Görüntü Verileriyle Arı Kovanı Davranışlarını Sınıflandıran Çok Modlu Derin Öğrenme Sistemi

Öz

Bu çalışmada, arı kovanlarında gözlemlenen davranışları sınıflandırmak amacıyla ses ve görüntü verilerine dayalı çok modlu bir derin öğrenme sistemi geliştirilmiştir. Literatürde genellikle tekil veri türü (yalnızca ses veya yalnızca görüntü) kullanılırken, bu çalışmada her iki modaliteyi bütünleştiren bir yapı önerilmiştir. Sistem, Raspberry Pi tabanlı donanım üzerinde çalışarak verileri uçta (edge computing) analiz edebilmekte, böylece bulut tabanlı çözümlerden farklı olarak düşük gecikmeli ve enerji verimli karar mekanizmaları sunmaktadır. Ses verileri 1 Boyutlu Konvolüsyonel Sinir Ağı (1D CNN), görüntü verileri ise 2 Boyutlu Konvolüsyonel Sinir Ağı (2D CNN) ile işlenmiş; “normal” ve “agresif” davranışların sınıflandırılmasında sırasıyla %88,24 ve %98,98 doğruluk elde edilmiştir. Bu oranlar, literatürde bildirilen benzer çalışmalara kıyasla daha yüksek başarım sergilemektedir. Ayrıca sınıflandırma sonuçları MQTT protokolü ile mobil uygulamalara ve merkezi sisteme iletilerek kolonilerin uzaktan gerçek zamanlı takibi sağlanmıştır. Çalışma, yapay zekâ destekli Edge tabanlı sistemlerin arıcılık teknolojilerine entegre edilebileceğini ortaya koymakta; sürdürülebilir, düşük maliyetli ve pratik bir izleme çözümü sunmaktadır.

Anahtar Kelimeler

• Çok modlu sistem
• iot tabanlı izleme
• uç birim (edge computing)
• mqtt protokolü
• arı davranışı

A Multimodal Deep Learning System for Classifying Beehive Behaviors Using Audio and Visual Data

Öz

In this study, a multimodal deep learning system was developed to classify honeybee colony behaviors using both audio and visual data. While previous research in the literature has mostly relied on a single data modality (either audio or image), this work proposes an integrated approach combining both modalities. The system operates on a Raspberry Pi-based hardware platform, enabling on-device (edge computing) data analysis and providing low-latency, energy-efficient decision-making compared to cloud-based solutions. Audio data were processed using a One-Dimensional Convolutional Neural Network (1D CNN), while image data were classified with a Two-Dimensional Convolutional Neural Network (2D CNN). The system achieved classification accuracies of 88.24% for audio and 98.98% for image data, outperforming similar studies reported in the literature. Furthermore, classification results were transmitted to mobile applications and a central monitoring system via the MQTT protocol, enabling remote and real-time colony monitoring. This work demonstrates the feasibility of integrating AI-powered, edge-based systems into apiculture technologies and provides a sustainable, low-cost, and practical monitoring solution.

Anahtar Kelimeler

• Multimodal system
• iot-based monitoring
• edge computing
• mqtt protocol
• bee behavior

Kaynakça

1. Costanza R. Valuing natural capital and ecosystem services toward the goals of efficiency, fairness, and sustainability. Ecosyst Serv 2020; 43:101096. doi:10.1016/j.ecoser.2020.101096.
2. Klein AM, Vaissiere BE, Cane JH, Steffan-Dewenter I, Cunningham SA, Kremen C, et al. Importance of pollinators in changing landscapes for world crops. Proc R Soc B Biol Sci 2006; 274(1608):303-313. doi:10.1098/rspb.2006.3721.
3. Kavak G. Son yıllarda görülen koloni kayıpları ve muhtemel sebepleri. Uludağ Arıcılık Dergisi 2016; 15(1): 33-40. doi: 10.31467/uluaricilik.377585
4. Hristov P, et al. Factors associated with honey bee colony losses: a mini-review. Vet Sci 2020; 7(4):166. doi:10.3390/vetsci7040166
5. Morfin N, et al. Varroa destructor and its impacts on honey bee biology. Bee Science 2023. doi:10.3389/frbee.2023.127293
6. Goulson D, Nicholls E, Botías C, Rotheray EL. Bee declines driven by combined stress from parasites, pesticides, and lack of flowers. Science 2015; 347(6229):1255957. doi:10.1126/science.1255957
7. Turyagyenda A, Katumba A, Akol R, Nsabagwa M, Mkiramweni ME. IoT and Machine Learning Techniques for Precision Beekeeping: A Review. AI. 2025;6(2):1–28. doi:10.3390/ai6020026
8. Kontogiannis S. Beehive Smart Detector Device for the Detection of Critical Conditions That Utilize Edge Device Computations and Deep Learning Inferences. Sensors. 2024;24(16):5444. doi:10.3390/s24165444

9. Zhuoqing C, Liu S, Xiong X, Cai Z, Tu G. A Survey of Recent Advances in Edge-Computing-Powered Artificial Intelligence of Things. IEEE Internet Things J. 2021;PP:1. doi:10.1109/JIOT.2021.3088875
10. A. Liang, “Developing an AI-based Integrated System for Bee Health Evaluation,” arXiv preprint, pp. 1–15, 2024, [Online]. Available: http://arxiv.org/abs/2401.09988
11. M. Abdollahi, et al., “UrBAN: Urban Beehive Acoustics and PheNotyping Dataset,” Scientific Data, vol. 12, no. 1, pp. 1–16, 2025, doi: 10.1038/s41597-025-04869-1
12. D. Peñaloza-Aponte, et al., “Automated entrance monitoring to investigate honey bee foraging trips using open-source wireless platform and fiducial tags,” HardwareX, vol. 20, Aug. 2024, p. e00609, doi: 10.1016/j.ohx.2024.e00609
13. D. Mrozek, K. Kaczmarek, A. Małysiak-Mrozek, and M. Kryś, “Edge-Based Detection of Varroosis in Beehives with IoT,” Appl. Sci., vol. 11, no. 22, p. 11078, 2021, doi: 10.3390/app112211078
14. Y. Zheng, H. Liu, and L. Chen, “Intelligent beehive monitoring system based on internet of things,” Smart Agric., vol. 6, no. 2, pp. 45–53, 2024, doi: 10.1016/j.saiot.2024.100189
15. Bird JJ, Faria DR, Premebida C, Ekart A, Vogiatzis G. Look and listen: A multi-modality late fusion approach to scene classification for autonomous machines. IEEE Int Conf Intell Robot Syst (IROS) 2020; 10380–10385. doi:10.1109/IROS45743.2020.9341557.
16. McFee B, Raffel C, Liang D, Ellis DPW, McVicar M, Battenberg E, Nieto O. Librosa: Audio and music signal analysis in Python. Proc 14th Python in Science Conf. 2015:18-24.
17. Hershey S, Chaudhuri S, Ellis DPW, Gemmeke JF, Jansen A, Moore RC, et al. CNN architectures for large-scale audio classification. Proc IEEE Int Conf Acoust Speech Signal Process (ICASSP). 2017:131-135. doi:10.1109/ICASSP.2017.7952132.
18. G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000. [Online]. Available: https://opencv.org
19. McLoughlin I, Zhang H, Xie Z, Song Y, Xiao W, Phan H. Continuous robust sound event classification using time-frequency features and deep learning. PLoS One 2017;12(9):e0182309. doi:10.1371/journal.pone.0182309.
20. Baltrusaitis T, Ahuja C, Morency L-P. Multimodal machine learning: A survey and taxonomy. IEEE Trans Pattern Anal Mach Intell 2019;41(2):423–443. doi:10.1109/TPAMI.2018.2798607.
21. Salman S, Liu X. Overfitting mechanism and avoidance in deep neural networks. arXiv preprint arXiv:1901.06566, 2019. Available: http://arxiv.org/abs/1901.06566.
22. Bisong E. Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners. Springer, 2019. doi:10.1007/978-1-4842-4470-8.
23. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60(6):84-90. doi:10.1145/3065386.
24. I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning, vol. 1, no. 2. MIT press Cambridge, 2016.
25. Bonomi F, Milito R, Zhu J, Addepalli S. Fog computing and its role in the internet of things. Proc 1st ACM Mob Cloud Comput Work (MCC’12); 2012; Helsinki, Finland. p. 13–16. doi:10.1145/2342509.2342513.
26. Yerlikaya Ö, Dalkılıç G. Security of message queue telemetry transport protocol. Proc 25th Signal Processing and Communications Applications Conference (SIU); 2017; Antalya, Turkey. p. 1–4.
27. Hunkeler U, Truong HL, Stanford-Clark A. MQTT-S — A publish/subscribe protocol for Wireless Sensor Networks. Proc 3rd Int Conf Communication Systems Software and Middleware and Workshops (COMSWARE’08); 2008; Bangalore, India. p. 791–798. doi:10.1109/COMSWA.2008.4554519.