Akıllı Buzdolaplarında Meyve ve Sebzelerin Bozulmasını Algılamak için Derin Öğrenme ve IoT Tabanlı Çift Aşamalı Sistem

Abstract

Sürdürülebilir bir gelecek ve yaşanabilir bir dünya sağlamak için sürdürülebilir üretim ve tüketim yöntemleri geliştirmek çok önemlidir. Ekonomik kalkınma ve sürdürülebilir yaşam, çevre ve evsel atıkları en aza indirerek ve gıda kaynaklarını verimli kullanarak sağlanabilir. Yapay zeka, bilgisayar görüşü, veri işleme ve entegre sistemler, bu tür akıllı çözümler geliştirme fırsatı sunmaktadır. Bu çalışmada, meyve ve sebzelerdeki bozulmayı erken tespit etmek için Raspberry Pi tabanlı akıllı buzdolabı modülü tasarlanmış ve uygulanmıştır. Çürümeye başlayan meyve ve sebzeler, çevrelerine çeşitli gazlar salar. Buna dayanarak, önerilen sistem iki aşamalı bir doğrulama yöntemi kullanır. İlk aşamada, buzdolaplarındaki meyve ve sebzelerin bozulması gaz sensörleri tarafından tespit edilir. Gaz sensörleri bozulmayı tespit ettiğinde, ikinci aşama tetiklenir; meyve ve sebzelerin görüntüleri ResNet50, DenseNet201, InceptionV3 ve VGG16 dahil olmak üzere CNN tabanlı modeller kullanılarak sınıflandırılır. Bozulma doğrulanırsa, belirlenen kullanıcıya bir bildirim gönderilir. Gaz algılama ile derin öğrenme tabanlı görüntü sınıflandırmasının entegrasyonu, önerilen sistemin temel yeniliğini oluşturur ve tek aşamalı yaklaşımlara kıyasla daha güvenilir ve erken algılama sağlar. Ayrıca, 20 sınıfta 12.000 görüntü içeren bir karşılaştırma veri seti üzerinde kapsamlı sınıflandırma deneyleri gerçekleştirilmiştir. Tüm CNN modellerinde ince ayar ve hiperparametre optimizasyonu gerçekleştirilmiş ve ResNet50 %98,00 ile en yüksek doğruluk oranına ulaşmıştır. Bu performans, aynı veri seti üzerinde yapılan bazı önceki çalışmalarda bildirilen sonuçları aşmaktadır. Önerilen prototip, sahip olduğu yetenekler sayesinde hem mevcut hem de yeni nesil buzdolaplarında yaygın olarak uygulanabilir.

Keywords

• Çürümüş meyve ve sebze tespiti
• Derin öğrenme
• Görüntü sınıflandırma
• Raspberry Pi
• Akıllı buzdolapları

A Deep Learning and IoT-Based Dual-Stage System for Detecting Fruit and Vegetable Spoilage in Smart Refrigerators

Abstract

It is critical to develop sustainable production and consumption methods to ensure a sustainable future and a livable world. Economic development and sustainable living can be achieved by minimizing environmental and household waste and by using food resources efficiently. Artificial intelligence, computer vision, data processing, and integrated systems offer the opportunity to develop such smart solutions. In this study, a Raspberry Pi-based smart refrigerator module was designed and implemented for the early detection of spoilage in fruits and vegetables. Fruits and vegetables that start to rot release various gases into the surrounding environment. Based on this, the proposed system uses a two-stage verification method. In the first stage, the spoilage of fruits and vegetables in refrigerators is detected by gas sensors. When the gas sensors detect spoilage, the second stage is triggered; images of fruits and vegetables are classified using CNN-based models, including ResNet50, DenseNet201, InceptionV3, and VGG16. If spoilage is confirmed, a notification is sent to the designated user. The integration of gas sensing with deep learning–based image classification constitutes the main novelty of the proposed system, enabling more reliable and early detection compared to single-stage approaches. Moreover, extensive classification experiments were carried out on a benchmark dataset containing 12,000 images across 20 classes. Fine-tuning and hyperparameter optimization were performed on all CNN models, with ResNet50 achieving the highest accuracy of 98.00%. This performance surpasses results reported in some of the earlier studies on the same dataset. Given its capabilities, the proposed prototype could be widely implemented in both existing and next-generation refrigerators.

Keywords

• Rotten fruit and vegetable detection
• Deep learning
• Image classification
• Raspberry Pi
• Smart refrigerators

References

1. [1] H. N. Schifferstein, Changes in appearance during the spoilage process of fruits and vegetables: Implications for consumer use and disposal, Cleaner and Responsible Consumption, 2024, 12, 100184.
2. [2] Bassi S. A., Christensen T. H., and Damgaard A., Environmental performance of household waste management in Europe-An example of 7 countries, Waste Management, 2017, 69, 545–557, DOI:10.1016/j.wasman.2017.07.042.
3. [3] Dos Santos S. F., et al., Post-harvest losses of fruits and vegetables in supply centers in Salvador, Brazil: Analysis of determinants, volumes and reduction strategies, Waste Management, 2020, 101, 161–170, DOI:10.1016/j.wasman.2019.10.007.
4. [4] Wang D., Zhang M., Li M., and Lin J., Fruits and vegetables preservation based on AI technology: research progress and application prospects, Computers and Electronics in Agriculture, 2024, 226, 109382, DOI:10.1016/j.compag.2024.109382.
5. [5] Nerella J. T., Nippulapalli V. K., Nancharla S., Vellanki L. P., and Suhasini P. S., Performance comparison of deep learning techniques for classification of fruits as fresh and rotten, in Int. Conf. Recent Adv. Elect. Electron. Ubiquitous Commun. Comput. Intell. (RAEEUCCI), 2023, Chennai, India, IEEE, DOI:10.1109/RAEEUCCI57140.2023.10134242.6] Palakodati S. S. S., Chirra V. R. R., Yakobu D., and Bulla S., Fresh and rotten fruits classification using cnn and transfer learning, Revue d'Intelligence Artificielle, 2020, 34 (5), 617–622, DOI:10.18280/ria.340512.
6. [7] Gao X., Ding X., Hou R., and Tao Y., Research on food recognition of smart refrigerator based on ssd target detection algorithm, in International Conference on Artificial Intelligence and Computer Science, 2019, Wuhan, China, ACM, 303–308, DOI:10.1145/3349341.3349421.
7. [8] Miah M. S., Tasnuva T., Islam M., Keya M., Rahman M. R., and Hossain S. A., An advanced method of identification fresh and rotten fruits using different convolutional neural networks, in International Conference on Computing Communication and Networking Technologies (ICCCNT), 2021, Kharagpur, India, IEEE, 1–7, DOI:10.1109/ICCCNT51594.2021.9523681.
8. [9] Valentino F., Cenggoro T. W., and Pardamean B., A design of deep learning experimentation for fruit freshness detection, IOP Conference Series: Earth and Environmental Science, 2021, 794 (1), 012023, DOI:10.1088/1755-1315/794/1/012023.

9. [10] Mukhiddinov M., Muminov A., and Cho J., Improved classification approach for fruits and vegetables freshness based on deep learning, Sensors, 2022, 22 (21), 8192, DOI:10.3390/s22218192.
10. [11] Karataş E., Open package detection with deep learning algorithms using thermal camera in heat sealed packages, El-Cezeri Journal of Science and Engineering, 2022, 9 (4), 1363–1374, DOI:10.31202/ecjse.1135411.
11. [12] Ramadan M., Hilles S. M. S., and Alkhedher M., Design and study of an ai-powered autonomous stair climbing robot, El-Cezeri Journal of Science and Engineering, 2023, 10 (3), 571–585, DOI:10.31202/ecjse.1272769.
12. [13] Nasir H., Aziz W. B. W., Ali F., Kadir K., and Khan S., The implementation of iot based smart refrigerator system, in International Conference on Smart Sensors and Application (ICSSA), 2018, Kuching, Malaysia, IEEE, 48–52, DOI:10.1109/ICSSA.2018.8535867.
13. [14] Cömert O., Hekim M., and Adem K., Faster r-cnn kullanarak elmalarda çürük tespiti, International Journal of Engineering Research and Development, 2019, 11 (1), 335–341, DOI:10.29137/umagd.469929.
14. [15] Wu A., Zhu J., and Ren T., Detection of apple defect using laser-induced light backscattering imaging and convolutional neural network, Computers and Electrical Engineering, 2020, 81, 106454, DOI:10.1016/j.compeleceng.2019.106454.
15. [16] Prabha S. and Kumar S., Assessment of banana fruit maturity by image processing technique, Journal of Food Science and Technology, 2015, 52 (3), 1316–1327, DOI:10.1007/s13197-013-1188-3.
16. [17] Ünal Z., Kızıldeniz T., Özden M., Aktaş H., and Karagöz Ö., Derin öğrenme teknikleri ile elmada (granny smith) kusur tespiti, Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi, 2023, 12 (4), 1119–1129, DOI:10.28948/ngumuh.1250012.
17. [18] Jana S., Parekh R., and Sarkar B., Detection of rotten fruits and vegetables using deep learning, in Computer Vision and Machine Learning in Agriculture, 2021, Springer, 31–49, DOI:10.1007/978-981-33-6424-0_3.
18. [19] Valentino F., Cenggoro T. W., and Pardamean B., A design of deep learning experimentation for fruit freshness detection, IOP Conference Series: Earth and Environmental Science, 2021, 794 (1), 012110, DOI:10.1088/1755-1315/794/1/012110.
19. [20] Yuan Y. and Chen X., Vegetable and fruit freshness detection based on deep features and principal component analysis, Current Research in Food Science, 2024, 8, 100656, DOI:10.1016/j.crfs.2023.100656.
20. [21] Amin U., Shahzad M. I., Shahzad A., Shahzad M., Khan U., and Mahmood Z., Automatic fruits freshness classification using cnn and transfer learning, Applied Sciences, 2023, 13 (14), 8087, DOI:10.3390/app13148087.
21. [22] Shweta A. S., Intelligent refrigerator using artificial intelligence, in International Conference on Intelligent Systems and Control (ISCO), 2017, Coimbatore, India, IEEE, 464–468, DOI:10.1109/ISCO.2017.7856036.
22. [23] Ananthanarayana T., Ptucha R., and Kelly S. C., Deep learning based fruit freshness classification and detection with cmos image sensors and edge processors, Electronic Imaging, 2020, 32, 1–7, DOI:10.2352/ISSN.2470-1173.2020.12.FAIS-172.
23. [24] Caya M. V. C., Cruz F. R. G., Fernando C. M. N., Lafuente R. M. M., Malonzo M. B., and Chung W.-Y., Monitoring and detection of fruits and vegetables spoilage in the refrigerator using electronic nose based on principal component analysis, in IEEE 11th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), 2019, Pasay, Philippines, IEEE, 1–6, DOI:10.1109/HNICEM48295.2019.9072715.
24. [25] Mukhiddinov M., Muminov A., and Cho J., Fruits and vegetables dataset, 03.10.2024, Available from: https://www.kaggle.com/datasets/muhriddinmuxiddinov/fruits-and-vegetables-dataset
25. [26] Goodfellow I., Bengio Y., Courville A., and Bengio Y., Deep Learning, 2016, MIT Press, Cambridge.
26. [27] Huang G., Liu Z., Van Der Maaten L., and Weinberger K. Q., Densely connected convolutional networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Honolulu, HI, USA, IEEE, 4700–4708, DOI:10.1109/CVPR.2017.243.
27. [28] Szegedy C., Vanhoucke V., Ioffe S., Shlens J., and Wojna Z., Rethinking the inception architecture for computer vision, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Las Vegas, NV, USA, IEEE, 2818–2826, DOI:10.1109/CVPR.2016.308.
28. [29] He K., Zhang X., Ren S., and Sun J., Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Las Vegas, NV, USA, IEEE, 770–778, DOI:10.1109/CVPR.2016.90.
29. [30] Simonyan K. and Zisserman A., Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014, DOI:10.48550/arXiv.1409.1556.
30. [31] Lu T., Han B., Chen L., Yu F., and Xue C., A generic intelligent tomato classification system for practical applications using densenet-201 with transfer learning, Scientific Reports, 2021, 11 (1), 15824, DOI:10.1038/s41598-021-98942-5.
31. [32] Google, Google colab, Google Research, 10.08.2024, Available from: https://colab.research.google.com.
32. [33] Kazi A. and Panda S. P., Determining the freshness of fruits in the food industry by image classification using transfer learning, Multimedia Tools and Applications, 2022, 81 (6), 7611–7624, DOI:10.1007/s11042-022-12150-5.
33. [34] Habek G. C., Tasdemir S., Basciftci F., and Yılmaz A., Transfer derin öğrenme teknikleri ile görüntü sınıflandırmada aktivasy on fonksiyonlarının performans üzerindeki etkisi, Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 2024, 24 (2), 294–307, DOI:10.35414/akufemubid.1334098.