Use of YOLOv5 Trained Model for Robotic Courgette Harvesting and Efficiency Analysis

Abstract

The utilization of machine learning in vegetable harvesting not only enhances efficiency and precision but also addresses labor shortages and improves overall agricultural productivity. In this study, a machine learning method was developed for harvesting courgette fruit. Courgette is a fruit that can take a long time to select and harvest in the agricultural area where it is grown. The YOLOv5 models (nano, small, medium, and large) were used as a deep learning method. All metric values of the models were analyzed. The most successful model was the one trained with the YOLOv5m algorithm using 20 batches and 160 epochs with 640x640 images. The results of the model scores were analyzed as "metrics/precision", "metrics/recall", "metrics/mAP_0.5" and "metrics/mAP_0.5: 0.95". These metrics are key indicators that measure the recognition success of a model and reflect the performance of the respective model on the validation dataset. The metrics data of the "YOLOv5 medium" model proved to be higher compared to the other models. The measured values were YOLOv5m = size: 640x640, batch: 20, epoch: 160, algorithm: YOLOv5m. It was concluded that "YOLOv5m" is the best recognition model that can be used in robotic courgette harvesting to separate the courgette from the branch.

Keywords

• deep learning
• courgette
• YOLOv5
• Product forecasting

References

1. Alam, M. S., Alam, M., Tufail, M., Khan, M. U., Güneş, A., Salah, B., Nasir, F. E., Saleem, W., & Khan, M. T. (2022). TobSet: A new tobacco crop and weeds image dataset and its utilization for vision-based spraying by agricultural robots. Applied Sciences, 12(3), 1308. https://doi.org/10.3390/app12031308
2. Arad, B., Balendonck, J., Barth, R., Ben‐Shahar, O., Edan, Y., Hellström, T., Hemming, J., Kurtser, P., Ringdahl, O., & Tielen, T. (2020). Development of a sweet pepper harvesting robot. Journal of Field Robotics, 37(6), 1027-1039. https://doi.org/10.1002/rob.21937
3. Atalay, M., & Çelik, E. (2017). Artificial intelligence and machine learning applications in big data analysis. Mehmet Akif Ersoy University Social Sciences Institute Journal, 9(22), 155-172. https://doi.org/https://doi.org/10.20875/makusobed.309727
4. Altınbilek, H. F., & Kızıl, Ü. (2022). Identification of Paddy Rice Diseases Using Deep Convolutional Neural Networks. Yuzuncu Yıl University Journal of Agricultural Sciences, 32(4), 705-713. https://doi.org/10.29133/yyutbd.1140911
5. Bai, W., Zhao, J., Dai, C., Zhang, H., Zhao, L., Ji, Z., & Ganchev, I. (2023). Two novel models for traffic sign detection based on YOLOv5s. Axioms, 12(2), 160. https://doi.org/10.3390/axioms12020160
6. Barman, U., Das, D., Sonowal, G., Dutta, M. (2024). Innovative Approaches to Rice (Oryza sativa) Crop Health: A Comprehensive Analysis of Deep Transfer Learning for Early Disease Detection. Yuzuncu Yıl University Journal of Agricultural Sciences, 34(2), 314-322. https://doi.org/10.29133/yyutbd.1402821
7. Bati, C. T., & Ser, G. (2023). Effects of Data Augmentation Methods on YOLO v5s: Application of Deep Learning with Pytorch for Individual Cattle Identification. Yuzuncu Yıl University Journal of Agricultural Sciences, 33(3), 363-376. https://doi.org/10.29133/yyutbd.1246901
8. Chen, W., Lu, S., Liu, B., Chen, M., Li, G., & Qian, T. (2022). CitrusYOLO: A algorithm for citrus detection under orchard environment based on YOLOv4. Multimedia Tools and Applications, 81, 31363–31389.https://doi.org/10.1007/s11042-022-12687-5

9. Darwin, B., Dharmaraj, P., Prince, S., Popescu, D. E., & Hemanth, D. J. (2021). Recognition of bloom/yield in crop images using deep learning models for smart agriculture: A review. Agronomy, 11(4), 646. https://doi.org/10.3390/agronomy11040646
10. Deng, L., & Yu, D. (2014). Deep learning: methods and applications. Foundations and Trends® in Signal Processing, 7(3–4), 197-387.https://doi.org/10.1561/2000000039
11. Droukas, L., Doulgeri, Z., Tsakiridis, N. L., Triantafyllou, D., Kleitsiotis, I., Mariolis, I., Giakoumis, D., Tzovaras, D., Kateris, D., & Bochtis, D. (2023). A Survey of Robotic Harvesting Systems and Enabling Technologies. Journal of Intelligent & Robotic Systems, 107(2), 21.https://doi.org/10.1007/s10846-022-01793-z
12. Du, F. J., & Jiao, S. J. (2022). Improvement of lightweight convolutional neural network model based on YOLO algorithm and its research in pavement defect detection. Sensors, 22(9), 3537. https://doi.org/10.3390/s22093537
13. Elavarasan, D., & Vincent, P. D. (2020). Crop yield prediction using deep reinforcement learning model for sustainable agrarian applications. IEEE Access, 8, 86886-86901. https://doi.org/10.1109/access.2020.2992480
14. Fountas, S., Mylonas, N., Malounas, I., Rodias, E., Hellmann Santos, C., & Pekkeriet, E. (2020). Agricultural robotics for field operations. Sensors, 20(9), 2672. https://doi.org/10.3390/s20092672
15. Gholipoor, M., & Fathollah, N. (2019). Fruit yield prediction of pepper using artificial neural network. Scientia Horticulturae, 250, 249-253.https://doi.org/10.1016/j.scienta.2019.02.040
16. Hong, W., Ma, Z., Ye, B., Yu, G., Tang, T., & Zheng, M. (2023). Detection of green asparagus in complex environments based on the improved YOLOv5 algorithm. Sensors, 23(3), 1562. https://doi.org/10.3390/s23031562
17. İmak, A., Doğan, G., Şengür, A., & Ergen, B. (2023). A new method based on extracting, combining and selecting deep features from natural and synthetic data for classification of grapevine leaf species. Int. J. Pure App. Sci., 9(1), 46-55. https://doi.org/10.29132/ijpas.1144615
18. Jaramillo-Hernández, J. F., Julian, V., Marco-Detchart, C., & Rincón, J. A. (2024). Application of machine vision techniques in low-cost devices to improve efficiency in precision farming. Sensors, 24(3), 937. https://doi.org/10.3390/s24030937
19. Kaldarova, M., Аkanova, A., Nazyrova, A., Mukanova, A., & Tynykulova, A. (2023). Identification Of Weeds In Fields Based On Computer Vision Technology. Eastern-European Journal of Enterprise Technologies, 124(2). https://doi.org/10.15587/1729-4061.2023.284600
20. Karahanlı, G., & Taşkın, C. (2024). Determining the growth stages of sunflower plants using deep learning methods. Journal of the Faculty of Engineering and Architecture of Gazi University, 39(3), 1455-1472. https://doi.org/10.17341/gazimmfd.1200615
21. Kini, A. S., Reddy, P. K., & Pai, S. N. (2023). Techniques of deep learning and image processing in plant leaf disease detection: A review. International Journal of Electrical and Computer Engineering (IJECE), 13(3), 3029-3040.
22. https://doi.org/10.11591/ijece.v13i3.pp3029-3040
23. Lu, D., Ye, J., Wang, Y., & Yu, Z. (2023). Plant detection and counting: Enhancing precision agriculture in UAV and general scenes. IEEE Access. https://doi.org/10.1109/ACCESS.2023.3325747
24. Luo, J., Li, B., & Leung, C. (2023). A survey of computer vision technologies in urban and controlled-environment agriculture. ACM Computing Surveys, 56(5), 1-39. https://doi.org/10.1145/1122445.1122456
25. Mao, S., Li, Y., Ma, Y., Zhang, B., Zhou, J., & Wang, K. (2020). Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion. Computers and Electronics in Agriculture, 170,105254.
26. https://doi.org/1016/j.compag.2020.105254
27. Nath, S. (2024). A vision of precision agriculture: Balance between agricultural sustainability and environmental stewardship. Agronomy Journal, 116(3), 1126-1143. https://doi.org/10.1002/agj2.21405
28. Palacios, F., Diago, M. P., Melo-Pinto, P., & Tardaguila, J. (2023). Early yield prediction in different grapevine varieties using computer vision and machine learning. Precision Agriculture, 24(2), 407-435.https://doi.org/10.1007/s11119-022-09950-y
29. Punithavathi, R., Rani, A. D. C., Sughashini, K., Kurangi, C., Nirmala, M., Ahmed, H. F. T., & Balamurugan, S. (2023). Computer Vision and Deep Learning-enabled Weed Detection Model for Precision Agriculture. Comput. Syst. Sci. Eng., 44(3), 2759-2774. https://doi.org/10.32604/csse.2023.027647
30. Rai, N., Mahecha, M. V., Christensen, A., Quanbeck, J., Zhang, Y., Howatt, K., Ostlie, M., & Sun, X. (2023). Multi-format open-source weed image dataset for real-time weed identification in precision agriculture. Data in Brief, 51, 109691. https://doi.org/10.1016/j.dib.2023.109691
31. Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.
32. Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. In computer vision and pattern recognition. arXiv preprint arXiv:1804.02767. https://doi.org/10.48550/arXiv.1804.02767
33. Rivera Zarate, G. (2023). LiDAR applications in precision agriculture for cultivating crops: A review of recent advances. Instituto de Ingeniería y Tecnología (207), 107737. https://doi.org/10.1016/j.compag.2023.107737
34. Roshanianfard, A., Noguchi, N., Ardabili, S., Mako, C., & Mosavi, A. (2022). Autonomous robotic system for pumpkin harvesting. Agronomy, 12(7), 1594. https://doi.org/10.3390/agronomy12071594
35. Rudenko, M., Plugatar, Y., Korzin, V., Kazak, A., Gallini, N., & Gorbunova, N. (2023). The use of computer vision to improve the affinity of rootstock-graft combinations and identify diseases of grape seedlings. Inventions, 8(4), 92. https://doi.org/10.3390/inventions8040092
36. Sapkota, R., Stenger, J., Ostlie, M., & Flores, P. (2023). Towards reducing chemical usage for weed control in agriculture using UAS imagery analysis and computer vision techniques. Scientific Reports, 13(1), 6548. https://doi.org/10.1038/s41598-023-33042-0
37. Shin, Y. H., Park, M. J., Lee, O. Y., & Kim, J. O. (2020). Deep orthogonal transform feature for image denoising. IEEE Access, 8, 66898-66909. https://doi.org/10.1109/ACCESS.2020.2986827
38. Soeb, M. J. A., Jubayer, M. F., Tarin, T. A., Al Mamun, M. R., Ruhad, F. M., Parven, A., Mubarak, N. M., Karri, S. L., & Meftaul, I. M. (2023). Tea leaf disease detection and identification based on YOLOv7 (YOLO-T). Scientific Reports, 13(1), 6078. https://doi.org/10.1038/s41598-023-33270-4
39. Štaka, Z., & Mišić, M. (2023). Leaf counting in the presence of occlusion in Arabidopsis thaliana plant using convolutional neural networks. Journal of Electronic Imaging, 32(5), 052407-052407. https://doi.org/10.1117/1.jei.32.5.052407
40. Ubaid, M. T., & Javaid, S. (2024). Precision agriculture: Computer vision-enabled sugarcane plant counting in the tillering phase. Journal of Imaging, 10(5), 102. https://doi.org/10.3390/jimaging10050102
41. Wang, H., Ji, C., Gu, B., & Tian, G. (2013). Cucumber image segmentation based on weighted connection coefficient pulse coupled neural network. Nongye Jixie Xuebao= Transactions of the Chinese Society for Agricultural Machinery, 44(3), 204-208. https://doi.org/10.6041/j.issn.1000-1298.2013.03.037
42. Wang, Y., Wang, Y., & Zhao, J. (2022). MGA-YOLO: A lightweight one-stage network for apple leaf disease detection. Frontiers in Plant Science, 13, 927424. https://doi.org/10.3389/fpls.2022.927424
43. Xiao, F., Wang, H., Li, Y., Cao, Y., Lv, X., & Xu, G. (2023). Object detection and recognition techniques based on digital image processing and traditional machine learning for fruit and vegetable harvesting robots: an overview and review. Agronomy, 13(3), 639. https://doi.org/10.3390/agronomy13030639
44. Xu, J., & Lu, Y. (2023). Openweedgui: an open-source graphical user interface for weed imaging and detection. Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping VIII. https://doi.org/10.1117/12.2664131
45. Zhu, L., Li, Z., Li, C., Wu, J., & Yue, J. (2018). High performance vegetable classification from images based on alexnet deep learning model. International Journal of Agricultural and Biological Engineering, 11(4),217-223. https://doi.org/10.25165/j.ijabe.20181104.2690
46. Zualkernan, I., Abuhani, D. A., Hussain, M. H., Khan, J., & El-Mohandes, M. (2023). Machine learning for precision agriculture using imagery from unmanned aerial vehicles (uavs): A survey. Drones, 7(6), 382. https://doi.org/10.3390/drones7060382