Non-destructive Weight Prediction Model of Spherical Fruits and Vegetables using U-Net Image Segmentation and Machine Learning Methods

Abstract

Artificial intelligence has become increasingly prominent in agriculture and other fields. Prediction of body weight in animals and plants has been done by humans using many different methods and observations from the past to the present. Although there has been extensive research on predicting the live body weight of animals, weight prediction of vegetables and fruits is not widely. As spherical or round-shaped fruits and vegetables are sold by weighing in the fields, markets and greengrocers, it is important to make weight predictions. Based on this, a model was developed to predict the weight of fruits and vegetables such as watermelons, melons, apples, oranges and tomatoes with the data obtained from their images. The fruit and vegetable weights were predicted by regression models using data obtained from images segmented by the U-Net architecture. Machine learning models such as Multi-Layer Perceptron (MLP), Random Forest (RF), Decision Trees (DT), Support Vector Machines (SVM), Linear and Stochastic Gradient Descent (SGD) regression models were used for weight predictions. The most effective regression models are the RF and DT models. For regression training, the best success rates were calculated as 0.9112 for watermelon, 0.9944 for apple, 0.9989 for tomato and 0.9996 for orange. In addition, the results were evaluated by comparing them to the studies of weight prediction. The weight prediction model will help to sell round-shaped fruits and vegetables in the fields, markets and gardens using the weight predictions from the images. It is also a guideline for studies that follow the growth of fruit and vegetables according to their weight.

Keywords

• Artificial intelligence techniques
• image segmentation
• Fruits and vegetables
• Weight prediction

References

1. Akkol S, Akilli A & Cemal I (2017). Comparison of artificial neural network and multiple linear regression for prediction of live weight in hair goats. Yyu J. Agric. Sci 27: 21-29. DOI: 10.29133/yyutbd.263968
2. Alzubaidi L, Zhang J, Humaidi A J, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaria J, Fadhel M A, Al-Amidie M & Farhan L (2021). Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of big Data 8(1): 1-74. DOI: 10.1186/s40537-021-00444-8
3. Babajide O, Hissam T, Anna P, Anatoliy G, Astrup A, Alfredo Martinez J, Oppert J M & Sørensen T I (2020). A machine learning approach to short-term body weight prediction in a dietary intervention program. In Computational Science–ICCS 2020: 20th International Conference, Proceedings, Part IV 20 (pp. 441-455). DOI: 10.1007/978-3-030-50423-6_33
4. Bangdiwala S I (2018). Regression: simple linear. International journal of injury control and safety promotion 25(1): 113-115. DOI: 10.1080/17457300.2018.1426702
5. Barbole D K, Jadhav P M & Patil S B (2021). A review on fruit detection and segmentation techniques in agricultural field. In International Conference on Image Processing and Capsule Networks (pp. 269-288). Springer, Cham. DOI: 10.1007/978-3-030-84760-9_24
6. Bargoti S & Underwood J P (2017). Image segmentation for fruit detection and yield estimation in apple orchards. Journal of Field Robotics 34(6): 1039-1060. DOI:10.1002/rob.21699
7. Breiman L (2001). Random forests. Machine learning, 45: 5-32. DOI: 10.1023/A:1010933404324
8. Castro C A D O, Resende R T, Kuki K N, Carneiro V Q, Marcatti G E, Cruz C D & Motoike S Y (2017). High-performance prediction of macauba fruit biomass for agricultural and industrial purposes using Artificial Neural Networks. Industrial Crops and Products, 108: 806-813. DOI: 10.1016/j.indcrop.2017.07.031

9. Chicchón Apaza M Á, Monzón H M B & Alcarria R (2020). Semantic segmentation of weeds and crops in multispectral images by using a Convolutional Neural Networks based on u-net. In International Conference on Applied Technologies (pp. 473-485). Springer, Cham. DOI: 10.1007/978-3-030-42520-3_38
10. Cornelis C, Deschrijver G & Kerre E E (2006). Advances and challenges in interval-valued fuzzy logic. Fuzzy sets and systems, 157(5): 622-627. DOI: 10.1016/j.fss.2005.10.007
11. Duc N T, Ramlal A, Rajendran A, Raju D, Lal S K, Kumar S, Sahoo R N & Chinnusamy V (2023). Image-based phenotyping of seed architectural traits and prediction of seed weight using machine learning models in soybean. Frontiers in Plant Science, 14: 1206357. DOI: 10.3389/fpls.2023.1206357
12. Faisal M, Albogamy F, Elgibreen H, Algabri M & Alqershi F A (2020). Deep learning and computer vision for estimating date fruits type, maturity level, and weight. IEEE Access, 8: 206770-206782. DOI: 10.1109/ACCESS.2020.3037948
13. Fernandes A F, Turra E M, de Alvarenga É R, Passafaro T L, Lopes F B, Alves G F, Singh V & Rosa G J (2020). Deep Learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile tilapia. Computers and electronics in agriculture, 170: 105274. DOI: 10.1016/j.compag.2020.105274
14. Friha O, Ferrag M A, Shu L, Maglaras L & Wang X (2021). Internet of Things for the Future of Smart Agriculture: A Comprehensive Survey of Emerging Technologies, IEEE/CAA J. Autom. Sinica, 8(4): 718-752. DOI: 10.1109/JAS.2021.1003925
15. Gondchawar N & Kawitkar R S (2016). IoT based smart agriculture. International Journal of advanced research in Computer and Communication Engineering, 5(6): 838-842. DOI: 10.1088/1757-899X/1212/1/012047
16. Guo Y, Liu Y, Georgiou T & Lew M S (2018). A review of semantic segmentation using deep neural networks. International journal of multimedia information retrieval, 7(2): 87-93. DOI: 10.1007/s13735-017-0141-z
17. Han H G & Qiao J F (2013). A structure optimisation algorithm for feedforward neural network construction. Neurocomputing, 99: 347-357. DOI: 10.1016/j.neucom.2012.07.023
18. Huynh T, Tran L & Dao S (2020). Real-time size and mass estimation of slender axi-symmetric fruit/vegetable using a single top view image. Sensors, 20(18): 5406. DOI: 10.3390/s20185406
19. Jeong H, Moon H, Jeong Y, Kwon H, Kim C, Lee Y, Yang S M & Kim S (2024). Automated Technology for Strawberry Size Measurement and Weight Prediction Using AI. IEEE Access. 12: 14157-14167. DOI: 10.1109/ACCESS.2024.3356118
20. Kang H & Chen C (2020). Fruit detection, segmentation and 3D visualisation of environments in apple orchards. Computers and Electronics in Agriculture, 171: 105302. DOI: 10.1016/j.compag.2020.105302
21. Kassim M R M (2020). Iot applications in smart agriculture: Issues and challenges. In 2020 IEEE conference on open systems (ICOS) (pp. 19-24). IEEE. DOI: 10.1109/ICOS50156.2020.9293672
22. Kamiwaki Y & Fukuda S (2024). A Machine Learning-Assisted Three-Dimensional Image Analysis for Weight Estimation of Radish. Horticulturae, 10(2): 142. DOI: 10.3390/horticulturae10020142
23. Lee C Y (2023). Fruit Weight Predicting by Using Hybrid Learning. In International Conference on Technologies and Applications of Artificial Intelligence, (pp. 81-91). Singapore. DOI: 26912554
24. Li J, Sarma K V, Ho K C, Gertych A, Knudsen B S & Arnold C W (2017). A multi-scale u-net for semantic segmentation of histological images from radical prostatectomies. In AMIA Annual Symposium Proceedings. (pp. 1140). American Medical Informatics Association. DOI: PMC5977596
25. Lin B W, Yoshida D, Quinn J & Strehlow M (2009). A better way to estimate adult patients' weights. The American journal of emergency medicine, 27(9): 1060-1064. DOI: 10.1016/j.ajem.2008.08.018
26. Mahesh B (2020). Machine learning algorithms-a review. International Journal of Science and Research (IJSR), 9: 381-386. DOI: 10.21275/ART20203995
27. Naroui Rad M R, Ghalandarzehi A & Koohpaygani J A (2017). Predicting eggplant individual fruit weight using an artificial neural network. International Journal of Vegetable Science, 23(4): 331-339. DOI: 10.1080/19315260.2017.1290001
28. Nyalala I, Okinda C, Nyalala L, Makange N, Chao Q, Chao L, Yousaf K & Chen K (2019). Tomato volume and mass estimation using computer vision and machine learning algorithms: Cherry tomato model. Journal of Food Engineering, 263: 288-298. DOI: 10.1016/j.jfoodeng.2019.07.012
29. O'Grady M J, Langton D & O'Hare G M P (2019). Edge computing: A tractable model for smart agriculture. Artificial Intelligence in Agriculture, 3: 42-51. DOI: 10.1016/j.aiia.2019.12.001
30. Ozkaya S (2013). The prediction of live weight from body measurements on female Holstein calves by digital image analysis. The Journal of Agricultural Science, 151(4): 570-576. DOI: 10.1017/S002185961200086X
31. Park J, Kwak Y H, Jung J Y, Lee J H, Jang H Y, Kim H B & Hong K J (2012). A new age-based formula for estimating weight of Korean children. Resuscitation, 83(9): 1129-1134. DOI: https://doi.org/10.1016/j.resuscitation.2012.01.023
32. Pathan M, Patel N, Yagnik H & Shah M (2020). Artificial cognition for applications in smart agriculture: A comprehensive review. Artificial Intelligence in Agriculture, 4: 81-95. DOI: 10.1016/j.aiia.2020.06.001
33. Pekel E (2020). Estimation of soil moisture using decision tree regression. Theoretical and Applied Climatology, 139(3-4): 1111-1119. DOI: 10.1007/s00704-019-03048-8
34. Rad M R N, Fanaei H R & Rad M R P (2015). Application of Artificial Neural Networks to predict the final fruit weight and random forest to select important variables in native population of melon (Cucumis melo L.). Scientia Horticulturae, 181: 108-112. DOI: https://doi.org/10.1016/j.scienta.2014.10.025
35. Rozario L J, Rahman T & Uddin M S (2016). Segmentation of the region of defects in fruits and vegetables. International Journal of Computer Science and Information Security, 14(5): 399-406. https://www.researchgate.net/publication/304253402
36. Rudenko O, Megel Y, Bezsonov O & Rybalka A (2020). Cattle breed identification and live weight evaluation on the basis of machine learning and computer vision. In CMIS (pp.939-954). https://ceur-ws.org/Vol-2608/paper70.pdf
37. Teoh C C & Syaifudin A M (2007). Image processing and analysis techniques for estimating weight of Chokanan mangoes. Journal of Tropical Agriculture and Food Science, 35(1): 183. DOI: http://jtafs.mardi.gov.my/jtafs/35-1/Chokanan%20mangoes.pdf
38. Xiao J & Zhou Z (2020). Research progress of RNN language model. In 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) (pp. 1285-1288). IEEE. DOI: 10.1109/ICAICA50127.2020.9182390
39. Xu J, Lu Y, Olaniyi E & Harvey L (2024). Online volume measurement of sweetpotatoes by A LiDAR-based machine vision system. Journal of Food Engineering, 361: 111725. DOI: 10.1016/j.jfoodeng.2023.111725
40. Xu W, Yang W, Chen S, Wu C, Chen P & Lan Y (2020). Establishing a model to predict the single boll weight of cotton in northern Xinjiang by using high resolution UAV remote sensing data. Computers and Electronics in Agriculture, 179: 105762. DOI: https://doi.org/10.1016/j.compag.2020.105762
41. Wu C H, Ho J M & Lee D T (2004). Travel-time prediction with support vector regression. IEEE transactions on intelligent transportation systems, 5(4): 276-281. DOI: 10.1109/TITS.2004.837813
42. Yan Q, Ding L, Wei H, Wang X, Jiang C & Degen A (2019). Body weight estimation of yaks using body measurements from image analysis. Measurement, 140: 76-80. DOI: 10.1016/j.measurement.2019.03.021
43. Ying-kai L, Feng-nan S, Qiao C, Ming-wei X, Chen-di L, Wen-tao L & Xue-cheng Z (2023). Dragon fruit weight estimation based on machine vision and machine learning. Food and Machinery, 39(7): 99-103. DOI: https://www.ifoodmm.cn/journal/vol39/iss7/15/
44. Yu Y, Si X, Hu C & Zhang J (2019). A review of recurrent neural networks: LSTM cells and network architectures. Neural computation 31(7): 1235-1270. DOI: 10.1162/neco_a_01199
45. Zhou X Y & Yang G Z (2019). Normalization in training U-Net for 2-D biomedical semantic segmentation. IEEE Robotics and Automation Letters, 4(2): 1792-1799. DOI: 10.1109/LRA.2019.2896518