Comparative analysis of mature tomato detection by feature extraction and machine learning for autonomous greenhouse robots

Abstract

Accurate detection of tomatoes grown in greenhouses is important for timely harvesting. In this way, it is ensured that mature tomatoes are collected by distinguishing them from the unripe ones. Insufficient light, occlusion, and overlapping adversely affect the detection of mature tomatoes. In addition, it is time consuming for people to detect mature tomatoes at certain periods in large greenhouses. For these reasons, high-performance automatic detection of tomatoes by greenhouse robots has become an increasingly studied area today. In this paper, two feature extraction methods, histogram of oriented gradients (HOG) and local binary patterns (LBP), which are effective in object recognition, and two important and commonly used classifiers of machine learning, support vector machines (SVM) and k-nearest neighbor (kNN), are comparatively used to detect and count tomatoes. The HOG and LBP features are classified separately and together by SVM or kNN, and the success of each case are compared. Performance of the detection is improved by eliminating false positive results at the postprocessing stage using color information.

Keywords

• Tomato detection
• harvesting robots
• machine learning
• smart agriculture

References

1. Liu, G., Mao, S., Kim, J. H., A mature-tomato detection algorithm using machine learning and color analysis, Sensors 2019, 19 (2023), https://doi.org/10.3390/s19092023.
2. Onishi, Y., Yoshida, T., Kurita, H. et al., An automated fruit harvesting robot by using deep learning, Robomech J., 6 (13) (2019).
3. Selvaraj, A., Shebiah, N., Nidhyananthan, S., Ganesan, L., Fruit recognition using color and texture features, J. Emerg. Trends Comput. Inf. Sci., 1 (2010), 90-94.
4. Bulanon, D. M., Kataoka, T., Ota, Y., Hiroma, T., AE-Automation and emerging technologies: A segmentation algorithm for the automatic recognition of fuji apples at harvest, Biosyst. Eng., 83 (4) (2002), 405-412.
5. Liu, X., Zhao, D., Jia, W., Ji, W., & Sun, Y., A detection method for apple fruits based on color and shape features, IEEE Access, 7 (2019), 67923–67933.
6. Mao, W., Ji, B., Zhan, J., Zhang, X. and Hu, X., Apple location method for the apple harvesting robot, 2009 2nd International Congress on Image and Signal Processing, (2009),1-5.
7. Tanigaki, K., Fujiura, T., Akase, A., Imagawa, J., Cherry-harvesting robot, Comput. Electron. Agric., 63 (1) (2008), 65-72.
8. Ji, W., Zhao, D., Cheng, F., Xu, B., Zhang, Y., Wang, J., Automatic recognition vision system guided for apple harvesting robot, Comput. Elec. Eng., 38 (5) (2012), 1186-1195.

9. Kurtulmus, F., Lee, W.S., Vardar, A., Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network, Precision Agric., 15 (2014) 57–79.
10. Song, Y., Glasbey, C. A., Horgan, G. W., Polder, G., Dieleman, J. A., van der Heijden, G. W. A. M., Automatic fruit recognition and counting from multiple images, Biosyst. Eng., 118 (2014), 203-215, https://doi.org/10.1016/j.biosystemseng.2013.12.008.
11. Islam, M. A., Yousuf, Md. S. I., Billah, M. M., Automatic plant detection using HOG and LBP features with SVM, Int. J. Comput., 33 (1) (2019), 26-38.
12. Hummel, R. A., Image enhancement by histogram transformation, Comput. Graphics Image Process., 6 (1977), 184-195.
13. Ketcham, D. J., Lowe, R. W. and Weber, J. W., Real-time image enhancement techniques, Seminar on Image Processing, (1976), 1-6.
14. Pizer, S. M., Intensity mappings for the display of medical images, Functional Mapping of Organ Systems and Other Computer Topics, Society of Nuclear Medicine (1981).
15. Pizer, S. M., Amburn, E. P., Austin, J. D., et al., Adaptive histogram equalization and its variations, Computer Vision, Graphics, and Image Processing, 39 (1987), 355-368.
16. Maison, Lestari, T., Luthfi, A., Retinal blood vessel segmentation using gaussian filter, J. Phys.: Conf. Ser., 1376 (2019), 012023, https://doi.org/10.1088/1742-6596/1376/1/012023.
17. Umri, B. K., Utami, E. and Kurniawan, M. P., Comparative analysis of CLAHE and AHE on application of CNN algorithm in the detection of Covid-19 patients, 2021 4th Int. Conf. on Inf. and Comm. Tech. (ICOIACT), (2021), 203-208, https://doi.org/10.1109/ICOIACT53268.2021.9563980.
18. Wang, Q., Lu, Y., Zhang, X., Hahn, J., Region of interest selection for functional features, Neurocomputing, 422 (2021), 235-244.
19. Zhang, L., Sun, Q. and Zhang, J., Region of interest extraction via common salient feature analysis and feedback reinforcement strategy for remote sensing images, GI Science Remote Sens., 55 (5) (2018), 745-762.
20. Vogt, P., Riitters, K. H., Estreguil, C., Kozak, J., Wade, T. G., Wickham, J. D. , Mapping spatial patterns with morphological, Image Processing, 22 (2) (2007), 171-177.
21. [21] Dalal, N. and Triggs, B., Histograms of oriented gradients for human detection, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), (2005), 886-893.
22. Tuncer, T. and Avcı, E., Yerel ikili örüntü tabanlı veri gizleme algoritması: LBP-LSB, Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 10 (1) (2017) 48-53.
23. Chen, J., Kellokumpu, V., Zhao, G., Pietikainen, M., RLBP: Robust Local Binary Pattern, Proceedings British Machine Vision Conference 2013, 122 (2014),1-11, http://dx.doi.org/10.5244/C.27.122.
24. Cortes, C. and Vapnik, V., Support-Vector Networks, Mach. Learn., 20 (1995), 273-297, http://dx.doi.org/10.1007/BF00994018.
25. [Kuang, Q. and Zhao, L., A practical GPU based KNN algorithm, Proceedings of the Second Symposium International Computer Science and Computational Technology (ISCSCT '09), (2009), 151-155.
26. Li, H., Chen, L., Removal of false positive in object detection with contour-based classifiers, 2010 IEEE International Conference on Image Processing, (2010), 3941-3944.
27. Rothe, R., Guillaumin, M., Van Gool, L., Non-maximum suppression for object detection by passing messages between windows, Computer Vision ACCV 2014, 9003 (2015), 290-306, https://doi.org/10.1007/978-3-319-16865-4_19.
28. Salscheider, N. O., FeatureNMS: Non-maximum suppression by learning feature embeddings, 25th International Conference on Pattern Recognition (ICPR), (2020), 7848-7854.
29. Liu, G., Mao, S., Open tomatoes dataset, (2019). Available at: https://github.com/pandalgx/Tomato-dataset. [Accessed February 2023].