TRANSFORMER NETWORKS TO CLASSIFY WEEDS AND CROPS IN HIGH-RESOLUTION AERIAL IMAGES FROM NORTH-EAST SERBIA

Abstract

The intricate backgrounds present in crop and field images, coupled with the minimal contrast between weedinfested areas and the background, can lead to considerable ambiguity. This, in turn, poses a significant challenge to the resilience and precision of crop identification models. Identifying and mapping weeds are pivotal stages in weed control, essential for maintaining crop health. A multitude of research efforts underscore the significance of leveraging remote sensing technologies and sophisticated machine learning algorithms to enhance weed management strategies. Deep learning techniques have demonstrated impressive effectiveness in a range of agricultural remote sensing applications, including plant classification and disease detection. High-resolution imagery was collected using a UAV equipped with a high-resolution camera, which was strategically deployed over weed, sunflower, tobacco and maize fields to collect data. The VIT models achieved commendable levels of accuracy, with test accuracies of 92.97% and 90.98% in their respective evaluations. According to the experimental results, transformers not only excel in crop classification accuracy, but also achieve higher accuracy with a smaller sample size. Swin-B16 achieved an accuracy of 91.65% on both the training and test datasets. Compared to the other two ViT models, the loss value is significantly lower by half, at 0.6450.

Keywords

• agriculture
• drone
• image classification
• multi-head attention
• remote sensing
• vision transformers

References

1. Abdalla, A., H. Cen, L. Wan, R. Rashid, H. Weng, W. Zhou and Y. He. 2019. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Comput Electron Agric 167. https://doi.org/10.1016/j.compag.2019.105091
2. Alzahrani, M.S., F.W. Alsaade. 2023. Transform and Deep Learning Algorithms for the Early Detection and Recognition of Tomato Leaf Disease. Agronomy 13. https://doi.org/10.3390/agronomy13051184
3. Bazi, Y., L. Bashmal, M. M. A. Rahhal, R. A. Dayil and N.A. Ajlan. 2021. Vision transformers for remote sensing image classification. Remote Sensing, 13(3), 516.
4. Beyer, L., Zhai, X., Kolesnikov, A., 2022. Better plain ViT baselines for ImageNet-1k.
5. Culpan, E. 2023. Effect of sowing dates on seed yield, yield traits and oil content of safflower in Northwest Turkey. Turkish Journal of Field Crops, 28(1), 87-93.
6. Czymmek, V., L. O. Harders, F. J. Knoll and S. Hussmann. 2019. Vision-based deep learning approach for real-time detection of weeds in organic farming. In 2019 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) (pp. 1-5). IEEE.
7. dos Santos Ferreira, A., D.M. Freitas, G.G. da Silva, H. Pistori, M.T. Folhes. 2019. Unsupervised deep learning and semi-automatic data labeling in weed discrimination. Comput Electron Agric 165, 104963. https://doi.org/10.1016/J.COMPAG.2019.104963
8. dos Santos Ferreira, A., D. Matte Freitas, G. Goncalves da Silva, H. Pistori, M. Theophilo Folhes, 2017. Weed detection in soybean crops using ConvNets. Comput Electron Agric 143, 314–324. https://doi.org/10.1016/j.compag.2017.10.027

9. Han, K., Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu, Z. Yang, Y. Zhang and D. Tao. 2020. A Survey on Visual Transformer. https://doi.org/10.1109/TPAMI.2022.3152247
10. Hand, D. J. 2009. “Measuring Classifier Performance: A Coherent Alternative to the Area under the ROC Curve.” Machine Learning 77 (1): 103–23. doi:10.1007/s10994-009-5119-5.
11. Hasan, A.S., M.M., F. Sohel, D. Diepeveen, H. Laga, M.G.K. Jones. 2021. A survey of deep learning techniques for weed detection from images. Comput Electron Agric. https://doi.org/10.1016/j.compag.2021.106067
12. He, K., X. Zhang, S. Ren and J. Sun. 2016. Deep residual learning for image recognition, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
13. Huang, H., J. Deng, Y. Lan, A. Yang, X. Deng, S. Wen, H. Zhang and Zhang, Y., 2018. Accurate weed mapping and prescription map generation based on fully convolutional networks using UAV imagery. Sensors (Switzerland) 18. https://doi.org/10.3390/s18103299
14. Iqbal, N., S. Manalil, B.S. Chauhan and S.W. Adkins. 2019. “Investigation of Alternate Herbicides for Effective Weed Management in Glyphosate-Tolerant Cotton.” Archives of Agronomy and Soil Science 65 (13). Taylor and Francis Ltd.: 1885–99. doi:10.1080/03650340.2019.1579904.
15. Iqbal, N., S. Manalil, B.S. Chauhan, S.W. Adkins. 2019. Investigation of alternate herbicides for effective weed management in glyphosate-tolerant cotton. Arch Agron Soil Sci 65, 1885–1899. https://doi.org/10.1080/03650340.2019.1579904
16. Kang, J., L. Liu, F. Zhang, C. Shen, N. Wang, L. Shao. 2021. Semantic segmentation model of cotton roots in-situ image based on attention mechanism. Comput Electron Agric 189. https://doi.org/10.1016/j.compag.2021.106370
17. Kayin, G.B., H. Kayin, A.T. Goksoy. 2024. Effects of Plant Density on Micronutrient Uptake in Sunflower (Helianthus annuus L.) Varieties. Turkish Journal of Field Crops 29, 9–17. https://doi.org/10.17557/tjfc.1349344
18. Lecun, Y., Y. Bengio, G. Hinton. 2015. Deep learning. Nature. https://doi.org/10.1038/nature14539
19. Li, X. and S. Li. 2022. Transformer Help CNN See Better: A Lightweight Hybrid Apple Disease Identification Model Based on Transformers. Agriculture (Switzerland) 12. https://doi.org/10.3390/agriculture12060884
20. Louargant, M., S. Villette, G. Jones, N. Vigneau, J.N. Paoli and C. Gée. 2017. Weed detection by UAV: simulation of the impact of spectral mixing in multispectral images. Precis Agric 18, 932–951. https://doi.org/10.1007/s11119-017-9528-3
21. Lu, Y. and S. Young. 2020. A survey of public datasets for computer vision tasks in precision agriculture. Comput Electron Agric. https://doi.org/10.1016/j.compag.2020.105760
22. Ma, H., L. Zhao, B. Li, R. Niu, and Y. Wang. 2023. Change Detection Needs Neighborhood Interaction in Transformer. Remote Sens (Basel) 15. https://doi.org/10.3390/rs15235459
23. Madsen, S.L., S.K. Mathiassen, M. Dyrmann, M.S. Laursen, L.C. Paz and R.N. Jørgensen. 2020. Open plant phenotype database of common weeds in Denmark. Remote Sens (Basel) 12. https://doi.org/10.3390/RS12081246
24. Maurício, J., I. Domingues and J. Bernardino. 2023. Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Applied Sciences (Switzerland). https://doi.org/10.3390/app13095521
25. Niu, Z., G. Zhong and H. Yu. 2021. A review on the attention mechanism of deep learning. Neurocomputing 452, 48–62. https://doi.org/10.1016/j.neucom.2021.03.091
26. Ozcift, A. and A. Gulten. 2011. Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms. Comput Methods Programs Biomed 104, 443–451. https://doi.org/10.1016/J.CMPB.2011.03.018
27. Radoglou-Grammatikis, P., P. Sarigiannidis, T. Lagkas and I. Moscholios. 2020. A compilation of UAV applications for precision agriculture. Computer Networks 172, 107148. https://doi.org/10.1016/J.COMNET.2020.107148
28. Reedha, R., E. Dericquebourg, R. Canals and A. Hafiane.2022. Transformer Neural Network for Weed and Crop Classification of High Resolution UAV Images. Remote Sens (Basel) 14. https://doi.org/10.3390/rs14030592
29. Shin, H., S. Jeon, Y. Seol, S. Kimand D. Kang. 2023. Vision Transformer Approach for Classification of Alzheimer’s Disease Using 18F-Florbetaben Brain Images. Applied Sciences (Switzerland) 13. https://doi.org/10.3390/app13063453
30. Suh, H.K., J. IJsselmuiden, J.W. Hofstee and van E.J. Henten. 2018. Transfer learning for the classification of sugar beet and volunteer potato under field conditions. Biosyst Eng 174, 50–65. https://doi.org/10.1016/j.biosystemseng.2018.06.017
31. Sunil, C.K., C.D. Jaidhar and N. Patil. 2023. Systematic study on deep learning-based plant disease detection or classification. Artif Intell Rev 56, 14955–15052. https://doi.org/10.1007/s10462-023-10517-0
32. Suravarapu, V.K., and H.Y. Patil. 2023. Person Identification and Gender Classification Based on Vision Transformers for Periocular Images. Applied Sciences (Switzerland) 13. https://doi.org/10.3390/app13053116
33. Szegedy, C., V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna. 2016. Rethinking the Inception Architecture for Computer Vision, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 2818–2826. https://doi.org/10.1109/CVPR.2016.308
34. Ustuner, T., A. Sakran and K. Almhemed. 2020. Effect of Herbicides on Living Organisms in The Ecosystem and Available Alternative Control Methods. International Journal of Scientific and Research Publications (IJSRP) 10, 622–632. https://doi.org/10.29322/ijsrp.10.08.2020.p10480
35. Thakur, P.S., S. Chaturvedi, P. Khanna, T. Sheorey and A. Ojha. 2023. Vision transformer meets convolutional neural network for plant disease classification. Ecol Inform 77. https://doi.org/10.1016/j.ecoinf.2023.102245
36. Vaswani, A., G. Brain, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser and .I. Polosukhin, 2017. Attention Is All You Need.
37. Vrbničanin, S., D. Pavlović and D. Božić. 2017. Weed Resistance to Herbicides, in: Herbicide Resistance in Weeds and Crops. InTech. https://doi.org/10.5772/67979
38. Wang, H., W. Chang, Y. Yao, Z. Yao, Y. Zhao, S. Li, Z. Liu and X. Zhang. 2023. Cropformer: A new generalized deep learning classification approach for multi-scenario crop classification. Front Plant Sci 14. https://doi.org/10.3389/fpls.2023.1130659
39. Wu, X., S. Aravecchia, P. Lottes, C. Stachniss and C. Pradalier. 2020. Robotic weed control using automated weed and crop classification. J Field Robot 37, 322–340. https://doi.org/10.1002/rob.21938
40. Xia, Z., X. Pan, S. Song, L. Erran Li and G. Huang, 2022. Vision Transformer with Deformable Attention.
41. Zhai, X., A. Kolesnikov, N. Houlsby and L. Beyer. 2021. Scaling Vision Transformers.
42. Zhao, J., T.W. Berge and J. Geipel. 2023. Transformer in UAV Image-Based Weed Mapping. Remote Sens (Basel) 15. https://doi.org/10.3390/rs15215165