TRANSFORMER NETWORKS TO CLASSIFY WEEDS AND CROPS IN HIGH-RESOLUTION AERIAL IMAGES FROM NORTH-EAST SERBIA

Year 2024, Volume: 29 Issue: 2, 112 - 120, 24.12.2024

Fatih Celik , Füsun Balık Şanlı , Dragana Boziç

https://doi.org/10.17557/tjfc.1511404

Abstract

The intricate backgrounds present in crop and field images, coupled with the minimal contrast between weedinfested areas and the background, can lead to considerable ambiguity. This, in turn, poses a significant
challenge to the resilience and precision of crop identification models. Identifying and mapping weeds are pivotal
stages in weed control, essential for maintaining crop health. A multitude of research efforts underscore the
significance of leveraging remote sensing technologies and sophisticated machine learning algorithms to enhance
weed management strategies. Deep learning techniques have demonstrated impressive effectiveness in a range
of agricultural remote sensing applications, including plant classification and disease detection. High-resolution
imagery was collected using a UAV equipped with a high-resolution camera, which was strategically deployed
over weed, sunflower, tobacco and maize fields to collect data. The VIT models achieved commendable levels of
accuracy, with test accuracies of 92.97% and 90.98% in their respective evaluations. According to the
experimental results, transformers not only excel in crop classification accuracy, but also achieve higher
accuracy with a smaller sample size. Swin-B16 achieved an accuracy of 91.65% on both the training and test
datasets. Compared to the other two ViT models, the loss value is significantly lower by half, at 0.6450.

Keywords

agriculture, drone, image classification, multi-head attention, remote sensing, vision transformers

References

• Abdalla, A., H. Cen, L. Wan, R. Rashid, H. Weng, W. Zhou and Y. He. 2019. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Comput Electron Agric 167. https://doi.org/10.1016/j.compag.2019.105091
• Alzahrani, M.S., F.W. Alsaade. 2023. Transform and Deep Learning Algorithms for the Early Detection and Recognition of Tomato Leaf Disease. Agronomy 13. https://doi.org/10.3390/agronomy13051184
• Bazi, Y., L. Bashmal, M. M. A. Rahhal, R. A. Dayil and N.A. Ajlan. 2021. Vision transformers for remote sensing image classification. Remote Sensing, 13(3), 516.
• Beyer, L., Zhai, X., Kolesnikov, A., 2022. Better plain ViT baselines for ImageNet-1k.
• Culpan, E. 2023. Effect of sowing dates on seed yield, yield traits and oil content of safflower in Northwest Turkey. Turkish Journal of Field Crops, 28(1), 87-93.
• Czymmek, V., L. O. Harders, F. J. Knoll and S. Hussmann. 2019. Vision-based deep learning approach for real-time detection of weeds in organic farming. In 2019 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) (pp. 1-5). IEEE.
• dos Santos Ferreira, A., D.M. Freitas, G.G. da Silva, H. Pistori, M.T. Folhes. 2019. Unsupervised deep learning and semi-automatic data labeling in weed discrimination. Comput Electron Agric 165, 104963. https://doi.org/10.1016/J.COMPAG.2019.104963
• dos Santos Ferreira, A., D. Matte Freitas, G. Goncalves da Silva, H. Pistori, M. Theophilo Folhes, 2017. Weed detection in soybean crops using ConvNets. Comput Electron Agric 143, 314–324. https://doi.org/10.1016/j.compag.2017.10.027
• Han, K., Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu, Z. Yang, Y. Zhang and D. Tao. 2020. A Survey on Visual Transformer. https://doi.org/10.1109/TPAMI.2022.3152247
• Hand, D. J. 2009. “Measuring Classifier Performance: A Coherent Alternative to the Area under the ROC Curve.” Machine Learning 77 (1): 103–23. doi:10.1007/s10994-009-5119-5.
• Hasan, A.S., M.M., F. Sohel, D. Diepeveen, H. Laga, M.G.K. Jones. 2021. A survey of deep learning techniques for weed detection from images. Comput Electron Agric. https://doi.org/10.1016/j.compag.2021.106067
• He, K., X. Zhang, S. Ren and J. Sun. 2016. Deep residual learning for image recognition, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
• Huang, H., J. Deng, Y. Lan, A. Yang, X. Deng, S. Wen, H. Zhang and Zhang, Y., 2018. Accurate weed mapping and prescription map generation based on fully convolutional networks using UAV imagery. Sensors (Switzerland) 18. https://doi.org/10.3390/s18103299
• Iqbal, N., S. Manalil, B.S. Chauhan and S.W. Adkins. 2019. “Investigation of Alternate Herbicides for Effective Weed Management in Glyphosate-Tolerant Cotton.” Archives of Agronomy and Soil Science 65 (13). Taylor and Francis Ltd.: 1885–99. doi:10.1080/03650340.2019.1579904.
• Iqbal, N., S. Manalil, B.S. Chauhan, S.W. Adkins. 2019. Investigation of alternate herbicides for effective weed management in glyphosate-tolerant cotton. Arch Agron Soil Sci 65, 1885–1899. https://doi.org/10.1080/03650340.2019.1579904
• Kang, J., L. Liu, F. Zhang, C. Shen, N. Wang, L. Shao. 2021. Semantic segmentation model of cotton roots in-situ image based on attention mechanism. Comput Electron Agric 189. https://doi.org/10.1016/j.compag.2021.106370
• Kayin, G.B., H. Kayin, A.T. Goksoy. 2024. Effects of Plant Density on Micronutrient Uptake in Sunflower (Helianthus annuus L.) Varieties. Turkish Journal of Field Crops 29, 9–17. https://doi.org/10.17557/tjfc.1349344
• Lecun, Y., Y. Bengio, G. Hinton. 2015. Deep learning. Nature. https://doi.org/10.1038/nature14539
• Li, X. and S. Li. 2022. Transformer Help CNN See Better: A Lightweight Hybrid Apple Disease Identification Model Based on Transformers. Agriculture (Switzerland) 12. https://doi.org/10.3390/agriculture12060884
• Louargant, M., S. Villette, G. Jones, N. Vigneau, J.N. Paoli and C. Gée. 2017. Weed detection by UAV: simulation of the impact of spectral mixing in multispectral images. Precis Agric 18, 932–951. https://doi.org/10.1007/s11119-017-9528-3
• Lu, Y. and S. Young. 2020. A survey of public datasets for computer vision tasks in precision agriculture. Comput Electron Agric. https://doi.org/10.1016/j.compag.2020.105760
• Ma, H., L. Zhao, B. Li, R. Niu, and Y. Wang. 2023. Change Detection Needs Neighborhood Interaction in Transformer. Remote Sens (Basel) 15. https://doi.org/10.3390/rs15235459
• Madsen, S.L., S.K. Mathiassen, M. Dyrmann, M.S. Laursen, L.C. Paz and R.N. Jørgensen. 2020. Open plant phenotype database of common weeds in Denmark. Remote Sens (Basel) 12. https://doi.org/10.3390/RS12081246
• Maurício, J., I. Domingues and J. Bernardino. 2023. Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Applied Sciences (Switzerland). https://doi.org/10.3390/app13095521
• Niu, Z., G. Zhong and H. Yu. 2021. A review on the attention mechanism of deep learning. Neurocomputing 452, 48–62. https://doi.org/10.1016/j.neucom.2021.03.091
• Ozcift, A. and A. Gulten. 2011. Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms. Comput Methods Programs Biomed 104, 443–451. https://doi.org/10.1016/J.CMPB.2011.03.018
• Radoglou-Grammatikis, P., P. Sarigiannidis, T. Lagkas and I. Moscholios. 2020. A compilation of UAV applications for precision agriculture. Computer Networks 172, 107148. https://doi.org/10.1016/J.COMNET.2020.107148
• Reedha, R., E. Dericquebourg, R. Canals and A. Hafiane.2022. Transformer Neural Network for Weed and Crop Classification of High Resolution UAV Images. Remote Sens (Basel) 14. https://doi.org/10.3390/rs14030592
• Shin, H., S. Jeon, Y. Seol, S. Kimand D. Kang. 2023. Vision Transformer Approach for Classification of Alzheimer’s Disease Using 18F-Florbetaben Brain Images. Applied Sciences (Switzerland) 13. https://doi.org/10.3390/app13063453
• Suh, H.K., J. IJsselmuiden, J.W. Hofstee and van E.J. Henten. 2018. Transfer learning for the classification of sugar beet and volunteer potato under field conditions. Biosyst Eng 174, 50–65. https://doi.org/10.1016/j.biosystemseng.2018.06.017
• Sunil, C.K., C.D. Jaidhar and N. Patil. 2023. Systematic study on deep learning-based plant disease detection or classification. Artif Intell Rev 56, 14955–15052. https://doi.org/10.1007/s10462-023-10517-0
• Suravarapu, V.K., and H.Y. Patil. 2023. Person Identification and Gender Classification Based on Vision Transformers for Periocular Images. Applied Sciences (Switzerland) 13. https://doi.org/10.3390/app13053116
• Szegedy, C., V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna. 2016. Rethinking the Inception Architecture for Computer Vision, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 2818–2826. https://doi.org/10.1109/CVPR.2016.308
• Ustuner, T., A. Sakran and K. Almhemed. 2020. Effect of Herbicides on Living Organisms in The Ecosystem and Available Alternative Control Methods. International Journal of Scientific and Research Publications (IJSRP) 10, 622–632. https://doi.org/10.29322/ijsrp.10.08.2020.p10480
• Thakur, P.S., S. Chaturvedi, P. Khanna, T. Sheorey and A. Ojha. 2023. Vision transformer meets convolutional neural network for plant disease classification. Ecol Inform 77. https://doi.org/10.1016/j.ecoinf.2023.102245
• Vaswani, A., G. Brain, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser and .I. Polosukhin, 2017. Attention Is All You Need.
• Vrbničanin, S., D. Pavlović and D. Božić. 2017. Weed Resistance to Herbicides, in: Herbicide Resistance in Weeds and Crops. InTech. https://doi.org/10.5772/67979
• Wang, H., W. Chang, Y. Yao, Z. Yao, Y. Zhao, S. Li, Z. Liu and X. Zhang. 2023. Cropformer: A new generalized deep learning classification approach for multi-scenario crop classification. Front Plant Sci 14. https://doi.org/10.3389/fpls.2023.1130659
• Wu, X., S. Aravecchia, P. Lottes, C. Stachniss and C. Pradalier. 2020. Robotic weed control using automated weed and crop classification. J Field Robot 37, 322–340. https://doi.org/10.1002/rob.21938
• Xia, Z., X. Pan, S. Song, L. Erran Li and G. Huang, 2022. Vision Transformer with Deformable Attention.
• Zhai, X., A. Kolesnikov, N. Houlsby and L. Beyer. 2021. Scaling Vision Transformers.
• Zhao, J., T.W. Berge and J. Geipel. 2023. Transformer in UAV Image-Based Weed Mapping. Remote Sens (Basel) 15. https://doi.org/10.3390/rs15215165