Deep learning-based vehicle detection from orthophoto and spatial accuracy analysis

Abstract

Deep Learning algorithms are used by many different disciplines for various purposes, thanks to their ever-developing data processing skills. Convolutional neural network (CNN) are generally developed and used for this integration purpose. On the other hand, the widespread usage of Unmanned Aerial Vehicles (UAV) enables the collection of aerial photographs for Photogrammetric studies. In this study, these two fields were brought together and it was aimed to find the equivalents of the objects detected from the UAV images using deep learning in the global coordinate system and to evaluate their accuracy over these values. For these reasons, v3 and v4 versions of the YOLO algorithm, which prioritizes detecting the midpoint of the detected object, were trained in Google Colab’s virtual machine environment using the prepared data set. The coordinate values read from the orthophoto and the coordinate values of the midpoints of the objects, which were derived according to the estimations made by the YOLO-v3 and YOLOV4-CSP models, were compared and their spatial accuracy was calculated. Accuracy of 16.8 cm was obtained with the YOLO-v3 and 15.5 cm with the YOLOv4-CSP. In addition, the mAP value was obtained as 80% for YOLOv3 and 87% for YOLOv4-CSP. F1-score is 80% for YOLOv3 and 85% for YOLOv4-CSP.

Keywords

• UAV
• Photogrammetry
• Deep Learning
• Orthophoto
• Object Detection

References

1. Luo, X., Tian, X., Zhang, H., Hou, W., Leng, G., Xu, W., ... & Zhang, J. (2020). Fast automatic vehicle detection in uav images using convolutional neural networks. Remote Sensing, 12(12), 1994.
2. Lu, J., Ma, C., Li, L., Xing, X., Zhang, Y., Wang, Z., & Xu, J. (2018). A vehicle detection method for aerial image based on YOLO. Journal of Computer and Communications, 6(11), 98-107.
3. Sang, J., Wu, Z., Guo, P., Hu, H., Xiang, H., Zhang, Q., & Cai, B. (2018). An improved YOLOv2 for vehicle detection. Sensors, 18(12), 4272.
4. Jazayeri, A., Cai, H., Zheng, J. Y., & Tuceryan, M. (2011). Vehicle detection and tracking in car video based on motion model. IEEE Transactions on Intelligent Transportation Systems, 12(2), 583-595.
5. Senkal, E., Kaplan, G., & Avdan, U. (2021). Accuracy assessment of digital surface models from unmanned aerial vehicles’ imagery on archaeological sites. International Journal of Engineering and Geosciences, 6(2), 81-89.
6. Şasi A., & Yakar, M. (2018). Photogrammetric modelling of hasbey dar'ülhuffaz (masjid) using an unmanned aerial vehicle. International Journal of Engineering and Geosciences, 3(1), 6-11.
7. Ulvi, A., Yakar, M., Yigit, A. Y., & Kaya, Y. (2020). İha ve Yersel Fotogrametrik Teknikler Kullanarak Aksaray Kızıl Kilisenin 3b Modelinin ve Nokta Bulutunun Elde Edilmesi. Geomatik, 5(1), 19-26.
8. Gönültaş, F., Atik, M. E. & Duran, Z. (2020). Extraction of Roof Planes from Different Point Clouds Using RANSAC Algorithm. International Journal of Environment and Geoinformatics, 7 (2), 165-171. https://doi.org/10.30897/ijegeo.715510

9. Atik, M. E., Donmez, S. O., Duran, Z., & İpbuker, C. (2018). Comparison Of Automatic Feature Extraction Methods for Building Roof Planes by Using Airborne Lidar Data and High-Resolution Satellite Image. In Proceedings, 7th International Conference on Cartography and GIS (pp. 857-863), 18-23 June 2018, Sozopol, Bulgaria.
10. Ulvi, A., Yakar, M., Yiğit, A. Y. & Kaya, Y. (2020). İha ve Yersel Fotogrametrik Teknikler Kullanarak Aksaray Kızıl Kilisenin 3b Modelinin ve Nokta Bulutunun Elde Edilmesi. Geomatik, 5 (1), 19-26. https://doi.org/10.29128/geomatik.560179
11. Agca. M., Gültekin, N., & Kaya, E. (2020). İnsansız hava aracından elde edilen veriler ile kaya düşme potansiyelinin değerlendirilmesi: Adam Kayalar örneği, Mersin. Geomatik, 5(2), 134-145.
12. Alptekin, A. & Yakar, M. (2020). Determination of pond volume with using an unmanned aerial vehicle. Mersin Photogrammetry Journal, 2 (2), 59-63.
13. Aykut, N. O. (2019). İnsansız Hava Araçlarının Kıyı Çizgisinin Belirlenmesinde Kullanılabilirliğinin Araştırılması. Geomatik, 4(2), 141-146.
14. Atik, S. O., & Ipbuker, C. (2021). Integrating Convolutional Neural Network and Multiresolution Segmentation for Land Cover and Land Use Mapping Using Satellite Imagery. Applied Sciences, 11(12), 5551.
15. Atik, M. E., Duran, Z., & Özgünlük, R. (2022). Comparison of YOLO Versions for Object Detection from Aerial Images. International Journal of Environment and Geoinformatics, 9(2), 87-93.
16. Bengio, Y., Goodfellow, I., & Courville, A. (2017). Deep learning (Vol. 1). Cambridge, MA, USA: MIT press.
17. Tasdemir, S., & Ozkan, I. A. (2019). ANN approach for estimation of cow weight depending on photogrammetric body dimensions. International Journal of Engineering and Geosciences, 4(1), 36-44.
18. Sariturk, B., Bayram, B., Duran, Z., & Seker, D. Z. (2020). Feature extraction from satellite images using segnet and fully convolutional networks (FCN). International Journal of Engineering and Geosciences, 5(3), 138-143.
19. Cepni, S., Atik, M. E., & Duran, Z. (2020). Vehicle detection using different deep learning algorithms from image sequence. Baltic Journal of Modern Computing, 8(2), 347-358.
20. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).
21. Zhao, X., Pu, F., Wang, Z., Chen, H., & Xu, Z. (2019). Detection, tracking, and geolocation of moving vehicle from uav using monocular camera. IEEE Access, 7, 101160-101170.
22. Atik, S. O., & Ipbuker, C. (2020). Instance Segmentation of Crowd Detection in The Camera Images. In Proceeding of 41st Asian Conference on Remote Sensing 2020 (ACRS2020), Deqing City, Virtual, 9-11 November 2020.
23. Cazzato, D., Cimarelli, C., Sanchez-Lopez, J. L., Voos, H., & Leo, M. (2020). A survey of computer vision methods for 2d object detection from unmanned aerial vehicles. Journal of Imaging, 6(8), 78.
24. Božić-Štulić, D., Kružić, S., Gotovac, S., & Papić, V. (2018). Complete model for automatic object detection and localisation on aerial images using convolutional neural networks. Journal of Communications Software and Systems, 14(1), 82-90.
25. Liu, M., Wang, X., Zhou, A., Fu, X., Ma, Y., & Piao, C. (2020). Uav-yolo: Small object detection on unmanned aerial vehicle perspective. Sensors, 20(8), 2238.
26. Zhang, H., Wang, G., Lei, Z., & Hwang, J. N. (2019, October). Eye in the sky: Drone-based object tracking and 3d localization. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 899-907).
27. Radovic, M., Adarkwa, O., & Wang, Q. (2017). Object recognition in aerial images using convolutional neural networks. Journal of Imaging, 3(2), 21.
28. Benjdira, B., Khursheed, T., Koubaa, A., Ammar, A., & Ouni, K. (2019, February). Car detection using unmanned aerial vehicles: Comparison between faster r-cnn and yolov3. In 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS) (pp. 1-6). IEEE.
29. Kim, S., & Kim, H. (2021). Zero-centered fixed-point quantization with iterative retraining for deep convolutional neural network-based object detectors. IEEE Access, 9, 20828-20839.
30. Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
31. Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
32. Atik, M. E., Duran, Z., & Seker, D. Z. (2021). Machine learning-based supervised classification of point clouds using multiscale geometric features. ISPRS International Journal of Geo-Information, 10(3), 187.
33. Duran, Z., & Aydar, U. (2012). Digital modeling of world's first known length reference unit: The Nippur cubit rod. Journal of cultural heritage, 13(3), 352-356.
34. Duran, Z., & Atik, M. E. (2021). Accuracy comparison of interior orientation parameters from different photogrammetric software and direct linear transformation method. International Journal of Engineering and Geosciences, 6(2), 74-80.
35. Zhu, H., & Yu, F. (2016). A cross-correlation technique for vehicle detections in wireless magnetic sensor network. IEEE Sensors Journal, 16(11), 4484-4494.