A benchmark dataset for deep learning-based airplane detection: HRPlanes

Abstract

Airplane detection from satellite imagery is a challenging task due to the complex backgrounds in the images and differences in data acquisition conditions caused by the sensor geometry and atmospheric effects. Deep learning methods provide reliable and accurate solutions for automatic detection of airplanes; however, huge amount of training data is required to obtain promising results. In this study, we create a novel airplane detection dataset called High Resolution Planes (HRPlanes) by using images from Google Earth (GE) and labeling the bounding box of each plane on the images. HRPlanes include GE images of several different airports across the world to represent a variety of landscape, seasonal and satellite geometry conditions obtained from different satellites. We evaluated our dataset with two widely used object detection methods namely YOLOv4 and Faster R-CNN. Our preliminary results show that the proposed dataset can be a valuable data source and benchmark data set for future applications. Moreover, proposed architectures and results of this study could be used for transfer learning of different datasets and models for airplane detection.

Keywords

• Airplane detection
• Deep learning
• YOLO
• Faster R-CNN
• Google Earth

References

1. Kaynarca, M., Demir, N., & San, B. T. (2020). Yeraltı Suyu Kaynaklarının Uzaktan Algılama ve CBS Teknikleri Kullanarak Modellenmesine Yönelik bir Yaklaşım: Kırkgöz Havzası (Antalya). Geomatik, 5(3), 241-245.
2. Efe, E., & Alganci, U. (2023). Çok zamanlı Sentinel 2 uydu görüntüleri ve makine öğrenmesi tabanlı algoritmalar ile arazi örtüsü değişiminin belirlenmesi. Geomatik, 8(1), 27-34.
3. Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020). Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS journal of photogrammetry and remote sensing, 159, 296-307.
4. Körez, A. (2022). Divide and conquer object detection (DACOD) method for runway detection in remote sensing images. International Journal of Engineering and Geosciences, 7(2), 154-160.
5. Biyik, M. Y., Atik, M. E., & Duran, Z. (2023). Deep learning-based vehicle detection from orthophoto and spatial accuracy analysis. International Journal of Engineering and Geosciences, 8(2), 138-145.
6. Alganci, U., Soydas, M., & Sertel, E. (2020). Comparative research on deep learning approaches for airplane detection from very high-resolution satellite images. Remote sensing, 12(3), 458.
7. Liu, G., Sun, X., Fu, K., & Wang, H. (2012). Aircraft recognition in high-resolution satellite images using coarse-to-fine shape prior. IEEE Geoscience and Remote Sensing Letters, 10(3), 573-577.
8. Xu, C., & Duan, H. (2010). Artificial bee colony (ABC) optimized edge potential function (EPF) approach to target recognition for low-altitude aircraft. Pattern Recognition Letters, 31(13), 1759-1772.

9. Cheng, G., & Han, J. (2016). A survey on object detection in optical remote sensing images. ISPRS journal of photogrammetry and remote sensing, 117, 11-28.
10. Sun, H., Sun, X., Wang, H., Li, Y., & Li, X. (2011). Automatic target detection in high-resolution remote sensing images using spatial sparse coding bag-of-words model. IEEE Geoscience and Remote Sensing Letters, 9(1), 109-113.
11. Zhang, W., Sun, X., Fu, K., Wang, C., & Wang, H. (2013). Object detection in high-resolution remote sensing images using rotation invariant parts based model. IEEE Geoscience and Remote Sensing Letters, 11(1), 74-78.
12. Zhang, W., Sun, X., Wang, H., & Fu, K. (2015). A generic discriminative part-based model for geospatial object detection in optical remote sensing images. ISPRS journal of photogrammetry and remote sensing, 99, 30-44.
13. Lei, Z., Fang, T., Huo, H., & Li, D. (2011). Rotation-invariant object detection of remotely sensed images based on texton forest and hough voting. IEEE Transactions on Geoscience and Remote Sensing, 50(4), 1206-1217.
14. Liu, L., & Shi, Z. (2014). Airplane detection based on rotation invariant and sparse coding in remote sensing images. Optik, 125(18), 5327-5333.
15. Ball, J. E., Anderson, D. T., & Chan, C. S. (2017). Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community. Journal of applied remote sensing, 11(4), 042609
16. Chen, Z., Zhang, T., & Ouyang, C. (2018). End-to-end airplane detection using transfer learning in remote sensing images. Remote Sensing, 10(1), 139.
17. Xu, Y., Zhu, M., Xin, P., Li, S., Qi, M., & Ma, S. (2018). Rapid airplane detection in remote sensing images based on multilayer feature fusion in fully convolutional neural networks. Sensors, 18(7), 2335.
18. Zhu, M., Xu, Y., Ma, S., Li, S., Ma, H., & Han, Y. (2019). Effective airplane detection in remote sensing images based on multilayer feature fusion and improved nonmaximal suppression algorithm. Remote Sensing, 11(9), 1062.
19. Wu, Z. Z., Weise, T., Wang, Y., & Wang, Y. (2020). Convolutional neural network based weakly supervised learning for aircraft detection from remote sensing image. IEEE Access, 8, 158097-158106.
20. Zhou, L., Yan, H., Shan, Y., Zheng, C., Liu, Y., Zuo, X., & Qiao, B. (2021). Aircraft detection for remote sensing images based on deep convolutional neural networks. Journal of Electrical and Computer Engineering, 2021, 1-16.
21. Ji, F., Ming, D., Zeng, B., Yu, J., Qing, Y., Du, T., & Zhang, X. (2021). Aircraft detection in high spatial resolution remote sensing images combining multi-angle features driven and majority voting CNN. Remote Sensing, 13(11), 2207.
22. Shi, L., Tang, Z., Wang, T., Xu, X., Liu, J., & Zhang, J. (2021). Aircraft detection in remote sensing images based on deconvolution and position attention. International Journal of Remote Sensing, 42(11), 4241-4260.
23. Wu, Q., Feng, D., Cao, C., Zeng, X., Feng, Z., Wu, J., & Huang, Z. (2021). Improved mask R-CNN for aircraft detection in remote sensing images. Sensors, 21(8), 2618.
24. Zeng, B., Ming, D., Ji, F., Yu, J., Xu, L., Zhang, L., & Lian, X. (2022). Top-Down aircraft detection in large-scale scenes based on multi-source data and FEF-R-CNN. International Journal of Remote Sensing, 43(3), 1108-1130.
25. Chen, X., Liu, J., Xu, F., Xie, Z., Zuo, Y., & Cao, L. (2022). A Novel Method of Aircraft Detection under Complex Background Based on Circular Intensity Filter and Rotation Invariant Feature. Sensors, 22(1), 319.
26. Xia, G. S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., ... & Zhang, L. (2018). DOTA: A large-scale dataset for object detection in aerial images. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3974-3983).
27. Waqas Zamir, S., Arora, A., Gupta, A., Khan, S., Sun, G., Shahbaz Khan, F., ... & Bai, X. (2019). isaid: A large-scale dataset for instance segmentation in aerial images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 28-37).
28. Lam, D., Kuzma, R., McGee, K., Dooley, S., Laielli, M., Klaric, M., ... & McCord, B. (2018). xview: Objects in context in overhead imagery. arXiv preprint arXiv:1802.07856.
29. Shermeyer, J., Hossler, T., Van Etten, A., Hogan, D., Lewis, R., & Kim, D. (2021). Rareplanes: Synthetic data takes flight. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 207-217).
30. HyperLabel. https://docs.hyperlabel.com/
31. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).
32. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).
33. Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7263-7271).
34. Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
35. Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017). Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2117-2125).
36. Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
37. Wang, C. Y., Liao, H. Y. M., Wu, Y. H., Chen, P. Y., Hsieh, J. W., & Yeh, I. H. (2020). CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 390-391).
38. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 37(9), 1904-1916.
39. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580-587).
40. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., ... & Darrell, T. (2014, November). Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675-678).
41. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448).
42. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
43. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., ... & Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 (pp. 740-755). Springer International Publishing.
44. Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International journal of computer vision, 88, 303-338.
45. Yaban, B., Alganci, U., & Sertel, E. (2022). Aircraft detection in very high-resolution satellite images using YOLO-based deep learning methods. Intercontinental Geoinformation Days, 4, 270-273.