Enhance or Leave It: An Investigation of the Image Enhancement in Small Object Detection in Aerial Images

Abstract

Recent years of object detection (OD), a fundamental task in computer vision, have witnessed the rise of numerous practical applications of this sub-field such as face detection, self-driving, security, and more. Although existing deep learning models show significant achievement in object detection, they are usually tested on datasets having mostly clean images. Thus, their performance levels were not measured on degraded images. In addition, images and videos in real-world scenarios often involve several natural artifacts such as noise, haze, rain, dust, and motion blur due to several factors such as insufficient light, atmospheric scattering, and faults in image sensors. This image acquisition-related problem becomes more severe when it comes to detecting small objects in aerial images. In this study, we investigate the small object identification performance of several state-of-the-art object detection models (Yolo 6/7/8) under three conditions (noisy, motion blurred, and rainy). Through this inspection, we evaluate the contribution of an image enhancement scheme so-called MPRNet. For this aim, we trained three OD algorithms with the original clean images of the VisDrone dataset. Followingly, we measured the detection performance of saved YOLO models against (1) clean, (2) degraded, and (3) enhanced counterparts. According to the results, MPRNet-based image enhancement promisingly contributes to the detection performance and YOLO8 outperforms its predecessors. We believe that this work presents useful findings for researchers studying aerial image-based vision tasks, especially under extreme weather and image acquisition conditions

Keywords

• Object Detection
• Image Restoration
• MPRNet
• Single Shot Object Detection

References

1. Cao, Y., He, Z., Wang, L., Wang, W., Yuan, Y., Zhang, D., & Liu, M. (2021). VisDrone-DET2021: The vision meets drone object detection challenge results. In Proceedings of the IEEE/CVF International conference on computer vision (pp. 2847-2854).
2. Dai, J., Li, Y., He, K., & Sun, J. (2016). R-fcn: Object detection via region-based fully convolutional networks. Advances in neural information processing systems, 29.
3. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. (2019). Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 6569-6578).
4. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580-587).
5. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448).
6. Fu, C. Y., Liu, W., Ranga, A., Tyagi, A., & Berg, A. C. (2017). Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659.
7. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 37(9), 1904-1916.
8. He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969).

9. Law, H., & Deng, J. (2018). Cornernet: Detecting objects as paired keypoints. In Proceedings of the European conference on computer vision (ECCV) (pp. 734-750).
10. Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2017). Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international conference on computer vision (pp. 4770-4778).
11. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., & Wei, X. (2022). YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976.
12. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14 (pp. 21-37). Springer International Publishing.
13. Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018). Path aggregation network for instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8759-8768).
14. Rajaei, B., Rajaei, S., & Damavandi, H. (2023). An Analysis of Multi-stage Progressive Image Restoration Network (MPRNet). Image Processing On Line, 13, 140-152.
15. Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7263-7271).
16. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
17. Terven, J., Córdova-Esparza, D. M., & Romero-González, J. A. (2023). A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Machine Learning and Knowledge Extraction, 5(4) (pp.1680-1716).
18. Uijlings, J. R., Van De Sande, K. E., Gevers, T., & Smeulders, A. W. (2013). Selective search for object recognition. International journal of computer vision, 104, 154-171.
19. Wang, C. Y., Bochkovskiy, A., & Liao, H. Y. M. (2023). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7464-7475).
20. Wang, X., Gao, H., Jia, Z., & Li, Z. (2023). BL-YOLOv8: An Improved Road Defect Detection Model Based on YOLOv8. Sensors, 23(20), 8361.
21. Wang, C. Y., Liao, H. Y. M., & Yeh, I. H. (2022). Designing Network Design Strategies Through Gradient Path Analysis. arXiv preprint arXiv:2211.04800.
22. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M. H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14821-14831).