Mask-Based IoU for Bounding Box Regression Using Medical Images

Abstract

Bounding box regression plays a pivotal role in the majority of object detection algorithms, significantly influencing the accuracy of object positioning and the regression speed of Convolutional Neural Networks (CNN). In object detection benchmarks, Intersection over Union (IoU) remains the widely adopted metric for evaluation. Traditional IoU-based loss functions often suffer from poor training outcomes and slow convergence, and they fail to account for situations where the predicted bounding box does not entirely capture the object’s mask. This study introduces the Mask-based Intersection over Union (MbIoU) metric for improving bounding box regression in object detection using medical images. The proposed MbIoU metric incorporates the object mask into the bounding box regression process, offering a more precise evaluation of how well the predicted bounding box encapsulates the object. The developed MbIoU metric was tested on the MNIST: HAM10000 dermoscopic skin images dataset, COVID-19 CT dataset, and Brain Tumor dataset and compared to traditional IoU metrics. The results show that MbIoU enhances the prediction by better capturing the object’s contained mask.

Keywords

• Object detection
• convolutional neural networks
• bounding box regression
• intersection over union.

References

1. Brain Tumor Segmentation, Available at: https://github.com/rastislavkopal/brain-tumor-segmentation/tree/main/brain_tumor_data. Accessed on: 2024-09-20.
2. Cai, Z., Vasconcelos, N., Cascade R-CNN: Delving Into High Quality Object Detection, Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 2018.
3. Cheng, G., Han, J., Zhou, P., Xu, D., Learning rotation-invariant and fisher discriminative convolutional neural networks for object detection, IEEE Transactions On Image Processing, 28(2018), 265–278.
4. COVID-19 CT Scan Lesion Segmentation Dataset. Available at: https://www.kaggle.com/datasets/maedemaftouni/covid19-ct-scan-lesion-segmentation-dataset. Accessed on: 2024-09-20.
5. Dai, J., Li, Y., He, K., Sun, J., R-fcn: Object detection via region-based fully convolutional network, Advances In Neural Information Processing Systems, 29(2016).
6. Dalal, N., Triggs, B., Histograms of oriented gradients for human detection, 2005 IEEE Computer Society Conference On Computer Vision And Pattern Recognition (CVPR’05), 2005.
7. Girshick, R., Fast r-cnn,Proceedings of the ieee international conference on computer vision, 2(2015), 1440–1448.
8. Girshick, R., Donahue, J., Darrell, T., Malik, J., Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 2014.

9. Girshick, R., Iandola, F., Darrell, T., Malik, J., Deformable Part Models are Convolutional Neural Networks, Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 2015.
10. Gong, Y., Yu, X., Ding, Y., Peng, X., Zhao, J. et all., Effective Fusion Factor in FPN for Tiny Object Detection,Proceedings Of The IEEE/CVF Winter Conference On Applications Of Computer Vision (WACV), 2021.
11. He, K., Zhang, X., Ren, S., Sun, J., Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition, IEEE Transactions On Pattern Analysis And Machine Intelligence, 37(2014), 1904–1916.
12. Jiang, H., Learned-Miller, E., Face detection with the faster R-CNN, 12th IEEE International Conference On Automatic Face & Gesture Recognition, 2017.
13. Jiang, P., Ergu, D., Liu, F., Cai, Y., Ma, B., A Review of Yolo algorithm developments, Procedia Computer Science, 199(2022), 1066–1073.
14. Junior, G., Ferreira, J., Mill´an-Arias, C., Daniel, R., Casado, A. et all. Ceramic Cracks Segmentation with Deep Learning, Applied Sciences, 11(6)(2021), 6017.
15. Li, Z., Liu, F., Yang,W., Peng, S., Zhou, J., A survey of convolutional neural networks: analysis, applications, and prospects, IEEETransactions On Neural Networks And Learning Systems, 2021.
16. Lin, T., Goyal, P., Girshick, R., He, K., Dollar, P., Focal Loss for Dense Object Detection, Proceedings Of The IEEE International Conference On Computer Vision (ICCV), 2017.
17. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S. et all., Ssd: Single shot multibox detector, Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, 21–37.
18. Liu, X., Hu, J., Wang, H., Zhang, Z., Lu, X., et all., Gaussian-IoU loss: Better learning for bounding box regression on PCB component detection, Expert Systems With Applications, 190(2022), 116178.
19. Lowe, D., Distinctive image features from scale-invariant keypoints, International Journal Of Computer Vision, 60(2004), 91–110.
20. Malik, J., Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, ACM: New York, NY, USA, 2014.
21. Ojala, T., Pietikainen, M., Maenpaa, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Transactions On Pattern Analysis And Machine Intelligence, 24(2002), 971–987.
22. Padilla, R., Netto, S., Da Silva, E., A survey on performance metrics for object-detection algorithms, 2020 International Conference On Systems, Signals And Image Processing (IWSSIP), 237–242.
23. Qian, X., Wu, B., Cheng, G., Yao, X., Wang, W. et all. Building a bridge of bounding box regression between oriented and horizontal object detection in remote sensing images, IEEE Transactions On Geoscience And Remote Sensing, 61(2023), 1–9.
24. Rahman, M., Wang, Y. Optimizing intersection-over-union in deep neural networks for image segmentation, International Symposium On Visual Computing, (2016), 234–244.
25. Redmon, J., Divvala, S., Girshick, R., Farhadi, A., You Only Look Once: Unified, Real-Time Object Detection, Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 2016.
26. Ren, S., He, K., Girshick, R., Sun, J., Faster r-cnn: Towards real-time object detection with region proposal networks, Advances In Neural Information Processing Systems, 28(2015).
27. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I. et all., Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression, Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition (CVPR), 2019.
28. Selamet, F., Cakar, S., Kotan, M., Automatic detection and classification of defective areas on metal parts by using adaptive fusion of faster R-CNN and shape from shading, IEEE Access, 10(2022), 126030–126038.
29. Shen, Y., Zhang, F., Liu, D., Pu, W., Zhang, Q., Manhattan-distance IOU loss for fast and accurate bounding box regression and object detection, Neurocomputing, 500(2022), 99–114.
30. Su, K., Cao, L., Zhao, B., Li, N., Wu, D. et all., N-IoU: better IoU-based bounding box regression loss for object detection, Neural Computing And Applications, (2023), 1–15.
31. Tschandl, P., Rosendahl, C., Kittler, H., The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Scientific Data, 5(1-9)(2018).
32. Vakili, E., Karimian, G., Shoaran, M., Yadipour, R., Sobhi, J., Valid-IoU: An Improved IoU-based Loss Function and Its Application to Detection of Defects on Printed Circuit Boards, 2023.
33. Wang, X.,Song, J., ICIoU: Improved loss based on complete intersection over union for bounding box regression, IEEE Access,9(2021), 105686–105695
34. Yu, J., Jiang, Y., Wang, Z., Cao, Z., Huang, T., Unitbox: An advanced object detection network, Proceedings Of The 24th ACM International Conference On Multimedia, (2016), 516–520.
35. Zhai, H., Cheng, J.,Wang, M., Rethink the IoU-based loss functions for bounding box regression, 2020 IEEE 9th Joint International Information Technology And Artificial Intelligence Conference (ITAIC), 9(2020), 1522–1528.
36. Zhang, Y., Sohn, K., Villegas, R., Pan, G., Lee, H., Improving Object Detection With Deep Convolutional Networks via Bayesian Optimization and Structured Prediction, Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 2015.
37. Zhang, Y., Ren, W., Zhang, Z., Jia, Z., Wang, L. et all., Focal and efficient IOU loss for accurate bounding box regression, arXiv 2021, ArXiv Preprint ArXiv:2101.08158.
38. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R. et all., Distance-IoU loss: Faster and better learning for bounding box regression, Proceedings Of The AAAI Conference On Artificial Intelligence, 34(2020), 12993–13000.