Evaluation of Cutting-Edge Object Detection Architectures on Multi-Object and Single-Object Datasets

Abstract

This study focuses on the performance evaluation of cutting-edge object detection models, namely, YOLO12X, Mask R-CNN, RT-DETR-X, and RF-DETR-Large on the Open Images (Multi-Object) and LaSOT (Single-Object) datasets. Current cutting-edge trend applications involve CNN-based and Transformer-based object detection models. CNN-based models can use one-pass (YOLO family) or two-pass (R-CNN family) implementations. One-pass object detection models can be faster but suffer from accuracy compared to the two-pass models. Transformer-based models can use Detection Transformers or Vision Transformers. Transformer-based models are gaining popularity, and their performance surpasses CNN-based models. This study evaluates YOLO12X, Mask R-CNN from CNN-based family, and RT-DETR-X, RF-DETR-Large transformer-based architectures in terms of accuracy and time on the Open Images and the LaSOT datasets. All models are the largest available models and pretrained on COCO dataset. Transformer-based models incorporate special types of self-attention and pose significant improvement both on accuracy and speed. The experimental results demonstrate that attention and transformer-based models perform better than the traditional CNN-based object detectors and YOLO12X is the fastest method with a far margin. On the LaSOT dataset, RT-DETR-X posts 0.8804 IoU, 0.7047 F1-score, 0.6597 mAP@0.5, 28.64 fps whereas YOLO12X achieves 0.8572 IoU, 0.6657 F1-score, 0.5357 mAP@0.5, and 49.78 fps.

Keywords

• Object detection
• Transformers
• Convolutional neural networks

References

1. Atik, M. E., Duran, Z., & Özgünlük, R. (2022). Comparison of YOLO versions for object detection from aerial images. International Journal of Environment and Geoinformatics, 9(2), 87–93. https://doi.org/10.30897/ijegeo.1010741
2. Bakır, H., & Bakır, R. (2023). Evaluating the robustness of YOLO object detection algorithm in terms of detecting objects in noisy environment. Journal of Scientific Reports-A, (54), 1–25. https://doi.org/10.59313/jsr-a.1257361
3. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. In A. Vedaldi, H. Bischof, T. Brox, & J. M. Frahm (Eds.), Computer Vision – ECCV 2020 (pp. 213–229). Springer. https://doi.org/10.1007/978-3-030-58452-8_13
4. Chen, W., Luo, J., Zhang, F., & Tian, Z. (2024). A review of object detection: Datasets, performance evaluation, architecture, applications and current trends. Multimedia Tools and Applications, 83(24), 65603–65661. https://doi.org/10.1007/s11042-023-17949-4
5. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), (pp. 886–893). IEEE. https://doi.org/10.1109/CVPR.2005.177
6. Dayıoğlu, M., Eyüboğlu, A. K., & Ünal, R. (2025). Performance analysis of YOLO11 models in PCB defect detection tasks. Kuzey Ege Teknik Bilimler ve Teknoloji Dergisi, 2(1), 33–50.
7. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale [Paper presentation]. International Conference on Learning Representations (ICLR), Virtual.
8. Ereken, Ö. F., & Tarhan, Ç. (2025). Modeling objects with artificial intelligence based image processing techniques: Object detection with Mask R-CNN. Academic Platform Journal of Engineering and Smart Systems, 13(1), 17–21. https://doi.org/10.21541/apjess.1542885

9. Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., & Ling, H. (2019). LaSOT: A high-quality benchmark for large-scale single object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 5374–5383). IEEE. https://doi.org/10.1109/CVPR.2019.00552
10. Girshick, R. (2015). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), (pp. 1440–1448). IEEE. https://doi.org/10.1109/ICCV.2015.169
11. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 580–587). IEEE. https://doi.org/10.1109/CVPR.2014.81
12. He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), (pp. 2961–2969). IEEE. https://doi.org/10.1109/ICCV.2017.322
13. Jocher, G. (2020). Ultralytics YOLOv5 (Version 7.0) [Computer software]. GitHub. https://github.com/ultralytics/yolov5
14. Jocher, G., & Qiu, J. (2024). Ultralytics YOLO11 (Version 11.0.0) [Computer software]. https://github.com/ultralytics/ultralytics
15. Karadağ, B., & Arı, A. (2023). Akıllı mobil cihazlarda YOLOv7 modeli ile nesne tespiti. Politeknik Dergisi, 26(3), 1207–1214. https://doi.org/10.2339/politeknik.1296541
16. Karakuş, O. F., Gülcü, A., & Karaca, A. C. (2025). Adapting vision transformer-based object detection model for handwritten text line segmentation task. Journal of Innovative Science and Engineering, 9(1), 28–38. https://doi.org/10.38088/jise.1471047
17. Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., & Ferrari, V. (2020). The Open Images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. Int J Comp Vis, 128(7), 1956–1981. https://doi.org/10.1007/s11263-020-01316-z
18. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer Vision – ECCV 2014 (pp. 740–755). Springer. https://doi.org/10.1007/978-3-319-10602-1_48
19. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision – ECCV 2016 (pp. 21–37). Springer. https://doi.org/10.1007/978-3-319-46448-0_2
20. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. Int J Comp Vis, 60(2), 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
21. Lv, W., Zhao, Y., Chang, Q., Huang, K., Wang, G., & Liu, Y. (2024). RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer. arXiv. https://arxiv.org/abs/2407.17140
22. Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 6517–6525). IEEE. https://doi.org/10.1109/CVPR.2017.690
23. Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv. https://doi.org/10.48550/arXiv.1804.02767
24. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 779–788). IEEE. https://doi.org/10.1109/CVPR.2016.91
25. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28, 91–99.
26. Robinson, I., Robicheaux, P., & Popov, M. (2025). RF-DETR [Computer software]. GitHub. https://github.com/roboflow/rf-detr
27. Sun, Y., Sun, Z., & Chen, W. (2024). The evolution of object detection methods. Engineering Applications of Artificial Intelligence, 133, 108458. https://doi.org/10.1016/j.engappai.2024.108458
28. Şengül, F., & Adem, K. (2025). Detection of military aircraft using YOLO and transformer-based object detection models in complex environments. Bilişim Teknolojileri Dergisi, 18(1), 85–97. https://doi.org/10.17671/gazibtd.1549034
29. Tian, Y., Ye, Q., & Doermann, D. (2025). YOLOv12: Attention-centric real-time object detectors. arXiv. https://doi.org/10.48550/arXiv.2502.12524
30. Ultralytics. (2023). Ultralytics YOLOv8 [Computer software]. https://github.com/ultralytics/ultralytics
31. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), 1, 511–518. IEEE. https://doi.org/10.1109/CVPR.2001.990517
32. Wang, C. Y., Bochkovskiy, A., & Liao, H. Y. M. (2023). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 7462–7471). IEEE. https://doi.org/10.1109/CVPR52729.2023.00721
33. Xia, G. S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 3974–3983). IEEE. https://doi.org/10.1109/CVPR.2018.00418
34. Yuille, A. L. (1991). Deformable templates for face recognition. Journal of Cognitive Neuroscience, 3(1), 59–70. https://doi.org/10.1162/jocn.1991.3.1.59