Research Article

Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery

Volume: 6 Number: 4 October 31, 2026

Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery

Abstract

Aircraft recognition in remote-sensing imagery is usually reported through detection or classification accuracy. Accuracy alone says nothing about how confident a model is when it is wrong, or how much of an apparent architectural advantage comes from the training budget rather than from the architecture. This study reports a reliability and calibration analysis on the MAR20 benchmark under a complete two-factor design. YOLOv8n and YOLOv8s were trained for aircraft detection, and MobileNetV3-Large, EfficientNet-B0 and ResNet50 were each trained under two training protocols across five fixed seeds, giving thirty classification runs. Classification is evaluated on ground-truth crops and therefore describes an oracle-crop setting rather than an end-to-end pipeline. The effect of the training protocol proved to depend on the architecture and to be of the same order as the differences between the architectures themselves: the longer schedule improved ResNet50 and MobileNetV3-Large on every seed, while EfficientNet-B0 was not separated from its own baseline. Once the protocol was held fixed, EfficientNet-B0 and ResNet50 both reached approximately 0.98 top-1 accuracy and could not be separated by any paired test at either the sample level or the seed level, although both separated from MobileNetV3-Large. Model ranking also depended on the calibration estimator, because the highest equal-width bin holds over 90 percent of the test crops: after temperature scaling, equal-width expected calibration error favored EfficientNet-B0 and equal-mass expected calibration error favored ResNet50. Temperature scaling left accuracy unchanged in all thirty runs while reducing wrong predictions issued above 0.95 confidence by between a quarter and a half. Code, configurations and per-run outputs are archived.

Keywords

References

  1. Gui, S., Song, S., Qin, R., & Tang, Y. (2024). Remote sensing object detection in the deep learning era—A review. Remote Sensing, 16(2), Article 327. https://doi.org/10.3390/rs16020327
  2. Zhang, L., Zhang, L., & Du, B. (2016). Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22–40. https://doi.org/10.1109/MGRS.2016.2540798
  3. Alganci, U., Soydas, M., & Sertel, E. (2020). Comparative research on deep learning approaches for airplane detection from very high-resolution satellite images. Remote Sensing, 12(3), Article 458. https://doi.org/10.3390/rs12030458
  4. Ji, F., Ming, D., Zeng, B., Yu, J., Qing, Y., Du, T., & Zhang, X. (2021). Aircraft detection in high spatial resolution remote sensing images combining multi-angle features driven and majority voting CNN. Remote Sensing, 13(11), Article 2207. https://doi.org/10.3390/rs13112207
  5. Yu, W., Cheng, G., Wang, M., Yao, Y., Xie, X., Yao, X., & Han, J. (2023). MAR20: A benchmark for military aircraft recognition in remote sensing images. National Remote Sensing Bulletin, 27(12), 2688–2696. https://doi.org/10.11834/jrs.20222139
  6. Zhang, Y., Sun, H., Zuo, J., Wang, H., Xu, G., & Sun, X. (2018). Aircraft type recognition in remote sensing images based on feature learning with conditional generative adversarial networks. Remote Sensing, 10(7), Article 1123. https://doi.org/10.3390/rs10071123
  7. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In D. Precup & Y. W. Teh (Eds.), Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 1321–1330). PMLR.
  8. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493%281950%29078%3C0001%3AVOFEIT%3E2.0.CO%3B2

Details

Primary Language

English

Subjects

Geomatic Engineering (Other)

Journal Section

Research Article

Publication Date

October 31, 2026

Submission Date

June 7, 2026

Acceptance Date

October 30, 2026

Published in Issue

Year 2026 Volume: 6 Number: 4

APA
Şengöz, N. (2026). Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery. Engineering Perspective, 6(4), 665-677. https://doi.org/10.64808/engineeringperspective.1965542
AMA
1.Şengöz N. Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery. engineeringperspective. 2026;6(4):665-677. doi:10.64808/engineeringperspective.1965542
Chicago
Şengöz, Nilgün. 2026. “Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery”. Engineering Perspective 6 (4): 665-77. https://doi.org/10.64808/engineeringperspective.1965542.
EndNote
Şengöz N (October 1, 2026) Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery. Engineering Perspective 6 4 665–677.
IEEE
[1]N. Şengöz, “Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery”, engineeringperspective, vol. 6, no. 4, pp. 665–677, Oct. 2026, doi: 10.64808/engineeringperspective.1965542.
ISNAD
Şengöz, Nilgün. “Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery”. Engineering Perspective 6/4 (October 1, 2026): 665-677. https://doi.org/10.64808/engineeringperspective.1965542.
JAMA
1.Şengöz N. Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery. engineeringperspective. 2026;6:665–677.
MLA
Şengöz, Nilgün. “Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery”. Engineering Perspective, vol. 6, no. 4, Oct. 2026, pp. 665-77, doi:10.64808/engineeringperspective.1965542.
Vancouver
1.Nilgün Şengöz. Reliability and Calibration Analysis of Deep Learning Models for Aircraft Recognition in Remote-Sensing Imagery. engineeringperspective. 2026 Oct. 1;6(4):665-77. doi:10.64808/engineeringperspective.1965542

download?token=eyJhdXRoX3JvbGVzIjpbXSwiZW5kcG9pbnQiOiJqb3VybmFsIiwib3JpZ2luYWxuYW1lIjoiQ2l0ZVNjb3JlMjAyNV9FbmdpbmVlcmluZ19QZXJzcGVjdC5wbmciLCJwYXRoIjoiMjk0YS81N2EyLzlmYzcvNmEyMDA1NTk1OTk0NjMuMTIyOTg1NTEucG5nIiwiZXhwIjoxNzgwNDg3MDE3LCJub25jZSI6ImFjOTI3ZGIxMTUyNGUwZWRhYTlmMzc4ZDk4ZWIwOWQ0In0.VU0eXdFjkzGH_RYBvG3PsXcGoypETO7r1qIe7dMExbU