Military camouflage classification with Mask R-CNN algorithm

Abstract

Camouflage, which is used as an art of hiding by living things in nature, started to be used in the military field in the 19th century with the widespread use of long-range firearms. When factors such as different nations, environment and climate are considered, we come across camouflages in various colors and patterns. Over time, the camouflage patterns adopted and used by countries or unions have become national identity. This study is on the classification and segmentation of camouflaged soldiers of 5 countries with deep learning. While the similarity of the camouflaged area with the background makes segmentation difficult, it becomes difficult to classify each camouflage pattern due to the cut of the fabric and the different locations of the pattern pieces on each soldier. There are different studies in the literature that are referred to as camouflage or pattern classification. The mentioned studies are in the form of segmentation of camouflaged object or classification of camouflaged objects of different types. Since the segmented and classified objects in this study are camouflaged soldiers, what is expected from the deep learning algorithm is to classify the objects mainly according to the camouflage pattern, not their outlines. In the study, 861 camouflaged soldier images were collected for 5 countries (Türkiye-Azerbaijan, USA, Russia, China, France) and polygonal labeling was made. Türkiye and Azerbaijan are considered a class as they have similar camouflages. For the solution of the problem, military camouflage classification was discussed with the Mask R-CNN algorithm, which is widely used today for object detection, segmentation and classification, and the importance of deep learning algorithms has been proven with such a difficult problem. The training resulted in 0.005219 classification loss and 0.03985 masking loss. The classification and segmentation success rate of the study is 95%.

Keywords

• Camouflage
• deep learning
• Mask R-CNN
• classification
• segmentation

References

1. Talas, L., Baddeley, R., Cuthill, I., Cultural evolution of military camouflage, Philos. Trans. R. Soc. B: Biol. Sci., 372 (2017), 20160351, https://doi.org/10.1098/rstb.2016.0351.
2. Hodson-Pressinger, S., Khaki uniform, 1848-1849: first introduction by Lumsden and Hodson, J. Soc. Army Hist. Res., 82 (2004), 341–347.
3. Singh, R., Study of Graphic Camouflage Patterns on Battle Uniform and Improving the Pattern Used by Indian Army, (2014). Master Thesis, PDPM Indian Institute of Information Technology, Design and Manufacturing Jabalpur, Jabalpur.
4. Doğan, F., Türkoğlu, I., Derin öğrenme algoritmalarının yaprak sınıflandırma başarımlarının karşılaştırılması, SAUCIS, 1 (2018), 10-21.
5. Gupta, A. and Gupta, U., Military surveillance with deep convolutional neural network, International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT), (2018), 1147-1152, https://doi.org/10.1109/ICEECCOT43722.2018.9001381.
6. Bayram, E. and Nabiyev, V., Classification of camouflage images using local binary patterns (LBP), 29th Signal Processing and Communications Applications Conference (SIU), (2021), 1-4, https://doi.org/10.1109/SIU53274.2021.9478040.
7. He, K., Gkioxari, G., Dollár, P., and Girshick, R., Mask R-CNN, IEEE International Conference on Computer Vision (ICCV), (2017), 2980-2988, https://doi.org/10.1109/ICCV.2017.322.
8. Ömeroğlu, A. N., Kumbasar, N., Oral, E. A. and Ozbek, I. Y., Mask R-CNN algoritması ile hangar tespiti hangar detection with Mask R-CNN algorithm, 27th Signal Processing and Communications Applications Conference (SIU), (2019), 1-4, https://doi.org/10.1109/SIU.2019.8806552.

9. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A., You only look once: unified, realtime object detection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 779-788, https://doi.org/10.1109/CVPR.2016.91.
10. Amri, M. B., Yedjour, D., El Amin Larabi, M. and Bakhti, K., Stadium detection from Alsat-2 and Google-Earth multispectral images using YOLOv5 and Mask R-CNN, 4th International Conference on Pattern Analysis and Intelligent Systems (PAIS), (2022), 1-4, https://doi.org/10.1109/PAIS56586.2022.9946887.
11. Torralba, A., Russell, B. C. and Yuen, J., LabelMe: online image annotation and applications, Proceedings of the IEEE, 98 (8) (2010), 1467-1484, https://doi.org/ 10.1109/JPROC.2010.2050290.
12. GIJOE SURPLUS, Turkish Army Digital Camo Uniform - Combat Shirt, (2021). Available at: https://gijoebcn.com/en/turkey/2427-18712-turkey-army-digital-camo-combat-shirt-used-condition.html. [Accessed: March 2023].
13. The New York Times, Would you consider serving in the U.S. Armed Forces?, (2020). Available at: https://www.nytimes.com/2020/01/15/learning/would-you-consider-serving-in-the-us-armed-forces.html. [Accessed: March 2023].
14. War on the Rocks, How is the Russian military responding to covid-19?, (2020). Available at: https://warontherocks.com/2020/05/how-is-the-russian-military-responding-to-covid-19/. [Accessed: March 2023].
15. South China Morning Post, South China Sea stand-off: Vietnam takes hard line while negotiating code of conduct but will Beijing listen?, (2018). Available at: https://www.scmp.com/news/asia/diplomacy/article/2180074/south-china-sea-stand-vietnam-takes-hard-line-while-negotiating. [Accessed: March 2023].
16. U.S. Army medical research and development command, First French Army Soldier Working at the ISR, (2012). Available at: https://mrdc.health.mil/index.cfm/media/articles/2012/first_french_army_soldier_worki ng_at_the_ISR. [Accessed: March 2023].
17. Wikipedia, Two Turkish soldiers salute in Kabul, (2013). Available at: https://tr.wikipedia.org/wiki/Dosya:Two_Turkish_soldiers_salute_in_Kabul_(46992773 63).jpg. [Accessed: March 2023].
18. Defense Visual Information Distribution Service, Brig. Gen. Moses Kaoiwi Jr. receives farewell gift, (2021). Available at: https://www.dvidshub.net/image/6890790/brig-gen-moses-kaoiwi-jr-receives-farewell-gift. [Accessed: March 2023].
19. 7sur7, Les troupes russes n'auraient pas enregistré d'avancées majeures ces dernières 24h, (2022). Available at: https://www.7sur7.be/monde/les-troupes-russes-n-auraient-pas-enregistre-d-avancees-majeures-ces-dernieres-24h~ae288a77/. [Accessed: March2023].
20. English.gov.cn, The state council the people's republic of China, President Xi reviews navy in South China Sea, (2018). Available at: http://english.www.gov.cn/news/top_news/2018/04/13/content_281476110326046.htm. [Accessed: March 2023].
21. Deep Learning Wizard, Learning rate scheduling (2019). Available at: https://www.deeplearningwizard.com/deep_learning/boosting_models_pytorch/lr_sched uling/. [Accessed: March 2023].
22. Ren, S., He, K., Girshick, R. and Sun, J., Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (6) (2017), 1137-1149, https://doi.org/10.1109/TPAMI.2016.2577031.
23. Yayla, R. and Şen, B., Region-based segmentation of terrain fields in SAR images, 28th Signal Processing and Communications Applications Conference (SIU), (2020), 1-4, https://doi.org/10.1109/SIU49456.2020.9302432.
24. Wu, Y., Kirillov, A., Massa, F., Lo, W. Y. and Girshick, R., Detectron2, July 2022.
25. Haber7, Türkiye'nin hangi ülkelerde askeri üssü var?, (2021). Available at: https://www.haber7.com/foto-galeri/66858-turkiyenin-hangi-ulkelerde-askeri-ussu-var. [Accessed: March 2023].
26. The Brooklyn News, Brookyn Times – Azerbaijan marks anniversary of victory in war, (2021). Available at: https://bklynnews.com/front-pages/front-pages-november-2021/the-brookyn-times-750-am-11-22-2021/. [Accessed: March 2023].
27. The Mike Church Show World, Tuesday Red Pill Diaries-New D-Day For U.S. Armed Forces and All Murican Gubbmint: CoronaDoom, (2021). Available at: https://mikechurch.com/wp-content/uploads/2021/08/New_DDAY_1920.jpg. [Accessed: March 2023].
28. 24 Канал, The French army is preparing for hostilities: the country has unprecedented military exercises, (2023). Available at: https://24tv.ua/ru/vo-francii-startovali-masshtabnye-voennye-uchenija-24-kanal_n2263123. [Accessed: March 2023].
29. Foreign Policy Magazine, China’s untested military could be a force-or a flop, (2018). Available at: https://foreignpolicy.com/2018/11/27/chinas-untested-military-could-be-aforce-or-a-flop/. [Accessed: March 2023].
30. News.ru, The head of the General Staff told Putin an anecdote, (2019). Available at: https://news.ru/russia/glava-genshtaba-rasskazal-putinu-anekdot/. [Accessed: March 2023].