Object Detection with Deep Learning in Simulation Environments

Abstract

With the concept of digital twins, the development cost and time are benefited by making simulation environments. Especially considering the high cost and other risks in image processing and unmanned aerial vehicle applications, the use of digital media in research for such applications allows easy testing of new and untested algorithms. A quadrotor unmanned aerial vehicle dynamic model twin was created in Matlab software, and the flight environment for this aircraft was simulated with Unreal game engine software. A camera model was created on the aircraft and images were taken from the digital media over the unmanned aerial vehicle. The images obtained were sent to the previously trained YOLOv4 deep learning network, and soldier figures in the field environment with different fog conditions were tried to be determined. In the tests carried out, it was observed that the number and accuracy of object detection decreased when fog conditions were difficult.

Keywords

• Digital twin
• Image processing
• UAV

Simülasyon Ortamlarda Derin Öğrenme ile Nesne Tespiti

Abstract

Dijital ikiz kavramı ile benzetim ortamlarının yapılması ile geliştirme maliyeti ve zamanından fayda sağlanmaktadır. Özellikle görüntü işleme ve insansız hava aracı uygulamalarındaki yüksek maliyet ve diğer riskler düşünüldüğünde bu tip uygulamalar için dijital ortamların araştırmalarda kullanılması yeni ve test edilmemiş algoritmalarında kolay bir şeklide denenmesine imkân vermektedir. Dört rotorlu bir insansız hava aracı dinamik model ikizi Matlab yazılımında oluşturulmuş ve bu hava aracı için uçuş ortamı Unreal oyun motoru yazılımı ile benzetilmiştir. Hava aracının üzerinde kamera modeli oluşturularak dijital ortamdan insansız hava aracı üzerinden görüntüler alınmıştır. Elde edilen görüntüler daha önce eğitilen YOLOv4 derin öğrenme ağına gönderilerek, faklı sis koşullarını içeren arazi ortamındaki asker figürleri tespit edilmeye çalışılmıştır. Yapılan testlerde sis koşullarının zor olduğu durumlarda nesne tespit sayısının ve doğruluğunun azaldığı gözlemlenmiştir.

Keywords

• Dijital ikiz
• Görüntü işleme
• İHA

References

1. Ashkir I, Roullier B, McQuade F, and Anjum A. (2021). 3D object recognition for virtual reality based digital twins. IEEE/ACM 8th International Conference on Big Data Computing, Applications and Technologies. United Kingdom, pp.9–17.
2. Bochkovskiy A, Wang C, & Liao H.M. (2020). YOLOv4: Optimal speed and accuracy of object detection. Computer Science ArXiv abs/2004.10934.
3. Fränti P, and Sieranoja S. (2018). K-means properties on six clustering benchmark datasets. Applied Intelligence 48: 4743–4759.
4. Hazbon O, et al. (2019). Digital twin concept for aircraft system failure detection and correction. AIAA Aviation 2019 Forum, Texas, pp.2019-2887.
5. Hendrik M, Ann-Kathrin Koschlik, and Raddatz F. (2022). Digital twin concept for aircraft components. 33rd Congress of The International Council of The Aeronautical Sciences, ICAS 2022. Sweden, ISSN 2958-4647.
6. Meng W. et al. (2023). DTUAV: a novel cloud–based digital twin system for unmanned aerial vehicles. Simulation 99(1): 69-87.
7. Lee Eung-Joo et al. (2021). Validation of object detection in UAV-based images using synthetic data. Proc. SPIE 11746: 584-601
8. Lei L, Shen G, Zhang L, and Li Z. (2021). Toward intelligent cooperation of uav swarms: when machine learning meets digital twin. IEEE Network 35(1): 386-392.

9. Li L, Aslam S, Wileman A, and Perinpanayagam S. (2022). Digital Twin in Aerospace Industry: A Gentle Introduction. IEEE Access 10: 9543-9562.
10. Redmon J, and Farhadi A. (2016). YOLO9000: Better, Faster, Stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp.6517-6525.
11. Shah S, Dey D, Lovett C, & Kapoor A. (2018). AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. Field and Service Robotics. Springer Proceedings in Advanced Robotics 5: 621-635.
12. Yang Y, Meng W, and Zhu S. (2020). A Digital Twin Simulation Platform for Multi-rotor UAV. 7th International Conference on Information, Cybernetics, and Computational Social Systems, China, pp.591-596.
13. Zhou X. et al. (2022). Intelligent Small Object Detection for Digital Twin in Smart Manufacturing With Industrial Cyber-Physical Systems. IEEE Transactions on Industrial Informatics 18(2) :1377-1386.
14. Zweber J.V, Kolonay R.M, Kobryn P, and Tuegel E.J. (2017). Digital Thread and Twin for Systems Engineering: Requirements to Design. 55th AIAA Aerospace Sciences Meeting, pp.0875