Image processing based autonomous landing zone detection for a multi-rotor drone in emergency situations

Abstract

Flight safety and reliability improvement is an important research issue in aerial applications. Multi-rotor drones are vulnerable to motor failures leading to potentially unsafe operations or collisions. Therefore, researchers are working on autonomous landing systems to safely recover and land the faulty drone in on a desired landing area. In such a case, a suitable landing zone should be detected rapidly in for emergency landing. Majority of the works related with autonomous landing utilize a marker and GPS signals to detect landing site. In this work, we propose a landing system framework that involves only the processing of images taken from the onboard camera of the vehicle. First, the objects in the image are determined by filtering and edge detection algorithm, then the most suitable landing zone is searched. The area that is free from obstacles and closest to the center of the image is defined as the most immediate and suitable landing zone. The method has been tested on 25 images taken from different heights and its performance has been evaluated in terms runtime on a single board computer and detection precision and recall values. The average measured runtime is 2.4923 seconds and 100% of precision and recall values are achieved for the images taken from 1m and 2m. The smallest precision and recall values are 79.1% and 81.2%, respectively.

Keywords

• autonomous landing
• image processing
• object detection
• UAV

Supporting Institution

Adana Alparslan Türkeş Science and Technology University

Project Number

19119001

References

1. AMAZON (2017). Prime Air. from https://www.amazon.com/Amazon-Prime-Air/b?ie=%20UTF8&node=8037720011.
2. Aydin B, Selvi E, Tao J & Starek M (2019). Use of Fire-Extinguishing Balls for a Conceptual System of Drone-Assisted Wildfire Fighting. Drones, 3(1). https://doi.org/10.3390/drones3010017
3. Barták R, Hraško A & Obdržálek D (2014). On autonomous landing of AR.Drone: Hands-on experience. Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference, FLAIRS 2014: 400-405.
4. Cabrera-Ponce A A & Martinez-Carranza J (2017). A vision-based approach for autonomous landing. 2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS). Linköping, Sweden. DOI: 10.1109/RED-UAS.2017.8101655
5. Canny J (1986). A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8(6): 679-698. DOI: 10.1109/TPAMI.1986.4767851
6. Chen W, Yue H, Wang J & Wu X (2014). An improved edge detection algorithm for depth map inpainting. Optics and Lasers in Engineering, 55: 69–77. https://doi.org/10.1016/j.optlaseng.2013.10.025
7. Fitzgerald D & Walker R (2005). Classification of Candidate Landing Sites for UAV Forced Landings. AIAA Guidance, Navigation, and Control Conference and Exhibit, San Francisco, California.
8. Fitzgerald D, Walker R & Campbell D (2005). A Vision Based Emergency Forced Landing System for an Autonomous UAV. Australian International Aerospace Congress, Melbourne, Australia.

9. Guo X, Denman S, Fookes C, Mejias L & Sridharan S (2014). Automatic UAV Forced Landing Site Detection Using Machine Learning. 2014 International Conference on Digital Image Computing: Techniques and Applications (DICTA). Wollongong, NSW, Australia. DOI: 10.1109/DICTA.2014.7008097
10. Heincke B, Jackisch R, Saartenoja A, Salmirinne H, Rapp S, Zimmermann R, Pirttijärvi M et al. (2019). Developing multi-sensor drones for geological mapping and mineral exploration: setup and first results from the MULSEDRO project. Geological Survey of Denmark and Greenland Bulletin 43. https://doi.org/10.34194/GEUSB-201943-03-02
11. Ho H W (2017). Autonomous landing of Micro Air Vehicles through bio-inspired monocular vision. PHD Thesis, ISBN: 978-94-6186-818-3
12. Hoffmann G M, Huang H, Waslander S L & Tomlin C (2007). Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment. AIAA Guidance, Navigation and Control Conference and Exhibit, South Carolina
13. Jakob S, Zimmermann R & Gloaguen R (2016). Processing of drone-borne hyperspectral data for geological applications. 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS). Los Angeles, CA, USA. DOI: 10.1109/WHISPERS.2016.8071689
14. Kim J & Sukkarieh S (2002). Flight Test Results of GPS/INS Navigation Loop for an Autonomous Unmanned Aerial Vehicle (UAV). Proceedings of the 15th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GPS 2002). Portland, OR, 510 - 517
15. Kumar K, Li J & Khan S (2015). Comparative Study on Various Edge Detection Techniques for 2-D Image. International Journal of Computer Applications 119(22), 6-10.
16. Lee D, Lim H, Kim H J & Kim Y (2012). Adaptive Image-Based Visual Servoing for an Underactuated Quadrotor System. Journal of Guidance, Control, and Dynamics, 35(4), 1335-1353.
17. Lee M R, Su S, Yeah J E, Huang H & Chen J (2014). Autonomous landing system for aerial mobile robot cooperation. 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS). Kitakyushu, Japan, DOI: 10.1109/SCIS-ISIS.2014.7044826
18. Liteye (2018). HK$1 million in damage caused by GPS jamming that caused 46 drones to plummet during Hong Kong show. from https://liteye.com/hk1-million-in-damage-caused-by-gps-jamming-that-caused-46-drones-to-plummet-during-hong-kong-show/.
19. Lopez-Franco C, Gomez-Avila J, Alanis A, Arana-Daniel N & Villaseñor C (2017). Visual Servoing for an Autonomous Hexarotor Using a Neural Network Based PID Controller. Sensors, 17(8), 1865. https://doi.org/10.3390/s17081865
20. Lu A, Ding W & Li H (2013). Multi-information Based Safe Area Step Selection Algorithm for UAV’s Emergency Forced Landing. Journal of Software, 8(4), 995-1002.
21. Lunghi P, Ciarambino M & Lavagna M (2015). Vision-Based Hazard Detection With Artificial Neural Networks for Autonomous Planetary Landing. In 13th ESA/ESTEC Symposium on Advanced Space Technologies in Robotics and Automation, ASTRA
22. Martin S, Bange J & Beyrich F (2010). Meteorological profiling of the lower troposphere using the research UAV "M2AV Carolo". Atmospheric Measurement Techniques Discussions 4, 705–716. DOI:10.5194/amt-4-705-2011
23. Mazeh H, Saied M, Shraim H and F. Clovis (2018). Fault-Tolerant Control of an Hexarotor Unmanned Aerial Vehicle Applying Outdoor Tests and Experiments. IFAC-PapersOnLine 51(22), 312-317. https://doi.org/10.1016/j.ifacol.2018.11.560
24. Nemati A, Sarim M, Hashemi M, Schnipke E et al. (2015). Autonomous Navigation of UAV through GPS-Denied Indoor Environment with Obstacles. AIAA SciTech, Kissimmee, Florida
25. Nguyen N P, Mung N X & Hong S K (2019). Actuator Fault Detection and Fault-Tolerant Control for Hexacopter. Sensors 19(21), 4721. https://doi.org/10.3390/s19214721
26. PropotoUAV (2019). Drones are being used for weather forecasting by meteorologists. from https://www.prophotouav.com/meteorologists-storm-weather-drones/.
27. Sani M F & Karimian G (2017). Automatic navigation and landing of an indoor AR. drone quadrotor using ArUco marker and inertial sensors. 2017 International Conference on Computer and Drone Applications (IConDA), Kuching, Malaysia. DOI: 10.1109/ICONDA.2017.8270408
28. Veroustraete F (2015). The Rise of the Drones in Agriculture. EC Agriculture 2(2), 325-327.
29. Zhao H & Wang Z (2012). Motion Measurement Using Inertial Sensors, Ultrasonic Sensors, and Magnetometers With Extended Kalman Filter for Data Fusion. IEEE Sensors Journal - IEEE SENS J 12(5), 943-953. DOI: 10.1109/JSEN.2011.2166066