Integrating Unmanned Aerial Vehicles in Airspace: A Systematic Review

Year 2024, , 89 - 115, 28.02.2024

Arif Tuncal , Ufuk Erol

https://doi.org/10.51785/jar.1393271

Abstract

In this article, a comprehensive review of the integration of Unmanned Aerial Vehicles (UAVs) into shared airspace is presented. By applying a systematic review methodology, the study clarifies the main challenges, problems, and possible fixes related to safety, coordination, and regulatory frameworks. The results demonstrate the critical role that several elements play in supporting the safety of UAV integration. These elements include multi-layered airspace models, careful path planning, secure communication networks, Conflict Detection and Resolution (CDR) strategies, and strong regulations. The paper explores the potential of Human-in-the-loop Reinforcement Learning (HRL) and Reinforcement Learning (RL) algorithms to train UAVs for maneuvering through complex terrain and adapting to changing circumstances. The study's conclusions highlight the importance of ongoing research projects, stakeholder cooperation and continuous support for technology developments-all of which are necessary to ensure the safe and orderly integration of UAVs into airspace.

Keywords

Reinforcement Learning, Unmanned Aerial Vehicle, Unmanned Traffic Management, Autonomous Vehicles, Artificial Intelligence

İnsansız Hava Araçlarının Hava Sahasına Entegrasyonu: Sistematik Bir İnceleme

Abstract

Bu makalede, İnsansız Hava Araçlarının (İHA) ortak hava sahasına entegrasyonu kapsamlı bir şekilde incelenmektedir. Sistematik inceleme metodolojisi kullanılarak çalışmada yasal düzenlemeler, uçuş emniyeti ve koordinasyon ile ilgili temel zorlukları, sorunları ve olası çözümleri ortaya koymaktadır. Bulgular çok katmanlı hava sahası modelleri, dikkatli rota planlama, güvenli iletişim ağları, çatışma tespiti ve çözümü stratejileri ile yapısal olarak güçlendirilmiş düzenlemeler dahil olmak üzere çeşitli unsurların İHA entegrasyonunda kritik bir rol oynadığını göstermektedir. Ayrıca İHA'ların karmaşık hava sahalarında ve değişken koşullara uyum sağlamalarını desteklemek adına önerilen çözümleri inceleyerek Reinforcement Learning (RL) ve Human-in-the-loop Reinforcement Learning (HRL) algoritmalarının potansiyeli ortaya konmuştur. Çalışmanın sonuçları, İHA'ların hava sahasına emniyetli ve düzenli bir şekilde entegre edilmesi için araştırma projelerinin sürekli olarak yürütülmesinin, paydaş işbirliğinin ve teknoloji geliştirmelerine kararlı desteğin önemini vurgulamaktadır.

Keywords

İnsansız Hava Aracı, İnsansız Hava Aracı Sistemleri Trafik Yönetimi, Reinforcement Learning, Otonom Araçlar, Yapay Zeka

References

• Abir, M. A. B. S., & Chowdhury, M. Z. (2023). Digital twin-based software-defined UAV networks using queuing model. In 2023 10th International Conference on Signal Processing and Integrated Networks (SPIN) (pp. 479-483). IEEE. DOI: 10.1109/SPIN57001.2023.10116319
• Abulkasim, H., Goncalves, B., Mashatan, A., & Ghose, S. (2022). Authenticated secure quantum-based communication scheme in Internet-of-drones deployment. IEEE Access, 10, 94963-94972. DOI: 10.1109/ACCESS.2022.3204793
• Acevedo, J. J., Capitán, C., Capitiin, J., Castaño, A. R., & Ollero, A. (2020). A geometrical approach based on 4D grids for conflict management of multiple UAVs operating in U-space. In 2020 International Conference on Unmanned Aircraft Systems (ICUAS). DOI: 10.1109/ICUAS48674.2020.9213929
• Adoni, W. Y. H., Lorenz, S., Fareedh, J. S., Gloaguen, R., & Bussmann, M. (2023). Investigation of autonomous multi-UAV systems for target detection in distributed environment: Current developments and open challenges. Drones, 7(4), 263. https://doi.org/10.3390/drones7040263
• Ahn, E., & Kang, H. (2018). Introduction to systematic review and meta-analysis. Korean journal of anesthesiology, 71(2), 103-112. https://doi.org/10.4097/kjae.2018.71. 2.103
• Alharasees, O., Abdalla, M. S., & Kale, U. (2022). Analysis of human factors analysis and classification system (HFACS) of UAV operators. In 2022 New Trends in Aviation Development (NTAD) (pp. 10-14). IEEE. DOI: 10.1109/NTAD57912.2022. 10013492
• Alharbi, A., Poujade, A., Malandrakis, K., Petrunin, I., Panagiotakopoulos, D., & Tsourdos, A. (2020). Rule-based conflict management for unmanned traffic management scenarios. In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC). DOI: 10.1109/DASC50938.2020.9256690
• Al-Shareeda, M. A., Saare, M. A., & Manickam, S. (2023). Unmanned aerial vehicle: A review and future directions. Indonesian Journal of Electrical Engineering and Computer Science (IJEECS), 30(2), 778-786. http://doi.org/10.11591/ijeecs.v30. i2.pp778-786
• Altin, U. C. (2020). Evolutionary reinforcement learning for the coordination of swarm UAVs. In 2020 28th Signal Processing and Communications Applications Conference (SIU) (pp. 1-4). IEEE. DOI: 10.1109/SIU49456.2020.9302227
• An, G., Wu, Z., Shen, Z., Shang, K., & Ishibuchi, H. (2023). Evolutionary multi-objective deep reinforcement learning for autonomous UAV navigation in large-scale complex environments. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 633-641). https://doi.org/10.1145/3583131.3590446
• Arani, A. H., Azari, M. M., Hu, P., Zhu, Y., Yanikomeroglu, H., & Safavi-Naeini, S. (2021). Reinforcement learning for energy-efficient trajectory design of UAVs. IEEE Internet of Things Journal, 9(11), 9060-9070. DOI: 10.1109/JIOT.2021.3118322
• Ayhan, B., Kwan, C., Um, Y. B., Budavari, B., & Larkin, J. (2018). Semi-automated emergency landing site selection approach for UAVs. IEEE Transactions on Aerospace and Electronic Systems, 55(4), 1892-1906. DOI: 10.1109/TAES.2018. 2879529
• Balestrieri, E., Daponte, P., De Vito, L., Picariello, F., & Tudosa, I. (2021). Guidelines for an Unmanned Aerial vehicle-based measurement instrument design. IEEE Instrumentation & Measurement Magazine, 24(4), 89-95. DOI: 10.1109/MIM.2021. 9448256
• Barnhart, R. K., Marshall, D. M., & Shappee, E. (2021). Introduction to unmanned aircraft systems, 3e. Boca Raton: CRC Press.
• Bartolomei, L., Kompis, Y., Teixeira, L., & Chli, M. (2022). Autonomous emergency landing for multicopters using deep reinforcement learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3392-3399). IEEE. DOI: 10.1109/IROS47612.2022.9981152
• Bian J., Xie, M., & Şeker, R. (2013). Towards a secure and reliable communication network for large-scale UAV systems deployed in hostile environments. Computer Science.
• Blasi, L., D’Amato, E., Notaro, I., & Raspaolo, G. (2023). Clothoid-based path planning for a formation of fixed-wing UAVs. Electronics, 12(10), 2204. https://doi.org/10.3390/ electronics12102204
• Bolz, K., & Nowacki, G. (2023). Air transport safety in UAV operational conditions. Journal of civil engineering and transport, 5 (1). https://doi.org/10.24136/tren.2023.001
• Bu, N., Ge, J., Yang, J., & Ru, H. (2022). Emergency landing system of rotor UAV in complex ground environment. In International Conference on Autonomous Unmanned Systems (pp. 2954-2964). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0479-2_273
• Callaghan, A., Mason, K., & Mannion, P. (2023). Evolutionary strategy guided reinforcement learning via multibuffer communication. arXiv preprint arXiv:2306.11535. https://doi.org/10.48550/arXiv.2306.11535
• Celestini, D., Primatesta, S., & Capello, E. (2022). Trajectory planning for UAVs based on interfered fluid dynamical system and Bézier curves. IEEE Robotics and Automation Letters, 7(4), 9620-9626. DOI: 10.1109/LRA.2022.3191855
• Chasanah, N., Rismayanti, I., Kusuma, W. T., Pranoto, F. S., Prabowo, Y., & Kusumoaji, D. (2022). Performance investigation of link failure line-of-sight (LOS) communication UAV. In 2022 IEEE International Conference on Aerospace Electronics and Remote Sensing Technology (ICARES) (pp. 1-6). IEEE. DOI: 10.1109/ICARES56907.2022. 9993526
• Chen, X., Zhao, N., Chang, Z., Hämäläinen, T., & Wang, X. (2023). UAV-aided secure short-packet data collection and transmission. IEEE Transactions on Communications. DOI: 10.1109/TCOMM.2023.3244954
• Chen, Y. Y., Chen, Y. L., & Zhou, B. H. (2014). Robust guidance law design for UAVs. In 11th IEEE International Conference on Control & Automation (ICCA) (pp. 44-49).
• Chin, C., Gopalakrishnan, K., Balakrishnan, H., Egorov, M., & Evans, A. (2021). Efficient and fair traffic flow management for on-demand air mobility. CEAS Aeronautical Journal, 1-11. https://link.springer.com/article/10.1007/s13272-021-00553-3
• Choi, J., Kim, H. M., Hwang, H. J., Kim, Y. D., & Kim, C. O. (2023). Modular reinforcement learning for autonomous UAV flight control. Drones, 7(7), 418. https://doi.org/10. 3390drones7070418
• Chronis, C., Anagnostopoulos, G., Politi, E., Garyfallou, A., Varlamis, I., & Dimitrakopoulos, G. (2023). Path planning of autonomous UAVs using reinforcement learning. In Journal of Physics: Conference Series (Vol. 2526, No. 1, p. 012088). IOP Publishing. DOI 10.1088/1742-6596/2526/1/012088
• Çınar, E., & Tuncal, A. (2023). A Comprehensive analysis of society's perspective on urban air mobility. Journal of Aviation, 7(3), 353-364. https://doi.org/10.30518/ jav.1324997
• Conrad, A., Isaac, S., Cochran, R., Sanchez-Rosales, D., Rezaei, T., Javid, T., Schroeder, A.J., Golba, G., Gauthier, D., & Kwiat, P. (2023). Drone-based quantum communication links. In Quantum Computing, Communication, and Simulation III (Vol. 12446, pp. 99-106). SPIE. https://doi.org/10.1117/12.2647923
• Cracknell, A. P. (2017). UAVs: Regulations and law enforcement. International Journal of Remote Sensing, 38(8-10), 3054-3067. https://doi.org/10.1080/01431161. 2017.1302115
• Davies, L., Vagapov, Y., Grout, V., Cunningham, S., & Anuchin, A. (2021). Review of air traffic management systems for UAV integration into urban airspace. In 2021 28th International Workshop on Electric Drives: Improving Reliability of Electric Drives (IWED) (pp. 1-6). IEEE. DOI: 10.1109/IWED52055.2021.9376343
• El Asslouj, A., Atkins, E., & Rastgoftar, H. (2023). Can a laplace PDE define air corridors through low-altitude airspace?. In 2023 International Conference on Unmanned Aircraft Systems (ICUAS) (pp. 1-8). IEEE. DOI: 10.1109/ICUAS57906. 2023.10180409
• Geister, D., & Korn, B. (2013). Operational integration of UAS into the ATM system. In AIAA Infotech@ Aerospace (I@ A) Conference (p. 5051). https://doi.org/10.2514/ 6.2013-5051
• Gong, S., Wang, M., Gu, B., Zhang, W., Hoang, D. T., & Niyato, D. (2023). Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in Multi-UAV networks. IEEE Transactions on Vehicular Technology. https://doi.org/10.48550/arXiv.2212.13396
• Han, X., Wang, J., Zhang, Q., Qin, X., & Sun, M. (2019). Multi-UAV automatic dynamic obstacle avoidance with experience-shared a2c. In 2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob) (pp. 330-335). IEEE. DOI: 10.1109/WiMOB.2019.8923344
• Ho, F., Geraldes, R., Goncalves, A., Cavazza, M., & Prendinger, H. (2018b). Simulating shared airspace for service UAVs with conflict resolution. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (pp. 2192-2194). https://dl.acm.org/doi/10.5555/3237383.3238117
• Ho, F., Geraldes, R., Gonçalves, A., Cavazza, M., & Prendinger, H. (2018a). Improved conflict detection and resolution for service UAVs in shared airspace. IEEE Transactions on Vehicular Technology, 68(2), 1231-1242. DOI: 10.1109/TVT. 2018.2889459
• Isufaj, R., Omeri, M., & Piera, M. A. (2022). Multi-UAV conflict resolution with graph convolutional reinforcement learning. Applied Sciences, 12(2), 610. https://doi.org /10.3390/app12020610.
• Jiang, X., Chen, X., Tang, J., Zhao, N., Zhang, X. Y., Niyato, D., & Wong, K. K. (2021). Covert communication in UAV-assisted air-ground networks. IEEE Wireless Communications, 28(4), 190-197. DOI: 10.1109/MWC.001.2000454
• Kainrath, K., Gruber, M., Hinze, A., Fluehr, H., & Leitgeb, E. (2022). Towards unmanned aerial vehicle UTM-integration using mobile radio networks. In 2022 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO) (pp. 465-469). IEEE. DOI: 10.23919/MIPRO55190. 2022.9803420
• Kang, H., Li, W., Mišić, J., Mišić, V. B., & Chang, X. (2022). Dual-UAV aided secure dynamic G2U communication. In 2022 IEEE Symposium on Computers and Communications (ISCC) (pp. 1-6). IEEE. DOI: 10.1109/ISCC55528.2022.9912939
• Khan, A., Ferramosca, M. L., Ivaki, N., & Madeira, H. (2022). Classifying fault category and severity of UAV flight controllers’ reported issues. In 2022 6th International Conference on System Reliability and Safety (ICSRS) (pp. 45-54). IEEE. DOI: 10.1109/ICSRS56243.2022.10067593
• Kim, Y. W., Lee, D. Y., Tahk, M. J., & Lee, C. H. (2020). A new path planning algorithm for forced landing of UAVs in emergency using velocity prediction method. In 2020 28th Mediterranean Conference on Control and Automation (MED) (pp. 62-66). IEEE. DOI: 10.1109/MED48518.2020.9183166
• Konert, A., & Kasprzyk, P. (2021). UAS safety operation–legal issues on reporting UAS incidents. Journal of Intelligent & Robotic Systems, 103(3), 51. https://link.springer. com/article/10.1007/s10846-021-01448-5
• Kumar, A., Krishnamurthi, R., Sharma, G., Jain, S., Srikanth, P., Sharma, K., & Aneja, N. (2023). Revolutionizing modern networks: Advances in AI, machine learning, and blockchain for quantum satellites and UAV-based communication. arXiv preprint arXiv:2303.11753. https://doi.org/10.48550/arXiv.2303.11753
• Labib, N. S., Danoy, G., Musial, J., Brust, M. R., & Bouvry, P. (2019a). A multilayer low-altitude airspace model for UAV traffic management. In Proceedings of the 9th ACM Symposium on Design and Analysis of Intelligent Vehicular
• Networks and Applications (pp. 57-63). https://doi.org/10.1145/3345838.3355998 Labib, N.S., Danoy, G., Musial, J., Brust, M. R., & Bouvry, P. (2019b). Internet of unmanned aerial vehicles—A multilayer low-altitude airspace model for distributed UAV traffic management. Sensors, 19(21), 4779. https://doi.org/10.3390/s19214779
• Lamba, M. A., Tangade, S. S., Nawaz, S. S., & Manvi, S. S. (2021). Path planning scheme for collision avoidance in unmanned aerial vehicle traffic management system. In 2021 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) (pp. 1-5). IEEE. DOI: 10.1109/CONECCT52877.2021. 9622656
• Li, G., Zuo, H., & Xu, J. (2023). Research on the influence of UAV Anti-collision device on aerodynamic shape. In Journal of Physics: Conference Series (Vol. 2477, No. 1, p. 012096). IOP Publishing. DOI:10.1088/1742-6596/2477/1/012096
• Li, Q., Zhang, D., Wang, H., Liu, K., & Liu, Y. (2022). A design method for the inspection network of over-the-horizon UAV based on 4G/5G communication network. In 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE) (pp. 240-244). IEEE. DOI: 10.1109/ICCECE54139.2022.9712840
• Li, X., Fang, J., Du, K., Mei, K., & Xue, J. (2023). UAV obstacle avoidance by human-in-the-loop reinforcement in arbitrary 3D environment. arXiv preprint arXiv:2304.05959. https://doi.org/10.48550/arXiv.2304.05959
• Lingyun, Z. H. O. U., Xiaotong, Z. H. A. O., Xin, G. U. A. N., Enbin, S. O. N. G., Xin, Z. E. N. G., & Qingjiang, S. H. I. (2022). Robust trajectory planning for UAV communication systems in the presence of jammers. Chinese Journal of Aeronautics, 35(10), 265-274. https://doi.org/10.1016/j.cja.2021.10.038
• Liu, X., Liu, Y., & Chen, Y. (2019). Reinforcement learning in multiple-UAV networks: Deployment and movement design. IEEE Transactions on Vehicular Technology, 68(8), 8036-8049. DOI: 10.1109/TVT.2019.2922849
• Liu, Z., Di, X., Wang, Q., & Wang, L. (2023). Path planning based on joint distribution of distribution vehicles and UAVs. In 2023 IEEE 3rd International Conference on Electronic Technology, Communication and Information (ICETCI) (pp. 1504-1508). IEEE. DOI: 10.1109/ICETCI57876.2023.10176924
• Lu, W., Mo, Y., Feng, Y., Gao, Y., Zhao, N., Wu, Y., & Nallanathan, A. (2022). Secure transmission for multi-UAV-assisted mobile edge computing based on reinforcement learning. IEEE Transactions on Network Science and Engineering, 10(3), 1270-1282. DOI: 10.1109/TNSE.2022.3185130
• Luo, B., Wu, Z., Zhou, F., & Wang, B. C. (2023). Human-in-the-loop reinforcement learning in continuous-action space. IEEE Transactions on Neural Networks and Learning Systems. DOI: 10.1109/TNNLS.2023.3289315
• Luo, H., Wu, Y., Sun, G., Yu, H., Xu, S., & Guizani, M. (2023). ESCM: An efficient and secure communication mechanism for UAV networks. arXiv preprint arXiv:2304.13244. https://doi.org/10.48550/arXiv.2304.13244
• Maurya, H. L., Singh, P., Yogi, S., Behera, L., & Verma, N. K. (2022). An intelligent game theory approach for collision avoidance of multi-UAVs. In Proceedings of International Conference on Computational Intelligence: ICCI 2021 (pp. 27-39). Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-19-2126-1_3
• McTegg, S. J., Tarsha Kurdi, F., Simmons, S., & Gharineiat, Z. (2022). Comparative approach of unmanned aerial vehicle restrictions in controlled airspaces. Remote Sensing, 14(4), 822. https://doi.org/10.3390/rs14040822
• Mohsan, S. A. H., Othman, N. Q. H., Li, Y., Alsharif, M. H., & Khan, M. A. (2023). Unmanned aerial vehicles (UAVs): Practical aspects, applications, open challenges, security issues, and future trends. Intelligent Service Robotics, 16(1), 109-137. https://link.springer.com/article/10.1007/s11370-022-00452-4
• Newman, M., & Gough, D. (2020). Systematic reviews in educational research: Methodology, perspectives and application. https://link.springer.com/chapter/ 10.1007/978-3-658-27602-7_1
• Patrikar, J., Dantas, J., Ghosh, S., Kapoor, P., Higgins, I., Aloor, J. J., Navarro, I., Sun, J., Stoler, B., Hamidi, M., Baijal, R., Moon, B., Oh, J., & Scherer, S. (2022). Challenges in close-proximity safe and seamless operation of manned and unmanned aircraft in shared airspace. arXiv preprint arXiv:2211.06932. https://doi.org/10.48550/arXiv. 2211.06932
• Picard, G. (2022). Trajectory Coordination based on distributed constraint optimization techniques in unmanned air traffic management. In 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2022). https://dl.acm.org/doi/abs/10.5555/3535850.3535969
• Quan, Y., Cheng, N., Wang, X., Shen, J., Ma, L., & Yin, Z. (2023). Interpretable and secure trajectory optimization for UAV-assisted communication. In 2023 IEEE/CIC International Conference on Communications in China (ICCC) (pp. 1-6). IEEE. https://doi.org/10.48550/arXiv.2307.02002
• Radanovic, M., Omeri, M., & Piera, M. A. (2019). Test analysis of a scalable UAV conflict management framework. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 233(16), 6076-6088. DOI: 10.1177/0954410019875241
• Raju, P., Rios, J., & Jordan, A. (2018). UTM—A complementary set of services to ATM. In 2018 Integrated Communications, Navigation, Surveillance Conference (ICNS) (pp. 2F2-1). IEEE. DOI: 10.1109/ICNSURV.2018.8384849
• Ralegankar, V. K., Bagul, J., Thakkar, B., Gupta, R., Tanwar, S., Sharma, G., & Davidson, I. E. (2021). Quantum cryptography-as-a-service for secure UAV communication: Applications, challenges, and case study. IEEE Access, 10, 1475-1492. DOI: 10.1109/ACCESS.2021.3138753
• Rithic, C. H., & Arulmozhi, N. (2023). Real-time implementation of RF-based mobile fleet localization and collision avoidance system in wireless sensor network for drones and gliders. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 1459-1465). IEEE. DOI: 10.1109/ICICCS56967.2023.10142713
• Saraçyakupoğlu, T., Delibaş, H. D., & Özçelik, A. D. (2022). A computational determination of a nozzle activated fixed-wing UAV. International Journal of 3D Printing Technologies and Digital Industry, 6(2), 292-306. https://doi.org/10.46519/ ij3dptdi.1128158
• Shan, L., Li, H. B., Miura, R., Matsuda, T., & Matsumura, T. (2023). A novel collision avoidance strategy with D2D communications for UAV systems. Drones, 7(5), 283. https://doi.org/10.3390/drones7050283
• Sharma, S., Kulkarni, P., & Pathak, P. (2022). Applications of unmanned aerial vehicles (UAVs) for improved business management. In 2022 International Interdisciplinary Humanitarian Conference for Sustainability (IIHC) (pp. 53-57). IEEE. DOI: 10.1109/IIHC55949.2022.10060638
• Shi, H. R., Lu, F. X., Wu, L., & Xia, J. W. (2022). Trajectory optimization of multi-UAVs for marine target tracking during approaching stage. Mathematical Problems in Engineering, 2022. https://doi.org/10.1155/2022/5472105
• Shrestha, R., Kim, D., Choi, J., & Kim, S. (2022). A novel E/E architecture for low altitude UAVs. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 346-350). IEEE. DOI: 10.1109/ISCAS48785.2022.9937942
• Shrestha, R., Zevenbergen, J., Panday, U. S., Awasthi, B., & Karki, S. (2019). Revisiting the current UAV regulations in Nepal: A step towards the legal dimension for UAVs' efficient application. The International Archives of the
• Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 107-114. https://doi.org/10.5194/isprs-archives-XLII-5-W3-107-2019
• Sun, J., Zhang, H., Xu, W., Li, H., Zhang, J., Ke, J., & Yan, H. (2021). Improving security performance of dual UAVs system with unknown eavesdropper location. In Proceedings of the International Conference on Internet-of-Things Design and Implementation (pp. 257-258). https://doi.org/10.1145/3450268.3453509
• Sun, S., & Dang, S. (2022). Study on collision avoidance strategy of multiple UAVs based on genetic algorithm. In 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022) (Vol. 12257, pp. 303-309). SPIE. DOI: 10.1117/12.2639508
• Sun, Y., Li, L., Zhou, C., Yang, S., Shi, D., & An, H. (2022). Design and Implementation of a collaborative air-ground unmanned system path planning framework. In China Intelligent Robotics Annual Conference (pp. 83-96). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0301-6_7
• Tang, G., Du, P., Lei, H., Ansari, I. S., & Fu, Y. (2021). Trajectory design and communication resources allocation for wireless powered secure UAV communication systems. IEEE Systems Journal, 16(4), 6300-6308. DOI: 10.1109/JSYST.2021.3132010
• Taylor, M. E. (2023). Reinforcement Learning Requires Human-in-the-Loop Framing and Approaches. In HHAI (pp. 351-360). https://alaworkshop2023.github.io/papers/ ala2023_paper_29.pdf
• Tovarnov, M. S., & Bykov, N. V. (2022). Reinforcement learning reward function in unmanned aerial vehicle control tasks. In Journal of Physics: Conference Series (Vol. 2308, No. 1, p. 012004). IOP Publishing. DOI 10.1088/1742-6596/2308/1/012004
• Volkert, A., Hackbarth, H., Lieb, T. J., & Kern, S. (2019). Flight tests of ranges and latencies of a threefold redundant C2 multi-link solution for small drones in VLL airspace. In 2019 Integrated Communications, Navigation and Surveillance Conference (ICNS) (pp. 1-14). IEEE. DOI: 10.1109/ICNSURV.2019.8735265
• Wang, T., Xiang, S., Men, Z., Ye, M., Zhang, Y., Xie, A., & Zhejiang Lab. (2023). An emergency landing spot detection algorithm based on semantic segmentation and safety evaluation. Presented at Forum 79. DOI: https://doi.org/10.4050/F-0079-2023-18018.
• Wang, W., Wei, X., Jia, Y., & Chen, M. (2023). UAV relay network deployment through the area with barriers. Ad Hoc Networks, 103222. https://doi.org/10.1016/ j.adhoc.2023.103222
• Wang, Y., Wang, H., Wen, J., Lun, Y., & Wu, J. (2020). Obstacle avoidance of UAV based on neural networks and interfered fluid dynamical system. In 2020 3rd International Conference on Unmanned Systems (ICUS) (pp. 1066-1071). DOI: 10.1109/ICUS50048.2020.9274988
• Wei, S., Li, L., Chen, G., Blasch, E., Chang, K. C., Clemons, T. M., & Pham, K. (2023). ROSIS: Resilience oriented security inspection system against false data injection attacks. In 2023 IEEE Aerospace Conference (pp. 1-11). IEEE. DOI: 10.1109/AERO55745.2023.10115584
• Wei, Y., Zhao, M., Zhang, F., & Hu, Y. (2004). Research of a heuristic reward function for reinforcement learning algorithms. In Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No. 04EX788) (Vol. 3, pp. 2676-2680). IEEE. DOI: 10.1109/WCICA.2004.1342083
• Wiedemann, M., Vij, A., & Banerjee, R. (2023). Validating the benefits of increased drone uptake for Australia: Geographic, demographic and social insights. Department of Infrastructure, Transport, Regional Development, Communications and the Arts (Australia). https://apo.org.au/node/322458
• Wijnker, D., van Dijk, T., Snellen, M., de Croon, G., & De Wagter, C. (2019). Hear-and-avoid for UAVs using convolutional neural networks. In Proceedings of the 11th International Micro Air Vehicle Competition and Conference (IMAV2019), Madrid, Spain (Vol. 30). https://www.imavs.org/papers/2019/19.pdf
• Wu, J., Yuan, W., & Hanzo, L. (2023). When UAVs meet ISAC: real-time trajectory design for secure communications. arXiv preprint arXiv:2306.14140. https://doi.org.10.48550/arXiv.2306.14140
• Wu, X., Lei, Y., Tong, X., Zhang, Y., Li, H., Qiu, C., Guo, C., Sun, Y., & Lai, G. (2022). A Non-rigid hierarchical discrete grid structure and its application to UAVs conflict detection and path planning. IEEE Transactions on Aerospace and Electronic Systems, 58(6), 5393-5411. DOI: 10.1109/TAES.2022.3170323
• Wubben, J., Calafate, C. T., Cano, J. C., & Manzoni, P. (2023). FFP: A force field protocol for the tactical management of UAV conflicts. Ad Hoc Networks, 140, 103078. https://doi.org/10.1016/j.adhoc.2022.103078
• Xiang, T., Jiang, F., Hao, Q., & Cong, W. (2016). Adaptive flight control for quadrotor UAVs with dynamic inversion and neural networks. In 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) (pp. 174-179). DOI: 10.1109/MFI.2016.7849485
• Xiao, Q., Zhang, X., Jiang, L., & Wang, M. (2022). Design of reward functions based on The DDQN Algorithm. In 2022 14th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA) (pp. 600-604). IEEE. DOI: 10.1109/ICMTMA54903.2022.00125
• Xie, R., Huang, C., Wang, Z., & Han, J. (2022). A deep reinforcement learning algorithm based on short-term advantage for air game decision-making. In International Conference on Autonomous Unmanned Systems (pp. 3884-3894). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0479-2_359
• Xu, J., Wu, W., & Sun, Y. (2022). Multi-UAVs trajectory planning method with coordinated attack angle-time constraints. In 2022 IEEE International Conference on Unmanned Systems (ICUS). DOI: 10.1109/ICUS55513.2022.9987057
• Xu, Z., Deng, D., Dong, Y., & Shimada, K. (2022). DPMPC-planner: A real-time UAV trajectory planning framework for complex static environments with dynamic obstacles. In 2022 International Conference on Robotics and Automation (ICRA) (pp. 250-256). IEEE. DOI: 10.1109/ICRA46639.2022.9811886
• Xue, J., Zhu, J., Du, J., Kang, W., & Xiao, J. (2023). Dynamic path planning for multiple UAVs with incomplete information. Electronics, 12(4), 980. https://doi.org/10.3390/ electronics12040980
• Yang, T., De Maio, A., Zheng, J., Su, T., Carotenuto, V., & Aubry, A. (2021). An adaptive radar signal processor for UAVs detection with super-resolution capabilities. IEEE Sensors Journal, 21(18), 20778-20787. DOI: 10.1109/JSEN.2021.3093779
• Ye, B., Li, J., Li, J., Liu, C., Li, J., & Yang, Y. (2023). Deep reinforcement learning-based diving/pull-out control for bioinspired morphing UAVs. Unmanned Systems, 11(02), 191-202. https://doi.org/10.1142/S2301385023410066
• Yin, S., & Yu, F. R. (2021). Resource allocation and trajectory design in UAV-aided cellular networks based on multiagent reinforcement learning. IEEE Internet of Things Journal, 9(4), 2933-2943. DOI: 10.1109/JIOT.2021.3094651
• Zhang, D., Li, X., Ren, G., Yao, J., Chen, K., & Li, X. (2023a). Three-dimensional path planning of UAVs in a complex dynamic environment based on environment exploration twin delayed deep deterministic policy gradient. Symmetry, 15(7), 1371. https://doi.org/10.3390/sym15071371
• Zhang, D., Xuan, Z., Zhang, Y., Yao, J., Li, X., & Li, X. (2023b). Path planning of unmanned aerial vehicle in complex environments based on state-detection twin delayed deep deterministic policy gradient. Machines, 11(1), 108. https://doi.org/10.3390/ machines11010108
• Zhang, S., Li, Y., Ye, F., Geng, X., Zhou, Z., & Shi, T. (2023). A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV motion planning for long trajectories with unpredictable obstacles. Drones, 7(5), 311. https://doi.org/10.3390/ drones7050311