İnsansız Hava Araçlarının Hava Sahasına Entegrasyonu: Sistematik Bir İnceleme

Abstract

Bu makalede, İnsansız Hava Araçlarının (İHA) ortak hava sahasına entegrasyonu kapsamlı bir şekilde incelenmektedir. Sistematik inceleme metodolojisi kullanılarak çalışmada yasal düzenlemeler, uçuş emniyeti ve koordinasyon ile ilgili temel zorlukları, sorunları ve olası çözümleri ortaya koymaktadır. Bulgular çok katmanlı hava sahası modelleri, dikkatli rota planlama, güvenli iletişim ağları, çatışma tespiti ve çözümü stratejileri ile yapısal olarak güçlendirilmiş düzenlemeler dahil olmak üzere çeşitli unsurların İHA entegrasyonunda kritik bir rol oynadığını göstermektedir. Ayrıca İHA'ların karmaşık hava sahalarında ve değişken koşullara uyum sağlamalarını desteklemek adına önerilen çözümleri inceleyerek Reinforcement Learning (RL) ve Human-in-the-loop Reinforcement Learning (HRL) algoritmalarının potansiyeli ortaya konmuştur. Çalışmanın sonuçları, İHA'ların hava sahasına emniyetli ve düzenli bir şekilde entegre edilmesi için araştırma projelerinin sürekli olarak yürütülmesinin, paydaş işbirliğinin ve teknoloji geliştirmelerine kararlı desteğin önemini vurgulamaktadır.

Keywords

• İnsansız Hava Aracı
• İnsansız Hava Aracı Sistemleri Trafik Yönetimi
• Reinforcement Learning
• Otonom Araçlar
• Yapay Zeka

Integrating Unmanned Aerial Vehicles in Airspace: A Systematic Review

Abstract

In this article, a comprehensive review of the integration of Unmanned Aerial Vehicles (UAVs) into shared airspace is presented. By applying a systematic review methodology, the study clarifies the main challenges, problems, and possible fixes related to safety, coordination, and regulatory frameworks. The results demonstrate the critical role that several elements play in supporting the safety of UAV integration. These elements include multi-layered airspace models, careful path planning, secure communication networks, Conflict Detection and Resolution (CDR) strategies, and strong regulations. The paper explores the potential of Human-in-the-loop Reinforcement Learning (HRL) and Reinforcement Learning (RL) algorithms to train UAVs for maneuvering through complex terrain and adapting to changing circumstances. The study's conclusions highlight the importance of ongoing research projects, stakeholder cooperation and continuous support for technology developments-all of which are necessary to ensure the safe and orderly integration of UAVs into airspace.

Keywords

• Reinforcement Learning
• Unmanned Aerial Vehicle
• Unmanned Traffic Management
• Autonomous Vehicles
• Artificial Intelligence

References

1. Abir, M. A. B. S., & Chowdhury, M. Z. (2023). Digital twin-based software-defined UAV networks using queuing model. In 2023 10th International Conference on Signal Processing and Integrated Networks (SPIN) (pp. 479-483). IEEE. DOI: 10.1109/SPIN57001.2023.10116319
2. Abulkasim, H., Goncalves, B., Mashatan, A., & Ghose, S. (2022). Authenticated secure quantum-based communication scheme in Internet-of-drones deployment. IEEE Access, 10, 94963-94972. DOI: 10.1109/ACCESS.2022.3204793
3. Acevedo, J. J., Capitán, C., Capitiin, J., Castaño, A. R., & Ollero, A. (2020). A geometrical approach based on 4D grids for conflict management of multiple UAVs operating in U-space. In 2020 International Conference on Unmanned Aircraft Systems (ICUAS). DOI: 10.1109/ICUAS48674.2020.9213929
4. Adoni, W. Y. H., Lorenz, S., Fareedh, J. S., Gloaguen, R., & Bussmann, M. (2023). Investigation of autonomous multi-UAV systems for target detection in distributed environment: Current developments and open challenges. Drones, 7(4), 263. https://doi.org/10.3390/drones7040263
5. Ahn, E., & Kang, H. (2018). Introduction to systematic review and meta-analysis. Korean journal of anesthesiology, 71(2), 103-112. https://doi.org/10.4097/kjae.2018.71. 2.103
6. Alharasees, O., Abdalla, M. S., & Kale, U. (2022). Analysis of human factors analysis and classification system (HFACS) of UAV operators. In 2022 New Trends in Aviation Development (NTAD) (pp. 10-14). IEEE. DOI: 10.1109/NTAD57912.2022. 10013492
7. Alharbi, A., Poujade, A., Malandrakis, K., Petrunin, I., Panagiotakopoulos, D., & Tsourdos, A. (2020). Rule-based conflict management for unmanned traffic management scenarios. In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC). DOI: 10.1109/DASC50938.2020.9256690
8. Al-Shareeda, M. A., Saare, M. A., & Manickam, S. (2023). Unmanned aerial vehicle: A review and future directions. Indonesian Journal of Electrical Engineering and Computer Science (IJEECS), 30(2), 778-786. http://doi.org/10.11591/ijeecs.v30. i2.pp778-786

9. Altin, U. C. (2020). Evolutionary reinforcement learning for the coordination of swarm UAVs. In 2020 28th Signal Processing and Communications Applications Conference (SIU) (pp. 1-4). IEEE. DOI: 10.1109/SIU49456.2020.9302227
10. An, G., Wu, Z., Shen, Z., Shang, K., & Ishibuchi, H. (2023). Evolutionary multi-objective deep reinforcement learning for autonomous UAV navigation in large-scale complex environments. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 633-641). https://doi.org/10.1145/3583131.3590446
11. Arani, A. H., Azari, M. M., Hu, P., Zhu, Y., Yanikomeroglu, H., & Safavi-Naeini, S. (2021). Reinforcement learning for energy-efficient trajectory design of UAVs. IEEE Internet of Things Journal, 9(11), 9060-9070. DOI: 10.1109/JIOT.2021.3118322
12. Ayhan, B., Kwan, C., Um, Y. B., Budavari, B., & Larkin, J. (2018). Semi-automated emergency landing site selection approach for UAVs. IEEE Transactions on Aerospace and Electronic Systems, 55(4), 1892-1906. DOI: 10.1109/TAES.2018. 2879529
13. Balestrieri, E., Daponte, P., De Vito, L., Picariello, F., & Tudosa, I. (2021). Guidelines for an Unmanned Aerial vehicle-based measurement instrument design. IEEE Instrumentation & Measurement Magazine, 24(4), 89-95. DOI: 10.1109/MIM.2021. 9448256
14. Barnhart, R. K., Marshall, D. M., & Shappee, E. (2021). Introduction to unmanned aircraft systems, 3e. Boca Raton: CRC Press.
15. Bartolomei, L., Kompis, Y., Teixeira, L., & Chli, M. (2022). Autonomous emergency landing for multicopters using deep reinforcement learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3392-3399). IEEE. DOI: 10.1109/IROS47612.2022.9981152
16. Bian J., Xie, M., & Şeker, R. (2013). Towards a secure and reliable communication network for large-scale UAV systems deployed in hostile environments. Computer Science.
17. Blasi, L., D’Amato, E., Notaro, I., & Raspaolo, G. (2023). Clothoid-based path planning for a formation of fixed-wing UAVs. Electronics, 12(10), 2204. https://doi.org/10.3390/ electronics12102204
18. Bolz, K., & Nowacki, G. (2023). Air transport safety in UAV operational conditions. Journal of civil engineering and transport, 5 (1). https://doi.org/10.24136/tren.2023.001
19. Bu, N., Ge, J., Yang, J., & Ru, H. (2022). Emergency landing system of rotor UAV in complex ground environment. In International Conference on Autonomous Unmanned Systems (pp. 2954-2964). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0479-2_273
20. Callaghan, A., Mason, K., & Mannion, P. (2023). Evolutionary strategy guided reinforcement learning via multibuffer communication. arXiv preprint arXiv:2306.11535. https://doi.org/10.48550/arXiv.2306.11535
21. Celestini, D., Primatesta, S., & Capello, E. (2022). Trajectory planning for UAVs based on interfered fluid dynamical system and Bézier curves. IEEE Robotics and Automation Letters, 7(4), 9620-9626. DOI: 10.1109/LRA.2022.3191855
22. Chasanah, N., Rismayanti, I., Kusuma, W. T., Pranoto, F. S., Prabowo, Y., & Kusumoaji, D. (2022). Performance investigation of link failure line-of-sight (LOS) communication UAV. In 2022 IEEE International Conference on Aerospace Electronics and Remote Sensing Technology (ICARES) (pp. 1-6). IEEE. DOI: 10.1109/ICARES56907.2022. 9993526
23. Chen, X., Zhao, N., Chang, Z., Hämäläinen, T., & Wang, X. (2023). UAV-aided secure short-packet data collection and transmission. IEEE Transactions on Communications. DOI: 10.1109/TCOMM.2023.3244954
24. Chen, Y. Y., Chen, Y. L., & Zhou, B. H. (2014). Robust guidance law design for UAVs. In 11th IEEE International Conference on Control & Automation (ICCA) (pp. 44-49).
25. Chin, C., Gopalakrishnan, K., Balakrishnan, H., Egorov, M., & Evans, A. (2021). Efficient and fair traffic flow management for on-demand air mobility. CEAS Aeronautical Journal, 1-11. https://link.springer.com/article/10.1007/s13272-021-00553-3
26. Choi, J., Kim, H. M., Hwang, H. J., Kim, Y. D., & Kim, C. O. (2023). Modular reinforcement learning for autonomous UAV flight control. Drones, 7(7), 418. https://doi.org/10. 3390drones7070418
27. Chronis, C., Anagnostopoulos, G., Politi, E., Garyfallou, A., Varlamis, I., & Dimitrakopoulos, G. (2023). Path planning of autonomous UAVs using reinforcement learning. In Journal of Physics: Conference Series (Vol. 2526, No. 1, p. 012088). IOP Publishing. DOI 10.1088/1742-6596/2526/1/012088
28. Çınar, E., & Tuncal, A. (2023). A Comprehensive analysis of society's perspective on urban air mobility. Journal of Aviation, 7(3), 353-364. https://doi.org/10.30518/ jav.1324997
29. Conrad, A., Isaac, S., Cochran, R., Sanchez-Rosales, D., Rezaei, T., Javid, T., Schroeder, A.J., Golba, G., Gauthier, D., & Kwiat, P. (2023). Drone-based quantum communication links. In Quantum Computing, Communication, and Simulation III (Vol. 12446, pp. 99-106). SPIE. https://doi.org/10.1117/12.2647923
30. Cracknell, A. P. (2017). UAVs: Regulations and law enforcement. International Journal of Remote Sensing, 38(8-10), 3054-3067. https://doi.org/10.1080/01431161. 2017.1302115
31. Davies, L., Vagapov, Y., Grout, V., Cunningham, S., & Anuchin, A. (2021). Review of air traffic management systems for UAV integration into urban airspace. In 2021 28th International Workshop on Electric Drives: Improving Reliability of Electric Drives (IWED) (pp. 1-6). IEEE. DOI: 10.1109/IWED52055.2021.9376343
32. El Asslouj, A., Atkins, E., & Rastgoftar, H. (2023). Can a laplace PDE define air corridors through low-altitude airspace?. In 2023 International Conference on Unmanned Aircraft Systems (ICUAS) (pp. 1-8). IEEE. DOI: 10.1109/ICUAS57906. 2023.10180409
33. Geister, D., & Korn, B. (2013). Operational integration of UAS into the ATM system. In AIAA Infotech@ Aerospace (I@ A) Conference (p. 5051). https://doi.org/10.2514/ 6.2013-5051
34. Gong, S., Wang, M., Gu, B., Zhang, W., Hoang, D. T., & Niyato, D. (2023). Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in Multi-UAV networks. IEEE Transactions on Vehicular Technology. https://doi.org/10.48550/arXiv.2212.13396
35. Han, X., Wang, J., Zhang, Q., Qin, X., & Sun, M. (2019). Multi-UAV automatic dynamic obstacle avoidance with experience-shared a2c. In 2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob) (pp. 330-335). IEEE. DOI: 10.1109/WiMOB.2019.8923344
36. Ho, F., Geraldes, R., Goncalves, A., Cavazza, M., & Prendinger, H. (2018b). Simulating shared airspace for service UAVs with conflict resolution. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (pp. 2192-2194). https://dl.acm.org/doi/10.5555/3237383.3238117
37. Ho, F., Geraldes, R., Gonçalves, A., Cavazza, M., & Prendinger, H. (2018a). Improved conflict detection and resolution for service UAVs in shared airspace. IEEE Transactions on Vehicular Technology, 68(2), 1231-1242. DOI: 10.1109/TVT. 2018.2889459
38. Isufaj, R., Omeri, M., & Piera, M. A. (2022). Multi-UAV conflict resolution with graph convolutional reinforcement learning. Applied Sciences, 12(2), 610. https://doi.org /10.3390/app12020610.
39. Jiang, X., Chen, X., Tang, J., Zhao, N., Zhang, X. Y., Niyato, D., & Wong, K. K. (2021). Covert communication in UAV-assisted air-ground networks. IEEE Wireless Communications, 28(4), 190-197. DOI: 10.1109/MWC.001.2000454
40. Kainrath, K., Gruber, M., Hinze, A., Fluehr, H., & Leitgeb, E. (2022). Towards unmanned aerial vehicle UTM-integration using mobile radio networks. In 2022 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO) (pp. 465-469). IEEE. DOI: 10.23919/MIPRO55190. 2022.9803420
41. Kang, H., Li, W., Mišić, J., Mišić, V. B., & Chang, X. (2022). Dual-UAV aided secure dynamic G2U communication. In 2022 IEEE Symposium on Computers and Communications (ISCC) (pp. 1-6). IEEE. DOI: 10.1109/ISCC55528.2022.9912939
42. Khan, A., Ferramosca, M. L., Ivaki, N., & Madeira, H. (2022). Classifying fault category and severity of UAV flight controllers’ reported issues. In 2022 6th International Conference on System Reliability and Safety (ICSRS) (pp. 45-54). IEEE. DOI: 10.1109/ICSRS56243.2022.10067593
43. Kim, Y. W., Lee, D. Y., Tahk, M. J., & Lee, C. H. (2020). A new path planning algorithm for forced landing of UAVs in emergency using velocity prediction method. In 2020 28th Mediterranean Conference on Control and Automation (MED) (pp. 62-66). IEEE. DOI: 10.1109/MED48518.2020.9183166
44. Konert, A., & Kasprzyk, P. (2021). UAS safety operation–legal issues on reporting UAS incidents. Journal of Intelligent & Robotic Systems, 103(3), 51. https://link.springer. com/article/10.1007/s10846-021-01448-5
45. Kumar, A., Krishnamurthi, R., Sharma, G., Jain, S., Srikanth, P., Sharma, K., & Aneja, N. (2023). Revolutionizing modern networks: Advances in AI, machine learning, and blockchain for quantum satellites and UAV-based communication. arXiv preprint arXiv:2303.11753. https://doi.org/10.48550/arXiv.2303.11753
46. Labib, N. S., Danoy, G., Musial, J., Brust, M. R., & Bouvry, P. (2019a). A multilayer low-altitude airspace model for UAV traffic management. In Proceedings of the 9th ACM Symposium on Design and Analysis of Intelligent Vehicular
47. Networks and Applications (pp. 57-63). https://doi.org/10.1145/3345838.3355998 Labib, N.S., Danoy, G., Musial, J., Brust, M. R., & Bouvry, P. (2019b). Internet of unmanned aerial vehicles—A multilayer low-altitude airspace model for distributed UAV traffic management. Sensors, 19(21), 4779. https://doi.org/10.3390/s19214779
48. Lamba, M. A., Tangade, S. S., Nawaz, S. S., & Manvi, S. S. (2021). Path planning scheme for collision avoidance in unmanned aerial vehicle traffic management system. In 2021 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) (pp. 1-5). IEEE. DOI: 10.1109/CONECCT52877.2021. 9622656
49. Li, G., Zuo, H., & Xu, J. (2023). Research on the influence of UAV Anti-collision device on aerodynamic shape. In Journal of Physics: Conference Series (Vol. 2477, No. 1, p. 012096). IOP Publishing. DOI:10.1088/1742-6596/2477/1/012096
50. Li, Q., Zhang, D., Wang, H., Liu, K., & Liu, Y. (2022). A design method for the inspection network of over-the-horizon UAV based on 4G/5G communication network. In 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE) (pp. 240-244). IEEE. DOI: 10.1109/ICCECE54139.2022.9712840
51. Li, X., Fang, J., Du, K., Mei, K., & Xue, J. (2023). UAV obstacle avoidance by human-in-the-loop reinforcement in arbitrary 3D environment. arXiv preprint arXiv:2304.05959. https://doi.org/10.48550/arXiv.2304.05959
52. Lingyun, Z. H. O. U., Xiaotong, Z. H. A. O., Xin, G. U. A. N., Enbin, S. O. N. G., Xin, Z. E. N. G., & Qingjiang, S. H. I. (2022). Robust trajectory planning for UAV communication systems in the presence of jammers. Chinese Journal of Aeronautics, 35(10), 265-274. https://doi.org/10.1016/j.cja.2021.10.038
53. Liu, X., Liu, Y., & Chen, Y. (2019). Reinforcement learning in multiple-UAV networks: Deployment and movement design. IEEE Transactions on Vehicular Technology, 68(8), 8036-8049. DOI: 10.1109/TVT.2019.2922849
54. Liu, Z., Di, X., Wang, Q., & Wang, L. (2023). Path planning based on joint distribution of distribution vehicles and UAVs. In 2023 IEEE 3rd International Conference on Electronic Technology, Communication and Information (ICETCI) (pp. 1504-1508). IEEE. DOI: 10.1109/ICETCI57876.2023.10176924
55. Lu, W., Mo, Y., Feng, Y., Gao, Y., Zhao, N., Wu, Y., & Nallanathan, A. (2022). Secure transmission for multi-UAV-assisted mobile edge computing based on reinforcement learning. IEEE Transactions on Network Science and Engineering, 10(3), 1270-1282. DOI: 10.1109/TNSE.2022.3185130
56. Luo, B., Wu, Z., Zhou, F., & Wang, B. C. (2023). Human-in-the-loop reinforcement learning in continuous-action space. IEEE Transactions on Neural Networks and Learning Systems. DOI: 10.1109/TNNLS.2023.3289315
57. Luo, H., Wu, Y., Sun, G., Yu, H., Xu, S., & Guizani, M. (2023). ESCM: An efficient and secure communication mechanism for UAV networks. arXiv preprint arXiv:2304.13244. https://doi.org/10.48550/arXiv.2304.13244
58. Maurya, H. L., Singh, P., Yogi, S., Behera, L., & Verma, N. K. (2022). An intelligent game theory approach for collision avoidance of multi-UAVs. In Proceedings of International Conference on Computational Intelligence: ICCI 2021 (pp. 27-39). Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-19-2126-1_3
59. McTegg, S. J., Tarsha Kurdi, F., Simmons, S., & Gharineiat, Z. (2022). Comparative approach of unmanned aerial vehicle restrictions in controlled airspaces. Remote Sensing, 14(4), 822. https://doi.org/10.3390/rs14040822
60. Mohsan, S. A. H., Othman, N. Q. H., Li, Y., Alsharif, M. H., & Khan, M. A. (2023). Unmanned aerial vehicles (UAVs): Practical aspects, applications, open challenges, security issues, and future trends. Intelligent Service Robotics, 16(1), 109-137. https://link.springer.com/article/10.1007/s11370-022-00452-4
61. Newman, M., & Gough, D. (2020). Systematic reviews in educational research: Methodology, perspectives and application. https://link.springer.com/chapter/ 10.1007/978-3-658-27602-7_1
62. Patrikar, J., Dantas, J., Ghosh, S., Kapoor, P., Higgins, I., Aloor, J. J., Navarro, I., Sun, J., Stoler, B., Hamidi, M., Baijal, R., Moon, B., Oh, J., & Scherer, S. (2022). Challenges in close-proximity safe and seamless operation of manned and unmanned aircraft in shared airspace. arXiv preprint arXiv:2211.06932. https://doi.org/10.48550/arXiv. 2211.06932
63. Picard, G. (2022). Trajectory Coordination based on distributed constraint optimization techniques in unmanned air traffic management. In 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2022). https://dl.acm.org/doi/abs/10.5555/3535850.3535969
64. Quan, Y., Cheng, N., Wang, X., Shen, J., Ma, L., & Yin, Z. (2023). Interpretable and secure trajectory optimization for UAV-assisted communication. In 2023 IEEE/CIC International Conference on Communications in China (ICCC) (pp. 1-6). IEEE. https://doi.org/10.48550/arXiv.2307.02002
65. Radanovic, M., Omeri, M., & Piera, M. A. (2019). Test analysis of a scalable UAV conflict management framework. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 233(16), 6076-6088. DOI: 10.1177/0954410019875241
66. Raju, P., Rios, J., & Jordan, A. (2018). UTM—A complementary set of services to ATM. In 2018 Integrated Communications, Navigation, Surveillance Conference (ICNS) (pp. 2F2-1). IEEE. DOI: 10.1109/ICNSURV.2018.8384849
67. Ralegankar, V. K., Bagul, J., Thakkar, B., Gupta, R., Tanwar, S., Sharma, G., & Davidson, I. E. (2021). Quantum cryptography-as-a-service for secure UAV communication: Applications, challenges, and case study. IEEE Access, 10, 1475-1492. DOI: 10.1109/ACCESS.2021.3138753
68. Rithic, C. H., & Arulmozhi, N. (2023). Real-time implementation of RF-based mobile fleet localization and collision avoidance system in wireless sensor network for drones and gliders. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 1459-1465). IEEE. DOI: 10.1109/ICICCS56967.2023.10142713
69. Saraçyakupoğlu, T., Delibaş, H. D., & Özçelik, A. D. (2022). A computational determination of a nozzle activated fixed-wing UAV. International Journal of 3D Printing Technologies and Digital Industry, 6(2), 292-306. https://doi.org/10.46519/ ij3dptdi.1128158
70. Shan, L., Li, H. B., Miura, R., Matsuda, T., & Matsumura, T. (2023). A novel collision avoidance strategy with D2D communications for UAV systems. Drones, 7(5), 283. https://doi.org/10.3390/drones7050283
71. Sharma, S., Kulkarni, P., & Pathak, P. (2022). Applications of unmanned aerial vehicles (UAVs) for improved business management. In 2022 International Interdisciplinary Humanitarian Conference for Sustainability (IIHC) (pp. 53-57). IEEE. DOI: 10.1109/IIHC55949.2022.10060638
72. Shi, H. R., Lu, F. X., Wu, L., & Xia, J. W. (2022). Trajectory optimization of multi-UAVs for marine target tracking during approaching stage. Mathematical Problems in Engineering, 2022. https://doi.org/10.1155/2022/5472105
73. Shrestha, R., Kim, D., Choi, J., & Kim, S. (2022). A novel E/E architecture for low altitude UAVs. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 346-350). IEEE. DOI: 10.1109/ISCAS48785.2022.9937942
74. Shrestha, R., Zevenbergen, J., Panday, U. S., Awasthi, B., & Karki, S. (2019). Revisiting the current UAV regulations in Nepal: A step towards the legal dimension for UAVs' efficient application. The International Archives of the
75. Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 107-114. https://doi.org/10.5194/isprs-archives-XLII-5-W3-107-2019
76. Sun, J., Zhang, H., Xu, W., Li, H., Zhang, J., Ke, J., & Yan, H. (2021). Improving security performance of dual UAVs system with unknown eavesdropper location. In Proceedings of the International Conference on Internet-of-Things Design and Implementation (pp. 257-258). https://doi.org/10.1145/3450268.3453509
77. Sun, S., & Dang, S. (2022). Study on collision avoidance strategy of multiple UAVs based on genetic algorithm. In 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022) (Vol. 12257, pp. 303-309). SPIE. DOI: 10.1117/12.2639508
78. Sun, Y., Li, L., Zhou, C., Yang, S., Shi, D., & An, H. (2022). Design and Implementation of a collaborative air-ground unmanned system path planning framework. In China Intelligent Robotics Annual Conference (pp. 83-96). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0301-6_7
79. Tang, G., Du, P., Lei, H., Ansari, I. S., & Fu, Y. (2021). Trajectory design and communication resources allocation for wireless powered secure UAV communication systems. IEEE Systems Journal, 16(4), 6300-6308. DOI: 10.1109/JSYST.2021.3132010
80. Taylor, M. E. (2023). Reinforcement Learning Requires Human-in-the-Loop Framing and Approaches. In HHAI (pp. 351-360). https://alaworkshop2023.github.io/papers/ ala2023_paper_29.pdf
81. Tovarnov, M. S., & Bykov, N. V. (2022). Reinforcement learning reward function in unmanned aerial vehicle control tasks. In Journal of Physics: Conference Series (Vol. 2308, No. 1, p. 012004). IOP Publishing. DOI 10.1088/1742-6596/2308/1/012004
82. Volkert, A., Hackbarth, H., Lieb, T. J., & Kern, S. (2019). Flight tests of ranges and latencies of a threefold redundant C2 multi-link solution for small drones in VLL airspace. In 2019 Integrated Communications, Navigation and Surveillance Conference (ICNS) (pp. 1-14). IEEE. DOI: 10.1109/ICNSURV.2019.8735265
83. Wang, T., Xiang, S., Men, Z., Ye, M., Zhang, Y., Xie, A., & Zhejiang Lab. (2023). An emergency landing spot detection algorithm based on semantic segmentation and safety evaluation. Presented at Forum 79. DOI: https://doi.org/10.4050/F-0079-2023-18018.
84. Wang, W., Wei, X., Jia, Y., & Chen, M. (2023). UAV relay network deployment through the area with barriers. Ad Hoc Networks, 103222. https://doi.org/10.1016/ j.adhoc.2023.103222
85. Wang, Y., Wang, H., Wen, J., Lun, Y., & Wu, J. (2020). Obstacle avoidance of UAV based on neural networks and interfered fluid dynamical system. In 2020 3rd International Conference on Unmanned Systems (ICUS) (pp. 1066-1071). DOI: 10.1109/ICUS50048.2020.9274988
86. Wei, S., Li, L., Chen, G., Blasch, E., Chang, K. C., Clemons, T. M., & Pham, K. (2023). ROSIS: Resilience oriented security inspection system against false data injection attacks. In 2023 IEEE Aerospace Conference (pp. 1-11). IEEE. DOI: 10.1109/AERO55745.2023.10115584
87. Wei, Y., Zhao, M., Zhang, F., & Hu, Y. (2004). Research of a heuristic reward function for reinforcement learning algorithms. In Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No. 04EX788) (Vol. 3, pp. 2676-2680). IEEE. DOI: 10.1109/WCICA.2004.1342083
88. Wiedemann, M., Vij, A., & Banerjee, R. (2023). Validating the benefits of increased drone uptake for Australia: Geographic, demographic and social insights. Department of Infrastructure, Transport, Regional Development, Communications and the Arts (Australia). https://apo.org.au/node/322458
89. Wijnker, D., van Dijk, T., Snellen, M., de Croon, G., & De Wagter, C. (2019). Hear-and-avoid for UAVs using convolutional neural networks. In Proceedings of the 11th International Micro Air Vehicle Competition and Conference (IMAV2019), Madrid, Spain (Vol. 30). https://www.imavs.org/papers/2019/19.pdf
90. Wu, J., Yuan, W., & Hanzo, L. (2023). When UAVs meet ISAC: real-time trajectory design for secure communications. arXiv preprint arXiv:2306.14140. https://doi.org.10.48550/arXiv.2306.14140
91. Wu, X., Lei, Y., Tong, X., Zhang, Y., Li, H., Qiu, C., Guo, C., Sun, Y., & Lai, G. (2022). A Non-rigid hierarchical discrete grid structure and its application to UAVs conflict detection and path planning. IEEE Transactions on Aerospace and Electronic Systems, 58(6), 5393-5411. DOI: 10.1109/TAES.2022.3170323
92. Wubben, J., Calafate, C. T., Cano, J. C., & Manzoni, P. (2023). FFP: A force field protocol for the tactical management of UAV conflicts. Ad Hoc Networks, 140, 103078. https://doi.org/10.1016/j.adhoc.2022.103078
93. Xiang, T., Jiang, F., Hao, Q., & Cong, W. (2016). Adaptive flight control for quadrotor UAVs with dynamic inversion and neural networks. In 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) (pp. 174-179). DOI: 10.1109/MFI.2016.7849485
94. Xiao, Q., Zhang, X., Jiang, L., & Wang, M. (2022). Design of reward functions based on The DDQN Algorithm. In 2022 14th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA) (pp. 600-604). IEEE. DOI: 10.1109/ICMTMA54903.2022.00125
95. Xie, R., Huang, C., Wang, Z., & Han, J. (2022). A deep reinforcement learning algorithm based on short-term advantage for air game decision-making. In International Conference on Autonomous Unmanned Systems (pp. 3884-3894). Singapore: Springer Nature Singapore. https://link.springer.com/chapter/10.1007/978-981-99-0479-2_359
96. Xu, J., Wu, W., & Sun, Y. (2022). Multi-UAVs trajectory planning method with coordinated attack angle-time constraints. In 2022 IEEE International Conference on Unmanned Systems (ICUS). DOI: 10.1109/ICUS55513.2022.9987057
97. Xu, Z., Deng, D., Dong, Y., & Shimada, K. (2022). DPMPC-planner: A real-time UAV trajectory planning framework for complex static environments with dynamic obstacles. In 2022 International Conference on Robotics and Automation (ICRA) (pp. 250-256). IEEE. DOI: 10.1109/ICRA46639.2022.9811886
98. Xue, J., Zhu, J., Du, J., Kang, W., & Xiao, J. (2023). Dynamic path planning for multiple UAVs with incomplete information. Electronics, 12(4), 980. https://doi.org/10.3390/ electronics12040980
99. Yang, T., De Maio, A., Zheng, J., Su, T., Carotenuto, V., & Aubry, A. (2021). An adaptive radar signal processor for UAVs detection with super-resolution capabilities. IEEE Sensors Journal, 21(18), 20778-20787. DOI: 10.1109/JSEN.2021.3093779
100. Ye, B., Li, J., Li, J., Liu, C., Li, J., & Yang, Y. (2023). Deep reinforcement learning-based diving/pull-out control for bioinspired morphing UAVs. Unmanned Systems, 11(02), 191-202. https://doi.org/10.1142/S2301385023410066
101. Yin, S., & Yu, F. R. (2021). Resource allocation and trajectory design in UAV-aided cellular networks based on multiagent reinforcement learning. IEEE Internet of Things Journal, 9(4), 2933-2943. DOI: 10.1109/JIOT.2021.3094651
102. Zhang, D., Li, X., Ren, G., Yao, J., Chen, K., & Li, X. (2023a). Three-dimensional path planning of UAVs in a complex dynamic environment based on environment exploration twin delayed deep deterministic policy gradient. Symmetry, 15(7), 1371. https://doi.org/10.3390/sym15071371
103. Zhang, D., Xuan, Z., Zhang, Y., Yao, J., Li, X., & Li, X. (2023b). Path planning of unmanned aerial vehicle in complex environments based on state-detection twin delayed deep deterministic policy gradient. Machines, 11(1), 108. https://doi.org/10.3390/ machines11010108
104. Zhang, S., Li, Y., Ye, F., Geng, X., Zhou, Z., & Shi, T. (2023). A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV motion planning for long trajectories with unpredictable obstacles. Drones, 7(5), 311. https://doi.org/10.3390/ drones7050311