Dağıtık pekiştirmeli öğrenme tabanlı çoklu insansız hava aracı ile ilgi çekici nokta kapsama

Yıl 2024, , 563 - 576, 21.08.2023

Aydın Çetin , Fatih Aydemir

https://doi.org/10.17341/gazimmfd.1172120

Öz

Mobil araçlar haritalama, trafiğin izlenmesi, arama-kurtarma operasyonları gibi çeşitli alan kapsama uygulamalarında yaygın olarak kullanılmaktadır. Kapsama sürecini geliştirmek için uygun konumlandırma modeli ve etkili öğrenme stratejisi gereklidir. Mobil araçlar hareket modeli içeren yönlendirme mekanizması ile dinamik ortamlara uyum sağlayabilir ve en uygun konumları bulabilirler. Konumlandırma sürecinin çok ajanlı mobil sistem temelinde yönetildiği çalışmalarda algılama, veri toplama ve gözetim gibi görevleri birden fazla ajanın işbirlikçi yaklaşım ile tamamlaması gerekir. Öğrenmeye dayalı bu süreç, bir görevi gerçek zamanlı optimize etmeyi öğrenebilen mobil ajanlar vasıtasıyla yürütülebilir. Bu çalışmada, bir grup insansız hava aracının (İHA) öğrenebilen çok ajanlı sistem temelinde modellenerek dinamik ortamda ilgi çekici noktaları (İÇN) etkin şekilde kapsaması hedeflenmektedir. Hedef alan, İÇN kapsamını en üst düzeye çıkarmak ve enerji tüketimini en aza indirmek için ızgaralara ayrıştırılır. Ayrıştırma, hedef alanın konumu ve mobil ajan olarak modellenen İHA’ların iletişim mesafesi göz önünde bulundurularak gerçekleştirilir. Bununla birlikte ızgaralara gidiş planlanması yapan mobil ajanlar çarpışmadan kaçınmayı da öğrenirler. Önerilen yöntem benzetim ortamında test edilmiş ve sonuçlar benzer çalışmalar ile kıyaslanarak sunulmuştur. Sonuçlar, önerilen yöntemin mevcut benzer çalışmalara göre daha iyi performans gösterdiğini ve alan kapsama uygulamaları için uygun olduğunu göstermektedir.

Anahtar Kelimeler

İnsansız Hava Aracı, Çok ajanlı sistem, Pekiştirmeli öğrenme, Dinamik alan kapsama, Izgara Ayrıştırma

Kaynakça

• Gupta, H., Verma, O.P., Monitoring and Surveillance of Urban Road Traffic Using Low Altitude Drone Images: A Deep Learning Approach, Multimedia Tools and Applications, 81 (14), 19683–19703, 2022.
• Lee H-R., Lee T., Multi-agent Reinforcement Learning Algorithm to Solve a Partially-observable Multi-agent Problem in Disaster Response, Eur. J. Oper. Res., 291 (1), 296-308, 2021.
• Drew, D.S., Multi-Agent Systems for Search and Rescue Applications, Curr Robot Rep, 2 (2), 189-200, 2021.
• Xiao J., Wang G., Zhang Y., Cheng L., A Distributed Multi-Agent Dynamic Area Coverage Algorithm Based on Reinforcement Learning, IEEE Access, 8 (1), 33511-33521, 2020.
• Dorri A., Kanhere S. S., Jurdak R., Multi-Agent Systems: A Survey, IEEE Access, 6, 28573-28593, 2018.
• Woolley A.W., Aggarwal I., Malone T.W., Collective Intelligence and Group Performance. Current Directions in Psychological Science, 24(6), 420-424, 2015.
• Gupta, S.K., Kuila, P., Jana, P.K., Genetic Algorithm Approach for K-coverage and M-connected Node Placement in Target Based Wireless Sensor Networks, Computers & Electrical Engineering, 56 (1), 544-556, 2016.
• Njoya A.N., Ari A.A.A., Awa M.N., Titouna C., Labraoui N., Effa J.Y., Abdou W., Gueroui A., Hybrid Wireless Sensors Deployment Scheme with Connectivity and Coverage Maintaining, Wireless Personal Communications, 112 (3), 544-556, 2020.
• Yue Y., Cao L., Luo Z., Hybrid Artificial Bee Colony Algorithm for Improving the Coverage and Connectivity of Wireless Sensor Networks, Wireless Personal Communications, 108 (3), 1719–1732, 2019.
• Jagtap A.M., Gomathi N., Minimizing Movement for Network Connectivity in Mobile Sensor Networks: An Adaptive Approach, Cluster Computing, 22 (1), 1373–1383, 2019.
• Shu T., Dsouza K.B., Bhargava V., Silva C., Using Geometric Centroid of Voronoi Diagram for Coverage and Lifetime Optimization in Mobile Wireless Sensor Networks, IEEE Canadian Conference of Electrical and Computer Engineering (CCECE), Edmonton AB-Kanada, 1-5, 05-08 Mayıs, 2019.
• Shi, W., Li, J., Xu, W., Zhou, H., Zhang, N., Zhang, S., Shen, X., Multiple Drone-cell Deployment Analyses and Optimization in Drone Assisted Radio Access Networks, IEEE Access, 6 (1), 12518-12529, 2018.
• Mozaffari M., Saad W., Bennis M., Debbah M., Efficient Deployment of Multiple Unmanned Aerial Vehicles for Optimal Wireless Coverage, IEEE Commun. Lett., 20 (8), 1647-1650, 2016.
• Zhang X., Duan L., Fast Deployment of UAV Networks for Optimal Wireless Coverage, IEEE Trans. Mob. Comput., 18 (3), 588-601, 2019.
• Sun J., Masouros C., Drone Positioning for User Coverage Maximization, IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Bologna-Italya, 318-322, 09-12 Eylül, 2018.
• Cabreira T.M., Ferreira P.R., Franco C.D., Buttazzo G.C., Grid-Based Coverage Path Planning With Minimum Energy Over Irregular-Shaped Areas With Uavs, International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta GA-ABD, 758-767, 11-14 Haziran, 2019.
• Kalantari E., Yanikomeroglu H., Yongacoglu A., On the Number and 3D Placement of Drone Base Stations in Wireless Cellular Networks, IEEE 84th Vehicular Technology Conference (VTC-Fall), Montreal QC-Kanada, 1-6, 18-21 Eylül, 2016.
• Chiu J-H., Kuo Y-C., Sheu J-P., Hong Y-W. P., Energy-Efficient UAV Deployment and IoT Device Association in Fixed-Wing Multi-UAV Networks, IEEE Global Communications Conference, Taipei-Tayvan, 1-6, 07-11 Aralık, 2020.
• Ganganath N., Cheng C., Tse C.K, Distributed Antiflocking Algorithms for Dynamic Coverage of Mobile Sensor Networks, IEEE Trans. Ind. Inf., 12 (5), 1795-1805, 2016.
• Krajník T., Nitsche M., Faigl J., Vaněk P., Saska M., Přeučil L, Duckett T., Mejail M., A Practical Multirobot Localization System, Journal of Intelligent & Robotic Systems, 76 (3), 539–562, 2014.
• Abidin H. Z., Din N.M., Yassin I.M., Omar H.A., Radzi N.A.M, Sadon S.K., Sensor Node Placement in Wireless Sensor Network Using Multi-objective Territorial Predator Scent Marking Algorithm, Arabian Journal of Science and Engineering, 39 (1), 6317–6325, 2014.
• Liu C.H., Chen Z., Tang J., Xu J., Piao C., Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning Approach, IEEE J. Sel. Areas Commun., 36 (9), 2059-2070, 2018.
• Lillicrap T.P., Hunt J.J, Pritzel A., Heess N., Erez T., Tassa Y., Silver D., Wierstra D., Continuous Control with Deep Reinforcement Learning, 4th International Conference on Learning Representations (ICLR), San Juan-Porto Riko, 1-14, 02-04 Mayıs, 2016.
• Liu C.H., Ma X., Gao X., Tang J., Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning, IEEE Trans. Mob. Comput., 19 (6), 1274-1285, 2020.
• Aydemir F., Çetin A., Multi-agent Dynamic Area Coverage Based on Reinforcement Learning with Connected Agents, Computer Systems Science and Engineering, 45 (1), 215–230, 2023.
• Lowe R., Wu Y., Tamar A., Harb J., Abbeel P., Mordatch I., Multiagent Actor-critic for Mixed Cooperative-competitive Environments, Advances in Neural Information Processing Systems, Long Beach CA-ABD, 6379-6390, 04-09 Aralık, 2017.
• Keith A.J., Ahner D.K., A Survey of Decision Making and Optimization Under Uncertainty, Annals of Operations Research, 300 (2), 319-353, 2021.
• Deng L., Yu D., Deep Learning: Methods and Applications, Foundations and Trends in Signal Processing, 7 (3-4), 197-387, 2014.
• Song H.A., Lee S. Y., Hierarchical Representation Using NMF, International Conference on Neural Information Processing (ICONIP), Daegu-Güney Kore, 466-473, 03-07 Kasım, 2013.
• Qie H., Shi D., Shen T., Xu X., Li Y., Wang L., Joint Optimization of Multi-UAV Target Assignment and Path Planning Based on Multi-Agent Reinforcement Learning, IEEE Access, 7 (1), 146264-146272, 2019.
• Jianqing F., Zhaoran W., Yuchen X., Zhuoran Y., A Theoretical Analysis of Deep Q-Learning, Proceedings of the 2nd Conference on Learning for Dynamics and Control, Berkeley CA-ABD, 486-489, 11-12 Haziran, 2020.
• Zoss B.M., Mateo D., Kuan Y.K., Toki´c G., Chamanbaz M., Goh L., Vallegra F., Bouffanais R., Yue D.K., Distributed System of Autonomous Buoys for Scalable Deployment and Monitoring of Large Waterbodies, Autonomous Robots, 42 (8), 1669-1689, 2018.
• H¨uttenrauch M., ˇSoˇsi´c A., Neumann G., Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning, 11th International Conference on Swarm Intelligence (ANTS), Roma-İtalya, 71-83, 29-31 Ekim, 2018.
• Jain R.K., Chiu D.M.W., Hawe W.R., A QuantitativeMeasure of Fairness And Discrimination for Resource Allocation In Shared Computer Systems, Eastern Research Laboratory Digital Equipment Corporation, 38 (1), 1984.