Pekiştirmeli Öğrenme Algoritmaları ile İnsansız Hava Aracının Duruş Dinamiklerinin Kontrolü

Öz

Bu çalışmada, dikey kalkış ve iniş yapabilen insansız bir hava aracının duruş dinamiklerinin kontrolü için modele bağlı ve modelden bağımsız öğrenme tabanlı kontrol tekniklerinin bazı uygulamaları sunulmaktadır. Bu amaç için, pekiştirmeli öğrenme kontrol algoritmaları incelenmiştir. Kontrol algoritmaları ele alınmış ve temel farklar sunulmuştur. Sistemin durum kontrolü üzerinde bir takım sayısal benzetimler gerçekleştirilmiş ve sonuçlar tartışılmıştır. Önerilen öğrenmeye dayalı kontrol yönteminin performans değerlendirmesi yapılmıştır.

Anahtar Kelimeler

• Modelden bağımsız kontrol
• Pekiştirmeli Öğrenme Algoritmaları
• Kontrol Uygulamaları
• Kararlılık
• İnsansız Hava Araçları

Control of Attitude Dynamics of an Unmanned Aerial Vehicle with Reinforcement Learning Algorithms

Abstract

In this study, some applications of model-dependent and model-free learning based control techniques are presented for the control of attitude dynamics of vertical take-off and landing unmanned aerial vehicle. Towards this goal, reinforcement learning control algorithms are examined. Control algorithms are discussed and the main differences are presented. A number of numerical simulations are carried out on the attitude control of the system and the results are discussed. Performance evaluation of the proposed learning-based control method has been carried out.

Keywords

• Model-free control
• Reinforcement Learning Algorithms
• Control Application
• Stability
• Unammaned Aerial Vehicles

Kaynakça

1. Boubakir, A., Labiod, S., Boudjema, F., & Plestan, F. (2014). Model-free controller with an observer applied in real-time to a 3-DOF helicopter. Turkish Journal of Electrical Engineering and Computer Sciences, 22(6), 1564–1581. https://doi.org/10.3906/elk-1204-54
2. Coelho, L. dos S., Pessôa, M. W., Rodrigues Sumar, R., & Augusto Rodrigues Coelho, A. (2010). Model-free adaptive control design using evolutionary-neural compensator. Expert Systems with Applications, 37(1), 499–508. https://doi.org/10.1016/j.eswa.2009.05.042
3. Çömert, R., Avdan, U., & Şenkal, E. (2012). İnsansiz hava araçlarinin kullanim alanlari ve gelecekteki̇ beklenti̇ler. 16–19.
4. Efe, M. Ö. (2011). Integral sliding mode control of a quadrotor with fractional order reaching dynamics. Transactions of the Institute of Measurement and Control, 33(8), 985–1003. https://doi.org/10.1177/0142331210377227
5. Ei, G. R. (n.d.). Model-free control of dc / dc converters.
6. Elhaki, O., & Shojaei, K. (2021). A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance. Aerospace Science and Technology, 119, 107128. https://doi.org/10.1016/j.ast.2021.107128
7. Hayat, S., Yanmaz, E., & Muzaffar, R. (2016). Survey on Unmanned Aerial Vehicle Networks for Civil Applications: A Communications Viewpoint. IEEE Communications Surveys and Tutorials, 18(4), 2624–2661. https://doi.org/10.1109/COMST.2016.2560343
8. Hwangbo, J., Sa, I., Siegwart, R., & Hutter, M. (2017). Control of a Quadrotor With Reinforcement Learning. IEEE Robotics and Automation Letters, 2(4), 2096–2103. https://doi.org/10.1109/LRA.2017.2720851

9. Lesecq, S., Gentil, S., & Daraoui, N. (2014). Quadrotor attitude estimation with data losses. 2009 European Control Conference, ECC 2009, 3851–3856. https://doi.org/10.23919/ecc.2009.7075000
10. Li, X., Zhang, J., & Han, J. (2021). Trajectory planning of load transportation with multi-quadrotors based on reinforcement learning algorithm. Aerospace Science and Technology, 116, 106887. https://doi.org/10.1016/j.ast.2021.106887
11. Loh, R., Yi Bian, & Roe, T. (2009). UAVs in civil airspace: Safety requirements. IEEE Aerospace and Electronic Systems Magazine, 24(1), 5–17. https://doi.org/10.1109/MAES.2009.4772749
12. Luque-Vega, L. F., Castillo-Toledo, B., Loukianov, A., & Gonzalez-Jimenez, L. E. (2014). Power line inspection via an unmanned aerial system based on the quadrotor helicopter. Proceedings of the Mediterranean Electrotechnical Conference - MELECON, (April), 393–397. https://doi.org/10.1109/MELCON.2014.6820566
13. Ozbek, N. S. (2019). An Evaluation of Model-Free Control Strategies for Quadrotor Type Unmanned Aerial Vehicles. 2019 International Conference on Applied Automation and Industrial Diagnostics (ICAAID), 1–6. https://doi.org/10.1109/ICAAID.2019.8935001
14. Özbek, N. S., Önkol, M., & Efe, M. Ö. (2014). Dönerkanat Tipinde Bir İnsansız Hava Aracının Farklı Yöntemlerle Kontrolü ve Performans Analizi. Otomatik Kontrol Ulusal Toplantısı, 11–13. Kocaeli.
15. Özbek, N. S., Önkol, M., & Efe, M. Ö. (2016). Feedback control strategies for quadrotor-type aerial robots: a survey. Transactions of the Institute of Measurement and Control, 38(5), 529–554. https://doi.org/10.1177/0142331215608427
16. Papachristos, C., Dang, T., Khattak, S., Mascarich, F., Khedekar, N., & Alexis, K. (2018). Modeling, Control, State Estimation and Path Planning Methods for Autonomous Multirotor Aerial Robots. Foundations and Trends in Robotics, 7(3), 180–250. https://doi.org/10.1561/2300000058
17. Polydoros, A. S., & Nalpantidis, L. (2017a). Survey of Model-Based Reinforcement Learning: Applications on Robotics. Journal of Intelligent & Robotic Systems, 86(2), 153–173. https://doi.org/10.1007/s10846-017-0468-y
18. Qi, D., Li, Z., Ren, B., Lei, P., & Yang, X. (2021). Detection and Tracking of a Moving Target for UAV Based on Machine Vision. 2021 7th International Conference on Control, Automation and Robotics (ICCAR), 173–178. https://doi.org/10.1109/ICCAR52225.2021.9463501
19. Rezoug, A., Hamerlain, M., Achour, Z., & Tadjine, M. (2015). Applied of an adaptive Higher order sliding mode controller to quadrotor trajectory tracking. 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), 353–358. https://doi.org/10.1109/ICCSCE.2015.7482211
20. Sadraey, M. (2017). Unmanned Aircraft Design: A Review of Fundamentals. Synthesis Lectures on Mechanical Engineering, 1(2), i–193. https://doi.org/10.2200/S00789ED1V01Y201707MEC004
21. Santoso, F., Garratt, M. A., & Anavatti, S. G. (2020). Hybrid PD-Fuzzy and PD Controllers for Trajectory Tracking of a Quadrotor Unmanned Aerial Vehicle: Autopilot Designs and Real-Time Flight Tests. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1–13. https://doi.org/10.1109/TSMC.2019.2906320
22. Wang, L. (2020). PID Control of Multi‐rotor Unmanned Aerial Vehicles. In PID Control System Design and Automatic Tuning using MATLAB/Simulink (pp. 305–326). https://doi.org/10.1002/9781119469414.ch10
23. Yoo, J., Jang, D., Kim, H. J., & Johansson, K. H. (2021). Hybrid Reinforcement Learning Control for a Micro Quadrotor Flight. IEEE Control Systems Letters, 5(2), 505–510. https://doi.org/10.1109/LCSYS.2020.3001663
24. Zhong, Y., Zhang, Y., Zhang, W., Zuo, J., & Zhan, H. (2018). Robust Actuator Fault Detection and Diagnosis for a Quadrotor UAV with External Disturbances. IEEE Access, 6, 48169–48180. https://doi.org/10.1109/ACCESS.2018.2867574
25. Zhu, P., Yao, S., Liu, Y., Liu, S., & Liang, X. (2020). Autonomous Reinforcement Control of Underwater Vehicles based on Monocular Depth Vision. IFAC-PapersOnLine, 53(2), 9201–9206. https://doi.org/10.1016/j.ifacol.2020.12.2186