Pekiştirmeli Öğrenme Algoritmaları ile İnsansız Hava Aracının Duruş Dinamiklerinin Kontrolü

Year 2021, Issue: 29, 351 - 357, 01.12.2021

Nurten Emer , Necdet Sinan Özbek

https://doi.org/10.31590/ejosat.1021970

Abstract

Bu çalışmada, dikey kalkış ve iniş yapabilen insansız bir hava aracının duruş dinamiklerinin kontrolü için modele bağlı ve modelden bağımsız öğrenme tabanlı kontrol tekniklerinin bazı uygulamaları sunulmaktadır. Bu amaç için, pekiştirmeli öğrenme kontrol algoritmaları incelenmiştir. Kontrol algoritmaları ele alınmış ve temel farklar sunulmuştur. Sistemin durum kontrolü üzerinde bir takım sayısal benzetimler gerçekleştirilmiş ve sonuçlar tartışılmıştır. Önerilen öğrenmeye dayalı kontrol yönteminin performans değerlendirmesi yapılmıştır.

Keywords

Modelden bağımsız kontrol, Pekiştirmeli Öğrenme Algoritmaları, Kontrol Uygulamaları, Kararlılık, İnsansız Hava Araçları

Control of Attitude Dynamics of an Unmanned Aerial Vehicle with Reinforcement Learning Algorithms

Abstract

In this study, some applications of model-dependent and model-free learning based control techniques are presented for the control of attitude dynamics of vertical take-off and landing unmanned aerial vehicle. Towards this goal, reinforcement learning control algorithms are examined. Control algorithms are discussed and the main differences are presented. A number of numerical simulations are carried out on the attitude control of the system and the results are discussed. Performance evaluation of the proposed learning-based control method has been carried out.

Keywords

Model-free control, Reinforcement Learning Algorithms, Control Application, Stability, Unammaned Aerial Vehicles

References

• Boubakir, A., Labiod, S., Boudjema, F., & Plestan, F. (2014). Model-free controller with an observer applied in real-time to a 3-DOF helicopter. Turkish Journal of Electrical Engineering and Computer Sciences, 22(6), 1564–1581. https://doi.org/10.3906/elk-1204-54
• Coelho, L. dos S., Pessôa, M. W., Rodrigues Sumar, R., & Augusto Rodrigues Coelho, A. (2010). Model-free adaptive control design using evolutionary-neural compensator. Expert Systems with Applications, 37(1), 499–508. https://doi.org/10.1016/j.eswa.2009.05.042
• Çömert, R., Avdan, U., & Şenkal, E. (2012). İnsansiz hava araçlarinin kullanim alanlari ve gelecekteki̇ beklenti̇ler. 16–19.
• Efe, M. Ö. (2011). Integral sliding mode control of a quadrotor with fractional order reaching dynamics. Transactions of the Institute of Measurement and Control, 33(8), 985–1003. https://doi.org/10.1177/0142331210377227
• Ei, G. R. (n.d.). Model-free control of dc / dc converters.
• Elhaki, O., & Shojaei, K. (2021). A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance. Aerospace Science and Technology, 119, 107128. https://doi.org/10.1016/j.ast.2021.107128
• Hayat, S., Yanmaz, E., & Muzaffar, R. (2016). Survey on Unmanned Aerial Vehicle Networks for Civil Applications: A Communications Viewpoint. IEEE Communications Surveys and Tutorials, 18(4), 2624–2661. https://doi.org/10.1109/COMST.2016.2560343
• Hwangbo, J., Sa, I., Siegwart, R., & Hutter, M. (2017). Control of a Quadrotor With Reinforcement Learning. IEEE Robotics and Automation Letters, 2(4), 2096–2103. https://doi.org/10.1109/LRA.2017.2720851
• Lesecq, S., Gentil, S., & Daraoui, N. (2014). Quadrotor attitude estimation with data losses. 2009 European Control Conference, ECC 2009, 3851–3856. https://doi.org/10.23919/ecc.2009.7075000
• Li, X., Zhang, J., & Han, J. (2021). Trajectory planning of load transportation with multi-quadrotors based on reinforcement learning algorithm. Aerospace Science and Technology, 116, 106887. https://doi.org/10.1016/j.ast.2021.106887
• Loh, R., Yi Bian, & Roe, T. (2009). UAVs in civil airspace: Safety requirements. IEEE Aerospace and Electronic Systems Magazine, 24(1), 5–17. https://doi.org/10.1109/MAES.2009.4772749
• Luque-Vega, L. F., Castillo-Toledo, B., Loukianov, A., & Gonzalez-Jimenez, L. E. (2014). Power line inspection via an unmanned aerial system based on the quadrotor helicopter. Proceedings of the Mediterranean Electrotechnical Conference - MELECON, (April), 393–397. https://doi.org/10.1109/MELCON.2014.6820566
• Ozbek, N. S. (2019). An Evaluation of Model-Free Control Strategies for Quadrotor Type Unmanned Aerial Vehicles. 2019 International Conference on Applied Automation and Industrial Diagnostics (ICAAID), 1–6. https://doi.org/10.1109/ICAAID.2019.8935001
• Özbek, N. S., Önkol, M., & Efe, M. Ö. (2014). Dönerkanat Tipinde Bir İnsansız Hava Aracının Farklı Yöntemlerle Kontrolü ve Performans Analizi. Otomatik Kontrol Ulusal Toplantısı, 11–13. Kocaeli.
• Özbek, N. S., Önkol, M., & Efe, M. Ö. (2016). Feedback control strategies for quadrotor-type aerial robots: a survey. Transactions of the Institute of Measurement and Control, 38(5), 529–554. https://doi.org/10.1177/0142331215608427
• Papachristos, C., Dang, T., Khattak, S., Mascarich, F., Khedekar, N., & Alexis, K. (2018). Modeling, Control, State Estimation and Path Planning Methods for Autonomous Multirotor Aerial Robots. Foundations and Trends in Robotics, 7(3), 180–250. https://doi.org/10.1561/2300000058
• Polydoros, A. S., & Nalpantidis, L. (2017a). Survey of Model-Based Reinforcement Learning: Applications on Robotics. Journal of Intelligent & Robotic Systems, 86(2), 153–173. https://doi.org/10.1007/s10846-017-0468-y
• Qi, D., Li, Z., Ren, B., Lei, P., & Yang, X. (2021). Detection and Tracking of a Moving Target for UAV Based on Machine Vision. 2021 7th International Conference on Control, Automation and Robotics (ICCAR), 173–178. https://doi.org/10.1109/ICCAR52225.2021.9463501
• Rezoug, A., Hamerlain, M., Achour, Z., & Tadjine, M. (2015). Applied of an adaptive Higher order sliding mode controller to quadrotor trajectory tracking. 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), 353–358. https://doi.org/10.1109/ICCSCE.2015.7482211
• Sadraey, M. (2017). Unmanned Aircraft Design: A Review of Fundamentals. Synthesis Lectures on Mechanical Engineering, 1(2), i–193. https://doi.org/10.2200/S00789ED1V01Y201707MEC004
• Santoso, F., Garratt, M. A., & Anavatti, S. G. (2020). Hybrid PD-Fuzzy and PD Controllers for Trajectory Tracking of a Quadrotor Unmanned Aerial Vehicle: Autopilot Designs and Real-Time Flight Tests. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1–13. https://doi.org/10.1109/TSMC.2019.2906320
• Wang, L. (2020). PID Control of Multi‐rotor Unmanned Aerial Vehicles. In PID Control System Design and Automatic Tuning using MATLAB/Simulink (pp. 305–326). https://doi.org/10.1002/9781119469414.ch10
• Yoo, J., Jang, D., Kim, H. J., & Johansson, K. H. (2021). Hybrid Reinforcement Learning Control for a Micro Quadrotor Flight. IEEE Control Systems Letters, 5(2), 505–510. https://doi.org/10.1109/LCSYS.2020.3001663
• Zhong, Y., Zhang, Y., Zhang, W., Zuo, J., & Zhan, H. (2018). Robust Actuator Fault Detection and Diagnosis for a Quadrotor UAV with External Disturbances. IEEE Access, 6, 48169–48180. https://doi.org/10.1109/ACCESS.2018.2867574
• Zhu, P., Yao, S., Liu, Y., Liu, S., & Liang, X. (2020). Autonomous Reinforcement Control of Underwater Vehicles based on Monocular Depth Vision. IFAC-PapersOnLine, 53(2), 9201–9206. https://doi.org/10.1016/j.ifacol.2020.12.2186