Real-Time Drone Command Processing: A Large Language Model Approach for IoD Systems

Abstract

One of the most critical steps toward autonomous capabilities, where natural language instructions can be successfully converted into executable API calls, is integrating Large Language Models (LLMs) into the ecosystem of the Internet of Drones (IoD). This study introduces an end-to-end LLM-based framework for enhancing real-time drone operation and problem handling in intent recognition, parameter extraction, and ambiguity resolution. It has resorted to a spectrum of methodologies in the form of Retrieval-Augmented Generation (RAG) and customized fine-tuning specific to each domain, towards accurate command interpretation and successful API generation. This paper evaluates the performance of the model on a carefully designed dataset containing 1,500 commands for the different scenarios of IoD with an average BLEU score of 89.6 and a cosine similarity of 0.94. Optimization techniques, such as parallel processing and better query handling, reduced the latency of this system by 15%, with an average query processing time of 0.9 seconds. This work gives considerable importance to the scalability and flexibility of the system, which is quite crucial for applications in domains like disaster response, precision agriculture, and surveillance. Proposed LLM-based framework thus tries to bridge gap between human intent and drone execution for intuitive, reliable, and efficient IoD deployments.

Keywords

• Large language models
• internet of drones
• natural language processing
• real-time command processing

Gerçek Zamanlı Drone Komut İşleme: IoD Sistemleri için Büyük Dil Modeli Yaklaşımı

Abstract

Doğal dil talimatlarının yürütülebilir API çağrılarına başarıyla dönüştürülebildiği otonom yeteneklere doğru atılan en kritik adımlardan biri, Büyük Dil Modellerinin (LLM) İnsansız Hava Araçları İnterneti (IoD) ekosistemine entegrasyonudur. Bu çalışma, amaç tanıma, parametre çıkarımı ve belirsizlik çözümleme alanlarında gerçek zamanlı drone operasyonlarını ve sorun çözümünü geliştirmek için uçtan uca bir büyük dil modeli tabanlı çerçeve sunmaktadır. Çalışmada, her alana özgü doğru komut yorumlama ve başarılı API oluşturma için Retrieval-Augmented Generation (RAG) yönteminden yararlanılmıştır. Çalışma, IoD'nin farklı senaryolarını kapsayan ve 1.500 komuttan oluşan bir veri kümesi üzerinde modeli değerlendirmiştir. Elde edilen sonuçlara göre, modelin ortalama BLEU skoru 89,6 ve kosinüs benzerliği 0,94 olarak ölçülmüştür. Paralel işlem ve daha iyi sorgu işleme gibi optimizasyon teknikleriyle sistemin gecikme süresi %15 azaltılmış ve ortalama sorgu işleme süresi 0,9 saniye olmuştur. Bu çalışma, özellikle afet müdahalesi, hassas tarım ve gözetim gibi alanlarda uygulamalar için oldukça kritik olan sistemin ölçeklenebilirliğine ve esnekliğine büyük önem vermektedir. Önerilen LLM tabanlı çerçeve, böylece insan amacı ile drone uygulamaları arasında sezgisel, güvenilir ve verimli IoD dağıtımları için bir köprü kurmayı amaçlamaktadır.

Keywords

• Büyük dil modelleri
• insansız hava araçları interneti
• doğal dil işleme
• gerçek zamanlı komut işleme

References

1. Sezgin A, Boyacı A. Securing the Skies: Exploring Privacy and Security Challenges in Internet of Drones. In: 2023 10th International Conference on Recent Advances in Air and Space Technologies (RAST); 2023; Istanbul, Turkiye, pp. 1-6.
2. Sezgin A, Boyacı A. Advancements in Object Detection for Unmanned Aerial Vehicles: Applications, Challenges, and Future Perspectives. In: 12th International Symposium on Digital Forensics and Security (ISDFS); 2024; San Antonio, TX, USA, pp. 1-6.
3. Zhou L, Yin H, Zhao H, Wei J, Hu D, Leung VCM. A Comprehensive Survey of Artificial Intelligence Applications in UAV-Enabled Wireless Networks. Digit Commun Netw 2024; 1-26.
4. Savenko I. Command interpretation for UAV using language models. In: 2024 IEEE 7th International Conference on Actual Problems of Unmanned Aerial Vehicles Development (APUAVD); 2024; Kyiv, Ukraine, pp. 228-231.
5. Sezgin A, Boyacı A. Rising Threats: Privacy and Security Considerations in the IoD Landscape. Journal of Aeronautics and Space Technologies (JAST) 2024; 17: 219-235.
6. Zhang C, Chen J, Li J, Peng Y, Mao Z. Large language models for human–robot interaction: A review. Biomim Intell Robot 2023; 3(4): 1-15.
7. Sun S, Li C, Zhao Z, Huang H, Xu W. Leveraging large language models for comprehensive locomotion control in humanoid robots design. Biomim Intell Robot 2024; 4(4): 1-16.
8. Du F, Ma X, Yang J, Liu Y, Luo C, Wang X, Jiang H, Jing X. A Survey of LLM Datasets: From Autoregressive Model to AI Chatbot. J Comput Sci Technol 2024; 39: 542-566.

9. Kim Y, Kim D, Choi J, Park J, Oh N, Park D. A survey on integration of large language models with intelligent robots. Intell Serv Robot 2024; 17: 1091-1107.
10. Jang D, Cho D, Lee W, Ryu S, Jeong B, Hong M, Jung M, Kim M, Lee M, Lee S, Choi H. Unlocking Robotic Autonomy: A Survey on the Applications of Foundation Models. Int J Control Autom Syst 2024; 22: 2341-2384.
11. Sauvola J, Tarkoma S, Klemettinen M, Riekki J, Doermann D. Future of software development with generative AI. Autom Softw Eng 2024; 31: 1-8.
12. Aharon U, Dubin R, Dvir A, Hajaj C. A classification-by-retrieval framework for few-shot anomaly detection to detect API injection. Comput Secur 2025; 150: 1-13.
13. Tlili F, Ayed S, Fourati LC. Advancing UAV security with artificial intelligence: A comprehensive survey of techniques and future directions. Internet Things 2024; 27: 1-27.
14. Ibrahum ADM, Hussain M, Hong J. Deep learning adversarial attacks and defenses in autonomous vehicles: a systematic literature review from a safety perspective. Artif Intell Rev 2024; 58: 1-53.
15. Luo H, Luo J, Vasilakos AV. BC4LLM: A perspective of trusted artificial intelligence when blockchain meets large language models. Neurocomputing 2024; 599: 1-20.
16. Oliveira F, Costa DG, Assis F, Silva I. Internet of Intelligent Things: A convergence of embedded systems, edge computing and machine learning. Internet Things 2024; 26: 1-20.
17. Fang H, Zhang D, Tan C, Yu P, Wang Y, Li W. Large Language Model Enhanced Autonomous Agents for Proactive Fault-Tolerant Edge Networks. In: IEEE INFOCOM 2024 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS); 2024; Vancouver, BC, Canada: IEEE. pp. 1-2.
18. Sabet M, Palanisamy P, Mishra S. Scalable modular synthetic data generation for advancing aerial autonomy. Robot Auton Syst 2023; 166: 1-17.
19. Fan H, Liu X, Fuh JYH, Lu WF, Li B. Embodied intelligence in manufacturing: leveraging large language models for autonomous industrial robotics. J Intell Manuf 2024; pp. 1-17.
20. Li X, Wang S, Zeng S, Wu Y, Yang Y. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth 2024; 1: 1-43.
21. Lykov A, Karaf S, Martynov M, Serpiva V, Fedoseev A, Konenkov M, Tsetserukou D. FlockGPT: Guiding UAV Flocking with Linguistic Orchestration. In: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct); 2024; Bellevue, WA, USA: IEEE. pp. 1-4.
22. Zhou J, Yi J, Yang Z, Pu H, Li X, Luo J, Gao L. A survey on vehicle–drone cooperative delivery operations optimization: Models, methods, and future research directions. Swarm Evol Comput 2025; 92: 1-29.
23. Maheriya K, Rahevar M, Mewada H, Parmar M, Patel A. Insights into aerial intelligence: assessing CNN-based algorithms for human action recognition and object detection in diverse environments. Multimed Tools Appl 2024; 1-43.
24. Zheng X, Wang G, Xu G, Yang J, Han B, Yu J. A LLM-driven and motif-informed linearizing graph transformer for Web API recommendation. Appl Soft Comput 2025; 169: 1-12.
25. Ma Z, An S, Xie B, Lin Z. Compositional API Recommendation for Library-Oriented Code Generation. In: 2024 IEEE/ACM 32nd International Conference on Program Comprehension (ICPC); 2024; Lisbon, Portugal: IEEE. pp. 1-12.
26. Park S, Kim A, Lee S, Kamyod C, Kim CG. Design of REST API Client for Conversational Agent using Large Language Model with Open API System. In: 2024 IEEE/ACIS 22nd International Conference on Software Engineering Research, Management and Applications (SERA); 2024; Honolulu, HI, USA: IEEE. pp. 1-4.
27. Hemberg E, Moskal S, O'Reilly U. Evolving code with a large language model. Genet Program Evolvable Mach 2024; 25: 1-36.
28. Hong J, Ryu S. Type-migrating C-to-Rust translation using a large language model. Empir Softw Eng 2024; 30: 1-38.
29. Balasundaram A, Aziz ABA, Gupta A, Shaik A, Kavitha MS. A fusion approach using GIS, green area detection, weather API and GPT for satellite image based fertile land discovery and crop suitability. Sci Rep 2024; 14: 1-16.
30. Gerstmayr J, Manzl P, Pieber M. Multibody Models Generated from Natural Language. Multibody Syst Dyn 2024; 62: 249-271.
31. Poth A, Rrjolli O, Arcuri A. Technology adoption performance evaluation applied to testing industrial REST APIs. Autom Softw Eng 2024; 32: 1-34.
32. Ishimizu Y, Li J, Yamauchi T, Chen S, Cai J, Hirano T, Tei K. Towards Efficient Discrete Controller Synthesis: Semantics-Aware Stepwise Policy Design via LLM. In: 2024 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia); 2024; Danang, Vietnam: IEEE. pp. 1-4.
33. Cao C, Wang F, Lindley L, Wang Z. Managing Linux servers with LLM-based AI agents: An empirical evaluation with GPT4. Mach Learn Appl 2024; 17: 1-13.
34. Luo H, Wu J, Liu J, Antwi-afari MF. Large language model-based code generation for the control of construction assembly robots: A hierarchical generation approach. Dev Built Environ 2024; 19: 1-18.