Detecting Robotic Cyber Attacks in Robot Operating System Networks

Abstract

The increasing deployment of robotic systems in critical sectors such as manufacturing, healthcare, and infrastructure necessitates robust cybersecurity measures. The Robot Operating System (ROS), a core middleware in modern robotics, is inherently susceptible to cyber threats due to its lack of integrated security mechanisms. This study presents a comprehensive benchmark evaluation of intrusion detection solutions for ROS-based environments. Utilizing the novel ROSIDS23 dataset, which includes realistic attack scenarios—such as Denial-of-Service (DoS), Unauthorized Publish, Unauthorized Subscribe, and Subscriber Flood—we rigorously evaluated and compared fifteen state-of-the-art Machine Learning (ML) and Deep Learning (DL) models. Using stratified 5-fold cross-validation, our results demonstrate that ensemble methods significantly outperform deep learning approaches in this context. Gradient Boosting achieved the highest performance, with 99.80% accuracy, precision, recall, and F1-score, followed by Light Gradient Boosting Machine (LightGBM) at 99.51% and Extreme Gradient Boosting (XGBoost) at 99.48%. Among DL models, the best-performing One-Dimensional Convolutional Neural Network (1D-CNN) reached 98.55%. Beyond overall metrics, we examine per-class performance, confusion matrices, and Receiver Operating Characteristic (ROC) curves, highlighting model-specific strengths and weaknesses, particularly in detecting minority attack classes.

Keywords

• Robot operating system
• cyber attacks
• machine learning
• deep learning
• robotic cybersecurity

References

1. Ahmad Yousef, K. M., AlMajali, A., Ghalyon, S. A., Dweik, W., & Mohd, B. J. (2018). Analyzing cyber-physical threats on robotic platforms. Sensors, 18(5), 1643. google scholar
2. An, T.-K., & Kim, M.-H. (2010). A new diverse AdaBoost classifier. 2010 International conference on artificial intelligence and computational intelligence, 359–363. google scholar
3. Bezemskij, A., Loukas, G., Anthony, R. J., & Gan, D. (2016). Behaviour-based anomaly detection of cyber-physical attacks on a robotic vehicle. 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS), 61–68. google scholar
4. Botta, A., Rotbei, S., Zinno, S., & Ventre, G. (2023). Cyber security of robots: A comprehensive survey. Intelligent Systems with Applications, 18, 200237. google scholar
5. Chaudhary, A., Kolhe, S., & Kamal, R. (2016). An improved random forest classifier for multi-class classification. Information Processing in Agriculture, 3(4), 215–222. google scholar
6. Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(1), 6. google scholar
7. Chicco, D., & Jurman, G. (2023). The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification. BioData Mining, 16(1), 4. google scholar
8. Değirmenci, E., Kırca, Y. S., Özçelik, İ., & Yazıcı, A. (2023). ROSIDS23: Network intrusion detection dataset for robot operating system. Data in Brief, 51, 109739. google scholar

9. Değirmenci, E., Özçelik, I., & Yazıcı, A. (2024). Adversarial Attack Detection Approach for Intrusion Detection Systems. IEEE Access, 12, 195996–196009. google scholar
10. Deo, T. Y., & Sanju, A. (2023). Data imputation and comparison of custom ensemble models with existing libraries like XGBoost, CATBoost, AdaBoost and Scikit learn for predictive equipment failure. Materials Today: Proceedings, 72, 1596–1604. google scholar
11. Dhanabal, L., & Shantharajah, S. (2015). A study on NSL-KDD dataset for intrusion detection system based on classification algorithms. International Journal of Advanced Research in Computer and Communication Engineering, 4(6), 446–452. google scholar
12. Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. 27th European Conference on IR Research, 345–359. google scholar
13. Heydarian, M., Doyle, T. E., & Samavi, R. (2022). MLCM: Multi-label confusion matrix. Ieee Access, 10, 19083–19095. google scholar
14. Holdbrook, R., Odeyomi, O., Yi, S., & Roy, K. (2024). Network-Based Intrusion Detection for Industrial and Robotics Systems: A Comprehensive Survey. Electronics, 13(22), 4440. google scholar
15. Hussein, N., & Zeebaree, S. R. (2024). Performance evaluation of extra trees classifier by using cpu parallel and non-parallel processing. The Indonesian Journal of Computer Science, 13(2), 1859–1872. google scholar
16. Ibrahim, A. A., Ridwan, R. L., Muhammed, M. M., Abdulaziz, R. O., & Saheed, G. A. (2020). Comparison of the CatBoost classifier with other machine learning methods. International Journal of Advanced Computer Science and Applications, 11(11), 738–748. google scholar
17. Joy, J., & Selvan, M. P. (2022). A comprehensive study on the performance of different Multi-class Classification Algorithms and Hyperparameter Tuning Techniques using Optuna. International Conference on Computing, Communication, Security and Intelligent Systems (IC3SIS), 1–5. google scholar
18. Kang, J., Kim, K., & Kwon, D. (2025). Watch Your Callback: Offline Anomaly Detection using Machine Learning in ROS 2. IEEE Access, 13, 60763-60775. google scholar
19. Krejčí, J., Babiuch, M., Suder, J., Krys, V., & Bobovský, Z. (2025). Internet of robotic things: Current technologies, challenges, applications, and future research topics. Sensors, 25(3), 765. google scholar
20. Lee, T., Singh, V. P., & Cho, K. H. (2021). Tensorflow and keras programming for deep learning. Deep learning for hydrometeorology and environmental science, 151–162. google scholar
21. Martín, F., Soriano, E., & Cañas, J. M. (2018). Quantitative analysis of security in distributed robotic frameworks. Robotics and Autonomous Systems, 100, 95–107. google scholar
22. Monoscalco, L., Simeoni, R., Maccioni, G., & Giansanti, D. (2022). Information security in medical robotics: A survey on the level of training, awareness and use of the physiotherapist. Healthcare, 10(1), 159–175. google scholar
23. Panigrahi, R., & Borah, S. (2018). A detailed analysis of CICIDS2017 dataset for designing Intrusion Detection Systems. International Journal of Engineering & Technology, 7(3.24), 479–482. google scholar
24. Peng, C.-Y. J., Lee, K. L., & Ingersoll, G. M. (2002). An introduction to logistic regression analysis and reporting. The Journal of Educational Research, 96(1), 3–14. google scholar
25. Prettenhofer, P., & Louppe, G. (2014). Gradient boosted regression trees in scikit-learn. PyData 2014. google scholar
26. Pu, H., He, L., Cheng, P., Sun, M., & Chen, J. (2022). Security of industrial robots: Vulnerabilities, attacks, and mitigations. IEEE Network, 37(1), 111–117. google scholar
27. Quinto, B. (2020). Next-generation machine learning with spark: Covers XGBoost, LightGBM, Spark NLP, distributed deep learning with keras, and more. Apress. google scholar
28. Rahman, H. A. A., Wah, Y. B., He, H., & Bulgiba, A. (2015). Comparisons of ADABOOST, KNN, SVM and logistic regression in classification of imbalanced dataset. International conference on soft computing in data science, 54–64. google scholar
29. Raza, A., Memon, S., Nizamani, M. A., Dhomeja, L. D., Memon, N., & Charan, K. (2024). Machine Learning Techniques for Cyber Security in Internet of Robotic Things. VFAST Transactions on Software Engineering, 12(3), 01–10. google scholar
30. Reddy, E. M. K., Gurrala, A., Hasitha, V. B., & Kumar, K. V. R. (2022). Introduction to Naive Bayes and a review on its subtypes with applications. Bayesian Reasoning and Gaussian Processes for Machine Learning Applications, 1–14. google scholar
31. Sharma, U., Medasetti, U. S., Deemyad, T., Mashal, M., & Yadav, V. (2024). Mobile robot for security applications in remotely operated advanced reactors. Applied Sciences, 14(6), 2552. google scholar
32. Shiri, F. M., Perumal, T., Mustapha, N., & Mohamed, R. (2023). A comprehensive overview and comparative analysis on deep learning models: CNN, RNN, LSTM, GRU. arXiv Preprint arXiv:2305.17473. google scholar
33. Szeghalmy, S., & Fazekas, A. (2023). A comparative study of the use of stratified cross-validation and distribution-balanced stratified cross-validation in imbalanced learning. Sensors, 23(4), 2333. google scholar
34. Tanimu, J. A., & Abada, W. (2025). Addressing cybersecurity challenges in robotics: A comprehensive overview. Cyber Security and Applications, 3, 100074. google scholar
35. Tsapin, D., Pitelinskiy, K., Suvorov, S., Osipov, A., Pleshakova, E., & Gataullin, S. (2024). Machine learning methods for the industrial robotic systems security. Journal of Computer Virology and Hacking Techniques, 20(3), 397–414. google scholar
36. Verma, N., Kumar, N., Sheikh, Z. A., Koul, N., & Ashish, A. (2025). Machine Learning for the Cybersecurity of Robotic Cyber-Physical Systems: A Review. Procedia Computer Science, 259, 1817–1826. google scholar
37. Verma, N., Kumar, N., Verma, C., Illés, Z., & Singh, D. (2025). A systematic review on cybersecurity of robotic systems: Vulnerabilities trends, threats, attacks, challenges, and proposed framework. International Journal of Information Security, 24(3), 127. google scholar
38. Yaacoub, J.-P. A., Noura, H. N., Salman, O., & Chehab, A. (2022). Robotics cyber security: Vulnerabilities, attacks, countermeasures, and recommendations. International Journal of Information Security, 21(1), 115–158. google scholar
39. Zafar, M. H., Langas, E. F., Aftab, M. F., & Sanfilippo, F. (2024). Enhanced intrusion detection in robot operating systems via grid search based multi-head attention stacked convolutional network. 20th International Conference on Automation Science and Engineering (CASE). 3880–3885. google scholar
40. Zhang, S. (2021). Challenges in KNN classification. IEEE Transactions on Knowledge and Data Engineering, 34(10), 4663–4675. google scholar