Detecting Robotic Cyber Attacks in Robot Operating System Networks

Year 2025, Volume: 9 Issue: 2, 682 - 701, 31.12.2025

Hamdullah Karamollaoğlu

https://doi.org/10.26650/acin.1798154

https://izlik.org/JA29LA62PT

Abstract

The increasing deployment of robotic systems in critical sectors such as manufacturing, healthcare, and infrastructure necessitates robust cybersecurity measures. The Robot Operating System (ROS), a core middleware in modern robotics, is inherently susceptible to cyber threats due to its lack of integrated security mechanisms. This study presents a comprehensive benchmark evaluation of intrusion detection solutions for ROS-based environments. Utilizing the novel ROSIDS23 dataset, which includes realistic attack scenarios—such as Denial-of-Service (DoS), Unauthorized Publish, Unauthorized Subscribe, and Subscriber Flood—we rigorously evaluated and compared fifteen state-of-the-art Machine Learning (ML) and Deep Learning (DL) models. Using stratified 5-fold cross-validation, our results demonstrate that ensemble methods significantly outperform deep learning approaches in this context. Gradient Boosting achieved the highest performance, with 99.80% accuracy, precision, recall, and F1-score, followed by Light Gradient Boosting Machine (LightGBM) at 99.51% and Extreme Gradient Boosting (XGBoost) at 99.48%. Among DL models, the best-performing One-Dimensional Convolutional Neural Network (1D-CNN) reached 98.55%. Beyond overall metrics, we examine per-class performance, confusion matrices, and Receiver Operating Characteristic (ROC) curves, highlighting model-specific strengths and weaknesses, particularly in detecting minority attack classes.

Keywords

Robot operating system , cyber attacks , machine learning , deep learning , robotic cybersecurity

References

• Ahmad Yousef, K. M., AlMajali, A., Ghalyon, S. A., Dweik, W., & Mohd, B. J. (2018). Analyzing cyber-physical threats on robotic platforms. Sensors, 18(5), 1643. google scholar
• An, T.-K., & Kim, M.-H. (2010). A new diverse AdaBoost classifier. 2010 International conference on artificial intelligence and computational intelligence, 359–363. google scholar
• Bezemskij, A., Loukas, G., Anthony, R. J., & Gan, D. (2016). Behaviour-based anomaly detection of cyber-physical attacks on a robotic vehicle. 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS), 61–68. google scholar
• Botta, A., Rotbei, S., Zinno, S., & Ventre, G. (2023). Cyber security of robots: A comprehensive survey. Intelligent Systems with Applications, 18, 200237. google scholar
• Chaudhary, A., Kolhe, S., & Kamal, R. (2016). An improved random forest classifier for multi-class classification. Information Processing in Agriculture, 3(4), 215–222. google scholar
• Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(1), 6. google scholar
• Chicco, D., & Jurman, G. (2023). The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification. BioData Mining, 16(1), 4. google scholar
• Değirmenci, E., Kırca, Y. S., Özçelik, İ., & Yazıcı, A. (2023). ROSIDS23: Network intrusion detection dataset for robot operating system. Data in Brief, 51, 109739. google scholar
• Değirmenci, E., Özçelik, I., & Yazıcı, A. (2024). Adversarial Attack Detection Approach for Intrusion Detection Systems. IEEE Access, 12, 195996–196009. google scholar
• Deo, T. Y., & Sanju, A. (2023). Data imputation and comparison of custom ensemble models with existing libraries like XGBoost, CATBoost, AdaBoost and Scikit learn for predictive equipment failure. Materials Today: Proceedings, 72, 1596–1604. google scholar
• Dhanabal, L., & Shantharajah, S. (2015). A study on NSL-KDD dataset for intrusion detection system based on classification algorithms. International Journal of Advanced Research in Computer and Communication Engineering, 4(6), 446–452. google scholar
• Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. 27th European Conference on IR Research, 345–359. google scholar
• Heydarian, M., Doyle, T. E., & Samavi, R. (2022). MLCM: Multi-label confusion matrix. Ieee Access, 10, 19083–19095. google scholar
• Holdbrook, R., Odeyomi, O., Yi, S., & Roy, K. (2024). Network-Based Intrusion Detection for Industrial and Robotics Systems: A Comprehensive Survey. Electronics, 13(22), 4440. google scholar
• Hussein, N., & Zeebaree, S. R. (2024). Performance evaluation of extra trees classifier by using cpu parallel and non-parallel processing. The Indonesian Journal of Computer Science, 13(2), 1859–1872. google scholar
• Ibrahim, A. A., Ridwan, R. L., Muhammed, M. M., Abdulaziz, R. O., & Saheed, G. A. (2020). Comparison of the CatBoost classifier with other machine learning methods. International Journal of Advanced Computer Science and Applications, 11(11), 738–748. google scholar
• Joy, J., & Selvan, M. P. (2022). A comprehensive study on the performance of different Multi-class Classification Algorithms and Hyperparameter Tuning Techniques using Optuna. International Conference on Computing, Communication, Security and Intelligent Systems (IC3SIS), 1–5. google scholar
• Kang, J., Kim, K., & Kwon, D. (2025). Watch Your Callback: Offline Anomaly Detection using Machine Learning in ROS 2. IEEE Access, 13, 60763-60775. google scholar
• Krejčí, J., Babiuch, M., Suder, J., Krys, V., & Bobovský, Z. (2025). Internet of robotic things: Current technologies, challenges, applications, and future research topics. Sensors, 25(3), 765. google scholar
• Lee, T., Singh, V. P., & Cho, K. H. (2021). Tensorflow and keras programming for deep learning. Deep learning for hydrometeorology and environmental science, 151–162. google scholar
• Martín, F., Soriano, E., & Cañas, J. M. (2018). Quantitative analysis of security in distributed robotic frameworks. Robotics and Autonomous Systems, 100, 95–107. google scholar
• Monoscalco, L., Simeoni, R., Maccioni, G., & Giansanti, D. (2022). Information security in medical robotics: A survey on the level of training, awareness and use of the physiotherapist. Healthcare, 10(1), 159–175. google scholar
• Panigrahi, R., & Borah, S. (2018). A detailed analysis of CICIDS2017 dataset for designing Intrusion Detection Systems. International Journal of Engineering & Technology, 7(3.24), 479–482. google scholar
• Peng, C.-Y. J., Lee, K. L., & Ingersoll, G. M. (2002). An introduction to logistic regression analysis and reporting. The Journal of Educational Research, 96(1), 3–14. google scholar
• Prettenhofer, P., & Louppe, G. (2014). Gradient boosted regression trees in scikit-learn. PyData 2014. google scholar
• Pu, H., He, L., Cheng, P., Sun, M., & Chen, J. (2022). Security of industrial robots: Vulnerabilities, attacks, and mitigations. IEEE Network, 37(1), 111–117. google scholar
• Quinto, B. (2020). Next-generation machine learning with spark: Covers XGBoost, LightGBM, Spark NLP, distributed deep learning with keras, and more. Apress. google scholar
• Rahman, H. A. A., Wah, Y. B., He, H., & Bulgiba, A. (2015). Comparisons of ADABOOST, KNN, SVM and logistic regression in classification of imbalanced dataset. International conference on soft computing in data science, 54–64. google scholar
• Raza, A., Memon, S., Nizamani, M. A., Dhomeja, L. D., Memon, N., & Charan, K. (2024). Machine Learning Techniques for Cyber Security in Internet of Robotic Things. VFAST Transactions on Software Engineering, 12(3), 01–10. google scholar
• Reddy, E. M. K., Gurrala, A., Hasitha, V. B., & Kumar, K. V. R. (2022). Introduction to Naive Bayes and a review on its subtypes with applications. Bayesian Reasoning and Gaussian Processes for Machine Learning Applications, 1–14. google scholar
• Sharma, U., Medasetti, U. S., Deemyad, T., Mashal, M., & Yadav, V. (2024). Mobile robot for security applications in remotely operated advanced reactors. Applied Sciences, 14(6), 2552. google scholar
• Shiri, F. M., Perumal, T., Mustapha, N., & Mohamed, R. (2023). A comprehensive overview and comparative analysis on deep learning models: CNN, RNN, LSTM, GRU. arXiv Preprint arXiv:2305.17473. google scholar
• Szeghalmy, S., & Fazekas, A. (2023). A comparative study of the use of stratified cross-validation and distribution-balanced stratified cross-validation in imbalanced learning. Sensors, 23(4), 2333. google scholar
• Tanimu, J. A., & Abada, W. (2025). Addressing cybersecurity challenges in robotics: A comprehensive overview. Cyber Security and Applications, 3, 100074. google scholar
• Tsapin, D., Pitelinskiy, K., Suvorov, S., Osipov, A., Pleshakova, E., & Gataullin, S. (2024). Machine learning methods for the industrial robotic systems security. Journal of Computer Virology and Hacking Techniques, 20(3), 397–414. google scholar
• Verma, N., Kumar, N., Sheikh, Z. A., Koul, N., & Ashish, A. (2025). Machine Learning for the Cybersecurity of Robotic Cyber-Physical Systems: A Review. Procedia Computer Science, 259, 1817–1826. google scholar
• Verma, N., Kumar, N., Verma, C., Illés, Z., & Singh, D. (2025). A systematic review on cybersecurity of robotic systems: Vulnerabilities trends, threats, attacks, challenges, and proposed framework. International Journal of Information Security, 24(3), 127. google scholar
• Yaacoub, J.-P. A., Noura, H. N., Salman, O., & Chehab, A. (2022). Robotics cyber security: Vulnerabilities, attacks, countermeasures, and recommendations. International Journal of Information Security, 21(1), 115–158. google scholar
• Zafar, M. H., Langas, E. F., Aftab, M. F., & Sanfilippo, F. (2024). Enhanced intrusion detection in robot operating systems via grid search based multi-head attention stacked convolutional network. 20th International Conference on Automation Science and Engineering (CASE). 3880–3885. google scholar
• Zhang, S. (2021). Challenges in KNN classification. IEEE Transactions on Knowledge and Data Engineering, 34(10), 4663–4675. google scholar