Understanding Machine Learning Model Behavior for Intrusion Detection Across Attacks

Year 2025, Volume: 9 Issue: 4, 768 - 778, 08.10.2025

Mehmet Erdi Özbek , Ece Gelal Soyak

https://doi.org/10.31127/tuje.1613468

Abstract

With the growing volume and variety of network traffic driven by various applications such as real-time communications and cloud services, combined with the increasing sophistication and frequency of malicious attempts, network administrators are facing greater challenges in securing their networks against malware. Over the past two decades, advances in machine learning and deep learning have led to a growing number of proposals for intelligent Network Intrusion Detection Systems (NIDS) that leverage these models to detect the unauthorized entry of security threats into the network. Existing studies focus on improving model accuracies, without a closer analysis of the underlying characteristics of the data. In this work, we analyze the effectiveness of NIDS mechanisms in different scenarios using different machine learning models. By examining classification performance across various data distributions -including scenarios with and without normal traffic and cases addressing class imbalance- we identify patterns in model behaviors and their correlation with attack characteristics. In our experiments, we have observed, (i) the kNN algorithm achieved the fastest training and testing times while maintaining adequate accuracy, (ii) XGBoost performed best in detecting the most commonly occurring attacks, (iii) MLP provided the highest improvement in minority class labels when resampling was applied in the dataset, and (iv) notably, while Reconnaissance attacks were consistently detected even with limited samples, detection of DoS attacks remained challenging with all models. We believe NIDS systems could benefit from the insights raised in this work based on the interplay between attack behaviors, data distributions, and model characteristics.

Keywords

intrusion detection systems , machine learning , malware classification , attack behavior , cybersecurity , system security , information security

References

• Alkashto, H., & Elewi, A. (2024). Integration of blockchain and machine learning for safe and efficient autonomous car systems: A survey. Turkish Journal of Engineering, 8(2), 282-299
• Ayas, M. Ş. (2021). A brief review on attack design and detection strategies for networked cyber-physical systems. Turkish Journal of Engineering, 5(1), 1-7.
• Bace, R., & Mell, P. (2001). Intrusion detection systems, special publication, National Institute of Standards and Technology (NIST), 16.
• Moustafa, N., & Slay, J. (2015, November). UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 Military Communications and Information Systems Conf. (MilCIS)(pp. 1-6). IEEE.
• Basholli, F., Daberdini, A., & Basholli, A. (2023). Possibility of protection against unauthorized interference in telecommunication systems. Engineering Applications, 2(3), 265-278.
• Liao, H. J., Lin, C. H. R., Lin, Y. C., & Tung, K. Y. (2013). Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications, 36(1), 16-24.
• Khraisat, A., Gondal, I., Vamplew, P., & Kamruzzaman, J. (2019). Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity, 2(1), 1-22.
• Singh, A. P., Singh, M., Bhatia, K., Pathak, H. (2024). Encrypted malware detection methodology without decryption using deep learning-based approaches. Turkish Journal of Engineering, 8(3), 498-509.
• Singh, A. (2025). Real Time Intrusion Detection In Edge Computing Using Machine Learning Techniques. Turkish Journal of Engineering, 9(2), 385-393.
• Moustafa, N., & Slay, J. (2016). The evaluation of Network Anomaly Detection Systems: Statistical analysis of UNSW-NB15 data set and the comparison with the KDD99 data set. Information Security Journal: A Global Perspective, 25(1-3), 18-31.
• Shafiq, M., Tian, Z., Bashir, A. K., Du, X., & Guizani, M. (2020). CorrAUC: a malicious bot-IoT traffic detection method in IoT network using machine-learning techniques. IEEE Internet of Things Journal, 8(5), 3242-3254.
• Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009, July). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (pp. 1-6). IEEE.
• Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1, 108-116.
• Koroniotis, N., Moustafa, N., Sitnikova, E., & Turnbull, B. (2019). Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-IoT dataset. Future Generation Computer Systems, 100, 779-796.
• Ferriyan, A., Thamrin, A. H., Takeda, K., & Murai, J. (2021). Generating network intrusion detection dataset based on real and encrypted synthetic attack traffic. Applied Sciences, 11(17), 7868.
• Guven, E. Y., Gulgun, S., Manav, C., Bakir, B., & Aydin, Z. G. (2022). Multiple classification of cyber attacks using machine learning. Electrica, vol. 22, no. 2, pp. 313–320, 2022.
• Fernandes, R., & Lopes, N. (2022, June). Network intrusion detection packet classification with the HIKARI-2021 dataset: a study on ML algorithms. In 2022 10th International Symposium on Digital Forensics and Security (ISDFS) (pp. 1-5). IEEE.
• Vinayakumar, R., et al., (2019). Deep learning approach for intelligent intrusion detection system. IEEE Access, 7, 41525-41550.
• Kilincer, I. F., Ertam, F., & Sengur, A. (2021). Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks, 188, 107840.
• Zou, L., Wei, Y., Ma, L., & Leng, S. (2022, May). Feature-attended multi-flow LSTM for anomaly detection in internet of things. In IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) (pp. 1-6). IEEE.
• Kiflay, A. Z., Tsokanos, A., & Kirner, R. (2021, October). A network intrusion detection system using ensemble machine learning. In 2021 International Carnahan Conference on Security Technology (ICCST) (pp. 1-6). IEEE.
• Wester, P., Heiding, F., & Lagerström, R. (2021, October). Anomaly-based intrusion detection using tree augmented naive bayes. In 2021 IEEE 25th International Enterprise Distributed Object Computing Workshop (EDOCW) (pp. 112-121).
• Huma, Z. E., Latif, S., Ahmad, J., Idrees, Z., Ibrar, A., Zou, Z., ... & Baothman, F. (2021). A hybrid deep random neural network for cyberattack detection in the Industrial Internet of Things. IEEE Access, 9, 55595-55605.
• Yang, S., Guo, H., & Moustafa, N. (2021, December). Hunter in the dark: Discover anomalous network activity using deep ensemble network. In 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS) (pp. 829-840). IEEE.
• Abou El Houda, Z., Hafid, A. S., & Khoukhi, L. (2021, Dec.). A novel machine learning framework for advanced attack detection using SDN. In 2021 IEEE Global Comm. Conference (GLOBECOM) (pp. 1-6).
• Azad, S., Naqvi, S. S., Sabrina, F., Sohail, S., & Thakur, S. (2021, December). Iot cybersecurity: On the use of machine learning approaches for unbalanced datasets. In IEEE Asia-Pacific Conf. on Computer Science and Data Engineering (CSDE) (pp. 1-6). IEEE.
• Arregoces, P., Vergara, J., Gutiérrez, S. A., & Botero, J. F. (2022, April). Network-based intrusion detection: A one-class classification approach. In NOMS 2022-IEEE/IFIP Network Operations and Management Symposium (pp. 1-6). IEEE.
• Guizani, N., & Ghafoor, A. (2020). A network function virtualization system for detecting malware in large IoT based networks. IEEE Journal on Selected Areas in Communications, 38(6), 1218-1228.
• Husain, A., Salem, A., Jim, C., & Dimitoglou, G. (2019, December). Development of an efficient network intrusion detection model using extreme gradient boosting (XGBoost) on the UNSW-NB15 dataset. In 2019 IEEE Int. Symposium on Signal Processing and Information Technology (ISSPIT) (pp. 1-7). IEEE.
• Al-Zewairi, M., Almajali, S., & Awajan, A. (2017, October). Experimental evaluation of a multi-layer feed-forward artificial neural network classifier for network intrusion detection system. In 2017 International Conference on New Trends in Computing Sciences (ICTCS) (pp. 167-172). IEEE.
• Ahmad, M., Riaz, Q., Zeeshan, M., Tahir, H., Haider, S. A., & Khan, M. S. (2021). Intrusion detection in internet of things using supervised machine learning based on application and transport layer features using UNSW-NB15 data-set. EURASIP Journal on Wireless Comm. and Networking, 1-23.
• Larriva-Novo, X., Villagrá, V. A., Vega-Barbas, M., Rivera, D., & Sanz Rodrigo, M. (2021). An IoT-focused intrusion detection system approach based on preprocessing characterization for cybersecurity datasets. Sensors, 21(2), 656.
• Bagui, S., Kalaimannan, E., Bagui, S., Nandi, D., & Pinto, A. (2019). Using machine learning techniques to identify rare cyber‐attacks on the UNSW‐NB15 dataset. Security and Privacy, 2(6), e91.
• Dainotti, A., Pescapé, A., & Ventre, G. (2007, June). Worm traffic analysis and characterization. In 2007 IEEE International Conference on Communications (ICC) (pp. 1435-1442). IEEE.
• M. Erdi Özbek (2022), “Malware analysis and identification with machine learning techniques”, MSc Thesis, Bahcesehir University, Turkey.