Comparative Performance Analysis of Machine Learning Algorithms: Random Cut Forest, Robust Random Cut Forest, and Amazon Sage Maker Random Cut Forest for Intrusion Detection Systems Using the CIS IDS 2017 Dataset

Abstract

Dynamic cyber threats are screaming for better anomaly detection techniques in Intrusion Detection Systems. Organizations today are hugely dependent on digital infrastructures for which effective security is priceless. The following research article does a critical and comparative analysis among three popular algorithms, namely Amazon Sage Maker Random Cut Forest, Robust Random Cut Forest, and traditional Random Cut Forest. Using the CIS IDS 2017 dataset with multifaceted network traffic features together with the labeled type of attack, this work rigorously tests the performance in anomaly detection that may show potential intrusion, robustness, scalability, and adaptability of each algorithm. The comparative analysis does the performance metrics of each algorithm based on accuracy, precision, recall, and F1-score in a real-world setting. The findings are expected to provide useful insights toward optimizing IDS frameworks for hi-tech cybersecurity resilience. Finally, an organization can make decisions on its strategy regarding cyber security by being enlightened on the strengths and weaknesses of algorithms. In essence, this paper contributes to the larger body of research on enhancing intrusion detection methodologies in an environment that is confronted by sophisticated cyber-attacks.

Keywords

• Cyber Threats
• Anomaly Detection
• Intrusion Detection Systems
• Amazon Sage Maker Random Cut Forest
• Robust Random Cut Forest

References

1. Khraisat, A., Gondal, I., Vamplew, P., & Kamruzzaman, J. (2019). Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity,2. https://doi.org/10.1186/s42400-019-0038-7.
2. Singh, R., Kumar, H., Singla, R. K., & Ketti, R. R. (2017). Internet attacks and intrusion detection system: A review of the literature. Online Information Review, 41(2), 171-184.
3. Rjoub, G., Bentahar, J., Wahab, O., Mizouni, R., Song, A., Cohen, R., Otrok, H., & Mourad, A. (2023). A Survey on Explainable Artificial Intelligence for Cybersecurity. IEEE Transactions on Network and Service Management, 20, 5115-5140. https://doi.org/10.1109/TNSM.2023.3282740.
4. Tidjon, L. N., Frappier, M., & Mammar, A. (2019). Intrusion detection systems: A cross-domain overview. IEEE Communications Surveys & Tutorials, 21(4), 3639-3681.
5. Milenkoski, A., Vieira, M., Kounev, S., Avritzer, A., & Payne, B. D. (2015). Evaluating computer intrusion detection systems: A survey of common practices. ACM Computing Surveys (CSUR), 48(1), 1-41.
6. Depren, Ö., Topallar, M., Anarim, E., & Ciliz, M. K. (2005). An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks. Expert Systems with Applications, 29(4), 713–722. https://doi.org/10.1016/j.eswa.2005.05.002
7. Kim, D., Yang, J., & Sim, K. (2004). Adaptive intrusion detection algorithm based on learning algorithm. 30th Annual Conference of IEEE Industrial Electronics Society, 2004. IECON 2004, 3, 2229-2233
8. Mittal, A., Gupta, A., & Agarwal, K. (2024, May). Anomaly Detection in Cybersecurity: Leveraging Machine Learning for Intrusion Detection. In 2024 International Conference on Communication, Computer Sciences and Engineering (IC3SE) (pp. 1-5). IEEE.

9. Mudigonda, N. (2022). A Method for Network Intrusion Detection Using Deep Learning, Journal of Student Research, 11(3).
10. Yaokumah, W., & Wiafe, I. (2020). Analysis of machine learning techniques for anomaly-based intrusion detection, International Journal of Distributed Artificial Intelligence (IJDAI), 12(1), 20-38.
11. Liu, Z., Su, N., Qin, Y., Lu, J., & Li, X. (2020). A deep random forest model on spark for network intrusion detection, Mobile Information Systems, 2020(1), 6633252.
12. Baykan, N. A., & Khorram, T. (2021). Network Intrusion Detection using Optimized Machine Learning Algorithms, Avrupa Bilim ve Teknoloji Dergisi, (25), 463-474.
13. Basholli, F., Mema, B., & Basholli, A. (2024). Training of information technology personnel through simulations for protection against cyber attacks. Engineering Applications, 3(1), 45-58.
14. İncekara, Çetin Önder . (2023). Industrial internet of things (IIoT) in energy sector. Advanced Engineering Science, 3, 21–30. Retrieved from https://publish.mersin.edu.tr/index.php/ades/article/view/839
15. Oliveira, N., Praça, I., Maia, E., & Sousa, O. (2021). Intelligent cyber attack detection and classification for network-based intrusion detection systems. Applied Sciences, 11(4), 1674. https://doi.org/10.3390/app11041674
16. A. Airoboman, I. Araga, and J. Mohammad-Ashafa, “Reliability Improvement of Distribution System Network using Network Reconfiguration,” Engineering Applications, vol. 3, no. 3, pp. 214–225, 2024. [Online]. Available: https://publish.mersin.edu.tr/index.php/enap/article/view/1581.
17. Nwafor, E. O., & Akintayo, F. O. (2024). Predicting trip purposes of households in Makurdi using machine learning: A comparative analysis of decision tree, CatBoost, and XGBoost algorithms. Engineering Applications, 3(3), 260–274. Retrieved from https://publish.mersin.edu.tr/index.php/enap/article/view/1605.
18. Mema, B., Basholli, F., & Hyka, D. (2024). Learning transformation and virtual interaction through ChatGPT in Albanian higher education. Advanced Engineering Science, 4, 130–140. Retrieved from https://publish.mersin.edu.tr/index.php/ades/article/view/1509.
19. Folino, G., Otranto Godano, C., & Pisani, F. S. (2023). An ensemble-based framework for user behaviour anomaly detection and classification for cybersecurity. Journal of Supercomputing, 79(9), 11660–11683. https://doi.org/10.1007/s11227-023-05049-x
20. Gonaygunta, H., Nadella, G. S., Pawar, P. P., & Kumar, D. (2024). Enhancing cybersecurity: The development of a flexible deep learning model for enhanced anomaly detection. Systems and Information Engineering Design Symposium (SIEDS), Charlottesville, VA, USA, 79–84. https://doi.org/10.1109/SIEDS61124.2024.10534661
21. Handa, R., Kumar, S., & Kumar, S. (2019). Machine learning in cybersecurity: A review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(1), e1306. https://doi.org/10.1002/widm.1306
22. Bukhari, O., Agarwal, P., Koundal, D., & Zafar, S. (2023). Anomaly detection using ensemble techniques for boosting the security of intrusion detection system. Procedia Computer Science, 218, 1003-1013. https://doi.org/10.1016/j.procs.2023.01.080
23. Habeeb, R. A. A., Nasaruddin, F., Gani, A., Hashem, I. A. T., Ahmed, E., & Imran, M. (2019). Real-time big data processing for anomaly detection: A survey. International Journal of Information Management, 45, 289-307.
24. Fernandes, G., Rodrigues, J. J., Carvalho, L. F., Al-Muhtadi, J. F., & Proença, M. L. (2019). A comprehensive survey on network anomaly detection. Telecommunication Systems, 70, 447-489.
25. Sharafaldin, I., Habibi Lashkari, A., Ghorbani, A.A. (2019). A Detailed Analysis of the CICIDS2017 Data Set. In: Mori, P., Furnell, S., Camp, O. (eds) Information Systems Security and Privacy. ICISSP 2018. Communications in Computer and Information Science, vol 977. Springer
26. Liu, C., Gu, Z., & Wang, J. (2021). A hybrid intrusion detection system based on scalable k-means+ random forest and deep learning. IEEE Access,9,74745–74756.
27. Amazon Web Services. (n.d.). Amazon SageMaker developer guide: Random Cut Forest algorithm (pp. 3567-3577). https://docs.aws.amazon.com/pdfs/sagemaker/latest/dg/sagemaker-dg.pdf#randomcutforest
28. Guha, S., Mishra, N., Roy, G., & Schrijvers, O. (2016, June). Robust random cut forest based anomaly detection on streams. In International conference on machine learning (pp. 2712-2721). PMLR.
29. Pang, Z., Cen, J., & Yi, M. (2023). Unsupervised concept drift detection method based on robust random cut forest. International Journal of Machine Learning and Cybernetics, 14(12), 4207-4222.
30. Yeom, S., & Jung, J. H. (2022). Weighted Isolation and Random Cut Forest Algorithms for Anomaly Detection. arXiv preprint arXiv:2202.01891.
31. Trawinski, I., Wimmer, H., & Kim, J. (2023). Anomaly detection in intrusion detection system using Amazon SageMaker. 2023 IEEE/ACIS 21st International Conference on Software Engineering Research, Management and Applications (SERA), 210–217. https://doi.org/10.1109/SERA57763.2023.10197735
32. Nigenda, D., Karnin, Z., Zafar, M. B., Ramesha, R., Tan, A., Donini, M., & Kenthapadi, K. (2022, August). Amazon sagemaker model monitor: A system for real-time insights into deployed machine learning models. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 3671-3681).
33. Jabbar, A. F., & Mohammed, I. J. (2020, November). Development of an optimized botnet detection framework based on filters of features and machine learning classifiers using CICIDS2017 dataset. In IOP Conference Series: Materials Science and Engineering (Vol. 928, No. 3, p. 032027). IOP Publishing.