Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection

Abstract

The rise in internet usage and data transfer rates has led to numerous anomalies. Hence, anomaly-based intrusion detection systems (IDS) are essential in cybersecurity because of their ability to identify unknown cyber-attacks, especially zero-day attacks that signature-based IDS cannot detect. This study proposes an ensemble classification for intrusion detection using a weighted soft voting system with KNN, XGBoost, and Random Forest base models. The base model weights are optimized using the Nelder-Mead simplex method to improve the overall ensemble performance. We propose a robust intrusion detection framework that uses soft-voting classifier-level weights optimized using the Nelder-Mead algorithm and feature selection. We evaluated the system's performance using the KDD99 and UNSW-NB15 datasets, which demonstrated that the proposed approach exceeded other existing methods in respect of accuracy and provided comparable results with fewer features. The proposed system and its hyperparameter optimization technique were compared with other cyber threat detection and mitigation systems to determine their relative effectiveness and efficiency.

Keywords

• Intrusion Detection Systems
• Ensemble Learning
• Soft Voting
• Hyperparameter Optimization
• Nelder-Mead Algorithm

Saldırı Tespiti İçin Nelder-Mead Algoritması ile Optimize Ağırlıklı Oylama Topluluk Öğrenmesi

Abstract

İnternet kullanımı ve veri aktarım hızlarındaki artış çok sayıda anomaliye yol açmıştır. Bu nedenle, anomali tabanlı saldırı tespit sistemleri (IDS), bilinmeyen siber saldırıları, özellikle de imza tabanlı IDS’lerin tespit edemediği sıfırıncı gün saldırılarını belirleme yetenekleri nedeniyle siber güvenlikte çok önemlidir. Bu çalışmada, KNN, XGBoost ve Rastgele Orman temel modelleri ile ağırlıklı bir yumuşak oylama sistemi kullanarak saldırı tespiti için bir topluluk sınıflandırması önerilmektedir. Temel modellerin ağırlıkları, genel topluluk performansını iyileştirmek için Nelder-Mead simpleks yöntemi kullanılarak optimize edilmiştir. Çalışmamızda, Nelder-Mead algoritması ve özellik seçimi kullanılarak optimize edilen yumuşak oylama sınıflandırıcı seviyesi ağırlıklarını kullanan sağlam bir saldırı tespit çerçevesi öneriyoruz. Sistemin performansı KDD99 ve UNSW-NB15 veri setleri kullanılarak değerlendirilmiş ve önerilen yaklaşımın doğruluk açısından mevcut diğer yöntemleri aştığını ve daha az özellik ile karşılaştırılabilir sonuçlar sağladığı tespit edilmiştir. Önerilen sistem ve hiperparametre optimizasyon tekniği, göreceli etkinlik ve verimliliğini belirlemek için diğer siber tehdit tespit ve sınırlama sistemleriyle karşılaştırılmıştır.

Keywords

• Saldırı Tespit Sistemleri
• Topluluk Öğrenme
• Yumuşak Oylama
• Hiperparametre Optimizasyonu
• Nelder-Mead Algoritması

References

1. [1] “Mid-Year Update: 2023 SonicWall Cyber Threat Report”, Accessed: Sep. 30, 2023. [Online]. Available: https://www.sonicwall.com/2023-mid-year-cyber-threat-report/
2. [2] Md Haris Uddin Sharif and Mehmood Ali Mohammed, “A literature review of financial losses statistics for cyber security and future trend,” World J. Adv. Res. Rev., vol. 15, no. 1, pp. 138–156, Jul. 2022, doi: 10.30574/wjarr.2022.15.1.0573.
3. [3] “Cyber Security Market Analysis Report | 2022 - 2030.” Accessed: Nov. 05, 2023. [Online]. Available: https://www.nextmsc.com/report/cyber-security-market
4. [4] P. Spadaccino and F. Cuomo, “Intrusion Detection Systems for IoT: opportunities and challenges offered by Edge Computing and Machine Learning,” 2020, doi: 10.48550/ARXIV.2012.01174.
5. [5] W. Yao, L. Hu, Y. Hou, and X. Li, “A Two-Layer Soft-Voting Ensemble Learning Model For Network Intrusion Detection,” in 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), Baltimore, MD, USA: IEEE, Jun. 2022, pp. 155–161. doi: 10.1109/DSN-W54100.2022.00034.
6. [6] Y. Shen, K. Zheng, Y. Yang, S. Liu, and M. Huang, “CBA-CLSVE: A Class-Level Soft-Voting Ensemble Based on the Chaos Bat Algorithm for Intrusion Detection,” Appl. Sci., vol. 12, no. 21, p. 11298, Nov. 2022, doi: 10.3390/app122111298.
7. [7] R. Swami, M. Dave, and V. Ranga, “Voting‐based intrusion detection framework for securing software‐defined networks,” Concurr. Comput. Pract. Exp., vol. 32, no. 24, p. e5927, Dec. 2020, doi: 10.1002/cpe.5927.
8. [8] Y. Zhou, G. Cheng, S. Jiang, and M. Dai, “Building an efficient intrusion detection system based on feature selection and ensemble classifier,” Comput. Netw., vol. 174, p. 107247, Jun. 2020, doi: 10.1016/j.comnet.2020.107247.

9. [9] J. Gu, L. Wang, H. Wang, and S. Wang, “A novel approach to intrusion detection using SVM ensemble with feature augmentation,” Comput. Secur., vol. 86, pp. 53–62, Sep. 2019, doi: 10.1016/j.cose.2019.05.022.
10. [10] X. Gao, C. Shan, C. Hu, Z. Niu, and Z. Liu, “An Adaptive Ensemble Machine Learning Model for Intrusion Detection,” IEEE Access, vol. 7, pp. 82512–82521, 2019, doi: 10.1109/ACCESS.2019.2923640.
11. [11] S. Seth, K. K. Chahal, and G. Singh, “A Novel Ensemble Framework for an Intelligent Intrusion Detection System,” IEEE Access, vol. 9, pp. 138451–138467, 2021, doi: 10.1109/ACCESS.2021.3116219.
12. [12] R. Zhang, “Dynamic Weighted Voting Classifier for Network Intrusion Detection,” in 2022 International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), Guangzhou, China: IEEE, Aug. 2022, pp. 350–354. doi: 10.1109/MLISE57402.2022.00076.
13. [13] A. Harbola, J. Harbola, and K. S. Vaisla, “Improved Intrusion Detection in DDoS Applying Feature Selection Using Rank & Score of Attributes in KDD-99 Data Set,” in 2014 International Conference on Computational Intelligence and Communication Networks, Bhopal, India: IEEE, Nov. 2014, pp. 840–845. doi: 10.1109/CICN.2014.179.
14. [14] N. Moustafa and J. Slay, “The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set,” Inf. Secur. J. Glob. Perspect., vol. 25, no. 1–3, pp. 18–31, Apr. 2016, doi: 10.1080/19393555.2015.1125974.
15. [15] A. I. Saleh, F. M. Talaat, and L. M. Labib, “A hybrid intrusion detection system (HIDS) based on prioritized k-nearest neighbors and optimized SVM classifiers,” Artif. Intell. Rev., vol. 51, no. 3, pp. 403–443, Mar. 2019, doi: 10.1007/s10462-017-9567-1.
16. [16] “A novel SVM-kNN-PSO ensemble method for intrusion detection system,” Appl. Soft Comput., vol. 38, pp. 360–372, Jan. 2016, doi: 10.1016/j.asoc.2015.10.011.
17. [17] S. Dhaliwal, A.-A. Nahid, and R. Abbas, “Effective Intrusion Detection System Using XGBoost,” Information, vol. 9, no. 7, p. 149, Jun. 2018, doi: 10.3390/info9070149.
18. [18] P. A. A. Resende and A. C. Drummond, “A Survey of Random Forest Based Methods for Intrusion Detection Systems,” ACM Comput. Surv., vol. 51, no. 3, pp. 1–36, May 2019, doi: 10.1145/3178582.
19. [19] N. Zhu, C. Zhu, L. Zhou, Y. Zhu, and X. Zhang, “Optimization of the Random Forest Hyperparameters for Power Industrial Control Systems Intrusion Detection Using an Improved Grid Search Algorithm,” Appl. Sci. Switz., vol. 12, no. 20, Oct. 2022, doi: 10.3390/app122010456.
20. [20] Md. Raihan-Al-Masud and H. A. Mustafa, “Network Intrusion Detection System Using Voting Ensemble Machine Learning,” in 2019 IEEE International Conference on Telecommunications and Photonics (ICTP), Dhaka, Bangladesh: IEEE, Dec. 2019, pp. 1–4. doi: 10.1109/ICTP48844.2019.9041736.
21. [21] A. Z. Kiflay, A. Tsokanos, and R. Kirner, “A Network Intrusion Detection System Using Ensemble Machine Learning,” in 2021 International Carnahan Conference on Security Technology (ICCST), Hatfield, United Kingdom: IEEE, Oct. 2021, pp. 1–6. doi: 10.1109/ICCST49569.2021.9717397.
22. [22] A. Mohammed and R. Kora, “A comprehensive review on ensemble deep learning: Opportunities and challenges,” J. King Saud Univ. - Comput. Inf. Sci., vol. 35, no. 2, pp. 757–774, Feb. 2023, doi: 10.1016/j.jksuci.2023.01.014.
23. [23] J. A. Nelder and R. Mead, “A Simplex Method for Function Minimization,” Comput. J., vol. 7, no. 4, pp. 308–313, Jan. 1965, doi: 10.1093/comjnl/7.4.308.
24. [24] P. C. Wang and T. E. Shoup, “Parameter sensitivity study of the Nelder–Mead Simplex Method,” Adv. Eng. Softw., vol. 42, no. 7, pp. 529–533, Jul. 2011, doi: 10.1016/j.advengsoft.2011.04.004.
25. [25] S.-K. S. Fan and E. Zahara, “A hybrid simplex search and particle swarm optimization for unconstrained optimization,” Eur. J. Oper. Res., vol. 181, no. 2, pp. 527–548, Sep. 2007, doi: 10.1016/j.ejor.2006.06.034.
26. [26] J.-O. Palacio-Niño and F. Berzal, “Evaluation Metrics for Unsupervised Learning Algorithms.” arXiv, May 23, 2019. Accessed: Nov. 21, 2023. [Online]. Available: http://arxiv.org/abs/1905.05667
27. [27] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, p. 6, Dec. 2020, doi: 10.1186/s12864-019-6413-7.
28. [28] C. P. Chai, “The Importance of Data Cleaning: Three Visualization Examples,” CHANCE, vol. 33, no. 1, pp. 4–9, Jan. 2020, doi: 10.1080/09332480.2020.1726112.
29. [29] S. Khalid, T. Khalil, and S. Nasreen, “A survey of feature selection and feature extraction techniques in machine learning,” in 2014 Science and Information Conference, London, UK: IEEE, Aug. 2014, pp. 372–378. doi: 10.1109/SAI.2014.6918213.
30. [30] M. Farajzadeh-Zanjani, R. Razavi-Far, and M. Saif, “A Critical Study on the Importance of Feature Extraction and Selection for Diagnosing Bearing Defects,” in 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS), Windsor, ON, Canada: IEEE, Aug. 2018, pp. 803–808. doi: 10.1109/MWSCAS.2018.8623823.
31. [31] “Version 1.2.2,” scikit-learn. Accessed: Nov. 29, 2023. [Online]. Available: https://scikit-learn/stable/whats_new/v1.2.html
32. [32] “scipy: Fundamental algorithms for scientific computing in Python.” Accessed: Nov. 29, 2023. [MacOS, Microsoft :: Windows, POSIX, POSIX :: Linux, Unix]. Available: https://scipy.org/
33. [33] N. Pham and B. M. Wilamowski, “Improved Nelder Mead’s Simplex Method and Applications,” vol. 3, no. 3, 2011.