Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection

Mustafa Burak Ürün; Yusuf Sönmez

doi:10.29130/dubited.1440640

EN TR

Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection

Abstract

The rise in internet usage and data transfer rates has led to numerous anomalies. Hence, anomaly-based intrusion detection systems (IDS) are essential in cybersecurity because of their ability to identify unknown cyber-attacks, especially zero-day attacks that signature-based IDS cannot detect. This study proposes an ensemble classification for intrusion detection using a weighted soft voting system with KNN, XGBoost, and Random Forest base models. The base model weights are optimized using the Nelder-Mead simplex method to improve the overall ensemble performance. We propose a robust intrusion detection framework that uses soft-voting classifier-level weights optimized using the Nelder-Mead algorithm and feature selection. We evaluated the system's performance using the KDD99 and UNSW-NB15 datasets, which demonstrated that the proposed approach exceeded other existing methods in respect of accuracy and provided comparable results with fewer features. The proposed system and its hyperparameter optimization technique were compared with other cyber threat detection and mitigation systems to determine their relative effectiveness and efficiency.

Keywords

Intrusion Detection Systems, Ensemble Learning, Soft Voting, Hyperparameter Optimization, Nelder-Mead Algorithm

Saldırı Tespiti İçin Nelder-Mead Algoritması ile Optimize Ağırlıklı Oylama Topluluk Öğrenmesi

Abstract

İnternet kullanımı ve veri aktarım hızlarındaki artış çok sayıda anomaliye yol açmıştır. Bu nedenle, anomali tabanlı saldırı tespit sistemleri (IDS), bilinmeyen siber saldırıları, özellikle de imza tabanlı IDS’lerin tespit edemediği sıfırıncı gün saldırılarını belirleme yetenekleri nedeniyle siber güvenlikte çok önemlidir. Bu çalışmada, KNN, XGBoost ve Rastgele Orman temel modelleri ile ağırlıklı bir yumuşak oylama sistemi kullanarak saldırı tespiti için bir topluluk sınıflandırması önerilmektedir. Temel modellerin ağırlıkları, genel topluluk performansını iyileştirmek için Nelder-Mead simpleks yöntemi kullanılarak optimize edilmiştir. Çalışmamızda, Nelder-Mead algoritması ve özellik seçimi kullanılarak optimize edilen yumuşak oylama sınıflandırıcı seviyesi ağırlıklarını kullanan sağlam bir saldırı tespit çerçevesi öneriyoruz. Sistemin performansı KDD99 ve UNSW-NB15 veri setleri kullanılarak değerlendirilmiş ve önerilen yaklaşımın doğruluk açısından mevcut diğer yöntemleri aştığını ve daha az özellik ile karşılaştırılabilir sonuçlar sağladığı tespit edilmiştir. Önerilen sistem ve hiperparametre optimizasyon tekniği, göreceli etkinlik ve verimliliğini belirlemek için diğer siber tehdit tespit ve sınırlama sistemleriyle karşılaştırılmıştır.

Keywords

Saldırı Tespit Sistemleri, Topluluk Öğrenme, Yumuşak Oylama, Hiperparametre Optimizasyonu, Nelder-Mead Algoritması

References

[1] “Mid-Year Update: 2023 SonicWall Cyber Threat Report”, Accessed: Sep. 30, 2023. [Online]. Available: https://www.sonicwall.com/2023-mid-year-cyber-threat-report/
[2] Md Haris Uddin Sharif and Mehmood Ali Mohammed, “A literature review of financial losses statistics for cyber security and future trend,” World J. Adv. Res. Rev., vol. 15, no. 1, pp. 138–156, Jul. 2022, doi: 10.30574/wjarr.2022.15.1.0573.
[3] “Cyber Security Market Analysis Report | 2022 - 2030.” Accessed: Nov. 05, 2023. [Online]. Available: https://www.nextmsc.com/report/cyber-security-market
[4] P. Spadaccino and F. Cuomo, “Intrusion Detection Systems for IoT: opportunities and challenges offered by Edge Computing and Machine Learning,” 2020, doi: 10.48550/ARXIV.2012.01174.
[5] W. Yao, L. Hu, Y. Hou, and X. Li, “A Two-Layer Soft-Voting Ensemble Learning Model For Network Intrusion Detection,” in 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), Baltimore, MD, USA: IEEE, Jun. 2022, pp. 155–161. doi: 10.1109/DSN-W54100.2022.00034.
[6] Y. Shen, K. Zheng, Y. Yang, S. Liu, and M. Huang, “CBA-CLSVE: A Class-Level Soft-Voting Ensemble Based on the Chaos Bat Algorithm for Intrusion Detection,” Appl. Sci., vol. 12, no. 21, p. 11298, Nov. 2022, doi: 10.3390/app122111298.
[7] R. Swami, M. Dave, and V. Ranga, “Voting‐based intrusion detection framework for securing software‐defined networks,” Concurr. Comput. Pract. Exp., vol. 32, no. 24, p. e5927, Dec. 2020, doi: 10.1002/cpe.5927.
[8] Y. Zhou, G. Cheng, S. Jiang, and M. Dai, “Building an efficient intrusion detection system based on feature selection and ensemble classifier,” Comput. Netw., vol. 174, p. 107247, Jun. 2020, doi: 10.1016/j.comnet.2020.107247.
[9] J. Gu, L. Wang, H. Wang, and S. Wang, “A novel approach to intrusion detection using SVM ensemble with feature augmentation,” Comput. Secur., vol. 86, pp. 53–62, Sep. 2019, doi: 10.1016/j.cose.2019.05.022.
[10] X. Gao, C. Shan, C. Hu, Z. Niu, and Z. Liu, “An Adaptive Ensemble Machine Learning Model for Intrusion Detection,” IEEE Access, vol. 7, pp. 82512–82521, 2019, doi: 10.1109/ACCESS.2019.2923640.

[11] S. Seth, K. K. Chahal, and G. Singh, “A Novel Ensemble Framework for an Intelligent Intrusion Detection System,” IEEE Access, vol. 9, pp. 138451–138467, 2021, doi: 10.1109/ACCESS.2021.3116219.
[12] R. Zhang, “Dynamic Weighted Voting Classifier for Network Intrusion Detection,” in 2022 International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), Guangzhou, China: IEEE, Aug. 2022, pp. 350–354. doi: 10.1109/MLISE57402.2022.00076.
[13] A. Harbola, J. Harbola, and K. S. Vaisla, “Improved Intrusion Detection in DDoS Applying Feature Selection Using Rank & Score of Attributes in KDD-99 Data Set,” in 2014 International Conference on Computational Intelligence and Communication Networks, Bhopal, India: IEEE, Nov. 2014, pp. 840–845. doi: 10.1109/CICN.2014.179.
[14] N. Moustafa and J. Slay, “The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set,” Inf. Secur. J. Glob. Perspect., vol. 25, no. 1–3, pp. 18–31, Apr. 2016, doi: 10.1080/19393555.2015.1125974.
[15] A. I. Saleh, F. M. Talaat, and L. M. Labib, “A hybrid intrusion detection system (HIDS) based on prioritized k-nearest neighbors and optimized SVM classifiers,” Artif. Intell. Rev., vol. 51, no. 3, pp. 403–443, Mar. 2019, doi: 10.1007/s10462-017-9567-1.
[16] “A novel SVM-kNN-PSO ensemble method for intrusion detection system,” Appl. Soft Comput., vol. 38, pp. 360–372, Jan. 2016, doi: 10.1016/j.asoc.2015.10.011.
[17] S. Dhaliwal, A.-A. Nahid, and R. Abbas, “Effective Intrusion Detection System Using XGBoost,” Information, vol. 9, no. 7, p. 149, Jun. 2018, doi: 10.3390/info9070149.
[18] P. A. A. Resende and A. C. Drummond, “A Survey of Random Forest Based Methods for Intrusion Detection Systems,” ACM Comput. Surv., vol. 51, no. 3, pp. 1–36, May 2019, doi: 10.1145/3178582.
[19] N. Zhu, C. Zhu, L. Zhou, Y. Zhu, and X. Zhang, “Optimization of the Random Forest Hyperparameters for Power Industrial Control Systems Intrusion Detection Using an Improved Grid Search Algorithm,” Appl. Sci. Switz., vol. 12, no. 20, Oct. 2022, doi: 10.3390/app122010456.
[20] Md. Raihan-Al-Masud and H. A. Mustafa, “Network Intrusion Detection System Using Voting Ensemble Machine Learning,” in 2019 IEEE International Conference on Telecommunications and Photonics (ICTP), Dhaka, Bangladesh: IEEE, Dec. 2019, pp. 1–4. doi: 10.1109/ICTP48844.2019.9041736.
[21] A. Z. Kiflay, A. Tsokanos, and R. Kirner, “A Network Intrusion Detection System Using Ensemble Machine Learning,” in 2021 International Carnahan Conference on Security Technology (ICCST), Hatfield, United Kingdom: IEEE, Oct. 2021, pp. 1–6. doi: 10.1109/ICCST49569.2021.9717397.
[22] A. Mohammed and R. Kora, “A comprehensive review on ensemble deep learning: Opportunities and challenges,” J. King Saud Univ. - Comput. Inf. Sci., vol. 35, no. 2, pp. 757–774, Feb. 2023, doi: 10.1016/j.jksuci.2023.01.014.
[23] J. A. Nelder and R. Mead, “A Simplex Method for Function Minimization,” Comput. J., vol. 7, no. 4, pp. 308–313, Jan. 1965, doi: 10.1093/comjnl/7.4.308.
[24] P. C. Wang and T. E. Shoup, “Parameter sensitivity study of the Nelder–Mead Simplex Method,” Adv. Eng. Softw., vol. 42, no. 7, pp. 529–533, Jul. 2011, doi: 10.1016/j.advengsoft.2011.04.004.
[25] S.-K. S. Fan and E. Zahara, “A hybrid simplex search and particle swarm optimization for unconstrained optimization,” Eur. J. Oper. Res., vol. 181, no. 2, pp. 527–548, Sep. 2007, doi: 10.1016/j.ejor.2006.06.034.
[26] J.-O. Palacio-Niño and F. Berzal, “Evaluation Metrics for Unsupervised Learning Algorithms.” arXiv, May 23, 2019. Accessed: Nov. 21, 2023. [Online]. Available: http://arxiv.org/abs/1905.05667
[27] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, p. 6, Dec. 2020, doi: 10.1186/s12864-019-6413-7.
[28] C. P. Chai, “The Importance of Data Cleaning: Three Visualization Examples,” CHANCE, vol. 33, no. 1, pp. 4–9, Jan. 2020, doi: 10.1080/09332480.2020.1726112.
[29] S. Khalid, T. Khalil, and S. Nasreen, “A survey of feature selection and feature extraction techniques in machine learning,” in 2014 Science and Information Conference, London, UK: IEEE, Aug. 2014, pp. 372–378. doi: 10.1109/SAI.2014.6918213.
[30] M. Farajzadeh-Zanjani, R. Razavi-Far, and M. Saif, “A Critical Study on the Importance of Feature Extraction and Selection for Diagnosing Bearing Defects,” in 2018 IEEE 61st International Midwest Symposium on Circuits and Systems (MWSCAS), Windsor, ON, Canada: IEEE, Aug. 2018, pp. 803–808. doi: 10.1109/MWSCAS.2018.8623823.
[31] “Version 1.2.2,” scikit-learn. Accessed: Nov. 29, 2023. [Online]. Available: https://scikit-learn/stable/whats_new/v1.2.html
[32] “scipy: Fundamental algorithms for scientific computing in Python.” Accessed: Nov. 29, 2023. [MacOS, Microsoft :: Windows, POSIX, POSIX :: Linux, Unix]. Available: https://scipy.org/
[33] N. Pham and B. M. Wilamowski, “Improved Nelder Mead’s Simplex Method and Applications,” vol. 3, no. 3, 2011.

Details

Primary Language

English

Subjects

Supervised Learning, Classification Algorithms

Journal Section

Research Article

Authors

Mustafa Burak Ürün ^*
0000-0003-2442-4214
Türkiye

Yusuf Sönmez
0000-0002-9775-9835
Türkiye

Publication Date

October 23, 2024

Submission Date

February 25, 2024

Acceptance Date

July 10, 2024

Published in Issue

Year 2024 Volume: 12 Number: 4

DOI

https://doi.org/10.29130/dubited.1440640

IZ

https://izlik.org/JA52KH66UX

APA

Ürün, M. B., & Sönmez, Y. (2024). Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection. Duzce University Journal of Science and Technology, 12(4), 2139-2158. https://doi.org/10.29130/dubited.1440640

AMA

1.Ürün MB, Sönmez Y. Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection. DUBİTED. 2024;12(4):2139-2158. doi:10.29130/dubited.1440640

Chicago

Ürün, Mustafa Burak, and Yusuf Sönmez. 2024. “Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection”. Duzce University Journal of Science and Technology 12 (4): 2139-58. https://doi.org/10.29130/dubited.1440640.

EndNote

Ürün MB, Sönmez Y (October 1, 2024) Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection. Duzce University Journal of Science and Technology 12 4 2139–2158.

IEEE

[1]M. B. Ürün and Y. Sönmez, “Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection”, DUBİTED, vol. 12, no. 4, pp. 2139–2158, Oct. 2024, doi: 10.29130/dubited.1440640.

ISNAD

Ürün, Mustafa Burak - Sönmez, Yusuf. “Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection”. Duzce University Journal of Science and Technology 12/4 (October 1, 2024): 2139-2158. https://doi.org/10.29130/dubited.1440640.

JAMA

1.Ürün MB, Sönmez Y. Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection. DUBİTED. 2024;12:2139–2158.

MLA

Ürün, Mustafa Burak, and Yusuf Sönmez. “Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection”. Duzce University Journal of Science and Technology, vol. 12, no. 4, Oct. 2024, pp. 2139-58, doi:10.29130/dubited.1440640.

Vancouver

1.Mustafa Burak Ürün, Yusuf Sönmez. Nelder-Mead Optimized Weighted Voting Ensemble Learning for Network Intrusion Detection. DUBİTED. 2024 Oct. 1;12(4):2139-58. doi:10.29130/dubited.1440640

Cited By

Exploring AI and quantum computing synergies in holographic counterpart frameworks for IoT security and privacy

The Journal of Supercomputing

https://doi.org/10.1007/s11227-025-07682-0