Comparison of hybrid binary GWO-PSO algorithm with feature selection methods by using machine learning classifiers

Abstract

In the field of machine learning, feature selection methods used in the pre-processing of data for the classifier have become very popular. Instead of the whole dataset, it is important to create a new sub-dataset by discarding the irrelevant and redundant variables in the dataset to make the data ready for analysis. In this way, both the performance of the learning classifier will increase, and cost and time savings will be achieved. In this study, the performance of the hybrid binary grey wolf optimization - particle swarm optimization (BHGWOPSO) algorithm with machine learning methods is investigated. In addition, a comparison was made between BHGWOPSO and other feature selection methods such as principial component analysis and filter methods in contrast to literature. Thus, it is aimed to show which of the different feature selection methods will work better. For this purpose, five different benchmark datasets with different number of features were selected. Both feature selection methods and machine learning classifiers were compared with each other using the accuracy metric. As a result of the comparisons, it was observed that a different feature selection method and a different classifier had higher accuracy values for each data set.

Keywords

• Binary hibrid optimization
• Feature selection
• Learning classifiers
• Wrapper method

Hibrit ikili GWO-PSO algoritmasının makine öğrenmesi sınıflandırıcıları kullanılarak özellik seçim yöntemleriyle karşılaştırılması

Abstract

Makine öğrenmesi alanında sınıflandırıcı için verinin ön işlemesinde kullanılan değişken seçme yöntemleri oldukça popüler bir hale gelmiştir. Tüm veri seti yerine, veri setindeki değişkenlerden ilgisiz ve gereksiz olanların atılarak yeni bir alt veri kümesi oluşturulması veriyi analize hazır hale getirmek için önemlidir. Bu sayede öğrenme sınıflandırıcının hem performansı artacak hem de maliyet ve zaman bakımından tasarruf sağlanabilecektir. Bu çalışmada hibrit ikili gri kurt optimizasyon-parçacık sürü optimizasyon (BHGWOPSO) algoritmasının makine öğrenmesi yöntemleriyle performansı araştırılmıştır. Ayrıca simülasyonlarda literatürden farklı olarak BHGWOPSO ile diğer özellik seçim yöntemlerinden temel bileşen analizi ve filtre yöntemler kullanılarak da karşılaştırma yapılmıştır. Böylelikle farklı özellik seçim yöntemlerinin hangisinin daha iyi çalışacağının gösterilmesi amaçlanmıştır. Bu amaçla farklı özellik sayılarına sahip beş farklı ölçüt veri seti seçilmiştir. Hem özellik seçim yöntemleri hem de makine öğrenmesi sınıflandırıcıları birbirleriyle doğruluk metriği kullanılarak karşılaştırılmıştır. Karşılaştırmalar sonucunda her bir veri seti için farklı bir özellik seçim yöntemin ve farklı bir sınıflandırıcının daha yüksek doğruluk değerine sahip olduğu görülmüştür.

Keywords

• İkili hibrit optimizasyon
• Özellik seçimi
• Öğrenme sınıflandırıcıları
• Sarmal yöntem.

References

1. Büyükkeçeci, M., Okur, M. C., A Comprehensive Review of Feature Selection and Feature Selection Stability in Machine Learning. Gazi University Journal of Science, 36, 4, (2022).
2. Cherrington, M., Thabtah, F., Lu, J., Xu, Q., Feature selection: filter methods performance challenges, Proceedings, International Conference on Computer and Information Sciences (ICCIS), 1-4. (2019).
3. Moradi, P., Gholampour, M., A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy, Applied Soft Computing, 43, 4, 117-130, (2016).
4. Zarshenas, A., Suzuki, K., Binary coordinate ascent: An efficient optimization technique for feature subset selection for machine learning, Knowledge-Based Systems, 110, 191-201, (2016).
5. Talbi, E.-G., Jourdan, L., Garcia-Nieto, J., Alba, E., Comparison of population based metaheuristics for feature selection: Application to microarray data classification, Proceedings, 2008 IEEE/ACS International Conference on Computer Systems and Applications, 45-52, (2008).
6. Xue, B., Zhang, M., Browne, W., Yao, X., A survey on evolutionary computation approaches to feature selection, IEEE Transactions on evolutionary computation, 20, 4, 606-626, (2015).
7. Nekkaa, M., Boughaci, D, Hybrid harmony search combined with stochastic local search for feature selection, Neural Processing Letters, 44, 199-220, (2016).
8. Remeseiro, B., Bolon-Canedo, V., A review of feature selection methods in medical applications, Computers in biology and medicine, 112, 103375, (2019).

9. Ghosh, M., Guha, R., Alam, I., Lohariwal, P., Jalan, D., Sarkar, R., Binary genetic swarm optimization: A combination of GA and PSO for feature selection, Journal of Intelligent Systems, 29, 1, 1598-1610, (2019).
10. Al-Tashi, Q., Kadir, S., Rais, H., Mirjalili, S., Alhussian, H., Binary optimization using hybrid grey wolf optimization for feature selection. Ieee Access, 7, 39496-39508, (2019).
11. El-Kenawy, E.-S., Eid, M., Hybrid gray wolf and particle swarm optimization for feature selection, International Journal of Innovative Computing, Information and Control, 16, 3, 831-844, (2020).
12. Allam, M., Malaiyappan, N., Wrapper based feature selection using integrative teaching learning based optimization algorithm, International Arab Journal of Information Technology, 17, 6, 885-894, (2020).
13. Sameer, F., Comparison study on the performance of the multi classifiers with hybrid optimal features selection method for medical data diagnosis, Multimedia Tools and Applications, 81, 13, 18073-18090, (2022).
14. Alpaydin, E., Introduction to machine learning. MIT press, (2020).
15. Abdi, H., & Williams, L., Principal component analysis, Wiley interdisciplinary reviews: computational statistics, 2, 4, 433-459, (2010).
16. Sánchez-Maroño, N., Alonso-Betanzos, A., Tombilla-Sanromán, M., Filter methods for feature selection-a comparative study, Proceeding, International Conference on Intelligent Data Engineering and Automated Learning, 2, 178-187, (2007).
17. Liu, H., Motoda, H., Setiono, R., Zhao, Z., Feature selection: An ever evolving frontier in data mining, Proceeding, Feature selection in data mining, 4-13, (2010).
18. Liu, H., Zhao, Z., Manipulating data and dimension reduction methods: Feature selection, Springer New York, (2012).
19. Farshi, T. R., Drake, J. H., Özcan, E., A multimodal particle swarm optimization-based approach for image segmentation, Expert Systems with Applications, 149, 113233, (2020).
20. Raschka, S., Liu, Y., Mirjalili, V., Dzhulgakov, D., Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python, Packt Publishing Ltd, (2022).
21. Sun, J., Rahman, M., Wong, Y., Hong, G., Multiclassification of tool wear with support vector machine by manufacturing loss consideration, International Journal of Machine Tools and Manufacture, 44, 11, 1179-1187, (2004).
22. Hastie, T., Tibshirani, R., Friedman, J., Friedman, J., The elements of statistical learning: data mining, inference, and prediction, Springer, (2009).
23. Bonaccorso, G., Machine learning algorithms, Packt Publishing Ltd, (2017).
24. Ayyadevara, V., Pro machine learning algorithms, Springer, (2018).
25. Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., Wu, Xindong, Top 10 algorithms in data mining, Knowledge and information systems, 14, 1-37, (2008).
26. M. Kelly, R. Longjohn, K. Nottingham, The UCI Machine Learning Repository, (2024). https://archive.ics.uci.edu, (26.01.2024).