Simultaneous Feature Selection and Hyperparameter Tuning of K-Nearest Neighbors for Glass Classification: A Comparative Study of Artificial Bee Colony, Simulated Annealing, and Hill Climbing

Year 2026, Volume: 9 Issue: 1, 433 - 443, 15.01.2026

Sertaç Savaş

https://doi.org/10.34248/bsengineering.1841886

https://izlik.org/JA99NW66EN

Abstract

Glass identification is essential in forensic science and industrial uses. However, the effectiveness of classification algorithms heavily relies on selecting the correct hyperparameters and relevant features. This study investigates the efficacy of three optimization algorithms—Artificial Bee Colony (ABC), Simulated Annealing (SA), and Hill Climbing (HC)—for optimizing the k value, which is the neighbor value of the k-Nearest Neighbors (k-NN) classifier, the distance measure expressing the proximity between two samples, and the feature subset of the UCI Glass Identification dataset. By reframing the classification problem as a multi-dimensional optimization task, the algorithms are assessed based on accuracy, precision, recall, and F1 score. The results show that the population-based ABC algorithm, as a meta-heuristic approach, outperforms local search methods by avoiding local optima and effectively reducing the dimensionality of the data. RI (refractive index) and Mg (magnesium oxide content) emerge as particularly discriminative features. The study demonstrates that combining automatic parameter tuning with feature selection can substantially enhance the predictive performance of k-NN in complex classification tasks.

Keywords

Glass classification , K-Nearest neighbors , Artificial bee colony , Simulated annealing , Hill climbing , Optimization

Ethical Statement

Ethics committee approval was not required for this study because of there was no study on animals or humans.

Supporting Institution

This study did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Project Number

Yok / Bulunmamaktadır

References

• Al-Betar, M. A., Hammouri, A. I., Awadallah, M. A., & Doush, I. A. (2021). Binary β-hill climbing optimizer with S-shape transfer function for feature selection. Journal of Ambient Intelligence and Humanized Computing, 12(7), 7637–7665. https://doi.org/10.1007/s12652-020-02484-z
• Ay, Ş., Ekinci, E., & Garip, Z. (2023). A comparative analysis of meta-heuristic optimization algorithms for feature selection on ML-based classification of heart-related diseases. The Journal of Supercomputing, 79(11), 11797–11826. https://doi.org/10.1007/s11227-023-05132-3
• Bai, X., Zheng, Y., Lu, Y., & Shi, Y. (2024). Chain hybrid feature selection algorithm based on improved Grey Wolf Optimization algorithm. PLOS ONE, 19(10), e0311602. https://doi.org/10.1371/journal.pone.0311602
• Bhowmick, S., & Saha, A. (2023). Enhancing the performance of kNN for glass identification dataset using inverse distance weight, ReliefF ranking, and SMOTE. AIP Conference Proceedings, 2754(1), 020021. https://doi.org/10.1063/5.0161083
• Bouziane, M., Bouziane, A., Larguech, S., Naima, K., Haque, M. S., & Menni, Y. (2025). High-performance glass classification using advanced machine learning and deep learning algorithms with a comprehensive feature analysis. AIP Advances, 15(5), 055013. https://doi.org/10.1063/5.0260868
• Chantar, H., Tubishat, M., Essgaer, M., & Mirjalili, S. (2021). Hybrid binary dragonfly algorithm with simulated annealing for feature selection. SN Computer Science, 2(4), 295. https://doi.org/10.1007/s42979-021-00687-5
• Chen, Z., Xu, Y., Zhang, C., & Tang, M. (2024). Prediction of glass chemical composition and type identification based on machine learning algorithms. Applied Sciences, 14(10), 4017. https://doi.org/10.3390/app14104017
• Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27. https://doi.org/10.1109/TIT.1967.1053964
• Deza, M. M., & Deza, E. (2016). Encyclopedia of distances (4th ed.). Springer. https://doi.org/10.1007/978-3-662-52844-0
• German, B. (1987). Glass identification dataset. UCI Machine Learning Repository. https://doi.org/10.24432/C5WW2P
• Hussien, A. G., Oliva, D., Houssein, E. H., Juan, A. A., & Yu, X. (2020). Binary whale optimization algorithm for dimensionality reduction. Mathematics, 8(10), 1821. https://doi.org/10.3390/math8101821
• Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization (Technical Report TR06). Erciyes University, Engineering Faculty, Computer Engineering Department. https://abc.erciyes.edu.tr/pub/tr06_2005.pdf
• Kaspi, O., Israelsohn-Azulay, O., Yigal, Z., Rosengarten, H., Krmpotić, M., Gouasmia, S., Radović, I. B., Jalkanen, P., Liski, A., Mizohata, K., Räisänen, J., Kasztovszky, Z., Harsányi, I., Acharya, R., Pujari, P. K., Mihály, M., Braun, M., Shabi, N., Girshevitz, O., & Senderowitz, H. (2023). Toward developing techniques—Agnostic machine learning classification models for forensically relevant glass fragments. Journal of Chemical Information and Modeling, 63(1), 87–100. https://doi.org/10.1021/acs.jcim.2c01362
• Khan, M. A., Mazhar, T., Yaqoob, M. M., Khan, M. B., Saudagar, A. K. J., Ghadi, Y. Y., Khattak, U. F., & Shahid, M. (2024). Optimal feature selection for heart disease prediction using modified artificial bee colony (M-ABC) and k-nearest neighbors (KNN). Scientific Reports, 14, 26241. https://doi.org/10.1038/s41598-024-78021-1
• Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680. https://doi.org/10.1126/science.220.4598.671
• Li, G., Shi, M., Hao, T., Wang, P., Ma, Y., & Han, Y. (2024). A hybrid framework for ancient glass classification and recognition: Combining binary nutcracker optimization algorithm with KNN method. In 2024 9th International Conference on Control and Robotics Engineering (ICCRE) (pp. 345–351). IEEE. https://doi.org/10.1109/ICCRE61448.2024.10589876
• Rachmatullah, M. I. C. (2022). The application of repeated SMOTE for multi-class classification on imbalanced data. MATRIK: Jurnal Manajemen, Teknik Informatika ve Rekayasa Komputer, 22(1), 13–24. https://doi.org/10.30812/matrik.v22i1.1803
• Russell, S. J., & Norvig, P. (2020). Artificial intelligence: A modern approach (4th ed.). Pearson.
• Schiezaro, M., & Pedrini, H. (2013). Data feature selection based on artificial bee colony algorithm. EURASIP Journal on Image and Video Processing, 2013, 47. https://doi.org/10.1186/1687-5281-2013-47
• Uzer, M. S., Yilmaz, N., & Inan, O. (2013). Feature selection method based on artificial bee colony algorithm and support vector machines for medical datasets classification. The Scientific World Journal, 2013, 419187. https://doi.org/10.1155/2013/419187
• Visweshwaran, S., Anbazhagan, M., & Ganesh, K. (2024). An empirical study on ML models with glass classification dataset. In H. Sharma, V. Shrivastava, A. K. Tripathi, & L. Wang (Eds.), Communication and Intelligent Systems. ICCIS 2023. Lecture Notes in Networks and Systems (Vol. 968). Springer. https://doi.org/10.1007/978-981-97-2079-8_30
• Wang, Z., Lu, Y., & Li, T. (2023). Ancient glass classification based on random forest algorithm and decision tree model. Highlights in Science, Engineering and Technology, 34, 344–351. https://doi.org/10.54097/hset.v34i.5492