The Effects of Kernel Functions and Optimal Hyperparameter Selection on Support Vector Machines

Abstract

Support Vector Machine (SVM) is a supervised machine learning method used for classification and regression. It is based on the Vapnik-Chervonenkis (VC) theory and Structural Risk Minimization (SRM) principle. Thanks to its strong theoretical background, SVM exhibits a high performance compared to many other machine learning methods. The selection of hyperparameters and the kernel functions is an important task in the presence of SVM problems. In this study, the effect of tuning hyperparameters and sample size for the kernel functions on SVM classification accuracy was investigated. For this, UCI datasets of different sizes and with different correlations were simulated. Grid search and 10-fold Cross-Validation methods were used to tune the hyperparameters. Then, SVM classification process was performed using three kernel functions, and classification accuracy values were examined.

Keywords

• Support vector machines
• kernel function
• tune parameter

References

1. B. E. Boser, I. M. Guyon, V. N. Vapnik, A Training Algorithm for Optimal Margin Classifiers, In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (1992) 144-152.
2. V. N. Vapnik, The Nature of Statistical Learning Theory, New York: Springer-Verlag, 1995.
3. V. Jakkula, Tutorial on Support Vector Machine (SVM), School of EECS, Washington State University, (2006) 37.
4. S. Han, C. Qubo, H. Meng, Parameter Selection in SVM with RBF Kernel Function, In World Automation Congress (2012) 1-4 Puerto Vallarta, Mexico.
5. X. Chen, J. He, X. Wu, W. Yan, W. Wie, Sleep Staging by Bidirectional Long Short-term Memory Convolution Neural Network, Future Generation Computer Systems 109 (2020) 188-196.
6. J. K. Appati, G. K. Gogovi, G. O. Fosu, On the Selection of Appropriate Kernel Function for SVM in Face Recognition, International Journal of Advanced Research in Computer Science and Software Engineering 4(3) (2014) 6-9.
7. W. M. Campbell, J. P. Campbell, D. A. Reynolds, E. Singer, P. A. Torres-Carrasquillo, Support Vector Machines for Speaker and Language Recognition, Computer Speech and Language 20(2-3) (2006) 210-229.
8. S. Bellamkonda, N. P. Gopalan, A Facial Expression Recognition Model Using Support Vector Machines, IJ Mathematical Sciences and Computing 4 (2018) 56-65.

9. M. Rezaei, E. Zereshki, H. Sharini, M. Gharib Salehi, F. Naleini, Detection of Alzheimer’s Disease Based on Magnetic Resonance Imaging of The Brain Using Support Vector Machine Model, Tehran University Medical Journal TUMS Publications 76(6) (2018) 410-416.
10. Ö. Y. Akşehirli, H. Ankaralı, D. Aydın, Ö. Saraçlı, An Alternative Approach in Medical Diagnosis: Support Vector Machines, Türkiye Klinikleri Journal of Biostatistics 5(1) (2013) 19-28.
11. F. E. Tay, L. Cao, Application of Support Vector Machines in Financial Time Series Forecasting, Omega 29(4) (2001) 309-317.
12. P. Wang, R. Mathieu, J. Ke, H. J. Cai, Predicting Criminal Recidivism with Support Vector Machine, International Conference on Management and Service Science (2010) Wuhan, China.
13. J. Karia, Stock Market Prediction Using Machine Learning, International Journal of Emerging Technology and Computer Science 3(2) (2018) 159-162.
14. J. Wainer, P. Fonseca, How to Tune The RBF SVM Hyperparameters?: An Empirical Evaluation of 18 Search Algorithms.” arXiv preprint arXiv:2008.11655 (2020).
15. D. Fradkin, I. Muchnik, Support Vector Machines for Classification, DIMACS Series in Discrete Mathematics and Theoretical Computer Science 70 (2006) 13-20.
16. S. R. Gunn, Support Vector Machines for Classification and Regression, ISIS Technical Report 14(1) (1998) 5-16.
17. C. F. Lin, S. D. Wang, Fuzzy Support Vector Machines, IEEE Transactions on Neural Networks 13(2) (2002) 464-471.
18. C. Savas, F. Dovis, The Impact of Different Kernel Functions on The Performance of Scintillation Detection Based on Support Vector Machines, Sensors 19(23) (2019) 1-16.
19. D. Dua, C. Graff, UCI Machine Learning Repository [http://archive.ics.uci.edu/ml], Irvine, CA: University of California, School of Information and Computer Science, (2019).