Gazi University Journal of Science 2147-1762 Gazi University 10.35378/gujs.1727816 Computational Statistics Statistical Analysis Statistical Data Science Hesaplamalı İstatistik İstatistiksel Analiz İstatistiksel Veri Bilimi Simulated Annealing Algorithm Based Ridge Estimator https://orcid.org/0000-0002-0068-5471 Kocasoy Gizem İklil GAZİ ÜNİVERSİTESİ, FEN FAKÜLTESİ https://orcid.org/0000-0003-4798-3422 Ebegil Meral GAZİ ÜNİVERSİTESİ, FEN FAKÜLTESİ https://orcid.org/0000-0002-4921-8209 Özdemir Muhlis GAZİ ÜNİVERSİTESİ, FEN FAKÜLTESİ Advanced Online Publication 06 26 2025 02 21 2026 Copyright © 1988, Gazi University Journal of Science 1988 Gazi University Journal of Science

Multicollinearity is a significant problem in multiple linear regression. Different researchers have suggested biased estimators as a possible solution to address the issue of multicollinearity, and an example of a biased estimator is the ridge regression estimator. Estimating the bias parameter is an essential problem for the ridge regression estimator. This paper presents a new solution method that utilizes simulated annealing optimization to determine the optimal bias parameter as an alternative to the ridge regression bias value proposed by Hoerl and Kennard. We obtained the bias parameter estimation values using the proposed solution method, considering various dependency structures, sample sizes, variance, and number of variables.

 Ridge regression Biasing parameter Simulated annealing Optimization Multicollinearity [1] Frisch, R., “Statistical confluence analysis by means of complete regression system”, The Economic Journal, 45(180):741–742, (1935). DOI: https://doi.org/10.2307/2225583 [2] Montgomery, D.C., Peck, E.A., Vining, G.G., Introduction to Linear Regression Analysis, John Wiley & Sons, (2012). [3] Khalaf, G., Iguernane, M., “Ridge regression and ill-conditioning”, Journal of Modern Applied Statistical Methods, 13(2):355–363, (2014). DOI: https://doi.org/10.22237/jmasm/1414815420 [4] Hoerl, A.E., Kennard, R.W., “Ridge regression: biased estimation for nonorthogonal problems”, Technometrics, 12(1):55–67, (1970). DOI: https://doi.org/10.2307/1267351 [5] Keijan, L., “A new class of biased estimate in linear regression”, Communications in Statistics - Theory and Methods, 22(2):393–402, (1993). DOI: https://doi.org/10.1080/03610929308831027 [6] Kibria, B.M.G., “Performance of some new ridge regression estimators”, Communications in Statistics Part B: Simulation and Computation, 32(2):419–435, (2003). DOI: https://doi.org/10.1081/SAC-120017499 [7] Khalaf, G., Shukur, G., “Choosing ridge parameter for regression problems”, Communications in Statistics - Theory and Methods, 34(5):1177–1182, (2006). DOI: https://doi.org/10.1081/STA-200056836 [8] Alkhamisi, A., Shukur, G., “A monte carlo study of recent ridge parameters”, Communications in Statistics - Simulation and Computation, 36(3):535–547, (2007). DOI: https://doi.org/10.1080/03610910701208619 [9] Dorugade, A.V., “New ridge parameters for ridge regression”, Journal of the Association of Arab Universities for Basic and Applied Sciences, 15(1):94–99, (2014). DOI: https://doi.org/10.1016/j.jaubas.2013.03.005 [10] Ahn, J.J., Byun, H.W., Oh, K.J., Kim, T.Y., “Using ridge regression with genetic algorithm to enhance real estate appraisal forecasting”, Expert Systems with Applications, 39(9):8369–8379, (2012). DOI: https://doi.org/10.1016/j.eswa.2012.01.183 [11] Kareem, Q. A., Algamal, Z. Y., “Generalized ridge estimator shrinkage estimation based on particle swarm optimization”, Iraqi Journal of Statistical Sciences, 17(2): 26–35, (2020). DOI: https://doi.org/10.33899/iqjoss.2020.167387 [12] Cagcag, O., Yolcu, U., Egrioglu, E., “A new robust regression method based on particle swarm optimization”, Communications in Statistics - Theory and Methods, 44(6):1270–1280, (2015). DOI: https://doi.org/10.1080/03610926.2012.718843 [13] Bas, E., Egrioglu, E., Uslu, V.R., “Shrinkage parameters for each explanatory variable found via particle swarm optimization in ridge regression”, Trends in Computer Science and Information Technology, 2(1):012–020, (2017). DOI: https://doi.org/10.17352/tcsit.000005 [14] Inan, D., Egrioglu, E., Sarica, B., Askin, O.E., Tez, M., “Particle swarm optimization based Liu-type estimator”, Communications in Statistics - Theory and Methods, 46(22):11358–11369, (2017). DOI: https://doi.org/10.1080/03610926.2016.1267759 [15] Inan, D., Sancar, N., “Particle swarm optimization based ridge logistic estimator”, Communications in Statistics - Simulation and Computation, 49(3):669–683, (2020). DOI: https://doi.org/10.1080/03610918.2020.1713361 [16] Kocasoy, G.İ., “Ridge ve Liu Regresyonda Tavlama Benzetimi Optimizasyonu Kullanılarak Yanlılık Parametrelerinin Elde Edilmesi ve Bazı Yanlılık Parametreleri ile Karşılaştırılması”, Master Thesis, Gazi University, Graduate School of Natural and Applied Sciences, Ankara, 11-14 (2023). [17] Kennedy, J., Eberhart, R., “Particle swarm optimization”, in Proceedings of IEEE International Conference on Neural Networks, Perth, WA, Australia, (1995). [18] Bozorg-Haddad, O., Solgi, M., Loaiciga, H.A., Meta-Heuristic and Evolutionary Algorithms for Engineering Optimization, John Wiley & Sons, (2017). [19] Uslu, V.R., Egrioglu, E., Bas, E., “Finding optimal value for the shrinkage parameter in ridge regression via particle swarm optimization”, American Journal of Intelligent Systems, 4(4):142–147, (2014). DOI: https://doi.org/10.5923/j.ajis.20140404.03 [20] Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P., “Optimization by simulated annealing”, Science, 220(4598):671–680, (1983). DOI: https://doi.org/10.1126/science.220.4598.671 [21] Kumar, K., Zindani, D., Davim, J.P., Optimizing Engineering Problems through Heuristic Techniques, Taylor & Francis, (2020). [22] Gad, A.G., “Particle swarm optimization algorithm and its applications: a systematic review”, Archives of Computational Methods in Engineering, 29(5):2531–2561, (2022). DOI: https://doi.org/10.1007/s11831-021-09694-4 [23] Hamami, M. G. M., Ismail, Z. H., “A systematic review on particle swarm optimization towards target search in the swarm robotics domain”, Archives of Computational Methods in Engineering, (2022). DOI: https://doi.org/10.1007/s11831-022-09819-3 [24] Ben-Ameur, W., “Computing the initial temperature of simulated annealing”, Computational Optimization and Applications, 29(3):369–385, (2003). DOI: https://doi.org/10.1023/B:COAP.0000044187.23143.bd [25] Islam, M. S., Khatoon, M. T., Siddiquee, K. N., Yong, W. H., Huda, M. H., “Performance analysis of simulated annealing and genetic algorithm on systems of linear equations”, F1000Research, 10:1297, (2021). DOI: https://doi.org/10.12688/f1000research.73581.1 [26] Woods, H., Steinour, H. H., Starke, H. R., “Effect of composition of portland cement on heat evolved during hardening”, Industrial & Engineering Chemistry, 24(11):1207-1214, (1932). DOI: https://doi.org/10.1021/IE50275A002 [27] Longley, J.W., “An appraisal of least squares programs for the electronic computer from the point of view of the user”, Journal of the American Statistical Association, 62(319):819–841, (1967). DOI: https://doi.org/10.2307/2283673