An Efficient Hybrid Algorithm with Particle Swarm Optimization and Nelder-Mead Algorithm for Parameter Estimation of Nonlinear Regression Modeling

Abstract

Nonlinear regression analysis is an important statistical method widely used in many fields of science to model the complex relationships between variables. Therefore, many studies have been conducted to estimate the parameters of nonlinear regression models using various iterative techniques. In this study, an efficient hybrid algorithm, namely PSONM, by combining the exploration capability of Particle Swarm Optimization (PSO) and the exploitation capability of the Nelder-Mead (NM) algorithm is proposed to obtain parameter estimates of nonlinear regression models. To show the performance of the proposed hybrid algorithm, 20 nonlinear regression tasks with various levels of difficulty, and real data sets in the agriculture field have been tested. The experimental results indicated that the suggested hybrid algorithm provides accurate estimates, and its performance is much superior to those of NM and PSO algorithms.

Keywords

• Nelder-Mead algorithm
• Particle swarm optimization
• Nonlinear regression
• Parameter estimation
• Particle swarm optimization

References

1. [1] Nash, J.C. and Walker-Smith, M., Nonlinear parameter estimation: An integrated system on BASIC. Marcel Dekker, New York, (1987).
2. [2] Křivý, I. and Tvrdík, J., Krpec, R., “Stochastic algorithms in nonlinear regression”, Computational Statistics Data Analysis, 33(3): 277–290, (2000).
3. [3] Yonar, A., Yapıcı Pehlivan, N., “A novel differential evolution algorithm approach for estimating the parameters of Gamma distribution: An application to the failure stresses of single carbon fibres”, Hacettepe Journal of Mathematics and Statistics, 49(4): 1493–1514, (2020).
4. [4] De los Cobos Silva, S.G., Andrade, M.Á.G., García, E.A.R., Velázquez, P.L., Cornejo, M.A., “Estimación de parámetros de regresión no lineal mediante colonia de abejas artificiales”, Revista de Matemática: Teoría y Aplicaciones, 20(1): 49–60, (2013).
5. [5] Tvrdík, J., “Adaptation in differential evolution: A numerical comparison”, Applied Soft Computing, 9(3): 1149–1155, (2009).
6. [6] Kapanoğlu, M., Ozan Koc I., and Erdogmus, S., “Genetic algorithms in parameter estimation for nonlinear regression models: an experimental approach”, Journal of Statistical Computation Simulation, 77(10): 851–867, (2007).
7. [7] Chen, J., “A New Hybrid Genetic Algorithm for Parameter Estimation of Nonlinear Regression Modeling”, Proceedings of the 15th International Conference on Man–Machine–Environment System Engineering, 261–266, (2015).
8. [8] Altunkaynak, B. and Alptekin, E., “The genetic algorithm method for parameter estimation in nonlinear regression”, Gazi University Journal of Science, 17(2): 43–51, (2004).

9. [9] Karr, C.L., Weck, B., Massart, D.-L., and Vankeerberghen, P., “Least median squares curve fitting using a genetic algorithm”, Engineering Applications of Artificial Intelligence 8(2): 177–189, (1995).
10. [10] De-los-Cobos-Silva, S., Terceño-Gómez, A., Gutiérrez-Andrade, M., Rincón-García, E., Lara-Velázquez, P., Aguilar-Cornejo, M., “Particle Swarm Optimization: An Alternative for Parameter Estimation in Regression”, Fuzzy Economic Review, 18(2): (2013).
11. [11] Cheng, S., Zhao, C., Wu, J., Shi, Y., “Particle swarm optimization in regression analysis: a case study”, International Conference in Swarm Intelligence, 55–63, (2013).
12. [12] Schwaab, M., Biscaia Jr, E.C., Monteiro, J.L., and Pinto, J.C., “Nonlinear parameter estimation through particle swarm optimization”, Chemical Engineering Science, 63(6): 1542–1552, (2008).
13. [13] Özsoy, V.S., Örkçü, H.H., “Estimating the Parameters of Nonlinear Regression Models Through Particle Swarm Optimization”, Gazi University Journal of Science, 29(1): (2016).
14. [14] Yonar, A., Yapıcı Pehlivan, N., “Artificial bee colony with levy flights for parameter estimation of 3-p Weibull distribution”, Iranian Journal of Science and Technology, Transactions A: Science, 44: 851–864, (2020).
15. [15] Wang, L., Intelligent optimization algorithms with applications. Tsinghua University Springer Press, Beijing, (2001).
16. [16] Li, L.-l., Wang, L., Liu, L.-h., “An effective hybrid PSOSA strategy for optimization and its application to parameter estimation”, Applied Mathematics Computation, 179(1): 135–146 (2006).
17. [17] NIST: The National Institute for Standard and Technology. https://www.itl.nist.gov/div898/strd/nls/nls_main.shtml. Access date: 07.10.2020.
18. [18] Seber, G.A., Wild, C.J., Nonlinear Regression. New Jersey: John Wiley Sons, 62–63, (2003).
19. [19] Neter, J., Kutner, M.H., Nachtsheim, C.J., Wasserman, W., Applied linear statistical models, WCB McGraw-Hill, (1996).
20. [20] Bates, D.M., Watts, D.G., Nonlinear regression analysis and its applications, Wiley New York, (1988).
21. [21] Gallant, A.R., “Nonlinear regression”, The American Statistician, 29(2): 73–81, (1975).
22. [22] Nelder, J.A., Mead, R., “A simplex method for function minimization”, The Computer Journal, 7(4): 308–313, (1965).
23. [23] Yang X.-S., Engineering optimization: an introduction with metaheuristic applications. John Wiley and Sons, (2010).
24. [24] Eberhart, R., Kennedy, J., “Particle swarm optimization”, Proceedings of the IEEE international conference on neural networks, 1942–1948, (1995).
25. [25] Acitas, S., Aladag, C.H., Senoglu, B., “A new approach for estimating the parameters of Weibull distribution via particle swarm optimization: an application to the strengths of glass fibre data”, Reliability Engineering System Safety, 183: 116–127, (2019).
26. [26] Örkcü, H.H., Özsoy, V.S., Aksoy, E., Dogan, M.I., “Estimating the parameters of 3-p Weibull distribution using particle swarm optimization: A comprehensive experimental comparison”, Applied Mathematics Computational Statistics Data Analysis, 268: 201–226, (2015).
27. [27] Talbi, E.-G., Metaheuristics: from design to implementation, vol. 74. John Wiley and Sons, (2009).
28. [28] Rezaee Jordehi, A., Jasni, J., “Parameter selection in particle swarm optimization: a survey”, Journal of Experimental Theoretical Artificial Intelligence, 25(4): 527–542, (2013).
29. [29] Yonar, A., “Metaheuristic approaches for estimating parameters of univariate and multivariate distributions”, Phd. Thesis, Selçuk University Institute of Science, Konya, 48–52, (2020).
30. [30] Desta, F., Mac Siurtain, M., and Colbert, J., “Parameter estimation of nonlinear growth models in forestry”, Silva Fennica, 33(4): 327–336, (1999).
31. [31] Mahanta, D.J., Borah, M., “Parameter Estimation of Weibull Growth Models in Foresty”, International Journal of Mathematics Trends and Technology, 8(3): 157–163, (2014).