ESTIMATING THE PARAMETERS OF NONLINEAR REGRESSION MODELS THROUGH PARTICLE SWARM OPTIMIZATION

Abstract

Nonlinear regression models are widely used for modeling of stochastic phenomena and the estimating parameters problem plays a central role in the inference in nonlinear regression models. In this paper, this problem has been briefly discussed and an effective approach based on the Particle Swarm Optimization (PSO) algorithm is proposed in order to enhance the estimation accuracy. The PSO algorithm is tested on the well-known 28 nonlinear regression tasks of various level of difficulty. The results show that PSO approach which exhibits a rapid convergence to the minimum value of the sum of squared error function in less iterations, provides accurate estimates and is satisfactory for the parameter estimation of the nonlinear regression models.

Keywords

• Particle swarm optimization
• nonlinear regression
• parameter estimation

References

1. D.A. Ratkowsy, Nonlinear regression modeling: a unified practical approach, M. Dekker, Newyork, 1983.
2. J.C. Nash, M. Walker-Smith, Nonlinear Parameter Estimation, Marcel Dekker, Inc., New York, Basel (1987).
3. G.A.F. Seber, C.J. Wild, Nonlinear regression, Wiley, 2005.
4. L. Li, L. Wang, L. Liu, An effective hybrid PSOSA strategy for optimization and its application to parameter estimation, Applied Mathematics and Computation, 179 (2006) 135-146.
5. J. Kennedy, R. Eberhart (1995). Particle swarm optimization, Proceedings of IEEE International Conference on Neural Networks, Vol. 4, pp. 1942–1948.
6. R. Eberhart, J. Kennedy (1995). A New Optimizer Using Particle Swarm Theory, Proceedings of 6th International Symposium on Micro Machine and Human Science, Nagoya, Japan. IEEE Service Center Piscataway NJ,1995:39-43
7. I. Krivy, J. Tvrdik, R. Krpec, Stochastic algorithms in nonlinear regression, Computational Statistics and Data Analysis, 33 (2000) 277-290.
8. M. Kaptanoğlu, I.O. Koc, S. Erdogmus, Genetic algorithms in parameter estimation for nonlinear regression models: a experimental approach, Journal of Statistical Computation, 77(10) 2007, 851-867.

9. J. Tvrdik, I. Krivy, L. Misık, Adaptive population-based search: application to estimation of nonlinear regression parameters, Computational Statistics and Data Analysis, 52 (2007) 713-724.
10. B. Aşıkgil, A. Erar, Polynomial tapered two-stage least square method in nonlinear regression, Applied Mathematics and Computation, 219 (2013) 9743-9754.
11. S. Das, A. Abraham, A. Konar, Particle Swarm Optimization and Differential Evolution Algorithms: Technical Analysis, Applications and Hybridization Perspectives, Studies in Computational Intelligence, 116 (2008) 1–38.
12. Y. Fukuyama, Modern Heuristic Optimization Techniques, Edited by K.Y.Lee and M.A. El-Sharkawi, Institute of Electrical and Electronics Engineers, 71-83, 2008.
13. R. Eberhart, Y. Shi (2001). Particle swarm optimization: developments, applications and resources, Proceedings of IEEE International Congress on Evolutionary Computation, 1, 81–86.
14. Y. Shi and R. Eberhart, A modified particle swarm optimizer, Proceedings of IEEE International Conference on Evolutionary Computation, Anchorage, May 1998, 69–73.
15. K.Y. Lee, M.A. El-Sharkawi, Modern Heuristic Optimization Techniques: Theory and Applications to Power Systems, IEEE press series on power engineering. Hoboken: Wiley Interscience; New Jersey, 2008.
16. W.N. Lee, J.B. Park (2011) Educational Simulator for Particle Swarm Optimization and Economic Dispatch Applications, MATLAB - A Ubiquitous Tool for the Practical Engineer, Prof. Clara Ionescu (Ed.), ISBN: 978-953-307-907-3, InTech, DOI: 10.5772/19873.
17. R.C. Eberhart, Y. Shi (2000), Comparing Inertia Weights and Constriction Factors in Particle Swarm Optimization, Proceedings of IEEE International Congress on Evolutionary Computation, 1, 84–88.
18. C. Lanczos (1956), Applied Analysis. Englewood Cliffs, NJ: Prentice Hall, pp. 272-280.
19. R.I. Jennrich, P.F. Sampson (1968), Application of stepwise regression to non-linear estimation, Technometrics, 10(1) 63-72.
20. R.R. Meyer, P.M. Roth (1972), Modified damped least squares: an algorithm for non-linear estimation. J. Inst. Math. Appl, 9, 218-233.
21. G.P. Box, W. G. Hunter, J. S. Hunter (1978), Statistics for Experimenters. New York, NY: Wiley, 483-487.
22. J.S. Kowalik, M. R. Osborne (1978), Methods for Unconstrained Optimization Problems. New York, NY: Elsevier North-Holland.
23. C. Daniel, F. S. Wood (1980), Fitting Equations to Data, Second Edition. New York, NY: John Wiley and Sons, pp. 428-431.
24. W. Nelson (1981), Analysis of Performance-Degradation Data. IEEE Transactions on Reliability, 2 (2), 149-155.
25. Kahaner, D., C. Moler, and S. Nash, (1989). Numerical Methods and Software. Englewood Cliffs, NJ: Prentice Hall, 441-445.
26. J. Kennedy , "The Particle Swarm: Social Adaptation of Knowledge, Proc" , IEEE International Conference on Evolutionary Computation , pp.303 -308 , 1997
27. Y. Shi and R. C. Eberhart, , "Empirical study of particle swarm optimization" , Proc. Congr. Evolutionary Computation , pp.1945 -1949 , 1999
28. Y. Shi and R. Eberhart , "Fuzzy adaptive particle swarm optimization" , Proc. IEEE Congr. Evol. Comput. , vol. 1 , pp.101 -106 , 2001
29. X. Hu , Y. Shi and R. Eberhart , "Recent advances in particle swarm" , Proc. IEEE Congr. Evol. Comput. , vol. 1 , pp.90 -97 , 2004
30. Y. del Valle, G. Venayagamoorthy, S. Mohagheghi, J.-C. Hernandez and R. Harley , "Particle swarm optimization: Basic concepts, variants and applications in power systems" , IEEE Trans. Evol. Comput. , vol. 12 , no. 2 , pp.171 -195 , 2008
31. C. A. C. Coello, G. T. Pulido, and M. S. Lechuga MS, , "Handling multiple objectives with particle swarm optimization" , IEEE Trans. Evol. Comput. , vol. 8 , no. 3 , pp.256 -279 , 2004
32. Wang X, Yang J et al (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recogn Lett 28:459–471
33. Jun Sun, Xiaojun Wu, Vasile Palade, Wei Fang, Yuhui Shi, Random drift particle swarm optimization algorithm: convergence analysis and parameter selection, Machine Learning, 101, 1-3, 345-376 (2015)