Araştırma Makalesi

Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems

Cilt: 31 26 Ekim 2026
PDF İndir
TR EN

Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems

Öz

This study compared Gradient Descent, Gauss–Newton, Levenberg–Marquardt, Nelder–Mead, and a hybrid approach in which a Genetic Algorithm was followed sequentially by Nelder–Mead for parameter estimation in nonlinear least squares problems. For each data–model combination, the methods were evaluated under the same SSE objective function and a common numerical parameter space using 30 controlled initialization scenarios generated in normalized coordinates. Computational burden was monitored through the number of function evaluations (NFE), Jacobian evaluations (NJE), and runtime, while attainment of the algorithmic termination criterion was assessed separately from the quality of the terminal solution. The methods were examined on a two-parameter synthetic problem, 57 weekly live-weight observations from female turkeys between 4 and 60 weeks of age, and 3783 age–weight records from female Boer goats. In the synthetic problem, GN and LM reached low error levels at low computational cost, whereas NM met its termination criterion at higher-error boundary solutions for some initializations. In the turkey data, GA→NM produced more consistent results across changes in starting values, while LM reached comparable error levels with substantially fewer evaluations for the Gompertz and Richards models. In the Boer analysis, GA→NM likewise showed consistent behavior across starting values, whereas GN was particularly sensitive to initialization for the Richards and Weibull models. The Boer Richards solution approached the Gompertz boundary and exhibited pronounced numerical ill-conditioning in this region. These findings indicate that NLS methods should be compared by jointly considering terminal error level, sensitivity to starting values, termination behavior, parameter boundaries, and computational cost.

Anahtar Kelimeler

Nonlinear least squares, Parameter estimation, Gauss–Newton, Levenberg–Marquardt, Gradient descent, Nelder–Mead, Genetic algorithm

Etik Beyan

This article has received Ethics Committee Approval from the Van Yüzüncü Yıl University Animal Experiments Local Ethics Committee, dated October 17, 2024, Meeting No: 2024-9, File No: 2024-99, and Decision No: 2024-9-99, for the animal experiment described herein.

Kaynakça

  1. Akgün, F. (2018). Doğrusal olmayan regresyon model parametrelerinin nokta ve aralık tahmini için bir yaklaşım [Master’s thesis, Ankara University].
  2. Arando, A., González-Ariza, A., Lupi, T. M., Nogales, S., León, J. M., Navas-González, F. J., Delgado, J. V., & Camacho, M. E. (2021). Comparison of non-linear models to describe the growth in the Andalusian turkey breed. Italian Journal of Animal Science, 20(1), 1156–1167. https://doi.org/10.1080/1828051X.2021.1950054
  3. Bates, D. M., & Watts, D. G. (1981). A relative offset orthogonality convergence criterion for nonlinear least squares. Technometrics, 23(2), 179–183. https://doi.org/10.1080/00401706.1981.10486262
  4. Bilgin, Ö. C., & Esenbuğa, N. (2003). Doğrusal-olmayan büyüme modellerinde parametre tahmini. Hayvansal Üretim, 44(2), 81–90.
  5. Björck, Å. (2024). Numerical methods for least squares problems (2nd ed.). Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611977950
  6. Brand, T. S., van der Westhuyzen, J. P., Hough, W., & van Zyl, J. H. C. (2024). Application of growth models to South African Boer goat castrates and does under feedlot conditions. Tropical Animal Health and Production, 56, Article 178. https://doi.org/10.1007/s11250-024-03973-5
  7. Bvirwa, W., & Washaya, S. (2025). Application of growth models to Boer × Mashona and Kalahari × Mashona goats under extensive semi-arid conditions. Tropical Animal Health and Production, 57, Article 244. https://doi.org/10.1007/s11250-025-04489-2
  8. Cochran, W. G. (1950). The comparison of percentages in matched samples. Biometrika, 37(3–4), 256–266. https://doi.org/10.1093/biomet/37.3-4.256
  9. Çelik, A. (2013). Multiphase growth functions and some applications [Unpublished master’s thesis]. Kahramanmaraş Sütçü İmam Üniversitesi.
  10. Conn, A. R., Gould, N. I., & Toint, P. L. (2000). Trust region methods. Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9780898719857

Kaynak Göster

APA
Deniz, S., Altun, Y., Canayaz, M., & Yağiz, B. (2026). Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 31. https://doi.org/10.53433/yyufbed.1928294
AMA
1.Deniz S, Altun Y, Canayaz M, Yağiz B. Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems. YYUFBED. 2026;31. doi:10.53433/yyufbed.1928294
Chicago
Deniz, Sinan, Yener Altun, Murat Canayaz, ve Beytullah Yağiz. 2026. “Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems”. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi 31 (Ekim). https://doi.org/10.53433/yyufbed.1928294.
EndNote
Deniz S, Altun Y, Canayaz M, Yağiz B (01 Ekim 2026) Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi 31
IEEE
[1]S. Deniz, Y. Altun, M. Canayaz, ve B. Yağiz, “Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems”, YYUFBED, c. 31, Eki. 2026, doi: 10.53433/yyufbed.1928294.
ISNAD
Deniz, Sinan - Altun, Yener - Canayaz, Murat - Yağiz, Beytullah. “Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems”. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi 31 (01 Ekim 2026). https://doi.org/10.53433/yyufbed.1928294.
JAMA
1.Deniz S, Altun Y, Canayaz M, Yağiz B. Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems. YYUFBED. 2026;31. doi:10.53433/yyufbed.1928294.
MLA
Deniz, Sinan, vd. “Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems”. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi, c. 31, Ekim 2026, doi:10.53433/yyufbed.1928294.
Vancouver
1.Sinan Deniz, Yener Altun, Murat Canayaz, Beytullah Yağiz. Comparison of Derivative-Based and Derivative-Free Optimization Methods for Parameter Estimation in Nonlinear Least Squares Problems. YYUFBED. 01 Ekim 2026;31. doi:10.53433/yyufbed.1928294