Modified Hestenes - Stiefel Conjugate Gradient (MHS-CG) method for solving unconstrained optimization

Year 2021, Volume: 4 Issue: 2, 32 - 42, 03.01.2023

Osama Alkassaab Khalil K. Abbo Ibrahim Saleh

Abstract

The conjugate gradient technique is one of the most effective methods for solving and minimizing unconstrained optimization problems, and it is widely utilized. In this research, we introduce a novel nonlinear conjugate gradient approach with excellent convergence for unconstrained minimization problems that is based on the nonlinear conjugate gradient method. The new algorithm has the property of descent as well as global convergence. Results from the numerical evaluations demonstrate that the new technique is very efficient in practical computing and outperforms previous comparable approaches in a wide range of conditions

Keywords

unconstrained optimization, nonlinear conjugate gradient method, global convergence

References

• H. M. Khudhur and K. K. Abbo, “A New Conjugate Gradient Method for Learning Fuzzy Neural Networks,” J. Multidiscip. Model. Optim., vol. 3, no. 2, pp. 57–69, 2021.
• O. M. T. Wais, “Two Modified Spectral Conjugate Gradient methods for Optimization,” Coll. Basic Educ. Res. J., vol. 14, no. 4, pp. 533–553, 2018.
• Z. M. Abdullah, M. Hameed, M. K. Hisham, and M. A. Khaleel, “Modified new conjugate gradient method for Unconstrained Optimization,” Tikrit J. Pure Sci., vol. 24, no. 5, pp. 86–90, 2019.
• K. K. Abbo and O. M. T. Waiss, “Investigation on Extended Conjugate Gradient Non-linear methods for solving Unconstrained Optimization.”
• K. K. Abbo, Y. A. Laylani, and H. M. Khudhur, “Proposed new Scaled conjugate gradient algorithm for Unconstrained Optimization,” Int. J. Enhanc. Res. Sci. Technol. Eng., vol. 5, no. 7, 2016.
• H. N. Jabbar, K. K. Abbo, and H. M. Khudhur, “Four--Term Conjugate Gradient (CG) Method Based on Pure Conjugacy Condition for Unconstrained Optimization,” kirkuk Univ. J. Sci. Stud., vol. 13, no. 2, pp. 101–113, 2018.
• M. A. Wolfe, Numerical methods for unconstrained optimization: An introduction. Van Nostrand Reinhold New York, 1978.
• A. Al-Bayati, B. Hassan, and H. Jabbar, “A new class of nonlinear conjugate gradient method for solving unconstrained minimization problems,” 2019, doi: 10.1109/ICCISTA.2019.8830657.
• Y. H. Dai and Y. Yuan, “A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property,” SIAM J. Optim., vol. 10, no. 1, pp. 177–182, 1999, doi: 10.1137/S1052623497318992.
• H. M. Khudhur, “Numerical and analytical study of some descent algorithms to solve unconstrained Optimization problems,” University of Mosul, 2015.
• Y. Dai and L. Liao, “New conjugacy conditions and related non-linear CG methods’ Appl,” Math optim.,(43). Spring-verlag, New York, 2001.
• M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, vol. 49, no. 1. NBS Washington, DC, 1952.
• E. Polak and G. Ribiere, “Note sur la convergence de méthodes de directions conjuguées,” ESAIM Math. Model. Numer. Anal. Mathématique Anal. Numérique, vol. 3, no. R1, pp. 35–43, 1969.
• R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” Comput. J., vol. 7, no. 2, pp. 149–154, 1964, doi: 10.1093/comjnl/7.2.149.
• R. Fletcher, Practical methods of optimization. John Wiley & Sons, 1987.
• Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms, part 1: theory,” J. Optim. Theory Appl., vol. 69, no. 1, pp. 129–137, 1991.
• B. A. Hassan, O. M. T. Wais, and A. A. Mahmood, “A Class of Descent Conjugate Gradient Methods for Solving Optimization Problems,” Appl. Math. Sci., vol. 13, no. 12, pp. 559–567, 2019.
• K. K. ABBO, Y. A. Laylani, and H. M. Khudhur, “A NEW SPECTRAL CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION,” Int. J. Math. Comput. Appl. Res., vol. 8, pp. 1–9, 2018.
• N. Andrei, “An unconstrained optimization test functions collection,” Adv. Model. Optim, vol. 10, no. 1, pp. 147–161, 2008.
• E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Math. Program., vol. 91, no. 2, pp. 201–213, 2002.