Halpern-type relaxed algorithms with alternated and multi-step inertia for split feasibility problems with applications in classification problems

Abstract

In this article, we construct two Halpern-type relaxed algorithms with alternated and multi-step inertial extrapolation steps for split feasibility problems in infinite-dimensional Hilbert spaces. The first is the most general inertial method that employs three inertial steps in a single algorithm, one of which is an alternated inertial step, while the others are multi-step inertial steps, representing the recent improvements over the classical inertial step. Besides the inertial steps, the second algorithm uses a three-term conjugate gradient-like direction, which accelerates the sequence of iterates toward a solution of the problem. In proving the convergence of the second algorithm, we dispense with some of the restrictive assumptions in some conjugate gradient-like methods. Both algorithms employ a self-adaptive and monotonic step-length criterion that does not require knowledge of the norm of the underlying operator or the use of any line search procedure. Moreover, we formulate and prove some strong convergence theorems for each of the algorithms based on the convergence theorem of an alternated inertial Halpern-type relaxed algorithm with perturbations in real Hilbert spaces. Further, we analyse their applications to classification problems for some real-world datasets based on the extreme learning machine (ELM) with the $\ell_{1}$-regularization approach (that is, the Lasso model) and the $\ell_{1}-\ell_{2}$ hybrid regularization approach. Furthermore, we investigate their performance in solving a constrained minimization problem in infinite-dimensional Hilbert spaces. Finally, the numerical results of all experiments show that our proposed methods are robust, computationally efficient and achieve better generalization performance and stability than some existing algorithms in the literature.

Keywords

• Relaxed CQ method
• Alternated inertial method
• Multi-step inertial method
• Conjugate gradient method
• Split feasibility problem
• Classification problem

References

1. J. Abubakar, P. Kumam, G. H. Taddele, A. H. Ibrahim and K. Sitthithakerngkiet: Strong convergence of alternated inertial CQ relaxed method with application in signal recovery, Comput. Appl. Math., 40 (8) (2021), Article ID: 310.
2. A. Adamu, D. Kitkuan and T. Seangwattana: An accelerated Halpern-type algorithm for solving variational inclusion problems with applications, Bangmod J. Math. Comp. Sci, 8 (2022), 37–55.
3. A. Ahmad, P. Kumam and M. H. Harbau: Convergence theorems for common solutions of nonlinear problems and applications, Carpathian J. Math., 40 (2), (2024) 207–241.
4. A. Ahmad, P. Kumam, M. H. Harbau and K. Sitthithakerngkiet: Inertial hybrid algorithm for generalized mixed equilibrium problems, zero problems, and fixed points of some nonlinear mappings in the intermediate sense, Math. Methods Appl. Sci., 47 (11) (2024), 8527–8550.
5. H. H. Bauschke, P. L. Combettes: Convex analysis and monotone operator theory in Hilbert spaces, 2nd Edition, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, Springer-Cham, New York (2017).
6. A. Beck, M. Teboulle: A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (1) (2009), 183–202.
7. C. Byrne: Iterative oblique projection onto convex sets and the split feasibility problem, Inverse problems, 18 (2) (2002), Article ID: 441.
8. F. Cao, Y. Tan and M. Cai: Sparse algorithms of random weight networks and applications, Expert System Appl., 41 (5) (2014), 2457–2462.

9. L.-C. Ceng, Q. H. Ansari and J.-C. Yao: An extragradient method for solving split feasibility and fixed point problems, Comput. Math. Appl., 64 (4) (2012), 633–642.
10. Y. Censor, T. Elfving: A multiprojection algorithm using bregman projections in a product space, Numer. Algorithms, 8 (1994), 221–239.
11. H. Che, Y. Zhuang, Y. Wang and H. Chen: A relaxed inertial and viscosity method for split feasibility problem and applications to image recovery, J. Global Optim., 87 (2-4) (2023), 619–639.
12. P. Chen, J. Huang and X. Zhang: A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration, Inverse Problems, 29 (2) (2013), Article ID: 025011.
13. W. Cholamjiak, Y. Shehu and J. C. Yao: Prediction of breast cancer through fast optimization techniques applied to machine learning, Optimization, 73 (2024), 1–29.
14. P. L. Combettes, L. E. Glaudin: Quasi-nonexpansive iterations on the affine hull of orbits: from Mann’s mean value algorithm to inertial methods, SIAM J. Optim., 27 (4) (2017), 2356–2380.
15. J. Demˇ sar: Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res., 7 (2006), 1–30.
16. Q.-L. Dong, H.-b. Yuan: Accelerated mann and cq algorithms for finding a fixed point of a nonexpansive mapping, Fixed Point Theory Appl., 2015 (2015), Article ID: 125.
17. Q.-L. Dong, Y. J. Cho and T. M. Rassias: General inertial Mann algorithms and their convergence analysis for nonexpansive mappings, in: Applications of nonlinear analysis, Springer Optim. Appl., Springer, Cham, New York, 134 (2018), 175–191.
18. Q. L. Dong, Y. C. Tang, Y. J. Cho and T. M. Rassias,: “Optimal” choice of the step length of the projection and contraction methods for solving the split feasibility problem, J. Global Optim., 71 (2) (2018), 341–360.
19. Q.-L. Dong, J. Huang, X. Li, Y.J. Cho and T. M. Rassias, Mikm: multi-step inertial krasnosel’skiˇı–mann algorithm and its applications, J. Global Optim., 73 (2019), 801–824.
20. Q.-L. Dong, X.-H. Li and T. M. Rassias: Two projection algorithms for a class of split feasibility problems with jointly constrained nash equilibrium models, Optimization, 70 (4) (2021), 871–897.
21. Q.-L. Dong, L. Liu and Y. Yao: Self-adaptive projection and contraction methods with alternated inertial terms for solving the split feasibility problem, J. Nonlinear Convex Anal., 23 (3) (2022), 591–605.
22. B. German: Glass Identification, (1987), DOI: 10.24432/C5WW2P.
23. A. Gibali, L.-W. Liu and Y.-C. Tang: Note on the modified relaxation CQ algorithm for the split feasibility problem, Optim. Lett., 12 (4) (2018), 817–830.
24. A. Gibali, D. V. Thong and N. T. Vinh: Three new iterative methods for solving inclusion problems and related problems, Comput. Appl. Math., 39 (3) (2020), Article ID: 187.
25. K. Goebel, S. Reich: Uniform convexity, hyperbolic geometry, and nonexpansive mappings, Monographs and Textbooks in Pure and Applied Mathematics, 83, Marcel Dekker, Inc., New York (1984).
26. M. H. Harbau, G. C. Ugwunnadi, L. O. Jolaoso and A. Abdulwahab: Inertial accelerated algorithm for fixed point of asymptotically nonexpansive mapping in real uniformly convex banach spaces, Axioms, 10 (3) (2021), Article ID: 147.
27. M. H. Harbau, A. Ahmad, B. Ali and G. C. Ugwunnadi: Inertial residual algorithm for fixed points of finite family of strictly pseudocontractive mappings in banach spaces, Int. J. Nonlinear Anal. Appl., 13 (2) (2022), 2257–2269.
28. S. He, C. Yang: Solving the variational inequality problem defined on intersection of finite level sets, in: Abstract Appl. Anal., 2013 (2013), Article ID: 942315.
29. G.-B. Huang, Q.-Y. Zhu and C.-K. Siew: Extreme learning machine: a new learning scheme of feedforward neural networks, in: 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), IEEE, 2 (2004), 985–990.
30. H. Iiduka, I. Yamada: A use of conjugate gradient direction for the convex optimization problem over the fixed point set of a nonexpansive mapping, SIAM J. Optim., 19 (4) (2009) 1881–1893.
31. H. Iiduka: Three-term conjugate gradient method for the convex optimization problem over the fixed point set of a nonexpansive mapping, Appl. Math. Comp., 217 (13) (2011), 6315–6327.
32. H. Iiduka: Fixed point optimization algorithms for distributed optimization in networked systems, SIAM J. Optim., 23 (1) (2013), 1–26.
33. H. Iiduka: Acceleration method for convex optimization over the fixed point set of a nonexpansive mapping, Math. Prog., 149 (1-2) (2015), 131–165.
34. A. Janosi, W. Steinbrunn, M. Pfisterer and R. Detrano: Heart Disease, (1988), DOI: 10.24432/C52P4X.
35. N. Jun-On, W. Cholamjiak: Enhanced Double Inertial Forward–Backward Splitting Algorithm for Variational Inclusion Problems: Applications in Mathematical Integrated Skill Prediction, Symmetry, 16 (8) (2024), Article ID: 1091.
36. A. Kiri, A. Abubakar: A family of conjugate gradient projection method for nonlinear monotone equations with applications to compressive sensing, Nonlinear Convex Anal. Optim., 1 (1) (2022), 47–65.
37. J. Liang: Convergence rates of first-order operator splitting methods, Ph.D. thesis, Normandie Université; GREYC CNRS UMR 6072 (2016).
38. D. A. Lorenz, T. Pock: An inertial forward-backward algorithm for monotone inclusions, J. Math. Imaging Vision, 51 (2) (2015), 311–325.
39. X. Ma, H. Liu: An inertial Halpern-type CQ algorithm for solving split feasibility problems in Hilbert spaces, J. Appl. Math. Comput., 68 (3) (2022), 1699–1717.
40. Z. Mu, Y. Peng: A note on the inertial proximal point method, Stat. Optim. Inf. Comput., 3 (3) (2015), 241–248.
41. S. Penfold, R. Zalas, M. Casiraghi, M. Brooke, Y. Censor and R. Schulte: Sparsity constrained split feasibility for dosevolume constraints in inverse planning of intensity-modulated photon or proton therapy, Phys. Med. Biol., 62 (9) (2017), Article ID: 3599.
42. P. Phairatchatniyom, H. Rehman, J. Abubakar, P. Kumam and J. Mart´ınez-Moreno: An inertial iterative scheme for solving split variational inclusion problems in real Hilbert spaces, Bangmod J. Math. Comp. Sci., 7 (2021), 35–52.
43. B. T. Polyak: Some methods of speeding up the convergence of iteration methods, USSR Comp. Math. Math. Phys., 4 (5) (1964), 1–17.
44. B. Qu, N. Xiu: A note on the CQ algorithm for the split feasibility problem, Inverse Problems, 21 (5) (2005), 1655–1665.
45. S. Reich, T. M. Tuyen and P. T. V. Huyen: Inertial proximal point algorithms for solving a class of split feasibility problems, J. Optim. Theory Appl., 200 (3) (2024), 951–977.
46. D. R. Sahu, Y. J. Cho, Q.-L. Dong, M. Kashyap and X. Li: Inertial relaxed cq algorithms for solving a split feasibility problem in hilbert spaces, Numer. Algorithms, 87 (2021), 1075–1095.
47. D. Serre: Matrices, Graduate Texts in Mathematics, 216 Springer-Verlag, New York (2002), Theory and Applications, Translated from the 2001 French original.
48. Y. Shehu, Q.-L. Dong, L.-L. Liu: Global and linear convergence of alternated inertial methods for split feasibility problems, Rev. R. Acad. Cienc. Exactas Fís. Nat. Ser. A Mat., 115 (2) (2021), Article ID: 53.
49. S. Suantai, B. Panyanak, S. Kesornprom and P. Cholamjiak: Inertial projection and contraction methods for split feasibility problem applied to compressed sensing and image restoration, Optim. Lett., 16 (6) (2022), 1725–1744.
50. S. Suantai, P. Peeyada, A. Fulga andW. Cholamjiak: Heart disease detection using inertial Mann relaxed CQalgorithms for split feasibility problems, AIMS Math., 8 (8) (2023), 18898–18918.
51. G. H. Taddele, A. G. Gebrie and J. Abubakar: An iterative method with inertial effect for solving multiple-sets split feasibility problem, Bangmod J. Math. Comp. Sci, 7 (2021), 53–73.
52. B. Tan, X. Qin and X. Wang: Alternated inertial algorithms for split feasibility problems, Numer. Algorithms, 95 (2023) 773–812.
53. B. Tan, X. Qin and X. Wang: Alternated inertial algorithms for split feasibility problems, Numer. Algorithms, 95 (2) (2024), 773–812.
54. R. Tibshirani: Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B, 58 (1) (1996), 267–288.
55. W. Wolberg, O. Mangasarian, N. Street and W. Street: Breast Cancer Wisconsin (Diagnostic), (1995), DOI: 10.24432/C5DW2B
56. H.-K. Xu: Iterative methods for the split feasibility problem in infinite-dimensional Hilbert spaces, Inverse Problems, 26 (10) (2010), Artile ID: 105018.
57. W. Yajai, P. Nabheerong and W. Cholamjiak: A double inertial Mann Algorithm for Split Equilibrium Problems Application to Breast Cancer Screening, J. Nonlinear Convex Anal., 25 (7) (2024), 1697–1716.
58. W. Yajai, S. Das, S. Yajai and W. Cholamjiak: A modified viscosity type inertial subgradient extragradient algorithm for nonmonotone equilibrium problems and application to cardiovascular disease detection, Discrete Contin. Dyn. Syst. Ser. S, (2024), DOI: 10.3934/dcdss.2024163
59. Q. Yang: The relaxed cq algorithm solving the split feasibility problem, Inverse problems, 20 (4) (2004), Article ID: 1261.
60. Q. Yang: On variable-step relaxed projection algorithm for variational inequalities, J. Math. Anal. Appl., 302 (1) (2005), 166–179.
61. H. Ye, F. Cao and D.Wang: A hybrid regularization approach for random vector functional-link networks, Expert Systems Appl., 140 (2020) Article ID: 112912.