A Fast Spectral Conjugate Gradient Method for Solving Nonlinear Optimization Problems

Ali Abbas Al-Arbo, Rana Z. Al-Kawaz


This paper proposes a new spectral conjugate gradient (SCG) approach for solving unregulated nonlinear optimization problems. Our approach proposes Using Wolfe's rapid line scan to adjust the standard conjugate descent (CD) algorithm. A new spectral parameter is a mixture of new gradient and old search path. The path provided by the modified method provides a path of descent for the solution of objective functions. The updated method fits the traditional CD method if the line check is correct. The stability and global convergence properties of the current new SCG are technically obtained from applying certain well-known and recent mild assumptions. We test our approach with eight recently published CD and SCG methods on 55 optimization research issues from the CUTE library. The suggested and all other algorithms included in our experimental research were implemented in FORTRAN language with double precision arithmetic and all experiments were conducted on a PC with 8 GB ram Processor Intel Core i7. The results indicate that our proposed solution outperforms recently reported algorithms by processing and performing fewer iterations in a shorter time.


Spectral Conjugate Gradient, Global Convergence, Unconstrained Optimization, Descent Direction, Line Search.


M. Hestenes, E. Stiefel, "Methods of the conjugate gradient for solving a linear system", Journal of Research of the National Bureau of Standards, 49, 409–436, 1952.

R. Fletcher and G. Reeves, "Function Minimization by Gradients", Computer Journal, 7, 149-154, 1964.

B. Hassan, "A new class of BFGS updating formula based on the new Quasi-Newton equation", Indonesian Journal of Electrical Engineering and Computer Science, 13(3), 945-953, 2019.

B. Hassan, "A new formula for conjugate parameter computation based on the quadratic form", Indonesian Journal of Electrical Engineering and Computer Science, 13(3), 954-961, 2019.

E. Polak, and G. Ribière, "Note Sur la Convergence de Directions Conjuguée", Rev. Francaise Informat Recherche Operationally, 3e Année, 16, 35–43, 1969.

R. Fletcher, "Practical Methods of Optimization Unconstrained Optimization", John Wiley & Sons, New York, USA, 1987.

P. Wolfe, "Convergence Conditions for Ascent Methods", SIAM Review, 11, 226–235, 1969.

H. Zhang, and W. Hager, "Algorithms 851: CG-descent Conjugate Gradient Method with Guaranteed Descent", ACM. Transactions on Mathematical Software, 32(1), 113–137 2006.

M. Powell, "Restart Procedure For The Conjugate Gradient Methods", Mathematics programming, 12,241-254, 1977.

J. Barzilai, and J. Borwein, "Two-Point Step-Size Gradient Methods", IMA Journal of Numerical Analysis, 8(1), 141-148, 1988.

A. Y. Al-Bayati, et al, "Spectral Three-Term Constrained Conjugate Gradient Algorithm for Function Minimizations", Hindawi Publishing Corporation, Journal of Applied Mathematics, Article ID 6378368, 1-6, 2019.

A. Y. Al-Bayati, et al, "New Investigation for The Liu-Story Scaled Conjugate Gradient Methods for Nonlinear Optimizations", Hindawi Publishing Corporation, Journal of Mathematics, Article ID 3615208, 1-12, 2020.

A.Y. Al-Bayati, and M. Mohammed Ali,. "New Mixed Multi Gradient Descent Steps with Variable Metric Updates for Unconstraint Optimization, Indonesian Journal of Electrical Engineering and Computer Science, Article ID: 21300, 15(3), Accepted, 2020.

A. Y. Al-Bayati, et al, "Interaction Between Updated FR-CG Algorithms With Optimal Cuckoo Algorithm, Indonesian Journal of Electrical Engineering and Computer Science, Article ID: 21279, 15(3), Accepted, 2020.

M. Raydan, "The Brazilian and Borwein Gradient Method For The Large Unconstrained Minimization The problem", SIAM Journal on Optimization, 7, 26–33, 1997.

A. Y. Al-Bayati, and R. Abdullah, "A New Family of Spectral CG-Algorithm", Rafidain Journal of Computer Science and Mathematics, Iraq, 5(1), 69-80, 2008.

A.Y Al-Bayati, and B. Hassan, "Spectral Conjugate Gradient Method with An Inexact Line Search", Iraqi Journal of Statistical Science, Iraq, 11(20), 155-163, 2011.

Y. Jiang, and J. Liu, "Global Convergence of a Spectral Conjugate Gradient Method for Unconstrained Optimization", Hindawi Publishing Corporation, Abstract, and Applied Analysis, 2012.

J. Liu, et al, "A Mixed Spectral CD-DY Conjugate Gradient Method", Hindawi Publishing Corporation, Journal of Applied Mathematics, 2012

I. Livieris and P. Pintelas, "New Class of Spectral Conjugate Gradient Methods Based on a Modified Secant Equation for Unconstrained Optimization", Journal of Computational and Applied Mathematics, 239, 396–405, 2012.

A. Y. Al-Bayati, and H. Al-Khayat, "A Global Convergent Spectral Conjugate Gradient Method", Australian Journal of Basic and Applied Sciences, 7(7), 302-309, 2013.

M. Ghanbari, et al, "Global Convergence of Two Spectral Conjugate Gradient Methods", Science Asia, 39, 306–311, 2013.

D. Liu, et al, "Spectral Method and Its Application to the Conjugate Gradient Method", Applied Mathematics and Computation, 240, 339–347, 2014.

K. Bongartz, et al, "CUTE constrained and unconstrained testing environments", ACM Trans. Math. Software, 21, 123-160, 1995.

N. Andrei, "Another Nonlinear Conjugate Gradient Algorithm for Unconstrained Optimization", Optimization Methods and Software, 24(1), 89-104, 2009.

DOI: http://doi.org/10.11591/ijeecs.v21.i1.pp%25p
Total views : 45 times


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

shopify stats IJEECS visitor statistics