Accelerated hybrid conjugate gradient algorithm with modified secant condition for unconstrained optimization

Research paper by Neculai Andrei

Indexed on: 25 Jul '09Published on: 25 Jul '09Published in: Numerical Algorithms

Abstract

An accelerated hybrid conjugate gradient algorithm represents the subject of this paper. The parameter βk is computed as a convex combination of $$\beta_k^{HS}$$ (Hestenes and Stiefel, J Res Nat Bur Stand 49:409–436, 1952) and $$\beta_k^{DY}$$ (Dai and Yuan, SIAM J Optim 10:177–182, 1999), i.e. $$\beta_k^C =\left({1-\theta_k}\right)\beta_k^{HS} + \theta_k \beta_k^{DY}$$. The parameter θk in the convex combinaztion is computed in such a way the direction corresponding to the conjugate gradient algorithm is the best direction we know, i.e. the Newton direction, while the pair (sk, yk) satisfies the modified secant condition given by Li et al. (J Comput Appl Math 202:523–539, 2007) Bk + 1sk = zk, where $$z_k =y_k +\left({{\eta_k} / {\left\| {s_k} \right\|^2}} \right)s_k$$, $$\eta_k =2\left( {f_k -f_{k+1}} \right)+\left( {g_k +g_{k+1}} \right)^Ts_k$$, sk = xk + 1 − xk and yk = gk + 1 − gk. It is shown that both for uniformly convex functions and for general nonlinear functions the algorithm with strong Wolfe line search is globally convergent. The algorithm uses an acceleration scheme modifying the steplength αk for improving the reduction of the function values along the iterations. Numerical comparisons with conjugate gradient algorithms show that this hybrid computational scheme outperforms a variant of the hybrid conjugate gradient algorithm given by Andrei (Numer Algorithms 47:143–156, 2008), in which the pair (sk, yk) satisfies the classical secant condition Bk + 1sk = yk, as well as some other conjugate gradient algorithms including Hestenes-Stiefel, Dai-Yuan, Polack-Ribière-Polyak, Liu-Storey, hybrid Dai-Yuan, Gilbert-Nocedal etc. A set of 75 unconstrained optimization problems with 10 different dimensions is being used (Andrei, Adv Model Optim 10:147–161, 2008).