Article Sidebar

Submitted
Apr 26, 2017
Published
Apr 1, 2012
Download
PDF
Statistic
Read Counter : 401 Download : 262

Main Article Content

Abstract

A class of damped quasi-Newton methods for nonlinear optimization has recently been proposed by extending the damped-technique of Powell for the BFGS method to the Broyden family of quasi-Newton methods. It has been shown that this damped class has the global and superlinear convergence property that a restricted class of 'undamped' methods has for convex objective functions in unconstrained optimization. To test this result, we applied several members of the Broyden family and their corresponding damped methods to a simple quadratic function and observed several useful features of the damped-technique. These observations and other numerical experiences are described in this paper. The important role of the damped-technique is shown not only for enforcing the above convergence property, but also for improving the performance of efficient, inefficient and divergent undamped methods substantially (significantly in the latter case). Thus, some appropriate ways for employing the damped-technique are suggested.

 

Keywords

Nonlinear optimization Quasi-Newton methods Damped-technique Line search framework.

Article Details

References

  1. AL-BAALI, M. 1993. Variational quasi-Newton methods for unconstrained optimization. JOTA, 77: 127-143.
  2. AL-BAALI, M. 2003. Quasi-Wolfe conditions for Newton-like methods. 18th International Symposium on Mathematical Programming, Copenhagen, Denmark, August 18-22, 2003.
  3. AL-BAALI, M. 2004. Quasi-Wolfe conditions for quasi-Newton methods for large-scale optimization. 40th Workshop on Large-Scale Nonlinear Optimization, Erice, Italy, June 22 - July 1, 2004.
  4. AL-BAALI, M. 2011. Convergence analysis of a family of damped quasi-Newton methods for nonlinear optimization. Research Report DOMAS 11/2, Sultan Qaboos University, Oman.
  5. AL-BAALI, M. and GRANDINETTI, L. 2009. On practical modifications of the quasi-Newton BFGS method. Advanced Modeling and Optimization, 11: 63-76.
  6. AL-BAALI, M. and KHALFAN, H.F. 2009. A combined class of self-scaling and modified quasi-Newton methods. Optimization Online, 07/2334 (to appear in COAP).
  7. BYRD, R.H., LIU, D.C. and NOCEDAL, J. 1992. On the behavior of Broyden's class of quasi-Newton methods. SIAM J. Optim., 2: 533-557.
  8. DENNIS, J.E. and MORÉ, J.J. 1974. A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comp., 28: 549-560.
  9. DENNIS, J.E. and SCHNABEL, R.B. 1996. Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM Publications, Philadelphia.
  10. FLETCHER, R. 1987. Practical Methods of Optimization (2nd edition). Wiley, Chichester, England.
  11. GILL, P.E. and LEONARD, M.W. 2003. Limited-memory reduced-Hessian methods for large-scale unconstrained optimization. SIAM J. Optim., 14: 380-401.
  12. GRATTON, S. and TOINT, P.L. 2010. Approximate invariant subspaces and quasi-Newton optimization methods. OMS, 25: 507-529.
  13. NOCEDAL, J. 1980. Updating quasi-Newton matrices with limited storage. Math. Comput., 35: 773-782.
  14. NOCEDAL, J. and WRIGHT, S.J. 1999. Numerical Optimization. Springer, London.
  15. POWELL, M.J.D. 1978. Algorithms for nonlinear constraints that use Lagrange functions. Math. Programming, 14: 224-248.
  16. POWELL, M.J.D. 1986. How bad are the BFGS and DFP methods when the objective function is quadratic?. Math. Programming, 34: 34-47.
  17. POWELL, M.J.D. 2009. On nonlinear optimization since 1959. Optimization Online, 11/2477.
  18. YABE, H., OGASAWARA, H. and YOSHINO, M. 2007. Local and superlinear convergence of quasi-Newton methods based on modified secant conditions. Journal of Computational and Applied Mathematics, 205: 617-632.