Conjugate Direction Methods in Optimization

I Newton's Method and the Gradient Method.- 1 Introduction.- 2 Fundamental Concepts.- 3 Iterative Methods for Solving g(x) = 0.- 4 Convergence Theorems.- 5 Minimization of Functions by Newton's Method.- 6 Gradient Methods-The Quadratic Case.- 7 General Descent Methods.- 8 Iterative Methods for Solving Linear Equations.- 9 Constrained Minima.- II Conjugate Direction Methods.- 1 Introduction.- 2 Quadratic Functions on En.- 3 Basic Properties of Quadratic Functions.- 4 Minimization of a Quadratic Function F on k-Planes.- 5 Method of Conjugate Directions (CD-Method).- 6 Method of Conjugate Gradients (CG-Algorithm).- 7 Gradient PARTAN.- 8 CG-Algorithms for Nonquadratic Functions.- 9 Numerical Examples.- 10 Least Square Solutions.- III Conjugate Gram-Schmidt Processes.- 1 Introduction.- 2 A Conjugate Gram-Schmidt Process.- 3 CGS-CG-Algorithms.- 4 A Connection of CGS-Algorithms with Gaussian Elimination.- 5 Method of Parallel Displacements.- 6 Methods of Parallel Planes (PARP).- 7 Modifications of Parallel Displacements Algorithms.- 8 CGS-Algorithms for Nonquadratic Functions.- 9 CGS-CG-Routines for Nonquadratic Functions.- 10 Gauss-Seidel CGS-Routines.- 11 The Case of Nonnegative Components.- 12 General Linear Inequality Constraints.- IV Conjugate Gradient Algorithms.- 1 Introduction.- 2 Conjugate Gradient Algorithms.- 3 The Normalized CG-Algorithm.- 4 Termination.- 5 Clustered Eigenvalues.- 6 Nonnegative Hessians.- 7 A Planar CG-Algorithm.- 8 Justification of the Planar CG-Algorithm.- 9 Modifications of the CG-Algorithm.- 10 Two Examples.- 11 Connections between Generalized CG-Algorithms and Stadard CG- and CD-Algorithm.- 12 Least Square Solutions.- 13 Variable Metric Algorithms.- 14 A Planar CG-Algorithm for Nonquadratic Functions.- References.