Least Squares Data Fitting with Applications

Preface
Symbols and Acronyms
1. The Linear Data Fitting Problem
1.1. Parameter estimation, data approximation
1.2. Formulation of the data fitting problem
1.3. Maximum likelihood estimation
1.4. The residuals and their properties
1.5. Robust regression
2. The Linear Least Squares Problem
2.1. Linear least squares problem formulation
2.2. The QR factorization and its role
2.3. Permuted QR factorization
3. Analysis of Least Squares Problems
3.1. The pseudoinverse
3.2. The singular value decomposition
3.3. Generalized singular value decomposition
3.4. Condition number and column scaling
3.5. Perturbation analysis
4. Direct Methods for Full-Rank Problems
4.1. Normal equations
4.2. LU factorization
4.3. QR factorization
4.4. Modifying least squares problems
4.5. Iterative refinement
4.6. Stability and condition number estimation
4.7. Comparison of the methods
5. Direct Methods for Rank-Deficient Problems
5.1. Numerical rank
5.2. Peters-Wilkinson LU factorization
5.3. QR factorization with column permutations
5.4. UTV and VSV decompositions
5.5. Bidiagonalization
5.6. SVD computations
6. Methods for Large-Scale Problems
6.1. Iterative versus direct methods
6.2. Classical stationary methods
6.3. Non-stationary methods, Krylov methods
6.4. Practicalities: preconditioning and stopping criteria
6.5. Block methods
7. Additional Topics in Least Squares
7.1. Constrained linear least squares problems
7.2. Missing data problems
7.3. Total least squares (TLS)
7.4. Convex optimization
7.5. Compressed sensing
8. Nonlinear Least Squares Problems
8.1. Introduction
8.2. Unconstrained problems
8.3. Optimality conditions for constrained problems
8.4. Separable nonlinear least squares problems
8.5. Multiobjective optimization
9. Algorithms for Solving Nonlinear LSQ Problems
9.1. Newton's method
9.2. The Gauss-Newton method
9.3. The Levenberg-Marquardt method
9.4. Additional considerations and software
9.5. Iteratively reweighted LSQ algorithms for robust data fitting problems
9.6. Variable projection algorithm
9.7. Block methods for large-scale problems
10. Ill-Conditioned Problems
10.1. Characterization
10.2. Regularization methods
10.3. Parameter selection techniques
10.4. Extensions of Tikhonov regularization
10.5. Ill-conditioned NLLSQ problems
11. Linear Least Squares Applications
11.1. Splines in approximation
11.2. Global temperatures data fitting
11.3. Geological surface modeling
12. Nonlinear Least Squares Applications
12.1. Neural networks training
12.2. Response surfaces, surrogates or proxies
12.3. Optimal design of a supersonic aircraft
12.4. NMR spectroscopy
12.5. Piezoelectric crystal identification
12.6. Travel time inversion of seismic data
Appendix A: Sensitivity Analysis
A.1. Floating-point arithmetic
A.2. Stability, conditioning and accuracy
Appendix B: Linear Algebra Background
B.1. Norms
B.2. Condition number
B.3. Orthogonality
B.4. Some additional matrix properties
Appendix C: Advanced Calculus Background
C.1. Convergence rates
C.2. Multivariable calculus
Appendix D: Statistics
D.1. Definitions
D.2. Hypothesis testing
References
Index