LQ Dynamic Optimization and Differential Games

Preface ix

Notation and symbols xi

1 Introduction 1

1.1 Historical perspective 1

1.2 How to use this book 10

1.3 Outline of this book 10

1.4 Notes and references 14

2 Linear algebra 15

2.1 Basic concepts in linear algebra 15

2.2 Eigenvalues and eigenvectors 21

2.3 Complex eigenvalues 23

2.4 Cayleyâ"Hamilton theorem 31

2.5 Invariant subspaces and Jordan canonical form 34

2.6 Semi-definite matrices 42

2.7 Algebraic Riccati equations 43

2.8 Notes and references 54

2.9 Exercises 55

2.10 Appendix 59

3 Dynamical systems 63

3.1 Description of linear dynamical systems 64

3.2 Existenceâ"uniqueness results for differential equations 70

3.2.1 General case 70

3.2.2 Control theoretic extensions 74

3.3 Stability theory: general case 78

3.4 Stability theory of planar systems 83

3.5 Geometric concepts 91

3.6 Performance specifications 96

3.7 Examples of differential games 105

3.8 Information, commitment and strategies 114

3.9 Notes and references 114

3.10 Exercises 115

3.11 Appendix 118

4 Optimization techniques 121

4.1 Optimization of functions 121

4.2 The Eulerâ"Lagrange equation 125

4.3 Pontryaginâs maximum principle 133

4.4 Dynamic programming principle 154

4.5 Solving optimal control problems 162

4.6 Notes and references 162

4.7 Exercises 163

4.8 Appendix 170

5 Regular linear quadratic optimal control 175

5.1 Problem statement 175

5.2 Finite-planning horizon 177

5.3 Riccati differential equations 192

5.4 Infinite-planning horizon 196

5.5 Convergence results 209

5.6 Notes and references 218

5.7 Exercises 219

5.8 Appendix 224

6 Cooperative games 229

6.1 Pareto solutions 230

6.2 Bargaining concepts 240

6.3 Nash bargaining solution 246

6.4 Numerical solution 251

6.5 Notes and references 253

6.6 Exercises 254

6.7 Appendix 259

7 Non-cooperative open-loop information games 261

7.1 Introduction 264

7.2 Finite-planning horizon 265

7.3 Open-loop Nash algebraic Riccati equations 278

7.4 Infinite-planning horizon 283

7.5 Computational aspects and illustrative examples 299

7.6 Convergence results 305

7.7 Scalar case 312

7.8 Economics examples 319

7.8.1 A simple government debt stabilization game 320

7.8.2 A game on dynamic duopolistic competition 322

7.9 Notes and references 326

7.10 Exercises 327

7.11 Appendix 340

8 Non-cooperative feedback information games 359

8.1 Introduction 359

8.2 Finite-planning horizon 362

8.3 Infinite-planning horizon 371

8.4 Two-player scalar case 383

8.5 Computational aspects 389

8.5.1 Preliminaries 390

8.5.2 A scalar numerical algorithm: the two-player case 393

8.5.3 The N-player scalar case 399

8.6 Convergence results for the two-player scalar case 403

8.7 Notes and references 412

8.8 Exercises 413

8.9 Appendix 421

9 Uncertain non-cooperative feedback information games 427

9.1 Stochastic approach 428

9.2 Deterministic approach: introduction 433

9.3 The one-player case 435

9.4 The one-player scalar case 444

9.5 The two-player case 450

9.6 A fishery management game 455

9.7 A scalar numerical algorithm 461

9.8 Stochastic interpretation 472

9.9 Notes and references 474

9.10 Exercises 475

9.11 Appendix 481

References 485

Index 495