Dynamic Programming

I Introduction.- 1 The Management of Agricultural and Natural Resource Systems.- 1.1 The Nature of Agricultural and Natural Resource Problems.- 1.2 Management Techniques Applied to Resource Problems.- 1.2.1 Farm management.- 1.2.2 Forestry management.- 1.2.3 Fisheries management.- 1.3 Control Variables in Resource Management.- 1.3.1 Input decisions.- 1.3.2 Output decisions.- 1.3.3 Timing and replacement decisions.- 1.4 A Simple Derivation of the Conditions for Intertemporal Optimality.- 1.4.1 The general resource problem without replacement.- 1.4.2 The general resource problem with replacement.- 1.5 Numerical Dynamic Programming.- 1.5.1 Types of resource problem.- 1.5.2 Links with simulation.- 1.5.3 Solution procedures.- 1.5.4 Types of dynamic programming problem.- 1.6 References.- 1.A Appendix: A Lagrangian Derivation of the Discrete Maximum Principle.- 1.B Appendix: A Note on the Hamiltonian Used in Control Theory.- II The Methods of Dynamic Programming.- 2 Introduction to Dynamic Programming.- 2.1 Backward Recursion Applied to the General Resource Problem.- 2.2 The Principle of Optimality.- 2.3 The Structure of Dynamic Programming Problems.- 2.4 A Numerical Example.- 2.5 Forward Recursion and Stage Numbering.- 2.6 A Simple Crop-irrigation Problem.- 2.6.1 The formulation of the problem.- 2.6.2 The solution procedure.- 2.7 A General-Purpose Computer Program for Solving Dynamic Programming Problems.- 2.7.1 An introduction to the GPDP programs.- 2.7.2 Data entry using DPD.- 2.7.3 Using GPDP to solve the least-cost network problem.- 2.7.4 Using GPDP to solve the crop-irrigation problem.- 2.8 References.- 3 Stochastic and Infinite-Stage Dynamic Programming.- 3.1 Stochastic Dynamic Programming.- 3.1.1 Formulation of the stochastic problem.- 3.1.2 A stochastic crop-irrigation problem.- 3.2 Infinite-stage Dynamic Programming for Problems With Discounting.- 3.2.1 Formulation of the problem.- 3.2.2 Solution by value iteration.- 3.2.3 Solution by policy iteration.- 3.3 Infinite-stage Dynamic Programming for Problems Without Discounting.- 3.3.1 Formulation of the problem.- 3.3.2 Solution by value iteration.- 3.3.3 Solution by policy iteration.- 3.4 Solving Infinite-stage Problems in Practice.- 3.4.1 Applications to agriculture and natural resources.- 3.4.2 The infinite-stage crop-irrigation problem.- 3.4.3 Solution to the crop-irrigation problem with discounting.- 3.4.4 Solution to the crop-irrigation problem without discounting.- 3.5 Using GPDP to Solve Stochastic and Infinite-stage Problems.- 3.5.1 Stochastic problems.- 3.5.2 Infinite-stage problems.- 3.6 References.- 4 Extensions to the Basic Formulation.- 4.1 Linear Programming for Solving Stochastic, Infinite-stage Problems.- 4.1.1 Linear programming formulations of problems with discounting.- 4.1.2 Linear programming formulations of problems without discounting.- 4.2 Adaptive Dynamic Programming.- 4.3 Analytical Dynamic Programming.- 4.3.1 Deterministic, quadratic return, linear transformation problems.- 4.3.2 Stochastic, quadratic return, linear transformation problems.- 4.3.3 Other problems which can be solved analytically.- 4.4 Approximately Optimal Infinite-stage Solutions.- 4.5 Multiple Objectives.- 4.5.1 Multi-attribute utility.- 4.5.2 Risk.- 4.5.3 Problems involving players with conflicting objectives.- 4.6 Alternative Computational Methods.- 4.6.1 Approximating the value function in continuous form.- 4.6.2 Alternative dynamic programming structures.- 4.6.3 Successive approximations around a nominal control policy.- 4.6.4 Solving a sequence of problems of reduced dimension.- 4.6.5 The Lagrange multiplier method.- 4.7 Further Information on GPDP.- 4.7.1 The format for user-written data files.- 4.7.2 Redimensioning arrays in FDP and IDP.- 4.8. References.- 4.A Appendix: The Slope and Curvature of the Optimal Return Function Vi{xi}.- III Dynamic Programming Applications to Agriculture.- 5 Scheduling, Replacement and Inventory Management.- 5.1 Critical Path Analysis.- 5.1.1 A farm example.- 5.1.2 Solution using GPDP.- 5.1.3 Selected applications.- 5.2 Farm Investment Decisions.- 5.2.1 Optimal tractor replacement.- 5.2.2 Formulation of the problem without tax.- 5.2.3 Formulation of the problem with tax.- 5.2.4 Discussion.- 5.2.5 Selected applications.- 5.3 Buffer Stock Policies.- 5.3.1 Stochastic yields: planned production and demand constant.- 5.3.2 Stochastic yields and demand: planned production constant.- 5.3.3 Planned production a decision variable.- 5.3.4 Selected applications.- 5.4 References.- 6 Crop Management.- 6.1 The Crop Decision Problem.- 6.1.1 States.- 6.1.2 Stages.- 6.1.3 Returns.- 6.1.4 Decisions.- 6.2 Applications to Water Management.- 6.3 Applications to Pesticide Management.- 6.4 Applications to Crop Selection.- 6.5 Applications to Fertilizer Management.- 6.5.1 Optimal rules for single-period carryover functions.- 6.5.2 Optimal rules for a multiperiod carryover function.- 6.5.3 A numerical example.- 6.5.4 Extensions.- 6.6 References.- 7 Livestock Management.- 7.1 Livestock Decision Problems.- 7.2 Livestock Replacement Decisions.- 7.2.1 Types of problem.- 7.2.2 Applications to dairy cows.- 7.2.3 Periodic revision of estimated yield potential.- 7.3 Combined Feeding and Replacement Decisions.- 7.3.1 The optimal ration sequence: an example.- 7.3.2 Maximizing net returns per unit of time.- 7.3.3 Replacement a decision option.- 7.4 Extensions to the Combined Feeding and Replacement Problem.- 7.4.1 The number of livestock.- 7.4.2 Variable livestock prices.- 7.4.3 Stochastic livestock prices.- 7.4.4 Ration formulation systems.- 7.5 References.- 7.A Appendix: Yield Repeatability and Adaptive Dynamic Programming.- 7.A.1 The concept of yield repeatability.- 7.A.2 Repeatability of average yield.- 7.A.3 Expected yield given average individual and herd yields.- 7.A.4 Yield probabilities conditional on recorded average yields.- IV Dynamic Programming Applications to Natural Resources.- 8 Land Management.- 8.1 The Theory of Exhaustible Resources.- 8.1.1 The simple theory of the mine.- 8.1.2 Risky possession and risk aversion.- 8.1.3 Exploration.- 8.2 A Pollution Problem.- 8.2.1 Pollution as a stock variable.- 8.2.2 A numerical example.- 8.3 Rules for Making Irreversible Decisions Under Uncertainty.- 8.3.1 Irreversible decisions and quasi-option value.- 8.3.2 A numerical example.- 8.3.3 The discounting procedure.- 8.4 References.- 9 Forestry Management.- 9.1 Problems in Forestry Management.- 9.2 The Optimal Rotation Period.- 9.2.1 Deterministic problems.- 9.2.2 Stochastic problems.- 9.2.3 A numerical example of a combined rotation and protection problem.- 9.3 The Optimal Rotation and Thinning Problem.- 9.3.1 Stage intervals.- 9.3.2 State variables.- 9.3.3 Decision variables.- 9.3.4 Objective function.- 9.4 Extensions.- 9.4.1 Allowance for distributions of tree sizes and ages.- 9.4.2 Alternative objectives.- 9.5 References.- 10 Fisheries Management.- 10.1 The Management Problem.- 10.2 Modelling Approaches.- 10.2.1 Stock dynamics.- 10.2.2 Stage return.- 10.2.3 Developments in analytical modelling.- 10.3 Analytical Dynamic Programming Approaches.- 10.3.1 Deterministic results.- 10.3.2 Stochastic results.- 10.4 Numerical Dynamic Programming Applications.- 10.4.1 An application to the southern bluefin tuna fishery.- 10.4.2 A review of applications.- 10.5 References.- V Conclusion.- 11 The Scope for Dynamic Programming Applied to Resource Management.- 11.1 Dynamic Programming as a Method of Conceptualizing Resource Problems.- 11.2 Dynamic Programming as a Solution Technique.- 11.3 Applications to Date.- 11.4 Expected Developments.- 11.5 References.- Appendices.- A1 Coding Sheets for Entering Data Using DPD.- A2 Program Listings.- A2.1 Listing of DPD.- A2.2 Listing of FDP.- A2.3 Listing of IDP.- A2.4 Listing of DIM.- Author Index.