Preface
PART ONE BASIC CONCEPTS AND TOOLS
1 Stochastic Processes 11
1.1 Introduction 11
1.2 Key Concepts in Stochastic Processes 11
1.3 Main Classes of Stochastic Processes 16
1.4 Inference, Prediction and Decision Making 21
1.5 Discussion 23
2 Bayesian Analysis 27
2.1 Introduction 27
2.2 Bayesian Statistics 28
2.3 Bayesian Decision Analysis 37
2.4 Bayesian Computation 39
2.5 Discussion 51
PART TWO MODELS
3 Discrete Time Markov Chains 61
3.1 Introduction 61
3.2 Important Markov Chain Models 62
3.3 Inference for First Order Chains 66
3.4 Special Topics 76
3.5 Case Study: Wind Directions at Gijon 87
3.6 Markov Decision Processes 94
3.7 Discussion 97
4 Continuous Time Markov Chains and Extensions 105
4.1 Introduction 105
4.2 Basic Setup and Results 106
4.3 Inference and Prediction for CTMCs 108
4.4 Case Study: Hardware Availability through CTMCs 112
4.5 Semi-Markovian Processes 118
4.6 Decision Making with Semi-Markovian Decision Processes 122
4.7 Discussion 128
5 Poisson Processes and Extensions 133
5.1 Introduction 133
5.2 Basics on Poisson Processes 134
5.3 Homogeneous Poisson Processes 138
5.4 Nonhomogeneous Poisson Processes 147
5.5 Compound Poisson Processes 153
5.6 Further Extensions of Poisson Processes 154
5.7 Case Study: Earthquake Occurrences 157
5.8 Discussion 162
6 Continuous Time Continuous Space Processes 169
6.1 Introduction 169
6.2 Gaussian Processes 170
6.3 Brownian Motion and Fractional Brownian Motion 174
6.4 Di®usions 181
6.5 Case Study: Prey-predator Systems 184
6.6 Discussion 190
PART THREE APPLICATIONS
7 Queueing Analysis 201
7.1 Introduction 201
7.2 Basic Queueing Concepts 201
7.3 The Main Queueing Models 204
7.4 Inference for Queueing Systems 208
7.5 Inference for M=M=1 Systems 209
7.6 Inference for Non Markovian Systems 220
7.7 Decision Problems in Queueing Systems 229
7.8 Case Study: Optimal Number of Beds in a Hospital 230
7.9 Discussion 235
8 Reliability 245
8.1 Introduction 245
8.2 Basic Reliability Concepts 246
8.3 Renewal Processes 249
8.4 Poisson Processes 251
8.5 Other Processes 259
8.6 Maintenance 262
8.7 Case Study: Gas Escapes 263
8.8 Discussion 271
9 Discrete Event Simulation 279
9.1 Introduction 279
9.2 Discrete Event Simulation Methods 280
9.3 A Bayesian View of DES 283
9.4 Case Study: A G=G=1 Queueing System 286
9.5 Bayesian Output Analysis 288
9.6 Simulation and Optimization 292
9.7 Discussion 294
10 Risk Analysis 301
10.1 Introduction 301
10.2 Risk Measures 302
10.3 Ruin Problems 316
10.4 Case Study: Ruin Probability Estimation 320
10.5 Discussion 327
Appendix A Main Distributions 337
Appendix B Generating Functions and the Laplace-Stieltjes Transform 347
Index
