Preface xv
Introduction 1
Why Study Probability? 1
Software Use in Probability 2
Modern Application of Classic Probability Problems 2
Applications to Business 3
Applications to the Life Sciences 4
Applications to Engineering and Operations Research 4
Applications to Finance 6
Probability in Everyday Life 7
1 Introduction to Probability 13
Introduction 13
1.1 Sample Spaces and Events 13
The Sample Space of an Experiment 13
Events 15
Some Relations from Set Theory 16
Exercises Section 1.1 (1–12) 18
1.2 Axioms Interpretations and Properties of Probability 19
Interpreting Probability 21
More Probability Properties 23
Contingency Tables 25
Determining Probabilities Systematically 26
Equally Likely Outcomes 27
Exercises Section 1.2 (13–30) 28
1.3 Counting Methods 30
The Fundamental Counting Principle 31
Tree Diagrams 32
Permutations 33
Combinations 34
Partitions 38
Exercises Section 1.3 (31–50) 39
Supplementary Exercises (51–62) 42
2 Conditional Probability and Independence 45
Introduction 45
2.1 Conditional Probability 45
The Definition of Conditional Probability 46
The Multiplication Rule for P(A ∩ B) 49
2.2 The Law of Total Probability and Bayes’ Theorem 52
The Law of Total Probability 52
Bayes’ Theorem 55
Exercises Section 2.2 (17–32) 59
2.3 Independence 61
The Multiplication Rule for Independent Events 63
Independence of More Than Two Events 65
Exercises Section 2.3 (33–54) 66
2.4 Simulation of Random Events 69
The Backbone of Simulation: Random Number Generators 70
Precision of Simulation 73
Exercises Section 2.4 (55–74) 74
Supplementary Exercises (75–100) 77
3 Discrete Probability Distributions:general Properties 82
Introduction 82
3.1 Random Variables 82
Two Types of Random Variables 84
Exercises Section 3.1 (1–10) 85
3.2 Probability Distributions for Discrete Random Variables 86
Another View of Probability Mass Functions 89
Exercises Section 3.2 (11–21) 90
3.3 The Cumulative Distribution Function 91
Exercises Section 3.3 (22–30) 95
3.4 Expected Value and Standard Deviation 96
The Expected Value of X 97
The Expected Value of a Function 99
The Variance and Standard Deviation of X 102
Properties of Variance 104
Exercises Section 3.4 (31–50) 105
3.5 Moments and Moment Generating Functions 108
The Moment Generating Function 109
Obtaining Moments from the MGF 111
Exercises Section 3.5 (51–64) 113
3.6 Simulation of Discrete Random Variables 114
Simulations Implemented in R and Matlab 117
Simulation Mean Standard Deviation and Precision 117
Exercises Section 3.6 (65–74) 119
Supplementary Exercises (75–84) 120
4 Families of Discrete Distributions 122
Introduction 122
4.1 Parameters and Families of Distributions 122
Exercises Section 4.1 (1–6) 124
4.2 The Binomial Distribution 125
The Binomial Random Variable and Distribution 127
Computing Binomial Probabilities 129
The Mean Variance and Moment Generating Function 130
Binomial Calculations with Software 132
Exercises Section 4.2 (7–34) 132
4.3 The Poisson Distribution 136
The Poisson Distribution as a Limit 137
The Mean Variance and Moment Generating Function 139
The Poisson Process 140
Poisson Calculations with Software 141
Exercises Section 4.3 (35–54) 142
4.4 The Hypergeometric Distribution 145
Mean and Variance 148
Hypergeometric Calculations with Software 149
Exercises Section 4.4 (55–64) 149
4.5 The Negative Binomial and Geometric Distributions 151
The Geometric Distribution 152
Mean Variance and Moment Generating Function 152
Alternative Definitions of the Negative Binomial Distribution 153
Negative Binomial Calculations with Software 154
Exercises Section 4.5 (65–78) 154
Supplementary Exercises (79–100) 156
5 Continuous Probability Distributions:general Properties 160
Introduction 160
5.1 Continuous Random Variables and Probability Density Functions 160
Probability Distributions for Continuous Variables 161
Exercises Section 5.1 (1–8) 165
5.2 The Cumulative Distribution Function and Percentiles 166
Using F(x) to Compute Probabilities 168
Obtaining f(x) fromF(x) 169
Percentiles of a Continuous Distribution 169
Exercises Section 5.2 (9–18) 171
5.3 Expected Values Variance and Moment Generating Functions 173
Expected Values 173
Variance and Standard Deviation 175
Properties of Expectation and Variance 176
Moment Generating Functions 177
Exercises Section 5.3 (19–38) 179
5.4 Transformation of a Random Variable 181
Exercises Section 5.4 (39–54) 185
5.5 Simulation of Continuous Random Variables 186
The Inverse CDF Method 186
The Accept–Reject Method 189
Precision of Simulation Results 191
Exercises Section 5.5 (55–63) 191
Supplementary Exercises (64–76) 193
6 Families of Continuous Distributions 196
Introduction 196
6.1 The Normal (Gaussian) Distribution 196
The Standard Normal Distribution 197
Arbitrary Normal Distributions 199
The Moment Generating Function 203
Normal Distribution Calculations with Software 204
Exercises Section 6.1 (1–27) 205
6.2 Normal Approximation of Discrete Distributions 208
Approximating the Binomial Distribution 209
Exercises Section 6.2 (28–36) 211
6.3 The Exponential and Gamma Distributions 212
The Exponential Distribution 212
The Gamma Distribution 214
The Gamma and Exponential MGFs 217
Gamma and Exponential Calculations with Software 218
Exercises Section 6.3 (37–50) 218
6.4 Other Continuous Distributions 220
The Weibull Distribution 220
The Lognormal Distribution 222
The Beta Distribution 224
Exercises Section 6.4 (51–66) 226
6.5 Probability Plots 228
Sample Percentiles 228
A Probability Plot 229
Departures from Normality 232
Beyond Normality 234
Probability Plots in Matlab and R 236
Exercises Section 6.5 (67–76) 237
Supplementary Exercises (77–96) 238
7 Joint Probability Distributions 242
Introduction 242
7.1 Joint Distributions for Discrete Random Variables 242
The Joint Probability Mass Function for Two Discrete Random Variables 242
Marginal Probability Mass Functions 244
Independent Random Variables 245
More Than Two Random Variables 246
Exercises Section 7.1 (1–12) 248
7.2 Joint Distributions for Continuous Random Variables 250
The Joint Probability Density Function for Two Continuous Random Variables 250
Marginal Probability Density Functions 252
Independence of Continuous Random Variables 254
More Than Two Random Variables 255
Exercises Section 7.2 (13–22) 257
7.3 Expected Values Covariance and Correlation 258
Properties of Expected Value 260
Covariance 261
Correlation 263
Correlation Versus Causation 265
Exercises Section 7.3 (23–42) 266
7.4 Properties of Linear Combinations 267
Expected Value and Variance of a Linear Combination 268
The PDF of a Sum 271
Moment Generating Functions of Linear Combinations 273
Exercises Section 7.4 (43–65) 275
7.5 The Central Limit Theorem and the Law of Large Numbers 278
Random Samples 278
The Central Limit Theorem 282
A More General Central Limit Theorem 286
Other Applications of the Central Limit Theorem 287
The Law of Large Numbers 288
Proof of the Central Limit Theorem 290
Exercises Section 7.5 (66–82) 290
7.6 Simulation of Joint Probability Distributions 293
Simulating Values from a Joint PMF 293
Simulating Values from a Joint PDF 295
Exercises Section 7.6 (83–90) 297
Supplementary Exercises (91–124) 298
8 Joint Probability Distributions:additional Topics 304
Introduction 304
8.1 Conditional Distributions and Expectation 304
Conditional Distributions and Independence 306
Conditional Expectation and Variance 307
The Laws of Total Expectation and Variance 308
Exercises Section 8.1 (1–18) 313
8.2 The Bivariate Normal Distribution 315
Conditional Distributions of X and Y 317
Regression to the Mean 318
The Multivariate Normal Distribution 319
Bivariate Normal Calculations with Software 319
Exercises Section 8.2 (19–30) 320
8.3 Transformations of Jointly Distributed Random Variables 321
The Joint Distribution of Two New Random Variables 322
The Distribution of a Single New RV 323
The Joint Distribution of More Than Two New Variables 325
Exercises Section 8.3 (31–38) 326
8.4 Reliability 327
The Reliability Function 327
Series and Parallel System Designs 329
Mean Time to Failure 331
The Hazard Function 332
Exercises Section 8.4 (39–50) 335
8.5 Order Statistics 337
The Distributions of Yn and Y1 337
The Distribution of the ith Order Statistic 339
The Joint Distribution of All n Order Statistics 340
Exercises Section 8.5 (51–60) 342
8.6 Further Simulation Tools for Jointly Distributed Random Variables 343
The Conditional Distribution Method of Simulation 343
Simulating a Bivariate Normal Distribution 344
Simulation Methods for Reliability 346
Exercises Section 8.6 (61–68) 347
Supplementary Exercises (69–82) 348
9 the Basics of Statistical Inference 351
Introduction 351
9.1 Point Estimation 351
Estimates and Estimators 352
Assessing Estimators: Accuracy and Precision 354
Exercises Section 9.1 (1–18) 357
9.2 Maximum Likelihood Estimation 360
Some Properties of MLEs 366
Exercises Section 9.2 (19–30) 367
9.3 Statistical Intervals 368
Constructing a Confidence Interval 369
Confidence Intervals for a Population Proportion 369
Confidence Intervals for a Population Mean 371
Further Comments on Statistical Intervals 375
Confidence Intervals with Software 375
Exercises Section 9.3 (31–48) 376
9.4 Hypothesis Tests 379
Hypotheses and Test Procedures 380
Hypothesis Testing for a Population Mean 381
Errors in Hypothesis Testing and the Power of a Test 385
Hypothesis Testing for a Population Proportion 388
Software for Hypothesis Test Calculations 389
Exercises Section 9.4 (49–71) 391
9.5 Bayesian Estimation 393
The Posterior Distribution of a Parameter 394
Inferences from the Posterior Distribution 397
Further Comments on Bayesian Inference 398
Exercises Section 9.5 (72–80) 399
9.6 Simulation-Based Inference 400
The Bootstrap Method 400
Interval Estimation Using the Bootstrap 402
Hypothesis Tests Using the Bootstrap 404
More on Simulation-Based Inference 405
Exercises Section 9.6 (81–90) 405
Supplementary Exercises (91–116) 407
10 Markov Chains 411
Introduction 411
10.1 Terminology and Basic Properties 411
The Markov Property 413
Exercises Section 10.1 (1–10) 416
10.2 The Transition Matrix and the Chapman–Kolmogorov Equations 418
The Transition Matrix 418
Computation of Multistep Transition Probabilities 419
Exercises Section 10.2 (11–22) 423
10.3 Specifying an Initial Distribution 426
A Fixed Initial State 428
Exercises Section 10.3 (23–30) 429
10.4 Regular Markov Chains and the Steady-State Theorem 430
Regular Chains 431
The Steady-State Theorem 432
Interpreting the Steady-State Distribution 433
Efficient Computation of Steady-State Probabilities 435
Irreducible and Periodic Chains 437
Exercises Section 10.4 (31–43) 438
10.5 Markov Chains with Absorbing States 440
Time to Absorption 441
Mean Time to Absorption 444
Mean First Passage Times 448
Probabilities of Eventual Absorption 449
Exercises Section 10.5 (44–58) 451
10.6 Simulation of Markov Chains 453
Exercises Section 10.6 (59–66) 459
Supplementary Exercises (67–82) 461
11 Random Processes 465
Introduction 465
11.1 Types of Random Processes 465
Classification of Processes 468
Random Processes and Their Associated Random Variables 469
Exercises Section 11.1 (1–10) 470
11.2 Properties of the Ensemble: Mean and Autocorrelation Functions 471
Mean and Variance Functions 471
Autocovariance and Autocorrelation Functions 475
The Joint Distribution of Two Random Processes 477
Exercises Section 11.2 (11–24) 478
11.3 Stationary and Wide-Sense Stationary Processes 479
Properties of WSS Processes 483
Ergodic Processes 486
Exercises Section 11.3 (25–40) 488
11.4 Discrete-Time Random Processes 489
Special Discrete Sequences 491
Exercises Section 11.4 (41–52) 493
Supplementary Exercises (53–64) 494
12 Families of Random Processes 497
Introduction 497
12.1 Poisson Processes 497
Relation to Exponential and Gamma Distributions 499
Combining and Decomposing Poisson Processes 502
Alternative Definition of a Poisson Process 504
Nonhomogeneous Poisson Processes 505
The Poisson Telegraphic Process 506
Exercises Section 12.1 (1–18) 507
12.2 Gaussian Processes 509
Brownian Motion 510
Brownian Motion as a Limit 512
Further Properties of Brownian Motion 512
Variations on Brownian Motion 514
Exercises Section 12.2 (19–28) 515
12.3 Continuous-Time Markov Chains 516
Infinitesimal Parameters and Instantaneous Transition Rates 518
Sojourn Times and Transitions 520
Long-Run Behavior of Continuous-Time Markov Chains 523
Explicit Form of the Transition Matrix 526
Exercises Section 12.3 (29–40) 527
Supplementary Exercises (41–51) 529
13 Introduction to Signal Processing 532
Introduction 532
13.1 Power Spectral Density 532
Expected Power and the Power Spectral Density 532
Properties of the Power Spectral Density 535
Power in a Frequency Band 538
White Noise Processes 539
Cross-Power Spectral Density for Two Processes 541
Exercises Section 13.1 (1–21) 542
13.2 Random Processes and LTI Systems 544
Properties of the LTI System Output 545
Ideal Filters 548
Signal Plus Noise 551
Exercises Section 13.2 (22–38) 554
13.3 Discrete-Time Signal Processing 556
Random Sequences and LTI Systems 558
Sampling Random Sequences 560
Exercises Section 13.3 (39–50) 562
A Statistical Tables A- 1
A. 1 Binomial CDF A- 1
A. 2 Poisson CDF A- 4
A. 3 Standard Normal CDF A- 5
A. 4 Incomplete Gamma Function A- 7
A. 5 Critical Values for t Distributions A- 7
A. 6 Tail Areas of t Distributions A- 9
B Background Mathematics A- 13
B. 1 Trigonometric Identities A- 13
B. 2 Special Engineering Functions A- 13
B. 3 o(h) Notation A- 14
B. 4 The Delta Function A- 14
B. 5 Fourier Transforms A- 15
B. 6 Discrete-Time Fourier Transforms A- 16
C Important Probability Distributions A- 18
C. 1 Discrete Distributions A- 18
C. 2 Continuous Distributions A- 20
C. 3 Matlab and R Commands A- 23
Bibliography B- 1
Answers to Odd-numbered Exercises S- 1
Index I- 1
