Features of this Text i
Preface vii
1 Experiments, Models, and Probabilities 1
Getting Started with Probability 1
1.1 Set Theory 3
1.2 Applying Set Theory to Probability 7
1.3 Probability Axioms 11
1.4 Conditional Probability 15
1.5 Partitions and the Law of Total Probability 18
1.6 Independence 24
1.7 Matlab 27
Problems 29
2 Sequential Experiments 35
2.1 Tree Diagrams 35
2.2 Counting Methods 40
2.3 Independent Trials 49
2.4 Reliability Analysis 52
2.5 Matlab 55
Problems 57
3 Discrete Random Variables 62
3.1 Definitions 62
3.2 Probability Mass Function 65
3.3 Families of Discrete Random Variables 68
3.4 Cumulative Distribution Function (CDF) 77
3.5 Averages and Expected Value 80
3.6 Functions of a Random Variable 86
3.7 Expected Value of a Derived Random Variable 90
3.8 Variance and Standard Deviation 93
3.9 Matlab 99
Problems 106
4 Continuous Random Variables 118
4.1 Continuous Sample Space 118
4.2 The Cumulative Distribution Function 121
4.3 Probability Density Function 123
4.4 Expected Values 128
4.5 Families of Continuous Random Variables 132
4.6 Gaussian Random Variables 138
4.7 Delta Functions, Mixed Random Variables 145
4.8 Matlab 152
Problems 154
5 Multiple Random Variables 162
5.1 Joint Cumulative Distribution Function 163
5.2 Joint Probability Mass Function 166
5.3 Marginal PMF 169
5.4 Joint Probability Density Function 171
5.5 Marginal PDF 177
5.6 Independent Random Variables 178
5.7 Expected Value of a Function of Two Random Variables 181
5.8 Covariance, Correlation and Independence 184
5.9 Bivariate Gaussian Random Variables 191
5.10 Multivariate Probability Models 195
5.11 Matlab 201
Problems 206
6 Probability Models of Derived Random Variables 218
6.1 PMF of a Function of Two Discrete Random Variables 219
6.2 Functions Yielding Continuous Random Variables 220
6.3 Functions Yielding Discrete or Mixed Random Variables 226
6.4 Continuous Functions of Two Continuous Random Variables 229
6.5 PDF of the Sum of Two Random Variables 232
6.6 Matlab 234
Problems 236
7 Conditional Probability Models 242
7.1 Conditioning a Random Variable by an Event 242
7.2 Conditional Expected Value Given an Event 248
7.3 Conditioning Two Random Variables by an Event 252
7.4 Conditioning by a Random Variable 256
7.5 Conditional Expected Value Given a Random Variable 262
7.6 Bivariate Gaussian Random Variables: Conditional PDFs 265
7.7 Matlab 268
Problems 269
8 Random Vectors 277
8.1 Vector Notation 277
8.2 Independent Random Variables and Random Vectors 280
8.3 Functions of Random Vectors 281
8.4 Expected Value Vector and Correlation Matrix 285
8.5 Gaussian Random Vectors 291
8.6 Matlab 298
Problems 300
9 Sums of Random Variables 306
9.1 Expected Values of Sums 306
9.2 Moment Generating Functions 310
9.3 MGF of the Sum of Independent Random Variables 314
9.4 Random Sums of Independent Random Variables 317
9.5 Central Limit Theorem 321
9.6 Matlab 328
Problems 331
10 The Sample Mean 337
10.1 Sample Mean: Expected Value and Variance 337
10.2 Deviation of a Random Variable from the Expected Value 339
10.3 Laws of Large Numbers 343
10.4 Point Estimates of Model Parameters 345
10.5 Confidence Intervals 352
10.6 Matlab 358
Problems 360
11 Hypothesis Testing 366
11.1 Significance Testing 367
11.2 Binary Hypothesis Testing 370
11.3 Multiple Hypothesis Test 384
11.4 Matlab 387
Problems 389
12 Estimation of a Random Variable 399
12.1 Minimum Mean Square Error Estimation 400
12.2 Linear Estimation of X given Y 404
12.3 MAP and ML Estimation 409
12.4 Linear Estimation of Random Variables from Random Vectors 414
12.5 Matlab 421
Problems 423
13 Stochastic Processes 429
13.1 Definitions and Examples 430
13.2 Random Variables from Random Processes 435
13.3 Independent, Identically Distributed Random Sequences 437
13.4 The Poisson Process 439
13.5 Properties of the Poisson Process 443
13.6 The Brownian Motion Process 446
13.7 Expected Value and Correlation 448
13.8 Stationary Processes 452
13.9 Wide Sense Stationary Stochastic Processes 455
13.10 Cross-Correlation 459
13.11 Gaussian Processes 462
13.12 Matlab 464
Problems 468
Appendix A Families of Random Variables 477
A.1 Discrete Random Variables 477
A.2 Continuous Random Variables 479
Appendix B A Few Math Facts 483
References 489
Index 491
