Probability

Prelude: Random Walks.- The Model.- Random variables.- Probability.- First calculations.- Issues and Approaches.- Issues.- Approaches.- Tools.- Functional of the Random Walk.- Times of returns to the origin.- Numbers of returns to the origin.- First passage times.- Maxima.- Time spent positive.- Limit Theorems.- Summary.- 1 Probability.- 1.1 Random Experiments and Sample Spaces.- 1.1.1 Random experiments.- 1.1.2 Sample spaces.- 1.2 Events and Classes of Sets.- 1.2.1 Events.- 1.2.2 Basic set operations.- 1.2.3 Indicator functions.- 1.2.4 Operations on sequences of sets.- 1.2.5 Classes of sets closed under set operations.- 1.2.6 Generated classes.- 1.2.7 The monotone class theorem.- 1.2.8 Events, bis.- 1.3 Probabilities and Probability Spaces.- 1.3.1 Probability.- 1.3.2 Elementary properties.- 1.3.3 More advanced properties.- 1.3.4 Almost sure and null events.- 1.3.5 Uniqueness.- 1.4 Probabilities on R.- 1.4.1 Distribution functions.- 1.4.2 Discrete probabilities.- 1.4.3 Absolutely continuous probabilities.- 1.4.4 Mixed distributions.- 1.5 Conditional Probability Given a Set.- 1.6 Complements.- 1.6.1 The extended real numbers.- 1.6.2 Measures.- 1.6.3 Lebesgue measure.- 1.6.4 Singular probabilities on R.- 1.6.5 Representation of probabilities on R.- 1.7 Exercises.- 2 Random Variables.- 2.1 Fundamentals.- 2.1.1 Random variables.- 2.1.2 Random vectors.- 2.1.3 Stochastic processes.- 2.1.4 Complex-valued random variables.- 2.1.5 The -algebra generated by a random variable.- 2.1.6 Simplified criteria.- 2.2 Combining Random Variables.- 2.2.1 Algebraic operations.- 2.2.2 Limiting operations.- 2.2.3 Transformations.- 2.2.4 Approximation of positive random variables.- 2.2.5 Monotone class theorems.- 2.3 Distributions and Distribution Functions.- 2.3.1 Random variables.- 2.3.2 Random vectors.- 2.4 Key Random Variables and Distributions.- 2.4.1 Discrete random variables.- 2.4.2 Absolutely continuous random variables.- 2.4.3 Random vectors.- 2.5 Transformation Theory.- 2.5.1 Random variables.- 2.5.2 Random vectors.- 2.6 Random Variables with Prescribed Distributions.- 2.6.1 Individual random variables.- 2.6.2 Random vectors.- 2.6.3 Sequences of random variables.- 2.7 Complements.- 2.7.1 Measurability with respect to sub-?-algebras.- 2.7.2 Borel measurable functions.- 2.8 Exercises.- 3 Independence.- 3.1 Independent Random Variables.- 3.1.1 Fundamentals.- 3.1.2 Criteria for independence.- 3.1.3 Examples.- 3.2 Functions of Independent Random Variables.- 3.2.1 Transformation properties.- 3.2.2 Sums of independent random variables.- 3.3 Constructing Independent Random Variables.- 3.3.1 Finite families.- 3.3.2 Sequences.- 3.4 Independent Events.- 3.5 Occupancy Models.- 3.5.1 Four occupancy models.- 3.5.2 Occupancy numbers.- 3.5.3 Asymptotics.- 3.6 Bernoulli and Poisson Processes.- 3.6.1 Bernoulli processes.- 3.6.2 Poisson processes.- 3.7 Complements.- 3.7.1 Independent ?-algebras.- 3.7.2 Products of probability spaces.- 3.8 Exercises.- 4 Expectation.- 4.1 Definition and Fundamental Properties.- 4.1.1 Simple random variables.- 4.1.2 Positive random variables.- 4.1.3 Integrable random variables.- 4.1.4 Complex-valued random variables.- 4.2 Integrals with respect to Distribution Functions.- 4.2.1 Generalities.- 4.2.2 Discrete distribution functions.- 4.2.3 Absolutely continuous distribution functions.- 4.2.4 Mixed distribution functions.- 4.3 Computation of Expectations.- 4.3.1 Positive random variables.- 4.3.2 Integrable random variables.- 4.3.3 Functions of random variables.- 4.3.4 Functions of random vectors.- 4.3.5 Functions of independent random variables.- 4.3.6 Sums of independent random variables.- 4.4 LP Spaces and Inequalities.- 4.4.1 LPspaces.- 4.4.2 Key inequalities.- 4.5 Moments.- 4.5.1 Moments of random variables.- 4.5.2 Variance and standard deviation.- 4.5.3 Covariance and correlation.- 4.5.4 Moments of random vectors.- 4.5.5 Multivariate normal distributions.- 4.6 Complements.- 4.6.1 Integration with respect to Lebesgue measure.- 4.6.2 Expectation for product probabilities.- 4.7 Exercises.- 5 Convergence of Sequences of Random Variables.- 5.1 Modes of Convergence.- 5.1.1 Convergence of random variables as functions.- 5.1.2 Convergence of distribution functions.- 5.1.3 Alternative criteria.- 5.2 Relationships Among the Modes.- 5.2.1 Implications always valid.- 5.2.2 Counterexamples.- 5.2.3 Implications of restricted validity.- 5.2.4 Implications involving subsequences.- 5.3 Convergence under Transformations.- 5.3.1 Algebraic operations.- 5.3.2 Continuous mappings.- 5.4 Convergence of Random Vectors.- 5.4.1 Convergence of random vectors as functions.- 5.4.2 Convergence in distribution.- 5.4.3 Continuous mappings.- 5.5 Limit Theorems for Bernoulli Summands.- 5.5.1 Laws of large numbers.- 5.5.2 Central limit theorems.- 5.5.3 The Poisson limit theorem.- 5.5.4 Approximation of continuous functions.- 5.6 Complements.- 5.6.1 LP Convergence of random variables.- 5.7 Exercises.- 6 Characteristic Functions.- 6.1 Definition and Basic Properties.- 6.1.1 Fundamentals.- 6.1.2 Elementary properties.- 6.2 Inversion and Uniqueness Theorems.- 6.2.1 The inversion theorem.- 6.2.2 The uniqueness theorem.- 6.2.3 Specialized inversion theorems.- 6.3 Moments and Taylor Expansions.- 6.3.1 Calculation of moments known to exist.- 6.3.2 Establishing existence of moments.- 6.3.3 Taylor expansions of characteristic functions.- 6.4 Continuity Theorems and Applications.- 6.4.1 Convergence in distribution.- 6.4.2 The Levy continuity theorem.- 6.4.3 Application to classical limit theorems.- 6.5 Other Transforms.- 6.5.1 Characteristic functions of random vectors.- 6.5.2 Laplace transforms.- 6.5.3 Moment generating functions.- 6.5.4 Generating functions.- 6.6 Complements.- 6.6.1 Helly's theorem.- 6.7 Exercises.- 7 Classical Limit Theorems.- 7.1 Series of Independent Random Variables.- 7.1.1 Kolmogorov's inequality.- 7.1.2 The three series theorem.- 7.2 The Strong Law of Large Numbers.- 7.3 The Central Limit Theorem.- 7.3.1 The Lyapunov condition.- 7.3.2 The Lindeberg condition.- 7.4 The Law of the Iterated Logarithm.- 7.4.1 Normally distributed summands.- 7.4.2 More general versions.- 7.5 Applications of the Limit Theorems.- 7.5.1 Monte Carlo integration.- 7.5.2 Maximum likelihood estimation.- 7.5.3 Empirical distribution functions.- 7.5.4 Random sums of independent random variables.- 7.5.5 Renewal processes.- 7.6 Complements.- 7.6.1 The Berry-Esseen theorem.- 7.7 Exercises.- 8 Prediction and Conditional Expectation.- 8.1 Prediction in L2.- 8.1.1 The inner product and norm.- 8.1.2 L2 as metric space.- 8.1.3 Orthogonality and orthonormality.- 8.1.4 The orthogonal decomposition theorem.- 8.1.5 Computation of MMSE predictors.- 8.1.6 Linear prediction.- 8.2 Conditional Expectation Given a Finite Set of Random Variables.- 8.2.1 Basics.- 8.2.2 Examples.- 8.2.3 Conditional probability.- 8.3 Conditional Expectation for X?L2.- 8.3.1 Conditional expectation as MMSE prediction.- 8.3.2 Properties of conditional expectation.- 8.4 Positive and Integrable Random Variables.- 8.5 Conditional Distributions.- 8.5.1 Generalities.- 8.5.2 Discrete random variables.- 8.5.3 Absolutely continuous random variables.- 8.6 Computational Techniques.- 8.6.1 General results.- 8.6.2 Special cases.- 8.7 Complements.- 8.7.1 Mixed conditional distributions.- 8.7.2 Conditional expectation given a ?-algebra.- 8.8 Exercises.- 9 Martingales.- 9.1 Fundamentals.- 9.1.1 Definitions.- 9.1.2 Examples.- 9.1.3 Compositions and transformations.- 9.2 Stopping Times.- 9.3 Optional Sampling Theorems.- 9.3.1 Optional sampling theorems for martingales.- 9.3.2 Applications of optional sampling theorems.- 9.4 Martingale Convergence Theorems.- 9.4.1 Upcrossings and almost sure convergence.- 9.4.2 Almost sure convergence of submartingales.- 9.4.3 Almost sure convergence of martingales.- 9.4.4 Uniformly integrable martingales.- 9.5 Applications of Convergence Theorems.- 9.5.1 The Radon-Nikodym theorem.- 9.5.2 Zero-one laws.- 9.5.3 Likelihood ratios.- 9.6 Complements.- 9.6.1 Conditioning on Yo,...,YT.- 9.6.2 Martingales with respect to filtrations.- 9.6.3 Reversed martingales.- 9.7 Exercises.- A Notation.- B Named Objects.