FOREWORD xi
PREFACE xiii
Patrick Billingsley 1925â"2011 xv
Chapter1 PROBABILITY 1
1. BORELâS NORMAL NUMBER THEOREM, 1
The Unit Interval
The Weak Law of Large Numbers
The Strong Law of Large Numbers
Strong Law Versus Weak
Length
The Measure Theory of Diophantine Approximation*
2. PROBABILITY MEASURES, 18
Spaces
Assigning Probabilities
Classes of Sets
Probability Measures
Lebesgue Measure on the Unit Interval
Sequence Space*
Constructing s-Fields*
3. EXISTENCE AND EXTENSION, 39
Construction of the Extension
Uniqueness and the pâ"? Theorem
Monotone Classes
Lebesgue Measure on the Unit Interval
Completeness
Nonmeasurable Sets
Two Impossibility Theorems*
4. DENUMERABLE PROBABILITIES, 53
General Formulas
Limit Sets
Independent Events
Subfields
The Borel-Cantelli Lemmas
The Zero-One Law
5. SIMPLE RANDOM VARIABLES, 72
Definition
Convergence of Random Variables
Independence
Existence of Independent Sequences
Expected Value
Inequalities
6. THE LAW OF LARGE NUMBERS, 90
The Strong Law
The Weak Law
Bernstein's Theorem
A Refinement of the Second Borel-Cantelli Lemma
7. GAMBLING SYSTEMS, 98
Gambler's Ruin
Selection Systems
Gambling Policies
Bold Play*
Timid Play*
8. MARKOV CHAINS, 117
Definitions
Higher-Order Transitions
An Existence Theorem
Transience and Persistence
Another Criterion for Persistence
Stationary Distributions
Exponential Convergence*
Optimal Stopping*
9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM, 154
Moment Generating Functions
Large Deviations
Chernoff's Theorem*
The Law of the Iterated Logarithm
Chapter2 MEASURE 167
10. GENERAL MEASURES, 167
Classes of Sets
Conventions Involving 8
Measures
Uniqueness
11. OUTER MEASURE, 174
Outer Measure
Extension
An Approximation Theorem
12. MEASURES IN EUCLIDEAN SPACE, 181
Lebesgue Measure
Regularity
Specifying Measures on the Line
Specifying Measures in Rk
Strange Euclidean Sets*
13. MEASURABLE FUNCTIONS AND MAPPINGS, 192
Measurable Mappings
Mappings into Rk
Limits and Measurability
Transformations of Measures
14. DISTRIBUTION FUNCTIONS, 198
Distribution Functions
Exponential Distributions
Weak Convergence
Convergence of Types*
Extremal Distributions*
Chapter3 INTEGRATION 211
15. THE INTEGRAL, 211
Definition
Nonnegative Functions
Uniqueness
16. PROPERTIES OF THE INTEGRAL, 218
Equalities and Inequalities
Integration to the Limit
Integration over Sets
Densities
Change of Variable
Uniform Integrability
Complex Functions
17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234
The Lebesgue Integral on the Line
The Riemann Integral
The Fundamental Theorem of Calculus
Change of Variable
The Lebesgue Integral in Rk
Stieltjes Integrals
18. PRODUCT MEASURE AND FUBINIâS THEOREM, 245
Product Spaces
Product Measure
Fubini's Theorem
Integration by Parts
Products of Higher Order
19. THE Lp SPACES*, 256
Definitions
Completeness and Separability
Conjugate Spaces
Weak Compactness
Some Decision Theory
The Space L2
An Estimation Problem
Chapter4 RANDOM VARIABLES AND EXPECTED VALUES 271
20. RANDOM VARIABLES AND DISTRIBUTIONS, 271
Random Variables and Vectors
Subfields
Distributions
Multidimensional Distributions
Independence
Sequences of Random Variables
Convolution
Convergence in Probability
The Glivenko-Cantelli Theorem*
21. EXPECTED VALUES, 291
Expected Value as Integral
Expected Values and Limits
Expected Values and Distributions
Moments
Inequalities
Joint Integrals
Independence and Expected Value
Moment Generating Functions
22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300
The Strong Law of Large Numbers
The Weak Law and Moment Generating Functions
Kolmogorov's Zero-One Law
Maximal Inequalities
Convergence of Random Series
Random Taylor Series*
23. THE POISSON PROCESS, 316
Characterization of the Exponential Distribution
The Poisson Process
The Poisson Approximation
Other Characterizations of the Poisson Process
Stochastic Processes
24. THE ERGODIC THEOREM*, 330
Measure-Preserving Transformations
Ergodicity
Ergodicity of Rotations
Proof of the Ergodic Theorem
The Continued-Fraction Transformation
Diophantine Approximation
Chapter5 CONVERGENCE OF DISTRIBUTIONS 349
25. WEAK CONVERGENCE, 349
Definitions
Uniform Distribution Modulo 1*
Convergence in Distribution
Convergence in Probability
Fundamental Theorems
Helly's Theorem
Integration to the Limit
26. CHARACTERISTIC FUNCTIONS, 365
Definition
Moments and Derivatives
Independence
Inversion and the Uniqueness Theorem
The Continuity Theorem
Fourier Series*
27. THE CENTRAL LIMIT THEOREM, 380
Identically Distributed Summands
The Lindeberg and Lyapounov Theorems
Dependent Variables*
28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394
Vague Convergence
The Possible Limits
Characterizing the Limit
29. LIMIT THEOREMS IN Rk, 402
The Basic Theorems
Characteristic Functions
Normal Distributions in Rk
The Central Limit Theorem
30. THE METHOD OF MOMENTS*, 412
The Moment Problem
Moment Generating Functions
Central Limit Theorem by Moments
Application to Sampling Theory
Application to Number Theory
Chapter6 DERIVATIVES AND CONDITIONAL PROBABILITY 425
31. DERIVATIVES ON THE LINE*, 425
The Fundamental Theorem of Calculus
Derivatives of Integrals
Singular Functions
Integrals of Derivatives
Functions of Bounded Variation
32. THE RADONâ"NIKODYM THEOREM, 446
Additive Set Functions
The Hahn Decomposition
Absolute Continuity and Singularity
The Main Theorem
33. CONDITIONAL PROBABILITY, 454
The Discrete Case
The General Case
Properties of Conditional Probability
Difficulties and Curiosities
Conditional Probability Distributions
34. CONDITIONAL EXPECTATION, 472
Definition
Properties of Conditional Expectation
Conditional Distributions and Expectations
Sufficient Subfields*
Minimum-Variance Estimation*
35. MARTINGALES, 487
Definition
Submartingales
Gambling
Functions of Martingales
Stopping Times
Inequalities
Convergence Theorems
Applications: Derivatives
Likelihood Ratios
Reversed Martingales
Applications: de Finetti's Theorem
Bayes Estimation
A Central Limit Theorem*
Chapter7 STOCHASTIC PROCESSES 513
36. KOLMOGOROV'S EXISTENCE THEOREM, 513
Stochastic Processes
Finite-Dimensional Distributions
Product Spaces
Kolmogorov's Existence Theorem
The Inadequacy of RT
A Return to Ergodic Theory
The Hewittâ"Savage Theorem*
37. BROWNIAN MOTION, 530
Definition
Continuity of Paths
Measurable Processes
Irregularity of Brownian Motion Paths
The Strong Markov Property
The Reflection Principle
Skorohod Embedding
Invariance*
38. NONDENUMERABLE PROBABILITIES, 558
Introduction
Definitions
Existence Theorems
Consequences of Separability*
APPENDIX 571
NOTES ON THE PROBLEMS 587
BIBLIOGRAPHY 617
INDEX 619
