Markov Chains

I. Discrete time.- 1. Introduction to Discrete Time.- 1. Foreword.- 2. Summary.- 3. The Markov and strong Markov properties.- 4. Classification of states.- 5. Recurrence.- 6. The renewal theorem.- 7. The limits of Pn.- 8. Positive recurrence.- 9. Invariant probabilities.- 10. The Bernoulli walk.- 11. Forbidden transitions.- 12. The Harris walk.- 13. The tail ?-field and a theorem of Orey.- 14. Examples.- 2. Ratio Limit Theorems.- 1. Introduction.- 2. Reversal of time.- 3. Proofs of Derman and Doeblin.- 4. Variations.- 5. Restricting the range.- 6. Proof of Kingman-Orey.- 7. An example of Dyson.- 8. Almost everywhere ratio limit theorems.- 9. The sum of a function over different j-blocks.- 3. Some Invariance Principles.- 1. Introduction.- 2. Estimating the partial sums.- 3. The number of positive sums.- 4. Some invariance principles.- 5. The concentration function.- 4. The Boundary.- 1. Introduction.- 2. Proofs.- 3. A convergence theorem.- 4. Examples.- 5. The last visit to i before the first visit to J{i}.- II. Continuous time.- 5. Introduction to Continuous Time.- 1. Semigroups and processes.- 2. Analytic properties.- 3. Uniform semigroups.- 4. Uniform substochastic semigroups.- 5. The exponential distribution.- 6. The step function case.- 7. The uniform case.- 6. Examples for the Stable Case.- 1. Introduction.- 2. The first construction.- 3. Examples on the first construction.- 4. The second construction.- 5. Examples on the second construction.- 6. Markov times.- 7. Crossing the infinities.- 7. The Stable Case.- 1. Introduction.- 2. Regular sample functions.- 3. The post-exit process.- 4. The strong Markov property.- 5. The minimal solution.- 6. The backward and forward equations.- 8. More Examples for the Stable Case.- 1. An oscillating semigroup.- 2. A semigroup with an infinite second derivative.- 3. Large oscillations in P(t, 1, 1).- 4. An example of Speakman.- 5. The embedded jump process is not Markov.- 6. Isolated infinities.- 7. The set of infinities is bad.- 9. The General Case.- 1. An example of Blackwell.- 2. Quasiregular sample functions.- 3. The sets of constancy.- 4. The strong Markov property.- 5. The post-exit process.- 6. The abstract case.- III..- 10. Appendix.- 1. Notation.- 2. Numbering.- 3. Bibliography.- 4. The abstract Lebesgue integral.- 5. Atoms.- 6. Independence.- 7. Conditioning.- 8. Martingales.- 9. Metric spaces.- 10. Regular conditional distributions.- 11. The Kolmogorov consistency theorem.- 12. The diagonal argument.- 13. Classical Lebesgue measure.- 14. Real variables.- 15. Absolute continuity.- 16. Convex functions.- 17. Complex variables.- Symbol Finder.