| List of figures | p. xi |
| Prologue to the Second edition, Peter W. Glynn | p. xiii |
| Preface to the second edition, Sean Meyn | p. xvii |
| Preface to the first edition | p. xxi |
| Communication and Regeneration | p. 1 |
| Heuristics | p. 3 |
| A range of Markovian environments | p. 3 |
| Basic models in practice | p. 6 |
| Stochastic stability for Markov models | p. 13 |
| Commentary | p. 19 |
| Markov models | p. 21 |
| Markov models in time series | p. 22 |
| Nonlinear state space models | p. *26 |
| Models in control and systems theory | p. 33 |
| Markov models with regeneration times | p. 38 |
| Commentary | p. *46 |
| Transition probabilities | p. 48 |
| Defining a Markovian Process | p. 49 |
| Foundations on a countable space | p. 51 |
| Specific transition matrices | p. 54 |
| Foundations for general state space chains | p. 59 |
| Building transition kernels for specific models | p. 67 |
| Commentary | p. 72 |
| Irreducibility | p. 75 |
| Communication and irreducibility: Countable spaces | p. 76 |
| -Irreducibility | p. 81 |
| -Irreducibility for random walk models | p. 87 |
| -Irreducible linear models | p. 89 |
| Commentary | p. 93 |
| Pseudo-atoms | p. 96 |
| Splitting -irreducible chains | p. 97 |
| Small sets | p. 102 |
| Small sets for specific models | p. 106 |
| Cyclic behavior | p. 110 |
| Petite sets and sampled chains | p. 115 |
| Commentary | p. 121 |
| Topology and continuity | p. 123 |
| Feller properties and forms of stability | p. 125 |
| T-chains | p. 130 |
| Continuous components for specific models | p. 134 |
| e-Chains | p. 139 |
| Commentary | p. 144 |
| The nonlinear state space model | p. 146 |
| Forward accessibility and continuous components | p. 147 |
| Minimal sets and irreducibility | p. 154 |
| Periodicity for nonlinear state space models | p. 157 |
| Forward accessible examples | p. 161 |
| Equicontinuity and the nonlinear state space model | p. 163 |
| Commentary | p. *165 |
| Stability Structures | p. 169 |
| Transience and recurrence | p. 171 |
| Classifying chains on countable spaces | p. 173 |
| Classifying -irreducible chains | p. 177 |
| Recurrence and transience relationships | p. 182 |
| Classification using drift criteria | p. 187 |
| Classifying random walk on R+ | p. 193 |
| Commentary | p. *197 |
| Harris and topological recurrence | p. 199 |
| Harris recurrence | p. 201 |
| Non-evanescent and recurrent chains | p. 206 |
| Topologically recurrent and transient states | p. 208 |
| Criteria for stability on a topological space | p. 213 |
| Stochastic comparison and increment analysis | p. 218 |
| Commentary | p. 228 |
| The existence of | p. 229 |
| Stationarity and invariance | p. 230 |
| The existence of : chains with atoms | p. 234 |
| Invariant measures for countable space models | p. *236 |
| The existence of : -irreducible chains | p. 241 |
| Invariant measures for general models | p. 247 |
| Commentary | p. 253 |
| Drift and regularity | p. 256 |
| Regular chains | p. 258 |
| Drift, hitting times and deterministic models | p. 261 |
| Drift, criteria for regularity | p. 263 |
| Using the regularity criteria | p. 272 |
| Evaluating non-positivity | p. 278 |
| Commentary | p. 285 |
| Invariance and tightness | p. 288 |
| Chains bounded in probability | p. 289 |
| Generalized sampling and invariant measures | p. 292 |
| The existence of a -finite invariant measure | p. 298 |
| Invariant measures for e-chains | p. 300 |
| Establishing boundedness in probability | p. 305 |
| Commentary | p. 308 |
| Convergence | p. 311 |
| Ergodicity | p. 313 |
| Ergodic chains on countable spaces | p. 316 |
| Renewal and regeneration | p. 320 |
| Ergodicity of positive Harris chains | p. 326 |
| Sums of transition probabilities | p. 329 |
| Commentary | p. *334 |
| f-Ergodicity and f-regularity | p. 336 |
| f-Properties: chains with atoms | p. 338 |
| f-Regularity and drift | p. 342 |
| f-Ergodicity for general chains | p. 349 |
| f-Ergodicity of specific models | p. 352 |
| A key renewal theorem | p. 354 |
| Commentary | p. 359 |
| Geometric ergodicity | p. 362 |
| Geometric properties: chains with atoms | p. 364 |
| Kendall sets and drift criteria | p. 372 |
| f-Geometric regularity of and its skeleton | p. 380 |
| f-Geometric ergodicity for general chains | p. 384 |
| Simple random walk and linear models | p. 388 |
| Commentary | p. *390 |
| V-Uniform ergodicity | p. 392 |
| Operator norm convergence | p. 395 |
| Uniform ergodicity | p. 400 |
| Geometric ergodicity and increment analysis | p. 407 |
| Models from queueing theory | p. 411 |
| Autoregressive and state space models | p. 414 |
| Commentary | p. *418 |
| Sample paths and limit theorems | p. 421 |
| Invariant -fields and the LLN | p. 423 |
| Ergodic theorems for chains possessing an atom | p. 428 |
| General Harris chains | p. 433 |
| The functional CLT | p. 443 |
| Criteria for the CLT and the LIL | p. 450 |
| Applications | p. 454 |
| Commentary | p. *456 |
| Positivity | p. 462 |
| Null recurrent chains | p. 464 |
| Characterizing positivity using Pn | p. 469 |
| Positivity and T-chains | p. 471 |
| Positivity and e-chains | p. 473 |
| The LLN for e-chains | p. 477 |
| Commentary | p. *480 |
| Generalized classification criteria | p. 482 |
| State-dependent drifts | p. 483 |
| History-dependent drift criteria | p. 491 |
| Mixed drift conditions | p. 498 |
| Commentary | p. *508 |
| Epilogue to the second edition | p. 510 |
| Geometric ergodicity and spectral theory | p. 510 |
| Simulation and MCMC | p. 521 |
| Continuous time models | p. 523 |
| Appendices | p. 529 |
| Mud maps | p. 532 |
| Recurrence versus transience | p. 532 |
| Positivity versus nullity | p. 534 |
| Convergence properties | p. 536 |
| Testing for stability | p. 538 |
| Glossary of drift conditions | p. 538 |
| The Scalar SETAR model: a complete classification | p. 540 |
| Glossary of models assumptions | p. 543 |
| Regenerative models | p. 543 |
| State space models | p. 546 |
| Some mathematical background | p. 552 |
| Some measure theory | p. 552 |
| Some probability theory | p. 555 |
| Some topology | p. 556 |
| Some real analysis | p. 557 |
| Convergence concepts for measures | p. 558 |
| Some martingale theory | p. 561 |
| Some results on sequences and numbers | p. 563 |
| Bibliography | p. 567 |
| Indexes | p. 587 |
| General index | p. 587 |
| Symbols | p. 593 |
| Table of Contents provided by Ingram. All Rights Reserved. |
