Applied Mathematical Sciences

0. Introduction and Summary.- 1. Discrete Time Markov Chains; Reversibility in Time.- x1.00. Introduction.- x1.0. Notation, Transition Laws.- x1.1. Irreducibility, Aperiodicity, Ergodicity; Stationary Chains.- x1.2. Approach to Ergodicity; Spectral Structure, Perron-Romanovsky-Frobenius Theorem.- x1.3. Time-Reversible Chains.- 2. Markov Chains in Continuous Time; Uniformization; Reversibility.- x2.00. Introduction.- x2.0. Notation, Transition Laws; A Review.- x2.1. Uniformizable Chains - A Bridge Between Discrete and Continuous Time Chains.- x2.2. Advantages and Prevalence of Uniformizable Chains.- x2.3. Ergodicity for Continuous Time Chains.- x2.4. Reversibility for Ergodic Markov Chains in Continuous Time.- x2.5. Prevalence of Time-Reversibility.- 3. More on Time-Reversibility; Potential Coefficients; Process Modification.- x3.00. Introduction.- x3.1. The Advantages of Time-Reversibility.- x3.2. The Spectral Representation.- x3.3. Potentials; Spectral Representation.- x3.4. More General Time-Reversible Chains.- x3.5. Process Modifications Preserving Reversibility.- x3.6. Replacement Processes.- 4. Potential Theory, Replacement, and Compensation.- x4.00. Introduction.- x4.1. The Green Potential.- x4.2. The Ergodic Distribution for a Replacement Process.- x4.3. The Compensation Method.- x4.4. Notation for the Homogeneous Random Walk.- x4.5. The Compensation Method Applied to the Homogeneous Random Walk Modified by Boundaries.- x4.6. Advantages of the Compensation Method. An Illustrative Example.- x4.7. Exploitation of the Structure of the Green Potential for the Homogeneous Random Walk.- x4.8. Similar Situations.- 5. Passage Time Densities in Birth-Death Processes; Distribution Structure.- x5.00. Introduction.- x5.1. Passage Time Densities for Birth-Death Processes.- x5.2. Passage Time Moments for a Birth-Death Process.- x5.3. PF?, Complete Monotonicity, Log-Concavity and Log-Convexity.- x5.4. Complete Monotonicity and Log-Convexity.- x5.5. Complete Monotonicity in Time-Reversible Processes.- x5.6. Some Useful Inequalities for the Families CM and PF?.- x5.7. Log-Concavity and Strong Unimodality for Lattice Distributions.- x5.8. Preservation of Log-Concavity and Log-Convexity under Tail Summation and Integration.- x5.9. Relation of CM and PF? to IFR and DFR Classes in Reliability.- 6. Passage Times and Exit Times for More General Chains.- x6.00. Introduction.- x6.1. Passage Time Densities to a Set of States.- x6.2. Mean Passage Times to a Set via the Green Potential.- x6.3. Ruin Probabilities via the Green Potential.- x6.4. Ergodic Flow Rates in a Chain.- x6.5. Ergodic Exit Times, Ergodic Sojourn Times, and Quasi-Stationary Exit Times.- x6.6. The Quasi-Stationary Exit Time. A Limit Theorem.- x6.7. The Connection Between Exit Times and Sojourn Times. A Renewal Theorem.- x6.8. A Comparison of the Mean Ergodic Exit Time and Mean Ergodic Sojourn Time for Arbitrary Chains.- x6.9. Stochastic Ordering of Exit Times of Interest for Time-Reversible Chains.- x6.10. Superiority of the Exit Time as System Failure Time; Jitter.- 7. The Fundamental Matrix, and Allied Topics.- x7.00. Introduction.- x7.1. The Fundamental Matrix for Ergodic Chains.- x7.2. The Structure of the Fundamental Matrix for Time-Reversible Chains.- x7.3. Mean Failure Times and Ruin Probabilities for Systems with Independent Markov Components and More General Chains.- x7.4. Covariance and Spectral Density Structure for Time-Reversible Processes.- x7.5. A Central Limit Theorem.- x7.6. Regeneration Times and Passage Times-Their Relation For Arbitrary Chains.- x7.7. Passage to a Set with Two States.- 8. Rarity and Exponentiality.- x8.0. Introduction.- x8.1. Passage Time Density Structure for Finite Ergodic Chains; the Exponential Approximation.- x8.2. A Limit Theorem for Ergodic Regenerative Processes.- x8.3. Prototype Behavior: Birth-Death Processes; Strongly Stable Systems.- x8.4. Limiting Behavior of the Ergodic and Quasi-stationary Exit Time Densities and Sojourn Time Densities for Birth-Death Processes.- x8.5. Limit Behavior of Other Exit Times for More General Chains.- x8.6. Strongly Stable Chains, Jitter; Estimation of the Failure Time Needed for the Exponential Approximation.- x8.7. A Measure of Exponentiality in the Completely Monotone Class of Densities.- x8.8. An Error Bound for Departure from Exponentiality in the Completely Monotone Class.- x8.9. The Exponential Approximation for Time-Reversible Systems.- x8.10. A Relaxation Time of Interest.- 9. Stochastic Monotonicity.- x9.00. Introduction.- x9.1. Monotone Markov Matrices and Monotone Chains.- x9.2. Some Monotone Chains in Discrete Time.- x9.3. Monotone Chains in Continuous Time.- x9.4. Other Monotone Processes in Continuous Time.- References.