Continuous-Time Markov Decision Processes

Theory and Applications

By: Xianping Guo, OnÃ©simo HernÃ¡ndez-Lerma

Write A Review

Hardcover | 1 October 2009

At a Glance

Hardcover
254 Pages

Dimensions(cm)
23.5 x 15.88 x 2.54

Hardcover

$169.00

or 4 interest-free payments of $42.25 with

or

Ships in 5 to 7 business days

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

Industry Reviews

From the reviews:

"The book consists of 12 chapters. ... this is the first monograph on continuous-time Markov decision process. ... This is an important book written by leading experts on a mathematically rich topic which has many applications to engineering, business, and biological problems. ... scholars and students interested in developing the theory of continuous-time Markov decision Processes or working on their applications should have this book." (E. A. Feinberg, Mathematical Reviews, Issue 2011 b)

Shipping

	Standard Shipping	Express Shipping
Metro postcodes:	$9.99	$14.95
Regional postcodes:	$9.99	$14.95
Rural postcodes:	$9.99	$14.95

Orders over $79.00 qualify for free shipping.

How to return your order

At Booktopia, we offer hassle-free returns in accordance with our returns policy. If you wish to return an item, please get in touch with Booktopia Customer Care.

Additional postage charges may be applicable.

Defective items

If there is a problem with any of the items received for your order then the Booktopia Customer Care team is ready to assist you.

For more info please visit our Help Centre.

More in Mathematics

Speed

How it Explains the World

Paperback

RRP $36.99

$29.75

20%
OFF

The Infinite Game

From the bestselling author of Start With Why

Paperback

RRP $29.99

$24.99

17%
OFF

Primary Mathematics

4th Edition - Integrating Theory with Practice

Paperback

RRP $122.95

$102.99

16%
OFF

The Art of Gathering

How We Meet and Why It Matters

Paperback

RRP $26.99

$22.99

15%
OFF

Collins Classics - The Republic

Collins Classics

Paperback

RRP $9.99

$9.75

How to Win At Chess

The Ultimate Guide for Beginners and Beyond

Hardcover

RRP $45.00

$35.75

21%
OFF

The Maths Book

Big Ideas Simply Explained

Hardcover

RRP $42.99

$32.99

23%
OFF

The Selfish Gene

40th Anniversary edition

Paperback

RRP $32.95

$26.99

18%
OFF

The Mending of Broken Bones

A Modern Guide to Classical Algebra

Hardcover

RRP $51.95

$40.75

22%
OFF

Fooled By Randomness

The Hidden Role of Chance in Life and in the Markets

Paperback

RRP $26.99

$22.99

15%
OFF

$Math with Bad Drawings : Illuminating the Ideas That Shape Our Reality - Ben Orlin$

Math with Bad Drawings

Illuminating the Ideas That Shape Our Reality

Paperback

RRP $32.99

$26.99

18%
OFF

$Math Games with Bad Drawings : 75 1/4 Simple, Challenging, Go-Anywhere Games & And Why They Matter - Ben Orlin$

$41.75

24%
OFF

Geometry

Grades 6 - 8

Paperback

RRP $24.99

$18.75

25%
OFF

Grade 3 Geometry and Measurement

Paperback

RRP $16.99

$13.75

19%
OFF

Intro to Geometry

Grades 6 - 8

Paperback

RRP $24.99

$18.99

24%
OFF

Algebra Workbook Grades 6-8 : Algebra - Kumon

$24.99

17%
OFF

Oxford Maths for Australian Schools Year 4 Value Pack : 3rd Edition - Annie Facchinetti

$61.75

Maths in Focus 11 Mathematics Advanced Student Book with Nelson MindTap - Margaret Grove

$13.75

19%
OFF

Pre-Algebra Workbook Grades 6-8

Algebra

Paperback

RRP $24.99

$18.99

24%
OFF

Introduction and Summary	p. 1
Introduction	p. 1
Preliminary Examples	p. 1
Summary of the Following Chapters	p. 6
Continuous-Time Markov Decision Processes	p. 9
Introduction	p. 9
The Control Model	p. 10
Continuous-Time Markov Decision Processes	p. 13
Basic Optimality Criteria	p. 16
Average Optimality for Finite Models	p. 19
Introduction	p. 19
n-bias Optimality Criteria	p. 20
Difference Formulas of n-biases	p. 23
Characterization of n-bias Policies	p. 29
Computation of n-bias Optimal Policies	p. 36
The Policy Iteration Algorithm for Average Optimality	p. 36
The 0-bias Policy Iteration Algorithm	p. 39
n-bias Policy Iteration Algorithms	p. 43
The Linear Programming Approach	p. 46
Linear Programming for Ergodic Models	p. 46
Linear Programming for Multichain Models	p. 49
Notes	p. 52
Discount Optimality for Nonnegative Costs	p. 55
Introduction	p. 55
The Nonnegative Model	p. 55
Preliminaries	p. 56
The Discounted Cost Optimality Equation	p. 60
Existence of Optimal Policies	p. 63
Approximation Results	p. 63
The Policy Iteration Approach	p. 66
Examples	p. 68
Notes	p. 69
Average Optimality for Nonnegative Costs	p. 71
Introduction	p. 71
The Average-Cost Criterion	p. 72
The Minimum Nonnegative Solution Approach	p. 73
The Average-Cost Optimality Inequality	p. 76
The Average-Cost Optimality Equation	p. 80
Examples	p. 81
Notes	p. 84
Discount Optimality for Unbounded Rewards	p. 87
Introduction	p. 87
The Discounted-Reward Optimality Equation	p. 89
Discount Optimal Stationary Policies	p. 95
A Value Iteration Algorithm	p. 98
Examples	p. 98
Notes	p. 102
Average Optimality for Unbounded Rewards	p. 105
Introduction	p. 105
Exponential Ergodicity Conditions	p. 106
The Existence of AR Optimal Policies	p. 109
The Policy Iteration Algorithm	p. 113
Examples	p. 119
Notes	p. 124
Average Optimality for Pathwise Rewards	p. 127
Introduction	p. 127
The Optimal Control Problem	p. 129
Optimality Conditions and Preliminaries	p. 129
The Existence of PAR Optimal Policies	p. 131
Policy and Value Iteration Algorithms	p. 138
An Example	p. 139
Notes	p. 142
Advanced Optimality Criteria	p. 143
Bias and Weakly Overtaking Optimality	p. 143
Sensitive Discount Optimality	p. 147
Blackwell Optimality	p. 159
Notes	p. 160
Variance Minimization	p. 163
Introduction	p. 163
Preliminaries	p. 164
Computation of the Average Variance	p. 164
Variance Minimization	p. 170
Examples	p. 171
Notes	p. 173
Constrained Optimality for Discount Criteria	p. 175
The Model with a Constraint	p. 175
Preliminaries	p. 177
Proof of Theorem 11.4	p. 182
An Example	p. 184
Notes	p. 186
Constrained Optimality for Average Criteria	p. 187
Average Optimality with a Constraint	p. 187
Preliminaries	p. 188
Proof of Theorem 12.4	p. 192
An Example	p. 192
Notes	p. 194
	p. 195
Limit Theorems	p. 195
Results from Measure Theory	p. 197
	p. 203
Continuous-Time Markov Chains	p. 203
Stationary Distributions and Ergodicity	p. 206
	p. 209
The Construction of Transition Functions	p. 209
Ergodicity Based on the Q-Matrix	p. 214
Dynkin's Formula	p. 218
References	p. 221
Index	p. 229
Table of Contents provided by Ingram. All Rights Reserved.

Continuous-Time Markov Decision Processes

Theory and Applications

At a Glance

Hardcover

Industry Reviews

Shipping

How to return your order

Defective items

You Can Find This Book In

Other Editions and Formats

Paperback

More in Mathematics

Speed

How it Explains the World

The Infinite Game

From the bestselling author of Start With Why

Primary Mathematics

4th Edition - Integrating Theory with Practice

The Art of Gathering

How We Meet and Why It Matters

Collins Classics - The Republic

Collins Classics

How to Win At Chess

The Ultimate Guide for Beginners and Beyond

The Maths Book

Big Ideas Simply Explained

The Selfish Gene

40th Anniversary edition

The Mending of Broken Bones

A Modern Guide to Classical Algebra

Fooled By Randomness

The Hidden Role of Chance in Life and in the Markets

Math with Bad Drawings

Illuminating the Ideas That Shape Our Reality

Math Games with Bad Drawings

75 1/4 Simple, Challenging, Go-Anywhere Games & And Why They Matter

The Art of Uncertainty

How to Navigate Chance, Ignorance, Risk and Luck

Geometry

Grades 6 - 8

Grade 3 Geometry and Measurement

Intro to Geometry

Grades 6 - 8

Algebra Workbook Grades 6-8

Algebra

Woo's Wonderful World of Maths

Oxford Maths for Australian Schools Year 4 Value Pack

3rd Edition

Linear Partial Differential and Difference Equations and Simultaneous Systems with Constant or Homogeneous Coefficients

Maths in Focus 11 Mathematics Advanced Student Book with Nelson MindTap

Abel-Gontcharoff Pseudopolynomials and Stochastic Applications

ISTE Invoiced

Grade 4 Geometry and Measurement

Kumon Math Workbooks

Pre-Algebra Workbook Grades 6-8

Algebra

This product is categorised by