"This text is unique in bringing together so many results hitherto found only in part in other texts and papers. . . . The text is fairly self-contained, inclusive of some basic mathematical results needed, and provides a rich diet of examples, applications, and exercises. The bibliographical material at the end of each chapter is excellent, not only from a historical perspective, but because it is valuable for researchers in acquiring a good perspective of the MDP research potential."
Zentralblatt fur Mathematik
". . . it is of great value to advanced-level students, researchers, and professional practitioners of this field to have now a complete volume (with more than 600 pages) devoted to this topic. . . . Markov Decision Processes: Discrete Stochastic Dynamic Programming represents an up-to-date, unified, and rigorous treatment of theoretical and computational aspects of discrete-time Markov decision processes."
Journal of the American Statistical Association
2. Model Formulation.
4. Finite-Horizon Markov Decision Processes.
5. Infinite-Horizon Models: Foundations.
6. Discounted Markov Decision Problems.
7. The Expected Total-Reward. Criterion.
8. Average Reward and Related Criteria.
9. The Average Reward Criterion-Multichain and Communicating Models.
10. Sensitive Discount Optimality.
11. Continuous-Time Models.
Appendix A. Markov Chains.
Appendix B. Semicontinuous Functions.
Appendix C. Normed Linear Spaces.
Appendix D. Linear Programming.