Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach (Hardcover, 2, 2013)

Prologue and Preliminaries : Introduction and overview- Mathematical preliminaries.- Markovian models.- Two-Time-Scale Markov Chains : Asymptotic Expansions of Solutions for Forward Equations.- Occupation Measures: Asymptotic Properties and Ramification.- Asymptotic Expansions of Solutions for Backward Equations.- Applications:MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching: Markov Decision Problems.- Stochastic Control of Dynamical Systems.- Numerical Methods for Control and Optimization.- Hybrid LQG Problems.- References.- Index.-