Fundamentals of Stochastic Models (Paperback, 1)

1. Introduction. Part I. Fundamentals of Stochastic Models. 2. Discrete-time Markov Chains. 3. Continuous-Time Markov Chains. 4. Structured Markov Chains. 5. Renewal Processes and Embedded Markov Chains. 6. Random Walks and Brownian Motions. 7. Reflected Brownian Motion Approximations to Simple Stochastic Systems. 8. Large Queueing Systems. 9. Static Optimization in Stochastic Models. 10. Dynamic Optimization in Stochastic Models. 11. Learning in Stochastic Models. Part II. Appendices: Elements of Probability and Stochastics. A. Basics of Probability Theory. B. Conditional Expectation and Martingales. C. Some Useful Bounds, Inequalities, and Limit Laws. D. Non-linear Programming in Stochastics. E. Change of Probability Measure for a Normal Random Variable. F. Convergence of Random Variables. G. Major Theorems for Stochastic Process Limits. H. A Brief Review on Stochastic Calculus. I. Comparison of Stochastic Processes - Stochastic Orders. J. Matrix Algebra and Markov Chains.