Essentials of Stochastic Processes (Hardcover, 3, 2016)

1) Markov Chains 1.1 Definitions and Examples 1.2 Multistep Transition Probabilities 1.3 Classification of States 1.4 Stationary Distributions 1.4.1 Doubly stochastic chains 1.5 Detailed balance condition 1.5.1 Reversibility 1.5.2 The Metropolis-Hastings algorithm 1.5.3 Kolmogorow cycle condition 1.6 Limit Behavior 1.7 Returns to a fixed state 1.8 Proof of the convergence theorem 1.9 Exit Distributions 1.10 Exit Times 1.11 Infinite State Spaces 1.12 Chapter Summary 1.13 Exercises 2) Poisson Processes 2.1 Exponential Distribution 2.2 Defining the Poisson Process 2.2.1 Constructing the Poisson Process 2.2.2 More realistic models 2.3 Compound Poisson Processes 2.4 Transformations 2.4.1 Thinning 2.4.2 Superposition 2.4.3 Conditioning 2.5 Chapter Summary 2.6 Exercises 3) Renewal Processes 3.1 Laws of Large Numbers 3.2 Applications to Queueing Theory 3.2.1 GI/G/1 queue 3.2.2 Cost equations 3.2.3 M/G/1 queue 3.3 Age and Residual Life 3.3.1 Discrete case 3.3.2 General case 3.4 Chapter Summary 3.5 Exercises 4) Continuous Time Markov Chains 4.1 Definitions and Examples 4.2 Computing the Transition Probability 4.2.1 Branching Processes 4.3 Limiting Behavior 4.3.1 Detailed balance condition 4.4 Exit Distributions and Exit Times 4.5 Markovian Queues 4.5.1 Single server queues 4.5.2 Multiple servers 4.5.3 Departure Processes 4.6 Queueing Networks 4.7 Chapter Summary 4.8 Exercises 5) Martingales 5.1 Conditional Expectation 5.2 Examples 5.3 Gambling Strategies, Stopping Times 5.4 Applications 5.4.1 Exit distributions 5.4.2 Exit times 5.4.3 Extinction and ruin probabilities 5.4.4 Positive recurrence of the GI/G/1 queue 5.5 Exercises 6) Mathematical Finance 6.1 Two Simple Examples 6.2 Binomial Model 6.3 Concrete Examples 6.4 American Options 6.5 Black-Scholes formula 6.6 Calls and Puts 6.7 Exercises A) Review of Probability A.1 Probabilities, Independence A.2 Random Variables, Distributions A.3 Expected Value, Moments A.4 Integration to the Limit