Modeling and Analysis of Stochastic Systems (Hardcover, 3)

Introduction
What in the World is a Stochastic Process?
How to Characterize a Stochastic Process
What Do We Do with a Stochastic Process?

Discrete-Time Markov Chains: Transient Behaviour
Definition and Characterization
Examples
DTMCs in Other Fields
Marginal Distributions
Occupancy Times
Computation of Matrix Powers
Modeling Exercises
Computational Exercises
Conceptual Exercises

Discrete-Time Markov Chains: First Passage Times
Definitions
Cumulative Distribution Function of T
Absorption Probabilities
Expectation of T
Generating Function and Higher Moments of T
Computational Exercises
Conceptual Exercises

Discrete-Time Markox Chains: Limiting Behaviour
Exploring the Limiting Behaviour by Examples
Classification of States
Determining Recurrence and Transience: Finite DTMCs
Determining Recurrence and Transience: Infinite DTMSc
Limiting Behaviour of Irreducible DTMCs
Examples: Limiting Behaviour of Infinite State-Space Irreducible DTMCs
Limiting Behaviour of Reducible DTMCs
DTMCs with Costs and Rewards
Reversibility
Computational Exercises
Conceptual Exercises

Poisson Processes
Exponential Distributions
Poisson Process: Definitions
Event Times in a Poisson Process
Superposition and Splitting of Poisson Processes
Non-Homogeneous Poisson Process
Compound Poisson Process
Computational Exercises
Conceptual Exercises

Continuous-Time Markov Chains
Definitions and Sample Path Properties
Examples
CTMCs in Other Fields
Transient Behaviour: Marginal Distribution
Transient Behaviour: Occupancy Times
Computation of P(t): Finite State-Space
Computation of P(t): Infinite State-Space
First-Passage Times
Exploring the Limiting Behaviour by Examples
Classification of States
Limiting Behaviour of Irreducible CTMCs
Limiting Behaviour of Reducible CTMCs
CTMCs with Costs and Rewards
Phase Type Distributions
Reversibility
Modeling Exercises
Computational Exercises
Conceptual Exercises

Queueing Models
Introduction
Properties of General Queueing Systems
Birth and Death Queues
Open Queueing Networks
Closed Queueing Networks
Single Server Queues
Retrial Queue
Infinite Server Queue
Modeling Exercises
Computational Exercises

Renewal Processes
Introduction
Properties of N(t)
The Renewal Function
Renewal-Type Equation
Key Renewal Theorem
Recurrence Times
Delayed Renewal Processes
Semi-Markov Processes
Renewal Processes with Costs/Rewards
Regenerative Processes
Computational Exercises
Conceptual Exercises

Markov Regenerative Processes
Definitions and Examples
Markov Renewal Process and Markov Renewal Function
Key Renewal Theorem for MRPs
Semi-Markov Processes: Further Results
Markov Regenerative Processes
Applications to Queues
Modeling Exercises
Computational Exercises
Conceptual Exercises

Diffusion Process
Brownian Motion
Sample Path Properties of BM
Kolmogorov Equations for Standard Brownian Motion
First Passage Times
Reflected SBM
Reflected BM and Limiting Distributions
BM and Martingales
Cost/Reward Models
Stochastic Integration
Stochastic Differential Equations and Ito's Formula
Applications to Finance
Computational Exercises
Conceptual Exercises

Epilogue

Probability of Events

Appendices
Univariate Random Variables
Multivariate Random Variables
Generating Functions
Laplace-Stieltjes Transforms
Laplace Transforms
Modes of Convergence
Results from Analysis
Difference and Differential Equations

Answers to Selected Problems

References

Index