Stochastic Processes

AY2018/2019 Semester 1

This course will enable you to analyze random systems and their long-run behavior by the computation of key quantities such as hitting probabilities and mean hitting times. On a more global scale, it aims at training undergraduates to design stochastic models with potential applications in various fields, such as biology, economics, finance, physics, analytics and data science. Course Content 1 Probability Background 1.1 Probability Spaces and Events 1.2 Probability Measures 1.3 Conditional Probabilities and Independence 1.4 Random Variables 1.5 Probability Distributions 1.6 Expectation of Random Variables 1.7 Moment and Probability Generating Functions 2 Gambling Problems 2.1 Constrained Random Walk 2.2 Ruin Probabilities 2.3 Mean Game Duration 3 Random Walks 3.1 Unrestricted Random Walk 3.2 Mean and Variance 3.3 Distribution 3.4 First Return to Zero 4 Discrete-Time Markov Chains 4.1 Markov Property 4.2 Transition matrix 4.3 Examples of Markov Chains 4.4 Higher-Order Transition Probabilities 4.5 The Two-State Discrete-Time Markov Chain 5 First Step Analysis 5.1 Hitting Probabilities 5.2 Mean Hitting and Absorption Times 5.3 First Return Times 5.4 Mean Number of Returns 6 Classification of States 6.1 Communicating States 6.2 Recurrent States 6.3 Transient States 6.4 Positive vs Null Recurrence 6.5 Periodicity and Aperiodicity 7 Long-Run Behavior of Markov Chains 7.1 Limiting Distributions 7.2 Stationary Distributions 7.3 Markov Chain Monte Carlo 8 Branching Processes 8.1 Definition and Examples 8.2 Probability Generating Functions 8.3 Extinction Probabilities 9 Continuous-Time Markov Chains 9.1 The Poisson Process 9.2 Continuous-Time Markov Chains 9.3 Transition Semigroup 9.4 Infinitesimal Generator 9.5 The Two-State Continuous-Time Markov Chain 9.6 Limiting and Stationary Distributions 9.7 The Discrete-Time Embedded Chain 9.8 Mean Absorption Time and Probabilities